论文总字数：24155字

摘 要

现在各行各业每天都在产生很多数据。中国房地产行业便是如此，尤其是比较发达的城市房源如今很是热门，产生了很多的数据。有很多人在各个网站上发布二手房的信息或者在网上查询二手房的信息。而在人们进行二手房信息搜索的过程中，往往要在各个不同的网页上去查看，耗时耗力很不方便。而且当搜集到相关的二手房信息后，对这些信息进行分类处理也要一条条一条的处理，很不方便，而且容易错。因此，像以前那样手动查询应当被更加高效的方式代替。通过python爬虫技术，不仅可以很快的收集到二手房的数据信息，而且相较于原来的人工一条一条的查询，信息资源利用效率可以大大提高，人们在上面所花的时间也会减少很多。将大量的数据处理之后得到的结果将会更加的可靠合理，而且通过数据可视化后，人们一下子就能知道这些数据在表达着什么。这样直观有有效的表达方式可以省去很多的中间环节，从而提高人们的效率，更有利于人们进行分析决策，挖掘出数据背后隐藏的信息，减少因为信息错误而导致相关问题的可能性。本文将以南京二手房例，以链家网站为二手房数据的来源，编写一个利用python爬虫技术的南京二手房数据抓取和可视化分析的程序。该程序将用python爬取链家二手房的数据，并将爬取到的数据进行清洗，再对处理后的二手房数据进行可视化，便于查看分析数据^【2】。

关键词：Python；爬虫；数据可视化；大数据；二手房

Abstract

Now all walks of life are generating a lot of data every day. This is the case in China's real estate industry, especially in the more developed urban housing market, which is now very popular and generates a lot of data. There are a lot of people in each website to release the information of second-hand houses or to query the information of second-hand houses on the Internet. In the process of second-hand house information search, people often have to view it on different web pages, which is time-consuming and labor-consuming. And when we collect the relevant second-hand house information, we need to deal with these information one by one, which is very inconvenient and easy to make mistakes. Therefore, as before, manual queries should be replaced in a more efficient way. Through the python crawler technology, not only can we quickly collect the data information of the second-hand house, but also compared with the original manual one by one query, the efficiency of information resource utilization can be greatly improved, and the time people spend on it will be reduced a lot. After a large number of data processing, the results will be more reliable and reasonable, and through data visualization, people can immediately know what these data are expressing. This intuitionistic and effective way of expression can save a lot of intermediate links, so as to improve people's efficiency, more conducive to people's analysis and decision-making, mining the hidden information behind the data, reducing the possibility of related problems caused by information errors. In this paper, we will take Nanjing second-hand house as an example, and take the chain home website as the source of second-hand house data, and write a program of data capture and visual analysis of Nanjing second-hand house using Python crawler technology. The program will use Python to crawl the data of the second-hand house of the chain house, clean the crawled data, and then visualize the processed data of the second-hand house, so as to view the analysis data [2].

Key Words：Python; Crawler; Data Visualization; big data; Second-hand house

目录

摘 要 I

Abstract II

目录 III

第一章 绪论 1

1.1研究背景和意义 1

1.2国内外现状 1

1.3研究目标与内容 4

1.3.1研究目标 4

1.3.2研究内容 4

第二章 研究基础 4

2.1主要应用技术 4

2.1.1 Python及科学计算数据分析库 4

2.1.2数据可视化 5

2.1.3聚类算法分析 5

2.2开发环境 6

第三章 数据采集与清洗 7

3.1创建爬虫项目 7

3.1.1分析链家网站结构 7

3.1.2爬虫模块设计 8

3.1.3整体爬虫程序........................................................................................10

3.1.4区域爬虫程序.........................................................................................10

3.2数据清洗 10

3.2.1部分数据修改清洗 14

3.2.2数据清洗的实现及结果 14

第四章 数据可视化分析 15

4.1读入数据 16

4.2南京二手房整体情况分析 16

4.2.1.南京二手房户型分布 16

4.2.2南京各区二手房面积分布 18

4.2.3南京二手房关注度情况 19

4.2.4南京二手房房价前十 19

4.2.5南京各区域房源占比............................................................................22

4.2.6南京各区域中位数价格........................................................................23

4.3南京二手房数据聚类分析 24

4.3.1 k-means算法基本原理 24

4.3.2 k-means算法工作方式 24

4.3.3 k-means缺点 25

4.3.4初始k质心选取 25

4.3.5南京二手房聚类效果散点图 26

4.3.6南京二手房聚类结果分析 28

第五章 结语 29

参考文献 30

致谢 31

第一章 绪论

1.1研究背景和意义

二手房交易随着社会经济的增长和城市化进程的加快日益流行。二手房不仅用于居住，还用于人们的投资。在中国实施住房市场改革后，专门做房产交易的人员出现了。 伴随着互联网的蓬勃发展，已经建立了房地产中介网站。 这些网站提供了大量信息和社区描述性信息，用于住宅物业的租赁和出售，构成了大数据的位置感知形式

请支付后下载全文，论文总字数：24155字