Python获取去哪儿旅游数据并作可视化攻略

嗨喽，大家好呀~这里是爱看美女的茜茜呐

开发环境：

• 版 本： python 3.8

• 编辑器：pycharm

  第三方库：

  • requests >>> pip install requests

  • parsel >>> pip install parsel

    模块安装:

    按住键盘 win + r, 输入cmd回车 打开命令行窗口, 在里面输入 pip install 模块名

    ---

    👇 👇 👇 更多精彩机密、教程，尽在下方，赶紧点击了解吧~

    python源码、视频教程、插件安装教程、资料我都准备好了，直接在文末名片自取就可

    ---

    需求分析

    确定需要的数据

    找数据真实来源

    https://travel.qunar.com/travelbook/list.htm?order=hot_heat

    静态数据

    代码实现步骤

    1. 发送请求

    2. 获取数据

    3. 提取数据

    4. 保存数据

    数据获取

    导入模块

    import requests     # 发送请求 代码当中用来访问网站的模块
    import parsel       # 解析数据的模块
    import csv
    

    with open('攻略.csv', mode='a', encoding='utf-8', newline='') as f:
        csv_writer = csv.writer(f)
        csv_writer.writerow(['title', 'date', 'days', 'photo_nums', 'fee', 'people', 'trip', 'view', 'love', 'comment', 'href'])
    for page in range(1, 201):
        url = f'https://travel.qunar.com/travelbook/list.htm?page={page}&order=hot_heat'
    

    1. 发送请求

        response = requests.get(url=url)
    

    1. 获取数据

        html_data = response.text
    

    1. 提取数据

        # re / css / xpath
        # css: ul.b_strategy_list > li
        # xpath: //ul[@]/li
        select = parsel.Selector(html_data)
        lis = select.xpath('//ul[@]/li')
        # 二次的数据提取
        for li in lis:
            # li.css('h2 > a::text').get()
            title = li.xpath('./h2/a/text()').get()
            date = li.xpath('./p[@]//span[@]/text()').get("")
            days = li.xpath('./p[@]//span[@]/text()').get("")
            photo_nums = li.xpath('./p[@]//span[@]/text()').get("")
            fee = li.xpath('./p[@]//span[@]/text()').get("")
            people = li.xpath('./p[@]//span[@]/text()').get("")
            trip = li.xpath('./p[@]//span[@]/text()').get("")
            view = li.xpath('./p[@]//span[@]/span/text()').get("")
            love = li.xpath('./p[@]//span[@]/span/text()').get("")
            comment = li.xpath('./p[@]//span[@]/span/text()').get("")
            href = li.xpath('./h2/a/@href').get()
            print(title, date, days, photo_nums, fee, people, trip, view, love, comment, href)
    

    1. 保存数据

            with open('攻略.csv', mode='a', encoding='utf-8', newline='') as f:
                csv_writer = csv.writer(f)
                csv_writer.writerow([title, date, days, photo_nums, fee, people, trip, view, love, comment, href])
    

    数据可视化

    import pandas as pd
    from pyecharts.commons.utils import JsCode
    from pyecharts.charts import *
    from pyecharts import options as opts
    

    data = pd.read_csv('去哪儿_数分.csv')
    data
    

    

    data.info()
    

    data = data[~data['地点'].isin(['攻略'])]
    data = data[~data['天数'].isin(['99+'])]
    data
    

    data.drop_duplicates(inplace=True)
    

    data['人均费用'].fillna(0, inplace=True)
    data['人物'].fillna('独自一人', inplace=True)
    data['玩法'].fillna('没有', inplace=True)
    

    data['天数'] = data['天数'].astype(int)
    

    data = data[data['人均费用'].values>200]
    data = data[data['天数']}
    for j in range(len(word_list)):
        for i in word_list[j]:
            if i not in dict:
                dict[i] = 1
            else:
                dict[i]+=1
    #print(dict)
    list = []
    for item in dict.items():
        list.append(item)
    for i in range(1,len(list)):
        for j in range(0,len(list)-1):
            if list[j][1]"normal": {
         "barBorderRadius": [30, 30, 30, 30],
         'shadowBlur': 10,
         'shadowColor': 'rgba(0,191,255,0.5)',
         'shadowOffsetY': 1,
         'opacity': 0.8
     }
                                           })
            
                            )
    pie.render('3.html')