selenium模块
selenium基本概念
selenium优势
- 便捷的获取网站中动态加载的数据
- 便捷实现模拟登陆
selenium使用流程：
1.环境安装：pip install selenium
2.下载一个浏览器的驱动程序（谷歌浏览器）
3.实例化一个浏览器对象
基本使用
代码
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
from selenium import webdriverfrom lxml import etreefrom time import sleepif __name__ == '__main__': bro = webdriver.Chrome(r"E:\google\Chrome\Application\chromedriver.exe") bro.get(url='http://scxk.nmpa.gov.cn:81/xk/') page_text = bro.page_source tree = etree.HTML(page_text) li_list = tree.xpath('//*[@id="gzlist"]/li') for li in li_list: name = li.xpath('./dl/@title')[0] print(name) sleep(5) bro.quit() |
基于浏览器自动化的操作
代码
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
#编写基于浏览器自动化的操作代码- 发起请求: get(url)- 标签定位: find系列的方法- 标签交互: send_ keys( 'xxx' )- 执行js程序: excute_script('jsCod')- 前进，后退: back(),forward( )- 关闭浏览器: quit() |
代码
https://www.taobao.com/
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
from selenium import webdriverfrom time import sleepbro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")bro.get(url='https://www.taobao.com/')#标签定位search_input = bro.find_element_by_id('q')sleep(2)#执行一组js代码，使得滚轮向下滑动bro.execute_script('window.scrollTo(0,document.body.scrollHeight)')sleep(2)#标签交互search_input.send_keys('女装')button = bro.find_element_by_class_name('btn-search')button.click()bro.get('https://www.baidu.com')sleep(2)bro.back()sleep(2)bro.forward()sleep(5)bro.quit() |
selenium处理iframe：
|
1
2
3
4
5
6
7
8
|
- 如果定位的标签存在于iframe标签之中，则必须使用switch_to.frame(id)- 动作链(拖动) : from selenium. webdriver import ActionChains - 实例化一个动作链对象: action = ActionChains (bro) - click_and_hold(div) :长按且点击操作 - move_by_offset(x,y) - perform( )让动作链立即执行 - action.release( )释放动作链对象 |
代码
https://www.runoob.com/try/try.php?filename=jqueryui-api-droppable
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
from selenium import webdriverfrom time import sleepfrom selenium.webdriver import ActionChainsbro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")bro.get('https://www.runoob.com/try/try.php?filename=jqueryui-api-droppable')bro.switch_to.frame('iframeResult')div = bro.find_element_by_id('draggable')#动作链action = ActionChains(bro)action.click_and_hold(div)for i in range(5): action.move_by_offset(17,0).perform() sleep(0.3)#释放动作链action.release()bro.quit() |
selenium模拟登陆QQ空间
代码
https://qzone.qq.com/
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
from selenium import webdriverfrom time import sleepbro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe")bro.get('https://qzone.qq.com/')bro.switch_to.frame("login_frame")switcher = bro.find_element_by_id('switcher_plogin')switcher.click()user_tag = bro.find_element_by_id('u')password_tag = bro.find_element_by_id('p')user_tag.send_keys('1234455')password_tag.send_keys('qwer123')sleep(1)but = bro.find_element_by_id('login_button')but.click() |
无头浏览器和规避检测
代码
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
from selenium import webdriverfrom time import sleep#实现无可视化界面from selenium.webdriver.chrome.options import Options#实现规避检测from selenium.webdriver import ChromeOptions#实现无可视化界面chrome_options = Options()chrome_options.add_argument('--headless')chrome_options.add_argument('--disable-gpu')#实现规避检测option = ChromeOptions()option.add_experimental_option('excludeSwitches',['enable-automation'])bro = webdriver.Chrome(executable_path=r"E:\google\Chrome\Application\chromedriver.exe",chrome_options=chrome_options,options=option)bro.get('https://www.baidu.com')print(bro.page_source)sleep(2)bro.quit() |
到此这篇关于python爬虫selenium模块详解的文章就介绍到这了,更多相关python爬虫selenium模块内容请搜索服务器之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持服务器之家！
原文链接：https://blog.csdn.net/qq_43710889/article/details/115269951
