提问人:AnLucKa 提问时间:10/23/2023 更新时间:10/23/2023 访问量:59
错误 pyppeteer.errors.PageError:使用 requests_html 时出现 net::ERR_SPDY_PROTOCOL_ERROR
Error pyppeteer.errors.PageError: net::ERR_SPDY_PROTOCOL_ERROR when working with requests_html
问:
我想从 https://announcements.bybit.com/en-US/?category=new_crypto&page=1 解析数据,我需要执行JS(这是测试用例中的条件)。 我使用以下代码:
from requests_html import HTMLSession
from fake_useragent import UserAgent
from bs4 import BeautifulSoup as BS
ua = UserAgent()
headers = {
'User-Agent': ua.random,
'Accept': '*/*'
}
url = "https://announcements.bybit.com/en-US/?category=new_crypto&page=1"
session = HTMLSession()
r = session.get(url, headers=headers)
r.html.render()
如果没有 html.render,它就不会出错,并返回以下内容: 但是当我尝试渲染JS时,会引发以下异常:
Traceback (most recent call last):
File "/home/anlucka/PycharmProjects/test_parse/test_parse/main.py", line 16, in <module>
r.html.render()
File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/requests_html.py", line 598, in render
content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/requests_html.py", line 512, in _async_render
await page.goto(url, options={'timeout': int(timeout * 1000)})
File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/pyppeteer/page.py", line 831, in goto
raise PageError(result)
pyppeteer.errors.PageError: net::ERR_SPDY_PROTOCOL_ERROR at https://announcements.bybit.com/en-US/?category=new_crypto&page=1
答: 暂无答案
评论