错误 pyppeteer.errors.PageError:使用 requests_html 时出现 net::ERR_SPDY_PROTOCOL_ERROR

Error pyppeteer.errors.PageError: net::ERR_SPDY_PROTOCOL_ERROR when working with requests_html

提问人:AnLucKa 提问时间:10/23/2023 更新时间:10/23/2023 访问量:59

问:

我想从 https://announcements.bybit.com/en-US/?category=new_crypto&page=1 解析数据,我需要执行JS(这是测试用例中的条件)。 我使用以下代码:

from requests_html import HTMLSession
from fake_useragent import UserAgent
from bs4 import BeautifulSoup as BS


ua = UserAgent()
headers = {
    'User-Agent': ua.random,
    'Accept': '*/*'
}
url = "https://announcements.bybit.com/en-US/?category=new_crypto&page=1"


session = HTMLSession()
r = session.get(url, headers=headers)
r.html.render()

如果没有 html.render,它就不会出错,并返回以下内容: 但是当我尝试渲染JS时,会引发以下异常: enter image description here

Traceback (most recent call last):
  File "/home/anlucka/PycharmProjects/test_parse/test_parse/main.py", line 16, in <module>
    r.html.render()
  File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/requests_html.py", line 598, in render
    content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/requests_html.py", line 512, in _async_render
    await page.goto(url, options={'timeout': int(timeout * 1000)})
  File "/home/anlucka/PycharmProjects/test_parse/.venv/lib/python3.10/site-packages/pyppeteer/page.py", line 831, in goto
    raise PageError(result)
pyppeteer.errors.PageError: net::ERR_SPDY_PROTOCOL_ERROR at https://announcements.bybit.com/en-US/?category=new_crypto&page=1
python-3.x 解析 python-requests-html spdy

评论


答: 暂无答案