提问人:Limaximy 提问时间:7/8/2023 最后编辑:Limaximy 更新时间:7/8/2023 访问量:47
我使用带有 selenium 和漂亮汤的 python 解析器对 VK 站点的页面进行排序。一个难以理解的错误出现在一个随机的地方 [重复]
I use python parser with selenium and beautiful soup to sort through pages from the VK site. An incomprehensible error appears in a random place [duplicate]
问:
我使用带有 selenium 和 beautiful soup 的 python 解析器来遍历来自一个社交网络的页面。最重要的是,该程序加载了大量指向用户页面的链接,然后在一个周期中分析每个页面并输出数据(姓名、朋友、订阅等)。在循环的某个时刻,会发生错误,不会加载下一页。此外,它似乎发生在一个随机的地方,即我在相同的数据上检查了它。
循环代码:
people = BSFile.find_all(attrs={"class": "fans_fan_lnk"})
for i in range(len(people)):
print(people[i]["href"])
x = int(input())
for j in range(len(people)):
time.sleep(0.1)
driver.get("https://vk.com"+str(people[j]['href']))
time.sleep(0.1)
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "AvatarRich__background")))
main_page = driver.page_source
time.sleep(0.1)
BSFile = bs(main_page, "html.parser")
MasForUnpack = []
if BSFile.find(attrs = {"class": "ClosedProfileBlock-module__title--eb2WU vkuiTitle vkuiTitle--l-3 vkuiTitle--w-2"}) != None:
print("Профиль закрыт")
Numbers = BSFile.find_all(attrs={"class": "vkuiHeader__indicator vkuiCaption vkuiCaption--l-1"})
TagOfName = BSFile.h2
for st in TagOfName.stripped_strings:
MasForUnpack.append(st)
if (len(MasForUnpack) == 2):
if (MasForUnpack[1].find('заходил') == -1) and (MasForUnpack[1].find('онлайн')== -1):
print(MasForUnpack[0] + " " + MasForUnpack[1])
else:
print(MasForUnpack[0])
if (len(MasForUnpack) == 3):
print(MasForUnpack[0] + " " + MasForUnpack[1])
#Names = BSFile.find_all(attrs={"class":"vkuiHeader__content-in"})
for i in range(len(Numbers)):
print("Друзья: "+ Numbers[i].string)
else:
print("Профиль открыт")
Numbers = []
Names = []
Numbers = BSFile.find_all(attrs={"class": "vkuiHeader__indicator vkuiCaption vkuiCaption--l-1 vkuiCaption--w-1"})
Names = BSFile.find_all(attrs={"class":"vkuiHeader__content-in"})
TagOfName = BSFile.h2
for st in TagOfName.stripped_strings:
MasForUnpack.append(st)
if (len(MasForUnpack) == 2):
if (MasForUnpack[1].find('заходил') == -1) and (MasForUnpack[1].find('онлайн')== -1):
print(MasForUnpack[0] + " " + MasForUnpack[1])
else:
print(MasForUnpack[0])
if (len(MasForUnpack) == 3):
print(MasForUnpack[0] + " " + MasForUnpack[1])
for i in range(len(Numbers)):
print(Names[i].string + ": "+ Numbers[i].string)
print()
错误消息:
Traceback (most recent call last):
File "G:\_prog_files\Html\proba\2.py", line 51, in <module>
driver.get("https://vk.com"+str(people[j]['href']))
File "G:\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 441, in get
self.execute(Command.GET, {'url': url})
File "G:\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "G:\Python\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
我试图将 chrome 浏览器更改为 edge,但我没有做任何其他事情,因为我什至不明白问题出在哪里。
还对 chrome 使用了此选项
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options )
在前面的问题中,我没有找到答案,因为我不在linux中工作,而且我没有很好地理解答案的所有要点。
答: 暂无答案
评论