SQLITE3 在错误的行上插入数据

SQLITE3 Inserting data on wrong row

提问人:Renan Liporaci 提问时间:10/10/2023 更新时间:10/11/2023 访问量:46

问:

因此,首次创建网络抓取。在获得积极的结果后,它应该将其写入 sqlite3。问题是:出于某种原因,它跳过了第二列上的一行。我尝试了很多更改,但对修复一无所知。

代码到网络吱吱作响:

from selenium import webdriver
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import time
from sqlprocessos import get_numproc, insert_data

options = webdriver.ChromeOptions()
options.add_argument("--headless=new")



def get_process_date(url, process):
    options = webdriver.ChromeOptions()
    options.add_argument("--headless=new")
    driver = webdriver.Chrome(options=options)
    print(url)
    driver.get(url)
    time.sleep(5)
    driver.switch_to.frame('mainframe')

    iframe_content = driver.page_source

    soup = BeautifulSoup(iframe_content, 'html.parser')

    body_element = soup.find('app-movimento')
    possibel_mov = ['Data da juntada: ', 'Data de recebimento: ', 'Data de remessa: ', 'Data do trânsito: ', 'Data de arquivamento: ', 'Data da conclusão: ', 'Data: ', 'Data de devolução: ']
    possibel_desc =['Juiz: ', 'Descrição da Juntada: ', 'Descrição', 'Tipo de arquivamento: ', 'Descrição: ']
    data_da_juntada = ''
    descricao_da_juntada = ''
 
    def get_data(possible_mov):
        data_da_juntada = soup.find('label', text= possibel_mov).find_next('label').text.strip()
        print(f"Data: {data_da_juntada}")
        return data_da_juntada
    for i in possibel_mov:
       try:
            data = get_data(i)
            if type(data) == str:
                insert_data("p" + str(process).replace('-','').replace('.',''),data,descricao_da_juntada)
                    
       except Exception as e:
            pass
        
    
    def get_descricao(possible_desc):
        descricao_da_juntada = soup.find('label', text= possibel_desc).find_next('label').text.strip()
        print(f"{possibel_desc}: {descricao_da_juntada}")
        return descricao_da_juntada
    for d in possibel_desc:
        try:
            data = get_descricao(d)
            if type(data) == str:
                insert_data("p" + str(process).replace('-','').replace('.',''),data_da_juntada,data)
                
            print(get_descricao(d))
            
        except Exception as e:
            pass

    driver.switch_to.default_content()
    driver.quit()


base_url = 'https://www3.tjrj.jus.br/consultaprocessual/#/consultapublica?numProcessoCNJ='


num_procs = get_numproc()
for num in num_procs:
    url = base_url + num
    get_process_date(url, num)
    time.sleep(5)

编写它的函数

def insert_data(tabela, data, descricao):
comando = f'INSERT INTO {tabela} (DATA, MOVIMENTO) VALUES (?, ?)'
valores = (data, descricao)
cursor.execute(comando,valores)
print(cursor)
conexao.commit()

结果:

enter image description here

关于这个主题有什么提示吗?我知道代码不漂亮,但应该可以工作

python-3.x sqlite sqlite3-python

评论

0赞 JonSG 10/10/2023
您传递的参数顺序在您进行的两次调用之间不同有关系吗?insert_data()
0赞 Renan Liporaci 10/10/2023
我不这么认为。就数据而言,看到的是正确的。你认为这应该是问题的原因吗?
1赞 JonSG 10/10/2023
你确定?一次你做,下一次你做insert_data("<table_name>", data, descricao_da_juntada) insert_data("<table_name>", data_da_juntada, data)
0赞 furas 10/10/2023
最大的错误是.你可能会遇到一些错误,这会让你的所有问题都变成你不知道的。也许这个错误可以解释一切。您至少应该显示错误except Exception as e: passexcept Exception as e: print(e)
0赞 Renan Liporaci 10/10/2023
@JonSG是的。顺序不同,因为第二个 insert_data() 将抓取数据放在第二列上。我更改了,只是为了验证,确实在第一列提交了数据

答:

1赞 Renan Liporaci 10/11/2023 #1

我发现问题出在两个for循环上。如果仅包含在一个 for 循环中,则它可以正常工作。

data2 = ''
for i in possivel_mov:
   try:
        data = get_data(i)
        if type(data) == str:
            for d in possivel_desc:
                try:
                    data2 = get_descricao(d)
                    if type(data2) == str:
                           insert_data("p" + str(process).replace('-','').replace('.',''),data,data2)
                except Exception as e:
                    pass
            #insert_data("p" + str(process).replace('-','').replace('.',''),data,descricao_da_juntada)
            
   except Exception as e:
        pass