提问人:HumanzIzDead 提问时间:10/6/2023 最后编辑:HumanzIzDead 更新时间:10/11/2023 访问量:43
如果处理时间过长,shutil.move 会引发错误
shutil.move throwing errors if processing time takes too long
问:
我有一个 python 程序,它拾取一些文件,通过机器学习模型运行它们,并将输出文件写入另一个文件夹。它还将未处理的文件移动到另一个文件夹,以指示它已被程序处理。下面是执行这些操作的代码。
for f in files:
try:
if path.exists(path.join(processed_folder, f)):
print(f'File {f} has already been processed')
shutil.move(path.join(folder_to_moniter, f), path.join(processed_folder, f))
else:
print(f'Started processing file "{f}"')
convert_questionniare_using_ml(f, folder_to_moniter, modelFilesPath, keywords_list)
shutil.move(path.join(folder_to_moniter, f), path.join(processed_folder, f))
time.sleep(10)
print(f'Completed processing file "{f}"')
except Exception:
shutil.move(path.join(folder_to_moniter, f), error_folder)
print(f'Error Occurred"{Exception.with_traceback()}"')
该模型还会从另一个文件夹中读取训练数据,并在每次调用“convert_questionnaire_using_ml”时训练自身。我将来会更新它,以便仅在我们想要更新模型时训练自己。我还在更新模型以对其中一个目标类进行过采样以提高其性能。但是,当我这样做时,训练数据会变大,因此模型需要更长的时间来训练。由于某种原因,这会导致“shutil.move”方法出现故障,引发一堆错误。
这些错误表明,从“folder_to_monitor”移动到“processed_folder”时找不到“folder_to_monitor”中的文件。但是,在文件系统中,一切都正确执行。将文件移动到应有的位置,并生成输出文件。
真的很困惑为什么会这样。在“convert_questionnaire_using_ml”功能完成之前,不应移动该文件。事实上,该文件确实会移动,但控制台输出却另有说明。
我对代码进行了几次更改,但最终确定在训练数据发生过采样之前不会发生此错误。我只能想象训练所需的时间长短在某种程度上影响了代码的执行。
如果有帮助,以下是编辑后的堆栈跟踪:
Traceback (most recent call last):
File "App.py", line 125, in main
shutil.move(path.join(folder_to_moniter, f), path.join(processed_folder, f))
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 564, in move
copy_function(src, real_dst)
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 263, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'folder_to_monitor\f'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 550, in move
os.rename(src, real_dst)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'folder_to_monitor\f' -> 'error_folder\f'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "App.py", line 135, in <module>
main()
File "App.py", line 130, in main
shutil.move(path.join(folder_to_moniter, f), error_folder)
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 564, in move
copy_function(src, real_dst)
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 263, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\tools\Anaconda3\envs\test_env\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'folder_to_monitor\f'
答: 暂无答案
评论
f
files