提问人:MattRickS 提问时间:10/14/2023 更新时间:10/14/2023 访问量:40
Python cdll inotify 监视描述符仅在异步任务中失败
Python cdll inotify watch descriptor fails only in async task
问:
我正在尝试在异步任务中运行文件系统观察程序,该任务运行良好,但是当我尝试将监视描述符创建移动到任务中时,它停止了工作。更改该线路的调用位置是导致问题的唯一更改。我在 asyncio 文档中看不到任何表明这是不可能的。
我在下面制作了最小的可重复示例。文件描述符的初始化位置似乎并不重要,重要的是在哪里调用监视。
import asyncio, ctypes, ctypes.util, io, logging, os
# https://man7.org/linux/man-pages/man7/inotify.7.html
class _INotifyEvent(ctypes.Structure):
_fields_ = [
("wd", ctypes.c_int),
("mask", ctypes.c_uint32),
("cookie", ctypes.c_uint32),
("len", ctypes.c_uint32),
# ("name", char[])
]
_libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True)
_libc.inotify_init1.argtypes = (ctypes.c_int,)
_libc.inotify_add_watch.argtypes = (ctypes.c_int, ctypes.c_char_p, ctypes.c_uint)
EVENT_SIZE = ctypes.sizeof(_INotifyEvent) + 1
class FilesystemMonitor:
def __init__(self, path: str, delayed=False):
self._file_descriptor = _libc.inotify_init1(os.O_CLOEXEC)
self._path = path
self._wd = None
if not delayed:
self.add_watch()
def add_watch(self):
self._wd = _libc.inotify_add_watch(self._file_descriptor, bytes(self._path, "utf-8"), 8)
print("WD =", self._wd)
def get(self):
buffer = io.BytesIO(os.read(self._file_descriptor, EVENT_SIZE))
event_buffer = buffer.read(EVENT_SIZE)
event = _INotifyEvent.from_buffer_copy(event_buffer)
return self._path if event.wd == self._wd else "unknown"
async def start(self, queue: asyncio.Queue):
ioloop = asyncio.get_running_loop()
if self._wd is None:
self.add_watch()
def _add_to_queue():
queue.put_nowait(self.get())
ioloop.add_reader(self._file_descriptor, _add_to_queue)
async def run_monitor(monitor):
queue = asyncio.Queue()
asyncio.gather(monitor.start(queue))
while True:
print("!!! EVENT:", await queue.get())
async def run(delayed):
print("Start, delayed =", delayed)
path = "/tmp/file.txt"
monitor = FilesystemMonitor(path, delayed=delayed)
asyncio.create_task(run_monitor(monitor))
with open(path, "w") as f:
f.write("content")
await asyncio.sleep(0.1) # yield so run_monitor reads from queue
print("End")
logging.basicConfig(level=logging.DEBUG)
for val in (True, False):
asyncio.run(run(val))
其输出为
$ touch /tmp/file.txt && PYTHONASYNCIODEBUG=1 python3.11 temp.py
DEBUG:asyncio:Using selector: EpollSelector
Start, delayed = True
WD = 1
End
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>
DEBUG:asyncio:Using selector: EpollSelector
Start, delayed = False
WD = 1
!!! EVENT: /tmp/file.txt
End
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>
在这两种情况下,将以下内容添加到显示的末尾,inotify 都创建了监视描述符。add_watch
with open(f"/proc/{os.getpid()}/fdinfo/{self._file_descriptor}") as f:
print(f.read())
阅读另一个 stackoverflow 问题和与之相关的问题,这意味着 async 对 epoll 的使用阻止了它的工作。但是,如果同一监视是在异步循环之外创建的,则它确实在异步中工作。只是为了检查,我尝试使用以下命令更改异步事件循环策略,并尝试所有其他可用的类,但没有任何改变结果。Selector
import selectors
class MyPolicy(asyncio.DefaultEventLoopPolicy):
def new_event_loop(self):
selector = selectors.SelectSelector()
return asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop_policy(MyPolicy())
尝试通过异步行为进行 PDB 无助于识别任何内容。我担心我只是误解了这里异步行为的一些基本信息。谁能解释为什么异步任务适用于在异步之外创建的手表,但对在其中创建的手表无效?有没有办法让它起作用?
答: 暂无答案
评论