Python cdll inotify 监视描述符仅在异步任务中失败

Python cdll inotify watch descriptor fails only in async task

提问人:MattRickS 提问时间:10/14/2023 更新时间:10/14/2023 访问量:40

问:

我正在尝试在异步任务中运行文件系统观察程序,该任务运行良好,但是当我尝试将监视描述符创建移动到任务中时,它停止了工作。更改该线路的调用位置是导致问题的唯一更改。我在 asyncio 文档中看不到任何表明这是不可能的。

我在下面制作了最小的可重复示例。文件描述符的初始化位置似乎并不重要,重要的是在哪里调用监视。

import asyncio, ctypes, ctypes.util, io, logging, os

# https://man7.org/linux/man-pages/man7/inotify.7.html
class _INotifyEvent(ctypes.Structure):
    _fields_ = [
        ("wd", ctypes.c_int),
        ("mask", ctypes.c_uint32),
        ("cookie", ctypes.c_uint32),
        ("len", ctypes.c_uint32),
        # ("name", char[])
    ]

_libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True)
_libc.inotify_init1.argtypes = (ctypes.c_int,)
_libc.inotify_add_watch.argtypes = (ctypes.c_int, ctypes.c_char_p, ctypes.c_uint)
EVENT_SIZE = ctypes.sizeof(_INotifyEvent) + 1

class FilesystemMonitor:
    def __init__(self, path: str, delayed=False):
        self._file_descriptor = _libc.inotify_init1(os.O_CLOEXEC)
        self._path = path
        self._wd = None
        if not delayed:
            self.add_watch()

    def add_watch(self):
        self._wd = _libc.inotify_add_watch(self._file_descriptor, bytes(self._path, "utf-8"), 8)
        print("WD =", self._wd)

    def get(self):
        buffer = io.BytesIO(os.read(self._file_descriptor, EVENT_SIZE))
        event_buffer = buffer.read(EVENT_SIZE)
        event = _INotifyEvent.from_buffer_copy(event_buffer)
        return self._path if event.wd == self._wd else "unknown"

    async def start(self, queue: asyncio.Queue):
        ioloop = asyncio.get_running_loop()

        if self._wd is None:
            self.add_watch()

        def _add_to_queue():
            queue.put_nowait(self.get())

        ioloop.add_reader(self._file_descriptor, _add_to_queue)

async def run_monitor(monitor):
    queue = asyncio.Queue()
    asyncio.gather(monitor.start(queue))
    while True:
        print("!!! EVENT:", await queue.get())

async def run(delayed):
    print("Start, delayed =", delayed)
    path = "/tmp/file.txt"
    monitor = FilesystemMonitor(path, delayed=delayed)
    asyncio.create_task(run_monitor(monitor))

    with open(path, "w") as f:
        f.write("content")

    await asyncio.sleep(0.1)  # yield so run_monitor reads from queue
    print("End")

logging.basicConfig(level=logging.DEBUG)
for val in (True, False):
    asyncio.run(run(val))


其输出为

$ touch /tmp/file.txt && PYTHONASYNCIODEBUG=1 python3.11 temp.py 
DEBUG:asyncio:Using selector: EpollSelector
Start, delayed = True
WD = 1
End
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>
DEBUG:asyncio:Using selector: EpollSelector
Start, delayed = False
WD = 1
!!! EVENT: /tmp/file.txt
End
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>

在这两种情况下,将以下内容添加到显示的末尾,inotify 都创建了监视描述符。add_watch

        with open(f"/proc/{os.getpid()}/fdinfo/{self._file_descriptor}") as f:
            print(f.read())

阅读另一个 stackoverflow 问题和与之相关的问题,这意味着 async 对 epoll 的使用阻止了它的工作。但是,如果同一监视是在异步循环之外创建的,则它确实在异步中工作。只是为了检查,我尝试使用以下命令更改异步事件循环策略,并尝试所有其他可用的类,但没有任何改变结果。Selector

import selectors

class MyPolicy(asyncio.DefaultEventLoopPolicy):
    def new_event_loop(self):
        selector = selectors.SelectSelector()
        return asyncio.SelectorEventLoop(selector)

asyncio.set_event_loop_policy(MyPolicy())

尝试通过异步行为进行 PDB 无助于识别任何内容。我担心我只是误解了这里异步行为的一些基本信息。谁能解释为什么异步任务适用于在异步之外创建的手表,但对在其中创建的手表无效?有没有办法让它起作用?

linux python-asyncio epoll inotify

评论


答: 暂无答案