Rust PyO3 - 如何执行未知模块导入的脚本?

Rust PyO3 - How to execute a script with unknown module imports?

提问人:Riccardo Fagiolo 提问时间:11/14/2023 更新时间:11/14/2023 访问量:61

问:

我想使用 Rust 在 Python 模块中执行一个函数。 该模块可以具有导入,这些导入在编程时是未知的。

要运行的文件示例:


import testmod

def main():
    print("Hello World!")
    testmod.test_fun()
    return 0

我想要一种将文件传递给 PyO3 并让它解析并导入文件中指定的所有依赖项的方法。它似乎应该在 PyO3 包中,但我在网上找不到任何文档。

现在我正在使用这段代码,它不起作用,因为 Python 模块没有根据它们的依赖关系执行:

pub fn python(args: &WatchCommand) -> PyResult<()> {
    let main = std::fs::canonicalize(&args.main)?;
    let parent = main.parent().unwrap();
    debug!("Loading Python Module...");
    // Load file contents
    let file_contents = std::fs::read_to_string(&args.main)?;
    // Run Python Code
    Python::with_gil(|py| -> PyResult<()> {
        // Load modules in the same folder
        debug!("Search Folder: {:?}", parent);
        for entry in WalkDir::new(&parent) {
            match entry {
                Ok(res) => {
                    if res.path().is_file() && res.path().extension().unwrap_or_default() == "py" {
                        debug!("Loading module: {:?}", res.path());
                        let file_content = std::fs::read_to_string(res.path())?;
                        let name = res.path().file_stem().unwrap().to_str().unwrap();
                        PyModule::from_code(py, &file_content, name, name)?;
                    }
                }
                Err(e) => error!("Error loading module: {}", e),
            }
        }
        let main = PyModule::from_code(py, &file_contents, "main.py", "main");
        // Build arguments
        let args = ();
        // let kwargs = [("slope", 0.2)].into_py_dict(py);clap
        let result: i16 = main?.getattr("main")?.call(args, None)?.extract()?;
        Ok(())
    })
}

这将导致以下输出:

DEBUG Loading Python Module...
DEBUG Search Folder: "..."
DEBUG Loading module: "...\main.py"
ERROR Error running python code: ModuleNotFoundError: No module named 'testmod'

非常感谢任何帮助,

里卡尔多

PYO3型

评论

0赞 Chayim Friedman 11/14/2023
您是否尝试过将文件名设置为完整路径?
0赞 Riccardo Fagiolo 11/14/2023
@ChayimFriedman 是的,输出中的椭圆只是为了消除问题的混乱。路径是绝对的,没有 IO 错误。
0赞 Adesoji Alu 11/14/2023
我修改了脚本并运行了它,但我认为我的 Linux 编译器链接有问题,我不知道它是否会在您身边正常工作而不会出现任何错误
0赞 Masklinn 11/14/2023
pyo3 没有什么可以支持的?您的导入会通过基于 PYTHONPATH 的正常导入机制(可能还有导入钩子)。由您来正确设置 PYTHONPATH(或 )。sys.path

答:

0赞 Adesoji Alu 11/14/2023 #1

当您使用 PyO3 从 Rust 执行 Python 代码时,尤其是在处理具有在编译时未知的导入的 Python 脚本时,您需要一种方法让 Python 解释器正确解析和导入这些依赖关系。这里的关键是确保 Python 的导入系统知道这些模块所在的目录。


use pyo3::types::PyString;
use pyo3::prelude::*;
use pyo3::types::{PyList, PyModule};
use walkdir::WalkDir;
use std::path::Path;

// Making `WatchCommand` public
pub struct WatchCommand {
    pub main: String,
}

pub fn python(args: &WatchCommand) -> PyResult<()> {
    let main = Path::new(&args.main);
    let parent = main.parent().unwrap();
    println!("Loading Python Module...");
    let file_contents = std::fs::read_to_string(&args.main)?;

    Python::with_gil(|py| -> PyResult<()> {
        let sys = py.import("sys")?;
        let path: &PyList = sys.getattr("path")?.extract()?;
        path.insert(0, pyo3::IntoPy::<Py<PyString>>::into_py(parent.to_str().unwrap(), py))?;

        for entry in WalkDir::new(parent) {
            let entry = entry.map_err(|e| PyErr::from(std::io::Error::new(std::io::ErrorKind::Other, e)))?;
            if entry.path().is_file() && entry.path().extension().unwrap_or_default() == "py" {
                let file_content = std::fs::read_to_string(entry.path())?;
                let name = entry.path().file_stem().unwrap().to_str().unwrap();
                PyModule::from_code(py, &file_content, name, name)?;
                println!("Loading module: {:?}", entry.path());
            }
        }

        let main = PyModule::from_code(py, &file_contents, "main.py", "main")?;
        let result: i16 = main.getattr("main")?.call0()?.extract()?;
        println!("Result of main(): {}", result);

        Ok(())
    })
}

fn main() -> PyResult<()> {
    let args = WatchCommand {
        main: "/home/adesoji/pyo3_test/main.py".to_string(),
    };

    python(&args)
}

my cargo.toml file below

[package]
name = "pyo3_test"
version = "0.1.0"
edition = "2021"
Author = "Adesoji Alu , [email protected]"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
pyo3 = { version = "0.20.0", features = ["extension-module"] }
walkdir = "2.3"

now what i did was i ensured the Rust code dynamically loads Python modules from the same directory where the rust project exists as you can see the main.py script's path. This means that any import statements in my main.py that reference other Python files in the same directory should work correctly, as these files are being dynamically loaded into the Python interpreter's module path.

For example, if main.py imports a module testmod, and both main.py and testmod.py are in the same directory (/home/adesoji/pyo3_test/ in my case), the Rust code will load testmod.py into the Python interpreter, allowing main.py to import testmod without encountering a "ModuleNotFoundError".

my approach is particularly useful when you have a Python project with multiple modules and you want to invoke it from Rust while ensuring all Python module dependencies are correctly resolved. i hope this works for you because i used a Linux ubuntu 20.04. you could look at this documentation here from rust: https://docs.rs/pyo3/latest/pyo3/

评论

0赞 Riccardo Fagiolo 11/15/2023
Thank you! I will try out this solution later today. What happens to the sys.path variable once the program is done? Is it still polluted or is the modification to the sys.path variable temporary?