AlphaPeel:numba 无法分配布局为 A 的数组

AlphaPeel: numba fails to allocate array with layout A

提问人:mugdi 提问时间:12/5/2022 最后编辑:mugdi 更新时间:1/18/2023 访问量:105

问:

我尝试使用AlphaPeel进行一些混合剥离。我按照工具文档创建基因型 .txt 和谱系 .txt 格式作为指定的输入。 来自 AlphaPeel 文档:

基因型文件

基因型文件包含每个基因型的输入基因型 个人。每行中的第一个值是个人的 ID。这 剩余值是每个基因座上个体的基因型, 0、1 或 2(如果缺失,则为 9)。例:

id1 1 1 2 0 1 1 1 1 1 0
id2 0 2 1 1 0 1 1 1 2 2
id3 1 2 0 1 2 1 0 1 2 0
id4 2 1 1 1 1 1 1 1 2 1

谱系文件

谱系文件的每一行都有三个值, 个人 ID、父亲 ID 和母亲 ID。 表示未知的 ID。例:

id1 0 0
id2 0 0
id3 id1 id2
id4 id1 id2

但是当我尝试用以下命令调用 AlphaPeel 时:AlphaPeel -genotypes genotype.txt -pedigree pedigree.txt -runtype='multi' -out 1kg_22_seq

我收到以下输出:

Reading in AlphaImpute Format: genotype.txt
Cycle  0
Peeling Down, Generation 0
Peeling Down, Generation 1
Traceback (most recent call last):
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/errors.py", line 823, in new_error_context
    yield
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 265, in lower_block
    self.lower_inst(inst)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 439, in lower_inst
    val = self.lower_assign(ty, inst)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 626, in lower_assign
    return self.lower_expr(ty, value)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 1368, in lower_expr
    res = self.context.special_ops[expr.op](self, expr)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/np/ufunc/array_exprs.py", line 405, in _lower_array_expr
    return npyimpl.numpy_ufunc_kernel(
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/np/npyimpl.py", line 360, in numpy_ufunc_kernel
    output = _build_array(context, builder, ret_ty, sig.args, arguments)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/np/npyimpl.py", line 286, in _build_array
    array_val = arrayobj._empty_nd_impl(context, builder, real_array_ty,
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/np/arrayobj.py", line 3923, in _empty_nd_impl
    raise NotImplementedError(
NotImplementedError: Don't know how to allocate array with layout 'A'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oem/anaconda3/bin/AlphaPeel", line 33, in <module>
    sys.exit(load_entry_point('AlphaPeel==1.1.0', 'console_scripts', 'AlphaPeel')())
  File "/home/oem/anaconda3/lib/python3.9/site-packages/tinypeel/tinypeel.py", line 191, in main
    runPeelingCycles(pedigree, peelingInfo, args, singleLocusMode = singleLocusMode)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/tinypeel/tinypeel.py", line 23, in runPeelingCycles
    peelingCycle(pedigree, peelingInfo, args = args, singleLocusMode = singleLocusMode)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/tinypeel/tinypeel.py", line 46, in peelingCycle
    Peeling.peel(family, Peeling.PEEL_DOWN, peelingInfo, singleLocusMode)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 487, in _compile_for_args
    raise e
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 420, in _compile_for_args
    return_val = self.compile(tuple(argtypes))
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 965, in compile
    cres = self._compiler.compile(args, return_type)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 125, in compile
    status, retval = self._compile_cached(args, return_type)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 139, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/dispatcher.py", line 152, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler.py", line 716, in compile_extra
    return pipeline.compile_extra(func)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler.py", line 452, in compile_extra
    return self._compile_bytecode()
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler.py", line 520, in _compile_bytecode
    return self._compile_core()
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler.py", line 499, in _compile_core
    raise e
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler.py", line 486, in _compile_core
    pm.run(self.state)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler_machinery.py", line 368, in run
    raise patched_exception
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/compiler_machinery.py", line 273, in check
    mangled = func(compiler_state)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/typed_passes.py", line 394, in run_pass
    lower.lower()
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 168, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 222, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 251, in lower_function_body
    self.lower_block(block)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/lowering.py", line 265, in lower_block
    self.lower_inst(inst)
  File "/home/oem/anaconda3/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/oem/anaconda3/lib/python3.9/site-packages/numba/core/errors.py", line 837, in new_error_context
    raise newerr.with_traceback(tb)
numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
Don't know how to allocate array with layout 'A'.

File "../../anaconda3/lib/python3.9/site-packages/tinypeel/Peeling/Peeling.py", line 86:
def peel(family, operation, peelingInfo, singleLocusMode) :
    <source elided>
        # We are estimating the parent's genotypes so the anterior term is ignored to avoid double counting.
        childValues = posterior[child,:,:] * penetrance[child,:,:]
        ^

During: lowering "childValues = arrayexpr(expr=(<built-in function mul>, [Var($530binary_subscr.17, Peeling.py:86), Var($550binary_subscr.29, Peeling.py:86)]), ty=array(float32, 2d, A))" at /home/oem/anaconda3/lib/python3.9/site-packages/tinypeel/Peeling/Peeling.py (86)

我的文件包含 4 个示例,每行有 1603397 列。 我在这里最好的猜测是,这可能是由于处理浮点数以加快计算速度的方式。有关相关问题,请参阅此处genotype.txtnumba

我的python版本是:和我的python包列表:python --version Python 3.9.12numba 0.56.4

aiohttp                       3.8.1
aiosignal                     1.2.0
alabaster                     0.7.12
AlphaPeel                     1.0.0
anaconda-client               1.9.0
anaconda-navigator            2.1.4
anaconda-project              0.10.2
anyio                         3.5.0
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.2
astroid                       2.6.6
astropy                       5.0.4
asttokens                     2.0.5
async-timeout                 4.0.1
atomicwrites                  1.4.0
attrs                         21.4.0
Automat                       20.2.0
autopep8                      1.6.0
Babel                         2.9.1
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
backports.tempfile            1.0
backports.weakref             1.0.post1
bcrypt                        3.2.0
beautifulsoup4                4.11.1
binaryornot                   0.4.4
bio                           1.4.0
biopython                     1.79
biothings-client              0.2.6
bitarray                      2.4.1
bkcharts                      0.2
black                         19.10b0
bleach                        4.1.0
bokeh                         2.4.2
boto3                         1.21.32
botocore                      1.24.32
Bottleneck                    1.3.4
brotlipy                      0.7.0
cachetools                    4.2.2
certifi                       2021.10.8
cffi                          1.15.0
chardet                       4.0.0
charset-normalizer            2.0.4
click                         8.0.4
cloudpickle                   2.0.0
clyent                        1.2.2
colorama                      0.4.4
colorcet                      2.0.6
conda                         4.12.0
conda-build                   3.21.8
conda-content-trust           0+unknown
conda-pack                    0.6.0
conda-package-handling        1.8.1
conda-repo-cli                1.0.4
conda-token                   0.3.0
conda-verify                  3.4.2
constantly                    15.1.0
cookiecutter                  1.7.3
cryptography                  3.4.8
cssselect                     1.1.0
cycler                        0.11.0
Cython                        0.29.28
cytoolz                       0.11.0
daal4py                       2021.5.0
dask                          2022.2.1
datashader                    0.13.0
datashape                     0.5.4
debugpy                       1.5.1
decorator                     5.1.1
defusedxml                    0.7.1
diff-match-patch              20200713
distributed                   2022.2.1
docutils                      0.17.1
entrypoints                   0.4
et-xmlfile                    1.1.0
executing                     0.8.3
fastjsonschema                2.15.1
filelock                      3.6.0
flake8                        3.9.2
Flask                         1.1.2
fonttools                     4.25.0
frozenlist                    1.2.0
fsspec                        2022.2.0
future                        0.18.2
gensim                        4.1.2
glob2                         0.7
gmpy2                         2.1.2
google-api-core               1.25.1
google-auth                   1.33.0
google-cloud-core             1.7.1
google-cloud-storage          1.31.0
google-crc32c                 1.1.2
google-resumable-media        1.3.1
googleapis-common-protos      1.53.0
greenlet                      1.1.1
grpcio                        1.42.0
h5py                          3.6.0
HeapDict                      1.0.1
holoviews                     1.14.8
hvplot                        0.7.3
hyperlink                     21.0.0
idna                          3.3
imagecodecs                   2021.8.26
imageio                       2.9.0
imagesize                     1.3.0
importlib-metadata            4.11.3
incremental                   21.3.0
inflection                    0.5.1
iniconfig                     1.1.1
intake                        0.6.5
intervaltree                  3.1.0
ipykernel                     6.9.1
ipython                       8.2.0
ipython-genutils              0.2.0
ipywidgets                    7.6.5
isal                          1.1.0
isort                         5.9.3
itemadapter                   0.3.0
itemloaders                   1.0.4
itsdangerous                  2.0.1
jdcal                         1.4.1
jedi                          0.18.1
jeepney                       0.7.1
Jinja2                        2.11.3
jinja2-time                   0.2.0
jmespath                      0.10.0
joblib                        1.1.0
json5                         0.9.6
jsonschema                    4.4.0
jupyter                       1.0.0
jupyter-client                6.1.12
jupyter-console               6.4.0
jupyter-core                  4.9.2
jupyter-server                1.13.5
jupyterlab                    3.3.2
jupyterlab-pygments           0.1.2
jupyterlab-server             2.10.3
jupyterlab-widgets            1.0.0
keyring                       23.4.0
kiwisolver                    1.3.2
lazy-object-proxy             1.6.0
libarchive-c                  2.9
llvmlite                      0.39.1
locket                        0.2.1
lxml                          4.8.0
Markdown                      3.3.4
MarkupSafe                    2.0.1
matplotlib                    3.5.1
matplotlib-inline             0.1.2
mccabe                        0.6.1
mistune                       0.8.4
mkl-fft                       1.3.1
mkl-random                    1.2.2
mkl-service                   2.4.0
mock                          4.0.3
mpmath                        1.2.1
msgpack                       1.0.2
multidict                     5.2.0
multipledispatch              0.6.0
munkres                       1.1.4
mygene                        3.2.2
mypy-extensions               0.4.3
navigator-updater             0.2.1
nbclassic                     0.3.5
nbclient                      0.5.13
nbconvert                     6.4.4
nbformat                      5.3.0
nest-asyncio                  1.5.5
networkx                      2.7.1
nltk                          3.7
nose                          1.3.7
notebook                      6.4.8
numba                         0.56.4
numexpr                       2.8.1
numpy                         1.21.5
numpydoc                      1.2
olefile                       0.46
openpyxl                      3.0.9
packaging                     21.3
pandas                        1.4.2
pandocfilters                 1.5.0
panel                         0.13.0
param                         1.12.0
parsel                        1.6.0
parso                         0.8.3
partd                         1.2.0
pathspec                      0.7.0
patsy                         0.5.2
pep8                          1.7.1
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.0.1
pip                           21.2.4
pkginfo                       1.8.2
plotly                        5.6.0
pluggy                        1.0.0
poyo                          0.5.0
prometheus-client             0.13.1
prompt-toolkit                3.0.20
Protego                       0.1.16
protobuf                      3.19.1
psutil                        5.8.0
ptyprocess                    0.7.0
pure-eval                     0.2.2
py                            1.11.0
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycodestyle                   2.7.0
pycosat                       0.6.3
pycparser                     2.21
pyct                          0.4.6
pycurl                        7.44.1
PyDispatcher                  2.0.5
pydocstyle                    6.1.1
pyerfa                        2.0.0
pyfaidx                       0.7.1
pyflakes                      2.3.1
Pygments                      2.11.2
PyHamcrest                    2.0.2
PyJWT                         2.1.0
pylint                        2.9.6
pyls-spyder                   0.4.0
pyodbc                        4.0.32
pyOpenSSL                     21.0.0
pyparsing                     3.0.4
pyrsistent                    0.18.0
pysam                         0.19.1
PySocks                       1.7.1
pytest                        7.1.1
python-dateutil               2.8.2
python-lsp-black              1.0.0
python-lsp-jsonrpc            1.0.0
python-lsp-server             1.2.4
python-slugify                5.0.2
python-snappy                 0.6.0
pytz                          2021.3
pyviz-comms                   2.0.2
PyWavelets                    1.3.0
pyxdg                         0.27
PyYAML                        6.0
pyzmq                         22.3.0
QDarkStyle                    3.0.2
qstylizer                     0.1.10
QtAwesome                     1.0.3
qtconsole                     5.3.0
QtPy                          2.0.1
queuelib                      1.5.0
regex                         2022.3.15
requests                      2.27.1
requests-file                 1.5.1
rope                          0.22.0
rsa                           4.7.2
Rtree                         0.9.7
ruamel-yaml-conda             0.15.100
s3transfer                    0.5.0
scikit-image                  0.19.2
scikit-learn                  1.0.2
scikit-learn-intelex          2021.20220215.212715
scipy                         1.7.3
Scrapy                        2.6.1
seaborn                       0.11.2
SecretStorage                 3.3.1
Send2Trash                    1.8.0
service-identity              18.1.0
setuptools                    61.2.0
sip                           4.19.13
six                           1.16.0
smart-open                    5.1.0
sniffio                       1.2.0
snowballstemmer               2.2.0
sortedcollections             2.1.0
sortedcontainers              2.4.0
soupsieve                     2.3.1
Sphinx                        4.4.0
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
spyder                        5.1.5
spyder-kernels                2.1.3
SQLAlchemy                    1.4.32
stack-data                    0.2.0
statsmodels                   0.13.2
style                         1.1.0
sympy                         1.10.1
tables                        3.6.1
tabulate                      0.8.9
TBB                           0.2
tblib                         1.7.0
tenacity                      8.0.1
terminado                     0.13.1
testpath                      0.5.0
text-unidecode                1.3
textdistance                  4.2.1
threadpoolctl                 2.2.0
three-merge                   0.1.1
tifffile                      2021.7.2
tinycss                       0.4
tldextract                    3.2.0
toml                          0.10.2
tomli                         1.2.2
toolz                         0.11.2
tornado                       6.1
tqdm                          4.64.0
traitlets                     5.1.1
Twisted                       22.2.0
typed-ast                     1.4.3
typing_extensions             4.1.1
ujson                         5.1.0
Unidecode                     1.2.0
update                        0.0.1
urllib3                       1.26.9
w3lib                         1.21.0
watchdog                      2.1.6
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              0.58.0
Werkzeug                      2.0.3
whatshap                      1.6
wheel                         0.37.1
widgetsnbextension            3.5.2
wrapt                         1.12.1
wurlitzer                     3.0.2
xarray                        0.20.1
xlrd                          2.0.1
XlsxWriter                    3.0.3
xopen                         1.6.0
yapf                          0.31.0
yarl                          1.6.3
zict                          2.0.0
zipp                          3.7.0
zope.interface                5.4.0

如何防止错误:不知道如何从 numba 分配布局为“A”的数组?

python 数组浮 numba

评论

0赞 Rafnus 12/5/2022
我不熟悉 alphapeel,但它没有提到对 numba 的任何支持。如果你试图给一个不熟悉的模块给numba,它将无法编译它,所以我认为你试图做的事情是不可能的。如果你分享你的代码,我也许能提供进一步的帮助
0赞 mugdi 12/5/2022
@Rafnus我不确定我是否理解你说的话。例如,Alphapeel 本身使用 numba see github.com/AlphaGenes/AlphaPeel/blob/master/src/tinypeel/...。Peeling.py 导入函数,我对 jit 编程的了解非常有限,所以我肯定不能自己做这些东西。numbanumba.experimental

答:

1赞 mugdi 12/5/2022 #1

我能够通过以下设置修复此错误:

  1. 创建python3.7venv
  2. 这里下载Alphapeel存储库
  3. 从AlphaPeel.zip中提取AlphaPeel-master和wheel文件
  4. 使用 wheel 文件安装 AlphaPeel-1.1.x
  5. 在命令行中运行 AlphaPeel
  6. 出现错误,指出mportError: cannot import name 'jitclass' from 'numba' (/home/oem/.venvs/my-venv-name/lib/python3.7/site-packages/numba/__init__.py)
  7. 这可以修复编辑冲突中的文件,使用一些编辑器,例如,从行中删除函数nanojitclassfrom numba import ...
  8. 添加新行from numba.experimental import jitclass
  9. 保存文件,然后对冲突中的每个文件重复上述步骤
  10. 运行 Alphapeel

PS:尽管声明Alphagenes的工具可以读取plink文件,但该软件包在我的设置中不起作用。因此,最好坚持使用文档中提供的文件格式。alphaplinkpython