Unable to install certain tree-sitter packages from pypi
if i try to install
tree-sitter-markdown, tree-sitter-json, tree-sitter-toml, tree-sitter-yaml, tree-sitter-html, tree-sitter-rust, tree-sitter-xml or tree-sitter-java from pypi, it just fails
(also is it possible to get python 3.13 on termux?)63 Replies
in Termux, those packages are supposed to be installed from commands like
pkg install tree-sitter-markdown , not pip install tree-sitter-markdownhow will it interface with python packages? i need it for
textual[syntax]@NSPG911 if you would like to have python 3.13, that is understandable, but it takes a very long time to update that in Termux. Here is the discussion page for that if you are interested in helping to do it. https://github.com/termux/termux-packages/pull/24679
I am planning to help do it in the future but only after I finish all of my other PRs that are currently open
i want to have python 3.13 because the thing im working on is locked to python >= 3.13
yes
but im fine to downgrade it if necessary, just need the tree sitter packages
huh, the pr is closed
You didn't mention what errors you are seeing.
@NSPG911 does this work for you? this works for me
i prefer to use uv for my packages and stuff
so not sure how i would install it into the project
ok great but keep in mind that pure-venvs often don't work in Termux so if it doesn't work you might need an unsandboxed-venv
could you show me an example of the commands that you use to set up your project?
i remove the
project.requires-python config from pyproject.toml then
error is this in
pip install tree-sitter-markdown, does not happen when using pip install . on the git repository
tree-sitter-markdown-inline/src/parser.c:3:10: fatal error: 'tree_sitter/parser.h' file not founduv build . in the directory fails, pip install . works
whats the difference?does this work? I tried this
why does that work ๐ญ
so like do i have to do that for everything else??
here is more information about that: https://github.com/termux/termux-packages/issues/23567#issuecomment-2692983425 what I referred to as "unsandboxed-venv", or what I mean by that, is the command "
python3 -m venv myenv --system-site-packages"
probably but it's possible for us to make python-tree-sitter-X packages for all of them by copying the way they are provided on Arch Linux AUR, but it can take a while to set up that
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=python-tree-sitter-markdownoh thats interesting, never used that before
should i proot-distro to archlinux?
if you would like to then you can yes, and arch linux has python 3.13
and normal venvs work there
is there any major differences between ubuntu (im assuming thats what termux uses) and arch, aside from apt and pacman?
i also have an issue with arch, it says
Warning: CPU doesn't support 32-but instructions, some software may not work.actually the only thing that is really similar between ubuntu and termux is that they both use
apt, everything else is more different from ubuntu than ubuntu is from arch linuxoh
if you compare them both to termux, then there actually aren't very many major differences between ubuntu and arch linux other than the package versions, the default settings of some things, the package manager, the preinstalled packages, and a bunch of other stuff that is actually less important than it seems when you compare both of them to Termux, which is significantly different from them on a deeper level
anyways so this wont be open again?
interesting, i have an 64-bit-ARM-only computer also where I am running Ubuntu and Termux, and I ran Arch Linux in a VM on it for a little while (and the VM would have also been affected by the 64-bit-only-ness), but I don't remember ever seeing that message yet
im on a oneplus 13r
no it will be reopened when someone is ready to finish it, the person who opened it closed it because they aren't ready to finish it yet
but anyone else can reopen it and finish it
also you can too if you know how to do it but it's very hard
oh im not at all interested in continuing it ๐
no clue about android app development in general
if you try arch linux in proot-distro and it says the same message again then let me know where you saw it and what command makes that message, and I can try it on my computer that is also 64 bit only and check if the same thing happens to me or not
wait I realized I can install the arch linux termux proot-distro on that computer right now too, I can follow along what you do and see if it happens to me from the exact thing you do, if you want
wow, thanks, I see the same message on my 64-bit only computer
it comes from proot-distro itself,
https://github.com/termux/proot-distro/blob/8f68053e346ee92219d325fa6d43e92922502a2e/proot-distro.sh#L689
arch linux ARM aarch64 is completely 64-bit I think, so I think it is just a warning and you are unlikely to see any real error from it
okay, so how hard would that be
should i just mark https://discord.com/channels/641256914684084234/1423158072683008070/1423200143376777277 as the solution?
you can if that is a good enough solution for you, but if you need better solutions then leave this open
ah okay, hope that the tree sitter packages get added
should i open an issue in any of the repositories?
yes you can,
in here, open a package request repository and post a link to the AUR package, and list all the packages you need
that would be good and help us keep track of the package request
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=python-tree-sitter-markdown
in where?
GitHub
termux/termux-packages
A package build system for Termux. Contribute to termux/termux-packages development by creating an account on GitHub.
im kinda confused on tree sitter markdown though
i have it installed
but uv cannot use it, even though pip can
uv still generates the insanely large log, but python -m pip gives this
nu is using a venv overlay provided by uv
did you use the commands I sent here exactly, including
uv pip install ~/code/tree-sitter-markdown/, but with the path to where you downloaded tree-sitter-markdown instead of my path, or did you do something else?
^the issue with that method is that uv sync will remove it right?
oh, does it? I didn't know
uv sync ensures your pyproject.toml is consistent with the venv
in that case, actually having
pkg install python-tree-sitter-markdown available wouldn't help your problem by itself either,
you need to find a way to get uv to allow the "unsandboxed venv"
that i mentioned here
i don't know right now if there is a way to get uv's venv to do that but there might bei did use
uv venv --system-site-packages, but it doesnt show up
if i use python -m venv, there isnt a venv activation for nushelloh i see,
unfortunately I am not familiar with nushell, so I don't really know what to do if that happens,
there is a discussion here about that,
GitHub
A
nushell wrapper for virtualenv ยท Issue #273 ยท nushell/nu_sc...Hi everyone ๐ ๐ i've been using virtualenv to manage python virtual environments for a long time, with projects such as virtualenvwrapper on bash or virtualfish on fish ๐ i could not find anyth...
but I don't really understand what they are talking about in that thread
or where to get the solution that it was closed as completed for
and the repo which was linked got obliterated
https://github.com/amtoine/virtnualenv
Commands | uv
uv is an extremely fast Python package and project manager, written in Rust.
so
uv still tries to build python tree-sitter-markdown even though you used --no-build-isolation? strangeoh wait i didnt use that lemme try again
still breaks
it makes it seem like
uv isn't respecting --no-build-isolation the way that regular venvs do
would you be able to show the code of the project you are trying to build? i could try it without uv to check what happens thenyeah it still gives me the same error even though im using a
--system-site-packages venv and installing with --no-build-isolationGitHub
GitHub - NSPC911/rovr: A post-modern terminal file manager.
A post-modern terminal file manager. Contribute to NSPC911/rovr development by creating an account on GitHub.
you might need to downgrade it, it uses python3.13, so i just remove uv.lock and .python-version and change pyproject.toml to use >=3.12
ok i changed pyproject.toml to have python 3.12 instead of 3.13, yeah,
now when I use
pip install . on your project, it shows this: Requirement already satisfied: tree-sitter-markdown>=0.3.0 in /data/data/com.termux/files/usr/lib/python3.12/site-packages (from textual[syntax]<7.0.0,>=6.0.0->rovr==0.3.0) (0.5.1)
and it does fail to build the other ones that aren't the markdown one that it still needs, but i would guess the manual install method for the rest of them might also work for meuv handles it differently than pip
yeah
very weird
might need to check for an open issue
you should open an issue for this in termux-packages also, by showing that
uv in termux installs differently from pip in termux, i did not create the termux uv package but someone else might know what to do in uv
but it is related to your issue about the python-tree-sitter-markdown, so you can put the explanation of this in the same issue where you requested those packages, if you want, because,
even if we add those packages for pkg install,, they won't work for your use case of uv until there is a way to get uv to find them in the folder /data/data/com.termux/files/usr/lib/python3.12/site-packagesGitHub
uv does not use system site packages like
tree-sitter-* ยท Issue ...Summary On termux, the mirror does not provide some tree-sitter packages, but one of it that it provides is tree-sitter-markdown. Attempting to install it via pip is successful ~/test $ python -m v...
good, I did not anticipate you might report it upstream but that's probably fine
I just added some of the commands from this discord thread that are part of the reproduction steps that you didn't add there, since I assumed you might have just forgotten to add them