Arch Linux User Repository

@shashank, I'm not an expert on this myself, but I'll attempt to answer based on my knowledge. When you use makepkg on the PKGBUILD, it creates two directories, src and pkg. The package sources, the tarballs, or git repository or whatever are extracted inside the src directory, which has the variable name $srcdir in the PKGBUILD. $srcdir and $pkgdir are automatically set by makepkg, we don't fiddle with it. The tarball when extracted usually has a directory inside it named in the format of $pkgname-$pkgver, or here, open-webui-0.6.2. So, to build the package, we move into that directory, by cd-ing into src/open-webui-0.6.2, but we just use the variables because it's easier. Then, we just use the standard tools for whatever language the package is built in to build it.

I'll explain with a C package as an example. For C packages using plain GNU make, in build(), we use make, to actually compile the program. The project usually has all the build-specific configurations set up in a Makefile or something, and the end result usually results in a binary, documentation, header files, man pages, and etc, being built and compiled and ready to put into its proper places, but being stored inside the source directory still.

Inside package(), we create a similar structure to the way our systems are, with pkg being the analog to /, and then use the standard tools to install the built stuff into the necessary places while considering pkg as /. So, for a C package, we usually use make DESTDIR="$pkgdir" PREFIX=/usr install, where we define the destination for the binaries as "$pkgdir", and place the binaries with prefix as /usr, not /usr/local or something else. make then usually creates the directories inside pkg as necessary and places the stuff inside them, so the binary goes to PREFIX/bin/, man pages go to PREFIX/share/man, etc. So $pkgdir pkg has stuff inside it like /usr/bin/<binary>, /usr/share/man/<man page>, /usr/lib/<library .so>.

The specifics for the places are standardised and predefined. Then for the stuff that the make tool doesn't deal with, we place them manually using the install command, similar to how make does it. Then finally, package() compresses just the pkg directory, and does some other Arch specific things like the MTREE file, etc and we get the zstandard tarball. Pacman then just uncompresses it and assumes $pkgdir as / and places those files as per the stuff inside it.

As for your intention, it is not particularly recommended, because many things might depend on the binaries being in those specific locations, for eg sagemath might not like it if it can't find its gp binary in /usr/bin, and instead finds the git-python gp binary, and stuff may break. But, just to try it out, when building using makepkg, you can do a mv $pkgdir/usr/bin/gp $pkgdir/opt/git-python/bin/gp or whatever you want to do, at the end of the package() function. Once again, things will break if this is done, and the correct way forward is for the colbert-ai maintainer to use the correct gitpython package.

The flaky issue would be best resolved by the people most knowledgeable in doing so, the project maintainers, or if it is something caused by the AUR package itself, by that package maintainer.

Gnome-keyring has been/was the defacto standard for libsecret implementations, so perhaps they didn't account for people using kwallet, or other implementations. Once again, bringing it up with the msal-extensions people would help. Also, libsecret is used by multiple things, so it's surprising that this is the first time you've had to use it.

No worries, the flow of doing this was more or less a learning experience and the frustration wasn't at all directed at you (from what I've read on here, you're highly engaged, responsive, and helpful). My goal was to document the test fails mentioned earlier, but also as a record and an idea of how to fix these issues for future folk who install it (either by search engine or LLMs). I finally finished installing it last night by the way.

I'm still trying to wrap my head around how yay and makepkg work together to set the install path/directory, since my conf files have no listed defaults (about 3-4 packages intermediate in the chain sagemath dependencies used PARI gp, and it seems to be an important element of sagemath). I imagine that means makepkg is unpacking an archive its assembled with the root directory set to /. So when I was trying to install python-git-python, I decided to take it as an opportunity to understand the building and installing mechanism. Before I uninstalled sagemath, I spent a long time trying to understand how setuptools and yay and makepkg were compiling the binaries and libraries and then placing them in the appropriate arch directories with the simple code:

build() { cd "${srcdir}/${_pkgname}-${pkgver}" python setup.py build }

package() { cd "${srcdir}/${_pkgname}-${pkgver}" python setup.py install --root="$pkgdir"/ --optimize=1 }

This produces a z-standard tar ball that has the binary in /usr/bin and libraries in /usr/lib, where if you decompressed it to the root directory ($pkgdir I guess?), it would install these files into the appropriate directories. So then my question becomes what's doing what?

I also looked at other packages that install to the /opt directory in the hopes that I could figure out how to modify the PKGBUILD script to install the binary /opt and see if I could patch colbert.ai to refer to the full path of his binary instead of my own $PATH. I saw that they used the GNU install tool to get the files to the /opt directory. Now that I look at your PKGBUILD, you do the same thing. Every where I looked trying to figure out how to use the install command while keeping yay/pacman updated on where everything is and what it all is, without first replacing the sagemath binary, the responses were rude and involved a refusal to answer. Because extracting the zstandard tarball would replace the conflicting binary (or more accurately cause the install to fail), I can't use the rm -r "$pkgdir/usr/lib/python3.13/site-packages/data" trick that you use.

But as you mentioned, git-python was absolutely not what colbert.ai is asking for. Instead it asks for gitpython (which is weirdly a required as dependency of python-git-python, but absolutely not mentioned anywhere in git-python repository.

Anyways, you're right. I'll reach out the colbert.ai package maintainer and advise them to switch to stay in line with the recent bug fix.

As to flaky tests, @pytest.mark.flaky means one thing (actually, probably the same thing but using different language for argument variable names) with specific arguments in pytest-rerunfailures, and this same class and method (with the same name of @pytest.mark.flaky) is implemented by the flaky with different names for its argument variables (reruns vs. max_runs). For whatever reason, this sort of conflict never occurs again for other dependency packages that have pytest use both plugins.

As to the msal-extensions testing issue, I think the major issue is trying to figure out how to get libsecret and ksecretservice/kwallet to work well together in KDE. I recently reinstalled EndeavourOS with KDE and this is the first time I've experienced almost being required to use kwallet/ksecretservices. I never experienced or used it in my last install. Now it has me making sure I've unlocked my wallet before I start several programs (like launching the Vivaldi browser and selecting a profile or launching Bitwwarden). So, I need to understand all of that better and whether I'm doing something wrong that libsecret didn't recognize ksecretservices/kwallet for these install tests, or if it's just something weird about kwallet.

@shashank, specifically for the colbert gitpython error, here's an upstream PR that might be helpful: https://github.com/stanford-futuredata/ColBERT/pull/388, do bring it up with the colbert-ai package maintainer.

The package is using the wrong git-python, it shouldn't be the python-git-python package, but rather the python-gitpython package.

@shashank, have you reported each issue you've found (other than the Ctranslate2 one) first to the package maintainer, and then to the upstream project? The msal-extensions one definitely seems like something the upstream project might want to know about. The google-crc32c one can be fixed by the AUR package maintainer, and the flaky package thing once you've bisected what exactly is the root cause also can be fixed by the packager if it's a packaging issue or by upstream, if it's a greater issue.

ColBERT AI is required in the following files in open-webui:

Module: colbert.infra
  Imported in:
    open_webui/retrieval/models/colbert.py
Module: colbert.modeling.checkpoint
  Imported in:
    open_webui/retrieval/models/colbert.py

As for why it needs specific models, presumably because it has developed specific functionality around them and that functionality cannot be used without them.

The git-python and PARI calculator thing can definitely be resolved by either one of the two changing the binary name, depending upon which one has less dominoes stacked on top of them, and that is once again, something for the specific package maintainer/upstream project to determine.

It's good to have these issues documented and potentially fixed, but these are only tangentially related to this package, and nothing I change here in this package will affect them.

Ok, I've run into a larger issue (more on the earlier topic, msal-extensions uses libsecret to handle working with your desktop's secret service and for whatever reason, it fails to recognize KDE's secret service). Now, I've run into a conflict between python-git-python, which installs the 5 year old gitpython library. Its binary gp conflicts with the name of a binary in the PARI calculator which is required dependency of the sagemath library. So, to move forward, I'll remove sagemath and PARI for now until I figure out how to handle this sort of conflict. It's weird that open-webui requires ColBERT AI, which is what requires python-git-python. Why does it need specific models?

Ok. Ctranslate2 is fixed for now (take a look over at that package for a discussion over there). Now I'm running in into test errors with python-msal-extensions.

Particularly LibSecretAgent object and using the save method failed at the point of _test_persistence_roundtrip.

So apparently, it will not work with the kwallet backend as a keyring provider. It seems to require gnome-keyring. Installing it allowed me to pass the the tests and succesfully install python-msal-extensions. Moving on...

So, one of these packages is requiring the flaky plugin for pytest. That's conflicting with the rerunfailures plugin when portalocker is running its tests. Particularly, @pytest.mark.flaky is calling the flaky plugin's method instead of the rerunfailures plugin's method as the portalocker dev intends. Removing the python-flaky package to install portalocker and then running the the install for this program fixes this specific issue.

Another issue I ran into is that when installing google-crc32c, the make requirement that the make version be greater than 3.1 throws an error and modifying the PKGBUILD so that it requires something > 3.5 fixes it (adding -DCMAKE_POLICY_VERSION_MINIMUM=3.5 argument to the build).

Still running into more errors along the way, so I will report back soon with my fixes or qeustions if any.

I think maybe python-anthropic is the library that might require both the flaky and rerunfailures plugins into pytest.

Speaking of more errors along the way, I've just run into Ctranslate2 problems. Time to start investigating that.

@Skwal, ah I see. Then I think I know what the problem was. Some of the dependencies required to build this package are broken, nothing is wrong with this package specifically. The packages below have this problem:

• python-chex
• python-opentracing
• python-opentelemetry

Some of these problems are because those packages upstream do not support Python 3.13 yet. It will be fixed in due time. Until then, we have to keep bypassing the Python tests, I guess.

Hello, unfortunately I didn't keep any logs or error messages, all I can tell you is that I had to bypass the python tests to be able to install it.

Sorry I can't tell you more...

@Skwal, did it work in the end? Also can you post the error message?