Arch Linux User Repository

@bossi, Thanks for your comments. I cannot disagree with you, it's just that we haven't seen this managed at a higher or more general level yet (except the maintainers of the perl package have included a similar hook). So yes, this is totally a kludge.

Hi @kseistrup,

Good thoughts there. I'd propose the non-invasive character of the inform-only approach that you suggest. The overall design somehow seems a tad fragmented and I think it's a matter of level and domain: Dependency resolution logic probably shouldn't * live in lower levels (e.g. python in this case). * span across domains (serve AUR space from within the main repo)

I'm sure the approach would work but it doesn't seem ideal. I wouldn't be too big of a fan of adding a separate package for this task that should, if at all: * live in pacman or rather the AUR pacman wrapper of choice * be more general. This seems like a use case that is not specific to the python world. Any package in a dependencies graph should be able to run into this situation (depending on language, build setup etc.) where it needs to do something (rebuild, reinstall, reconfigure) when certain parameters in one of its dependencies have changed.

The last point I would add is that the direction seems upside down: Higher level packages should track changes in their lower level dependencies, but lower level stuff shouldn't concern itself (in any way) with higher levels. Ideally that's done by third party, where the package managers come into play and I think I'd stick with my impression that this should be either done by package management where dependency resolution and all that happens or nowhere, but not fragment out into package space.

Those are just my thoughts that come to mind. I like the "inform only" approach for this use case but it ultimately seems like something that should be addressed at a more general level.

Okay, so I've made a stab at what a hook can look like: https://aur.archlinux.org/packages/detect-old-python-modules

@bossi Thanks for your comments, I mostly agree with you.

Having looked at the perl hook, I now envision something that purely has a notification nature: it doesn't require a certain version, nor does it ever change anything or require an re-install — it purely notifies.

I'm talking about an AUR package that depends on core/python (and that can eventually be merged into that package, should the core developers feel like it), that does one thing: every time core/python is updated — even on minor updates — it queries pacman to see if there are any packages that still “owns” /usr/lib/python3.(N-1), where N is the current major version of python, and if so spits out the relevant package names. Nothing more and nothing less.

E.g., for the current version of python (3.12.*) it will check if there are still any packages that own /usr/lib/python3.11 and writes out those names to standard output. If there aren't any such packages, it will just keep its mouth shut.

That way the user/admin will be alerted to the fact that there may be packages that depends on python who still resides in a directory where the currently installed python package cannot reach them, and they can choose to act accordingly by reinstalling those packages (or ignore the notification altogether). Completely unintrusive.

(The perl hook goes further than that, but that is beyond the scope of “our” python hook.)

PS.: I see you currently specify no lower bound to python. I would probably set one for the one (and only, as far as I can think) reason to make sure that pacman's dependency solver builds the dependant (pyephem) guaranteed after the new python update, not before (causing a rebuild but still to the old python installation, rendering it pointless).

TL;DR: Automation doesn't seem to exist (brief research) and the concept argued against in a few places with the more explicit solution (user rebuilds or dependant releases a thin new package release to trigger the rebuild) being the de-facto way of going about it.

@kseistrup, thanks for your elaborate response. My initial question was fairly naive as I'm not all too familiar with AUR/arch packaging and had been wondering if there might be a relatively straight forward and widely used method to do these auto rebuilds.

Dependant packages from the main repository were all upgraded alongside python-3.12 with at least a package-release bump and I assume that this is all coordinated within main repo space to not run into broken dependency where a dependant has a hard upper bound but no immediate update for the dependency's new release.

I would have guessed that the most obvious way of going about this would be to just do another package release to force a rebuild but it seems a bit like a poor man's solution if there is a more elegant way to just trigger the rebuild on dependency's upgrade.

Regarding upper bounds: I agree, upper bounds to python's minor version would require the maintainer to be fully in sync with the python minor release cycle and I would wonder if it's even something that might be strongly advised against/prohibited by the arch release team as I do think pacman will run into a broken dependency situation where it won't be able to proceed (I don't think it'd force its way through by dropping the broken dependant).

So, long story short - triggers would be great if they existed but the more I search the less I believe they do (or are even mean to), I'm running into various discussions explicitly reminding of rebuilding AUR packages as dependency updates were released. Seems like the main repo solves this through tight coordination while AUR users just needs to understand and rebuild manually, with the option of maintainers releasing a new pkgrel to streamline this.

Looking at the libalpm hook you mention it seems like this would be something the python package would need to implement but I doubt that to be an option, seems error prone if Python rebuilt all its dependant modules that happen to be installed on a system, including AURs - highly doubt it. The only type of hook I could imagine would be on the dependants' side, specifying the exact version bump level (patch, minor, major) to act on but that doesn't seem to exist. Maybe it's the cleanest and least error-prone way to create a new pkgrel.

Thanks for thoughts!

PS: Here's a suggestion for a PKGBUILD that requires python>=3.12, has the correct license, uses a SHA256 checksum, and use a more modern way of building the python module:

https://gist.github.com/kseistrup/b7a3ee92fb2e639488a3c49bbcfb6d76

It doesn't solve the general problem, but it makes for a more correct PKGBUILD, IMHO.

@bossi

I assume we all run into this whenever there's a major python upgrade, and presumably people handle this in each their own way.

I wonder if it can be solved for an individual package by a clever combination of pkgrel (or epoch) and depends. Let's say we had python-pyephem 4.1.5-2 just before Python was upgraded from v3.11 to v3.12. The packager would then bump depends and pkgrel to:

depends=('python>=3.12' 'python<3.13')
pkgrel=3

Because the pkgrel has been bumped, then AUR helper will assume that the package should be updated, but it will refuse to do so unless the python dependency is satisfied. Once Python is updated to v3.13 it will refuse to do so unless PyEphem is also updated (assuming the maintainer is up on their toes and has changed depends to `depends=('python>=3.13' 'python<3.14').

I think this will work. The downside is that if the maintainer is not up on their toes — or has gone completely AWOL — then a single unmaintained AUR package will be able to block the upgrade of python + potentially hundreds of other python-packages.

Perhaps depends=python>=3.12 is sufficient. It will trigger an update, but it will not blocjk the whole thing when Python is updated to v3.13.

Perhaps just bumping pkgrel is the better solution. However, this will fail if a user opts to just update AUR packages and not the “real” Arch packages.

Personally, I have mitigated the situation around Python upgrades by asking:

pacman --query --owns /usr/lib/python3.11  # latest version _before_ upgrade

Package names can be extracted by e.g.:

pacman -Qo /usr/lib/python3.11 | awk '{print $5}'

And I then let the AUR helper reinstall packages that still resided in /usr/lib/python3.11.

Since we're all in the same boat here, and since this is a recurrent event, it would certainly be helpful if it could be fully automated. I wonder if we could have an AUR package that was essentially a hook into core/python, and run every time Python was updated — even for minor updates. If the hook sees that we have a major update, it will extract the names of all packages that still reside in the old Python lib so that the user is alerted to the fact that some packages must be reinstalled.

The package dashbinsh re-symlinks dash -> sh every time bash is updated. Perhaps that package could serve as inspiration.

I believe Perl packages are regularly checked for old locations, but I haven't taken the time to see where the trigger is. Perhaps this mechanism can also be used for Python.

We are not alone.

The perl package has the following hook:

/usr/share/libalpm/hooks/detect-old-perl-modules.hook

Perhaps we could just re-model this for use with python (either by the core maintainers, or as an individual AUR package).

FYI: I had to rebuild and reinstall (pikaur -Sy --reinstall python-pyephem) this package for the now current python-3.12 to install it into its own site-packages. Is there possibly a way to automate this during python's version upgrades?

It appears the entire thing was actually MIT licensed 3 years ago. See https://github.com/brandon-rhodes/pyephem and https://pypi.org/project/ephem/ .