Arch Linux User Repository

For people not using redis: there is now an "automated patching mechanism" that will instead use the next best fork of redis that is available on your system as the Requires= dependency in *.service. This way, anyone can use their favorite fork of it. Keep in mind that upstream does not explicitly say that forks of redis work as well, and this further makes the built paperless-ngx-venv package non-portable. Happy experimenting :)

Thank you very much for your work on the python-celery package, @hashworks!

Paperless-ngx 1.17.0 now works just fine without any package modification or downgrades!

I've released python-celery 5.3.1 with some test skips.

@hashworks thanks for celery custom build. works great on my paperless-ngx just installed. before i had paperless-task-queue.service fails issue cause the billiard version 4.1.0

many many thanks :)

FYI: I tested 1.16.2 with python-billiard 4.1.0 and python-celery 5.3.1 (custom build, some tests are failing ATM), seems to work fine.

Update to paperless-1.16.1 is now available. The new version of celery containing the fix for python-billiard has finally been published! We still have to wait for the archlinux maintainers of python-celery to update the package... You might therefore want to wait before upgrading python-billiard.

@amo

@dp20eic: Getting the python-environment right needs triggering a rebuilt of python packages, as advised in the pinned comment. Also, it seems that we need to apply the patch that @atomicfs kindly provided. This is due to celery taking ages to stay up-to-date as mentioned in the other pinned comment. As for the migration: It should be enough to put your old configuration file, your old sqlite file and the old persistent directories under /var/lib/paperless/ in place before starting the systemd units.

I was yesterday frustrated, so I stop my tests, today I saw, that the installation at my new LXC Container was finished correct, but I'm not able to add/upload new pdf to paperless-ngx, after I read all comments here, I think it is python-billiard-4.x which make problems. I remove this Version and try to install the old one, but anytime I get this error

================================================================================================================= FAILURES ==================================================================================================================
___________________________________________________________________________________________________________ test_spawn.test_start ___________________________________________________________________________________________________________

self = <t.unit.test_spawn.test_spawn object at 0x7f8fad938050>

    def test_start(self):
        ctx = get_context('spawn')

        p = ctx.Process(target=task_from_process, args=('opa',))
>       p.start()

t/unit/test_spawn.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
billiard/process.py:124: in start
    self._popen = self._Popen(self)
billiard/context.py:341: in _Popen
    return Popen(process_obj)
billiard/popen_spawn_posix.py:39: in __init__
    super(Popen, self).__init__(process_obj)
billiard/popen_fork.py:24: in __init__
    self._launch(process_obj)
billiard/popen_spawn_posix.py:49: in _launch
    tracker_fd = semaphore_tracker.getfd()
billiard/semaphore_tracker.py:38: in getfd
    self.ensure_running()
billiard/semaphore_tracker.py:62: in ensure_running
    spawnv_passfds(exe, args, fds_to_pass)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = '/usr/bin/python', args = [['/usr/bin/python', '-c', 'from billiard.semaphore_tracker import main;main(20)'], [b'/usr/bin/python'], True, (8, 20), None, None, ...], passfds = [8, 20]

    def spawnv_passfds(path, args, passfds):
        passfds = sorted(passfds)
        errpipe_read, errpipe_write = os.pipe()
        try:
            args = [
                args, [fsencode(path)], True, tuple(passfds), None, None,
                -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
                False, False]
            if sys.version_info >= (3, 9):
                args.extend((None, None, None, -1))  # group, extra_groups, user, umask
            args.append(None)  # preexec_fn
>           return _posixsubprocess.fork_exec(*args)
E           TypeError: fork_exec() takes exactly 23 arguments (21 given)

billiard/compat.py:230: TypeError
============================================================================================================= warnings summary ==============================================================================================================
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
t/unit/test_spawn.py::test_spawn::test_start
  /home/dp20eic/.cache/paru/clone/python-billiard3/src/billiard-3.6.4.0/billiard/pool.py:1753: DeprecationWarning: isSet() is deprecated, use is_set() instead
    return self._event.isSet()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================================================== short test summary info ==========================================================================================================
FAILED t/unit/test_spawn.py::test_spawn::test_start - TypeError: fork_exec() takes exactly 23 arguments (21 given)
=========================================================================================== 1 failed, 18 passed, 51 skipped, 8 warnings in 1.30s ============================================================================================
==> FEHLER: Ein Fehler geschah in check().
    Breche ab...
Fehler: kann ‚python-billiard3-3.6.4.0-1‘ nicht bauen: 
Fehler: kann Pakete nicht bauen: python-billiard3-3.6.4.0-1

So I comment the check at PKGBUILD and I was able to install python-billiard3-3.6.4.0-1 after restart paperless-ngx I was able to add/upload new PDF Files

So now I try again to copy, as you describe, from the old to the new installation.

With kind regards Bernd

Edit: The restore from the old installation into the new was successful :)

1.14.5 replaces python-django-ipware with python-ipware in its dependencies. Since these two are mutually exclusive and therefore conflicting with one another, you will be prompted to replace python-django-ipware with python-ipware, which I just submitted to the AUR. If you run into a deadlock where you can't remove python-django-ipware because it is still required by the (yet to be upgraded) paperless-ngx package, just force-remove python-django-ipware with pacman -Rdd python-django-ipware before relaunching the paperless package upgrade.

@dp20eic: Getting the python-environment right needs triggering a rebuilt of python packages, as advised in the pinned comment. Also, it seems that we need to apply the patch that @atomicfs kindly provided. This is due to celery taking ages to stay up-to-date as mentioned in the other pinned comment. As for the migration: It should be enough to put your old configuration file, your old sqlite file and the old persistent directories under /var/lib/paperless/ in place before starting the systemd units.

@amo kommentierte 2023-05-14 21:42 (CEST)

Hi, I try it out today on my production LXC Container, but I could not migrate the old installation, I'm an end user and have no idea about python I only install it, using package manager. After install the new concurrent-log-handler I get the next missing dependents corsheaders so I install django-cors-header and then magic_ahupp but package magic was installed in pythen-3.10 and python-3.11. So I stop this test and restore the last running backup.

I create a new LXC Container with arch-linux, did an update and install paperless-ngx from the scratch, and it works without a problem, in this LXC Container is no old python-3.10 so I think in my other installation is the python Environment not correct, as I say I'm an end user :) Now I must check how to migrate the Data to the new Container.

with kind regards Bernd

I had to do a bit of back-porting on python-billiard-3.6.4.0-3 to get it working with python 3.11, so in case somebody finds it usefull, here are the steps.

Download python-billiard-3.6.4.0-3 PKGBUILD file and patch it:

diff --git a/PKGBUILD b/PKGBUILD
index 47eac64..5a31009 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -12,12 +12,19 @@ license=('BSD')
 depends=('python')
 makedepends=('python-setuptools')
 checkdepends=('python-pytest-runner' 'python-case' 'python-psutil')
-source=("https://pypi.io/packages/source/b/billiard/billiard-$pkgver.tar.gz")
-sha512sums=('9f2a44024e45c4e7667472a0a3ade7caae755ec7c163929433f818e4b87165218b9cc9545bdee2254e7159c206e525306e383d38906b00fd5428bd1616c39b8f')
+source=(
+  "https://pypi.io/packages/source/b/billiard/billiard-$pkgver.tar.gz"
+  "python_311.diff"
+)
+sha512sums=(
+  '9f2a44024e45c4e7667472a0a3ade7caae755ec7c163929433f818e4b87165218b9cc9545bdee2254e7159c206e525306e383d38906b00fd5428bd1616c39b8f'
+  "SKIP"
+)

 prepare() {
   cd billiard-$pkgver
   sed -i 's/< *5/<6/' requirements/test.txt
+  patch --forward -p1 < "$srcdir/python_311.diff"
 }

 build() {

Create a patch file called python_311.diff with content:

diff --git a/billiard/compat.py b/billiard/compat.py
index b5ce7c7..20c93d5 100644
--- a/billiard/compat.py
+++ b/billiard/compat.py
@@ -3,6 +3,7 @@ from __future__ import absolute_import
 import errno
 import numbers
 import os
+import subprocess
 import sys

 from .five import range, zip_longest
@@ -224,9 +225,13 @@ else:
                 args, [fsencode(path)], True, tuple(passfds), None, None,
                 -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
                 False, False]
+            if sys.version_info >= (3, 11):
+                args.append(-1)  # process_group
             if sys.version_info >= (3, 9):
                 args.extend((None, None, None, -1))  # group, extra_groups, user, umask
             args.append(None)  # preexec_fn
+            if sys.version_info >= (3, 11):
+                args.append(subprocess._USE_VFORK)
             return _posixsubprocess.fork_exec(*args)
         finally:
             os.close(errpipe_read)

Build it makepkg -s