Package Details: python-safetensors 0.5.2-1

Git Clone URL: https://aur.archlinux.org/python-safetensors.git (read-only, click to copy)
Package Base: python-safetensors
Description: Simple, safe way to store and distribute tensors
Upstream URL: https://github.com/huggingface/safetensors
Licenses: Apache-2.0
Submitter: Henry-ZHR
Maintainer: envolution
Last Packager: envolution
Votes: 7
Popularity: 1.60
First Submitted: 2023-01-08 09:54 (UTC)
Last Updated: 2025-01-10 04:18 (UTC)

Dependencies (28)

Sources (1)

Latest Comments

1 2 3 4 Next › Last »

envolution commented on 2025-01-03 23:55 (UTC)

thanks for the update, glad it's working

actionless commented on 2025-01-03 23:46 (UTC) (edited on 2025-01-03 23:47 (UTC) by actionless)

@envolution

thanks! after these changes i've got a new error, same as in this thread https://aur.archlinux.org/packages/python-jaxlib-cuda#comment-1003697 (and the same solution helped here as well)

envolution commented on 2025-01-03 23:17 (UTC)

@actionless can you try this new version?

actionless commented on 2024-12-20 13:53 (UTC)

weird, i've rebuilt related libraries from (flax, jax, jaxlib), and i still have exception in the same test, although this time it's with stacktrace instead of being just a crash:

tests/test_flax_comparison.py [genesect:2646977:0:2647130] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x61616274)
==== backtrace (tid:2647130) ====
 0 0x000000000004c822 ucs_event_set_fd_get()  ???:0
 1 0x000000000004c9ed ucs_event_set_fd_get()  ???:0
 2 0x000000000003d1d0 __sigaction()  ???:0
 3 0x00000000000b0681 memcpy()  ???:0
 4 0x000000000730b167 PyInit_xla_extension()  ???:0
 5 0x000000000730ead0 PyInit_xla_extension()  ???:0
 6 0x000000000077ce25 PyInit_xla_extension()  ???:0
 7 0x00000000015d5b78 PyInit_xla_extension()  ???:0
 8 0x0000000004cdc3af PyInit_xla_extension()  ???:0
 9 0x0000000004cdc105 PyInit_xla_extension()  ???:0
10 0x0000000004cddd6b PyInit_xla_extension()  ???:0
11 0x0000000004cc2737 PyInit_xla_extension()  ???:0
12 0x0000000004cc3959 PyInit_xla_extension()  ???:0
13 0x0000000004cc76a8 PyInit_xla_extension()  ???:0
14 0x0000000004cd5840 PyInit_xla_extension()  ???:0
15 0x0000000004cbbe9f PyInit_xla_extension()  ???:0
16 0x0000000004cbaf02 PyInit_xla_extension()  ???:0
17 0x0000000004cc3ed6 PyInit_xla_extension()  ???:0
18 0x0000000004cc4663 PyInit_xla_extension()  ???:0
19 0x0000000004cc4a67 PyInit_xla_extension()  ???:0
20 0x0000000004cc4d3a PyInit_xla_extension()  ???:0
21 0x00000000015c60d6 PyInit_xla_extension()  ???:0
22 0x000000000078837e PyInit_xla_extension()  ???:0
23 0x000000000078019f PyInit_xla_extension()  ???:0
24 0x0000000007fdc0b5 PyInit_xla_extension()  ???:0
25 0x0000000007fdb95e PyInit_xla_extension()  ???:0
26 0x0000000007fd15da PyInit_xla_extension()  ???:0
27 0x000000000009439d pthread_condattr_setpshared()  ???:0
28 0x000000000011949c __clone()  ???:0
=================================
Fatal Python error: Segmentation fault

Thread 0x0000764a61b22b80 (most recent call first):
  File "/usr/lib/python3.12/ast.py", line 257 in iter_fields
  File "/usr/lib/python3.12/ast.py", line 380 in walk
  File "/usr/lib/python3.12/site-packages/_pytest/_code/source.py", line 148 in get_statement_startend2
<... here the same as before ...>

[mutex.cc : 453] RAW: Lock blocking 0x598c93524070   @ 0x74bca89619dc 0x74bca8961701 0x74bca0d29ad3 0x74bca1dd5b1a 0x74bca54dc3af 0x74bca54dc105 0x74bca54ddd6b 0x74bca54c2737 0x74bca54c3959 0x74bca54c76a8 0x74bca54d5840 0x74bca54bbe9f 0x74bca54baf02 0x74bca54c3ed6 0x74bca54c4663 0x74bca54c4a67 0x74bca54c4d3a 0x74bca1dc60d6 0x74bca0f8837e 0x74bca0f8019f 0x74bca87dc0b5 0x74bca87db95e 0x74bca87d15da 0x74bcbeaa339d 0x74bcbeb2849c

and sometimes:

tests/test_flax_comparison.py corrupted size vs. prev_size while consolidating
Fatal Python error: Aborted

do you have any further ideas where to debug that? although i'm not using jax/flax backend so just skipping that particular test only, make the other tests pass, so prolly i don't even need fixing that

actionless commented on 2024-12-18 19:37 (UTC)

thanks for confirming the expected result, i'll try it, i suspect python-flax pkg

envolution commented on 2024-12-18 19:36 (UTC)

@actionless I can't reproduce on clean chroot, nor on my normal environment. I'd suggest trying to rebuild a handful of the dependencies and - failing that - report upstream:

pikaur -S python-numpy python-torch python-jaxlib python-zstandard python-yaml python-pytest

my clean chroot log:

==> Starting check()...
============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /build/python-safetensors/src/safetensors-0.4.5/bindings/python
configfile: setup.cfg
plugins: anyio-4.6.2.post1, benchmark-4.0.0, hypothesis-6.118.7
collected 57 items

tests/test_flax_comparison.py ...                                        [  5%]
tests/test_mlx_comparison.py sss                                         [ 10%]
tests/test_pt_comparison.py s..s...s......sss....                        [ 47%]
tests/test_pt_model.py .............                                     [ 70%]
tests/test_simple.py .................                                   [100%]

=============================== warnings summary ===============================
... trimmed ...

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 48 passed, 9 skipped, 3 warnings in 4.41s ===================

actionless commented on 2024-12-18 16:54 (UTC) (edited on 2025-01-04 00:03 (UTC) by actionless)

the log doesn't change

python pkgs list:

envolution commented on 2024-12-18 16:42 (UTC)

@actionless can you please try without the aur helper (pikaur)? I'm not able to reproduce - can you confirm you're using python-h5py and not python-h5py-openmpi? 

pip list > /tmp/python-packages.txt && pacman -Q | grep '^python' >> /tmp/python-packages.txt && curl -F'file=@/tmp/python-packages.txt' https://0x0.st

if you could run that command and share the link it would be appreciated

actionless commented on 2024-12-18 16:18 (UTC)

here it is:

Successfully built safetensors-0.4.5-cp312-cp312-linux_x86_64.whl
==> Starting check()...
================================================================ test session starts =================================================================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/lie/.cache/pikaur/build/python-safetensors/src/safetensors-0.4.5/bindings/python
configfile: setup.cfg
plugins: hypothesis-6.118.7, benchmark-4.0.0, xonsh-0.18.3, typeguard-4.3.0, anyio-4.6.2.post1
collected 57 items

tests/test_flax_comparison.py double free or corruption (out)
Fatal Python error: Aborted

Thread 0x00007b1c41085b80 (most recent call first):
  File "/usr/lib/python3.12/ast.py", line 52 in parse
  File "/usr/lib/python3.12/site-packages/_pytest/_code/source.py", line 181 in getstatementrange_ast
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 262 in getsource
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 828 in _getentrysource
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 930 in repr_traceback_entry
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 992 in repr_traceback
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 1058 in repr_excinfo
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 690 in getrepr
  File "/usr/lib/python3.12/site-packages/_pytest/nodes.py", line 456 in _repr_failure_py
  File "/usr/lib/python3.12/site-packages/_pytest/python.py", line 1669 in repr_failure
  File "/usr/lib/python3.12/site-packages/_pytest/reports.py", line 376 in from_item_and_call
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 368 in pytest_runtest_makereport
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 244 in call_and_report
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 337 in _main
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/usr/lib/python3.12/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/usr/bin/pytest", line 8 in <module>

Extension modules: ujson, jaxlib.cpu_feature_guard, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, zstandard.backend_c, msgpack._cmsgpack, yaml._yaml, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special (total: 18)
/home/lie/.cache/pikaur/build/python-safetensors/PKGBUILD: line 55: 1400329 Aborted                 (core dumped) PYTHONPATH="${PWD}/test_build/usr/lib/python${python_version}/site-packages" pytest tests/ --ignore=tests/test_paddle_comparison.py --ignore=tests/test_tf_comparison.py
==> ERROR: A failure occurred in check().
    Aborting...

envolution commented on 2024-12-18 15:41 (UTC)

@actionless - it would be great if you'd share the actual failed check. It's possible one of the optional depends should be included in checkdepends - but without seeing your error it's difficult to know which.

If you don't want to bother helping troubleshoot, just install the optional depends and let us know if it's solved.

1 2 3 4 Next › Last »