Package Details: python-safetensors 0.4.5-1

Git Clone URL: https://aur.archlinux.org/python-safetensors.git (read-only, click to copy)
Package Base: python-safetensors
Description: Simple, safe way to store and distribute tensors
Upstream URL: https://github.com/huggingface/safetensors
Licenses: Apache
Submitter: Henry-ZHR
Maintainer: None
Last Packager: mane.andrea
Votes: 5
Popularity: 0.79
First Submitted: 2023-01-08 09:54 (UTC)
Last Updated: 2024-10-02 00:55 (UTC)

Dependencies (27)

Sources (1)

Latest Comments

« First ‹ Previous 1 2 3 Next › Last »

h3ss commented on 2023-12-05 03:34 (UTC)

The problem is that the AUR package includes a .gitignore file, and the maturin build backend will ignore anything specified in any .gitignore file in the path.

See: - Wheels for mixed Python/Rust project does not include Python source code #885

The contents of the AUR .gitignore are:

**/*
!.SRCINFO
!.gitignore
!PKGBUILD

That first line is the problem, it basically excludes everything from inclusion (except what the Rust part of the build creates apparently).

I have been able to get around this by editing the src/safetensors-0.4.0/bindings/python/pyproject.toml file. Specifically, by adding this under the [tool.maturin] section:

include = ["**/*"]

See: - Maturin User Guide - Configuration - Specifically the include part

When this is done, the resulting .whl file reliably contains the needed Python source files.

So, how can we get around this in the PKGBUILD? By using sed to add the include line to the pyproject.toml in the build() function:

build() {
  cd "safetensors-${pkgver}/bindings/python"
  sed -i 's/\[tool.maturin\]/[tool.maturin]\ninclude = \["**\/*"\]/g' pyproject.toml
  python -m build --wheel --no-isolation
}

After adding that sed line, the package reliably passes all checks and builds.

@mane.andrea Can we get this or something equivalent incorporated into the PKGBUILD? I'm confident this should fix this longstanding issue.

PhotonX commented on 2023-11-26 18:14 (UTC) (edited on 2023-11-26 18:15 (UTC) by PhotonX)

I'm getting the check() errors as well. I don't understand what the underlying problem is, though. One of the errors is, exemlarily:

________________ ERROR collecting tests/test_flax_comparison.py ________________
ImportError while importing test module '/home/photon/.cache/yay/python-safetensors/src/safetensors-0.4.0/bindings/python/tests/test_flax_comparison.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_flax_comparison.py:12: in <module>
    from safetensors import safe_open
E   ImportError: cannot import name 'safe_open' from 'safetensors' (unknown location)

So it searches for safe_open from... itself? I think, I have all the necessary dependencies installed, but none of them provides this...

t3nzor commented on 2023-11-20 02:22 (UTC) (edited on 2023-11-20 02:22 (UTC) by t3nzor)

The built package doesn't seem to install the Python bindings, only the shared library. I was able to work around this as follows:

makepkg -si --nocheck

sudo cp src/safetensors-0.4.0/bindings/python/py_src/safetensors/* /usr/lib/python3.11/site-packages/safetensors

65a commented on 2023-11-18 02:52 (UTC) (edited on 2023-11-18 02:59 (UTC) by 65a)

@phoenix17, I'm glad it works for you, but it doesn't work for me. The result is still failing to import safetensors.torch or crashing in the check() (this depends on which machine I use?!), using a clean git clone of the AUR source then makepkg -si on multiple machines. Here is one failure log: https://pastebin.com/mppuQw0H

phoenix17 commented on 2023-11-16 20:07 (UTC)

In case this helps someone, I was able to build/install this manually via makepkg -si without making any changes to the build script here: https://aur.archlinux.org/cgit/aur.git/plain/PKGBUILD?h=python-safetensors

I can confirm it also worked with python-transformers package on the aur and pytorch (https://archlinux.org/packages/extra/x86_64/python-pytorch/)

pulsarGuy commented on 2023-11-16 04:00 (UTC) (edited on 2023-11-16 04:03 (UTC) by pulsarGuy)

I wound up in @tomsod's situation, and their solution worked for me. Skipping checks did allow the package to be installed, but attempting to import it within python failed. Removing --wheel caused the checks to be successful, the package to be installed, and allowed importing within python. Unfortunately, I do not use torch, so I can't comment on that aspect.

65a commented on 2023-11-14 02:13 (UTC)

This package doesn't properly provide safetensors.torch import. If I follow @tomsod's advice, I can't even import safetensors. I suspect it's something between src and py-src directories, but I don't understand python packaging well enough to understand why this is isn't working (PKGBUILD changes look reasonable, but maybe something new is needed to support an upstream change? idk)

Tomsod commented on 2023-11-08 04:10 (UTC) (edited on 2023-11-08 04:10 (UTC) by Tomsod)

Tests failed for me too, so I skipped them, but the build package didn't work either. Turns out the --wheel option in build() somehow prevents these files from being installed. Without it, everything seems to work fine.

Henry-ZHR commented on 2023-08-01 05:06 (UTC)

@ruro

there doesn't seem to be a way to permanently disable check for a single package

AFAIK, you're right.

What I usually do:

  1. yay, skip these packages

  2. yay -Syu --mflags --nocheck

Maybe someone can issue a feature request to AUR helpers, but not me.

ruro commented on 2023-07-30 15:29 (UTC)

@Henry-ZHR

That's what you think, but not what arch devs think. See https://man.archlinux.org/man/PKGBUILD.5.

Huh. Damn, it seems that you are right. I must have been remembering it wrong.

Well, I suggest simply disabling the tests if you don't want those huge packages to be installed.

Please, correct me if I am wrong, but there doesn't seem to be a way to permanently disable check for a single package? So you either have to disable check globally for all packages in makepkg.conf or you have to do it manually each time the package is updated (which doesn't play well with AUR helpers).

« First ‹ Previous 1 2 3 Next › Last »