Package Details: python-transformers 4.45.2-1

Git Clone URL: https://aur.archlinux.org/python-transformers.git (read-only, click to copy)
Package Base: python-transformers
Description: State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow
Upstream URL: https://github.com/huggingface/transformers
Keywords: huggingface transformers
Licenses: Apache
Submitter: filipg
Maintainer: daskol
Last Packager: daskol
Votes: 11
Popularity: 1.43
First Submitted: 2021-10-23 09:30 (UTC)
Last Updated: 2024-10-08 16:44 (UTC)

Dependencies (20)

Sources (1)

Latest Comments

1 2 3 Next › Last »

BluePyTheDeer251 commented on 2024-10-04 02:35 (UTC)

The thing failed and threw this:

[code] python-transformers - exit status 8 python-optax - exit status 8 python-flax - exit status 8 python-orbax-checkpoint - exit status 8 python-chex - exit status 8 python-safetensors - exit status 8 python-jax - exit status 8 [/code]

BluePyTheDeer251 commented on 2024-10-03 23:48 (UTC)

I hope this helps an LLM I'm working on, it's a coding AI assistant I'm making as a passion project, like Linus Torvalds said: "It won't be as big as GNU", GNU in this case being GitHub Copilot.

daskol commented on 2024-05-07 09:58 (UTC)

@carsme Typo is fixed.

carsme commented on 2024-05-07 09:53 (UTC)

Tests fail both on my system and in a chroot:

==> Starting check()...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'transformers'

daskol commented on 2023-09-20 20:25 (UTC) (edited on 2023-09-20 20:25 (UTC) by daskol)

@rekman Enforced constraint on python-tokenizers in order to prevent its updates with system update.

Thank you for important news.

rekman commented on 2023-09-20 20:07 (UTC) (edited on 2023-09-20 20:09 (UTC) by rekman)

transformers 4.33.2 requires tokenizers<0.14. Currently the AUR has tokenizers 0.14.0.. Downgrade to 0.13.3 to use transformers for now.

This will be fixed once upstream provides a release including this pull request (it has been merged, just not released yet).

daskol commented on 2023-07-08 12:11 (UTC) (edited on 2023-07-08 12:15 (UTC) by daskol)

@ttc0419 Exactly. Most of dependencies of transformers are optional (see repo). By design HuggingFace is aimed at major deep learning frameworks TF, PT, JAX but the issue is that a user usually adhere to a greater extent to a single framework. So, enumeration of only required dependencies gives a user a freedom to manage installed packages on user's system in more fine-grained way.

The errors I listed are NOT optional.

They are actually optional. Just check transformers repo. It is a nightmare actually. The list of actual dependencies depends on a framework and model. In general it is impossible to manage this kind of projects properly.

ttc0419 commented on 2023-07-08 12:04 (UTC) (edited on 2023-07-08 12:07 (UTC) by ttc0419)

@xiota Why avoiding "listing hundreds of packages"? One purpose of the package is dependency management. It's not something nice to let users to track down decencies themselves. If it's optional, list it as optional. The errors I listed are NOT optional. It occurs immediately after an import, which is required packages missing. BTW, you can refer to pip for the dependencies of a package, the list of transformers is actually not very long:

transformers==4.31.0.dev0
├── filelock [required: Any, installed: 3.12.2]
├── huggingface-hub [required: >=0.14.1,<1.0, installed: 0.16.3]
│   ├── filelock [required: Any, installed: 3.12.2]
│   ├── fsspec [required: Any, installed: 2023.6.0]
│   ├── packaging [required: >=20.9, installed: 23.1]
│   ├── PyYAML [required: >=5.1, installed: 6.0]
│   ├── requests [required: Any, installed: 2.31.0]
│   │   ├── certifi [required: >=2017.4.17, installed: 2023.5.7]
│   │   ├── charset-normalizer [required: >=2,<4, installed: 3.1.0]
│   │   ├── idna [required: >=2.5,<4, installed: 3.4]
│   │   └── urllib3 [required: >=1.21.1,<3, installed: 2.0.3]
│   ├── tqdm [required: >=4.42.1, installed: 4.65.0]
│   │   └── colorama [required: Any, installed: 0.4.6]
│   └── typing-extensions [required: >=3.7.4.3, installed: 4.7.1]
├── numpy [required: >=1.17, installed: 1.25.0]
├── packaging [required: >=20.0, installed: 23.1]
├── PyYAML [required: >=5.1, installed: 6.0]
├── regex [required: !=2019.12.17, installed: 2023.6.3]
├── requests [required: Any, installed: 2.31.0]
│   ├── certifi [required: >=2017.4.17, installed: 2023.5.7]
│   ├── charset-normalizer [required: >=2,<4, installed: 3.1.0]
│   ├── idna [required: >=2.5,<4, installed: 3.4]
│   └── urllib3 [required: >=1.21.1,<3, installed: 2.0.3]
├── safetensors [required: >=0.3.1, installed: 0.3.1]
├── tokenizers [required: >=0.11.1,<0.14,!=0.11.3, installed: 0.13.3]
└── tqdm [required: >=4.27, installed: 4.65.0]
    └── colorama [required: Any, installed: 0.4.6]

1 2 3 Next › Last »