Package Details: llama.cpp-vulkan b5054-1

Git Clone URL: https://aur.archlinux.org/llama.cpp-vulkan.git (read-only, click to copy)
Package Base: llama.cpp-vulkan
Description: Port of Facebook's LLaMA model in C/C++ (with Vulkan GPU optimizations)
Upstream URL: https://github.com/ggerganov/llama.cpp
Licenses: MIT
Conflicts: llama.cpp
Provides: llama.cpp
Submitter: txtsd
Maintainer: txtsd
Last Packager: txtsd
Votes: 4
Popularity: 0.89
First Submitted: 2024-10-26 20:10 (UTC)
Last Updated: 2025-04-04 21:51 (UTC)

Pinned Comments

txtsd commented on 2024-10-26 20:15 (UTC) (edited on 2024-12-06 14:15 (UTC) by txtsd)

Alternate versions

llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip

Latest Comments

txtsd commented on 2025-02-14 05:14 (UTC)

@Sherlock-Holo I added python-pytorch as an optional dependency.

Sherlock-Holo commented on 2025-02-14 02:54 (UTC)

the convert_hf_to_gguf.py script miss dependency

Traceback (most recent call last):
  File "/usr/bin/convert_hf_to_gguf.py", line 22, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

txtsd commented on 2024-12-20 04:07 (UTC)

Thanks for reporting @greyltc and @envolution.

I pushed a fix.

envolution commented on 2024-12-20 03:43 (UTC)

the -git version compiled for me on intel igpu, this one failed similarly to how @greyltc's failed

greyltc commented on 2024-12-19 06:29 (UTC) (edited on 2024-12-19 06:30 (UTC) by greyltc)

This isn't building for me, /usr/bin/ld: attempted static link of dynamic object '/lib/libvulkan.so':

-- Found Vulkan: /lib/libvulkan.so (found version "1.4.303") found components: glslc glslangValidator
-- Vulkan found
-- GL_NV_cooperative_matrix2 not supported by glslc
-- Including Vulkan backend
-- Found CURL: /usr/lib/libcurl.so (found version "8.11.1")
-- Configuring done (2.8s)
-- Generating done (0.2s)
-- Build files have been written to: /build/llama.cpp-vulkan/src/build
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  4%] Linking CXX static library libggml-base.a
[  4%] Built target ggml-base
[  4%] Building CXX object ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[  5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
/usr/bin/ld: attempted static link of dynamic object `/lib/libvulkan.so'
collect2: error: ld returned 1 exit status
make[2]: *** [ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/build.make:102: bin/vulkan-shaders-gen] Error 1
make[1]: *** [CMakeFiles/Makefile2:2382: ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

Any ideas?

txtsd commented on 2024-10-26 20:15 (UTC) (edited on 2024-12-06 14:15 (UTC) by txtsd)

Alternate versions

llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip