| Git Clone URL: | https://aur.archlinux.org/koboldcpp-hipblas.git (read-only, click to copy) |
|---|---|
| Package Base: | koboldcpp-hipblas |
| Description: | An easy-to-use AI text-generation software for GGML and GGUF models (with HIPBLAS, for ROCM) |
| Upstream URL: | https://github.com/YellowRoseCx/koboldcpp-rocm |
| Licenses: | AGPL-3.0-only |
| Conflicts: | koboldcpp |
| Provides: | koboldcpp, koboldcpp-rocm |
| Submitter: | alpindale |
| Maintainer: | TheBill2001 |
| Last Packager: | TheBill2001 |
| Votes: | 8 |
| Popularity: | 0.71 |
| First Submitted: | 2023-10-23 18:05 (UTC) |
| Last Updated: | 2024-12-05 01:39 (UTC) |
@random_click KoboldCpp does not use tk directly, but via customtkinter to display the GUI Launcher. It is optional as you can just use KoboldCpp directly from the terminal without ever touching the launcher.
If you install either python-customtkinter or python-customtkinter-git, tk should be automatically install with it. If it isn't, please report to those packages' maintainers.
Isn't this package missing dependecy to tk? I couldn't start the app without it
I try to run this through Distrobox and I do see the hipblas (ROCm) option, but it crashes with a Cuda error:
ggml_cuda_compute_forward: ADD failed
CUDA error: shared object initialization failed
current device: 0, in function ggml_cuda_compute_forward at ggml/src/ggml-cuda.cu:2365
err
ggml/src/ggml-cuda.cu:107: CUDA error
Vulkan seems to work though, not sure if that's effectively the same? I really wish ROCm would not be such a mess to get running. It's the one thing I really hate when it comes to AMD gpus on Linux.
@Smae Also check if the beginning of the console has these lines:
Initializing dynamic library: koboldcpp_hipblas.so
It should be koboldcpp_hipblas.so.
@Smae Yes, counterintuitively, you have to use --usecublas flag to enable hipblas with this fork. However, looking through the source code here, it will only use hipblas if and only if:
/usr/share/koboldcpp/koboldcpp_cublas.so does not exist./usr/share/koboldcpp/koboldcpp_hipblas.so exists.Could check if you some how has the straggler koboldcpp_cublas.so file? And what is the output of ldd /usr/share/koboldcpp/koboldcpp_hipblas.so?
@TheBill2001 Pretty much the moment after launch, in all fairness I'm after the hipblas as I do have an AMD GPU, but it's not showing up. From a screenshot on YellowRose's github (which admittedly is just copypasta) it seems to imply to select 'cuBlas' but I'm assuming this is incorrect and where my confusion is stemming from.
Thus my real problem is more, 'why isn't the hipblas backend showing up'.
If i just launch koboldcpp from the command line, it comes up with the GUI, so I alter my preferences accordingly and set it to cuBlas, then click launch. Within around 1-5 milliseconds it exits out as it can't find the nvidia stuff. But lemme wind back a second.
As per previous the back ends I have available (despite having installed koboldcpp-hipblas) are 'OpenBlas, CLBlas, cuBlas, Vulkan, NoBlas....' no hipblas present.
Additional info AMD RX 590 series.
@Smae That should not be the case. libcuda.so.1 belongs to CUDA in the nvidia-utils package and this fork does not use really use anything from NVIDIA. I have just tested building the package in a clean chroot and none of the libraries in koboldcpp is linked against it. However, I cannot test further as I don't have AMD GPU. Where and when in the execution of koboldcpp did you encounter this error message?
I can't seem to get this to work? I've installed it but I get
'libcuda.so.1 cannot open shared object file: no such file or directory' . What else do I need to install?
cblas is 3.12.0-5 rocblas is 6.0.2-1 clbast is 1.6.3 hipblas 6.0.2-1 (koboldcpp-hipblas 1.71.1.yr0-1) openblas 0.3.27-1
The options I have are: OpenBLAS, CLBlas, cuBlas, Vulkan, NoBlas, CLBlas NoAVX2 (old cpu) NoAVX2 mode (old cup) Failsafe mode
There is no hipblas option surprisingly, if I hover over, it does say that it's not available, but as far as I'm aware I've installed all the dependencies.
NOTE: This package is a fork of koboldcpp by YellowRoseCx, which has slower releases. Please check first before flagging this package as out-of-date. https://github.com/YellowRoseCx/koboldcpp-rocm/releases
DISCLAIMER: I only maintain this package. Since I don't have an AMD GPU, I can only test the build process in a clean chroot. I cannot test if the result build would actually run. If this is not working for you, you can install koboldcpp and use Vulkan instead. It's faster to build and smaller to install.
Build configuration could be changed with environment variables. Or set via KEY=VALUE in ${XDG_CONFIG_HOME}/koboldcpp_build.conf file (default is ~/.config/koboldcpp_build.conf):
KOBOLDCPP_NO_PORTABLE - Disabling portable build (including failsafe and noavx2):
As of 1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.
Pinned Comments
TheBill2001 commented on 2024-07-25 10:07 (UTC) (edited on 2024-11-06 03:49 (UTC) by TheBill2001)
NOTE: This package is a fork of koboldcpp by YellowRoseCx, which has slower releases. Please check first before flagging this package as out-of-date. https://github.com/YellowRoseCx/koboldcpp-rocm/releases
DISCLAIMER: I only maintain this package. Since I don't have an AMD GPU, I can only test the build process in a clean chroot. I cannot test if the result build would actually run. If this is not working for you, you can install
koboldcppand use Vulkan instead. It's faster to build and smaller to install.Build configuration could be changed with environment variables. Or set via
KEY=VALUEin${XDG_CONFIG_HOME}/koboldcpp_build.conffile (default is~/.config/koboldcpp_build.conf):KOBOLDCPP_NO_PORTABLE- Disabling portable build (includingfailsafeandnoavx2):TheBill2001 commented on 2024-07-03 14:09 (UTC)
As of
1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to installcustomtkinterfor it.