Package Details: koboldcpp-cuda 1.80.3-1

Git Clone URL: https://aur.archlinux.org/koboldcpp-cuda.git (read-only, click to copy)
Package Base: koboldcpp-cuda
Description: An easy-to-use AI text-generation software for GGML and GGUF models (with CUDA)
Upstream URL: https://github.com/LostRuins/koboldcpp
Keywords: ai llm,
Licenses: AGPL-3.0-only
Conflicts: koboldcpp
Provides: koboldcpp
Submitter: alpindale
Maintainer: TheBill2001
Last Packager: TheBill2001
Votes: 6
Popularity: 0.51
First Submitted: 2023-10-23 17:51 (UTC)
Last Updated: 2024-12-27 20:18 (UTC)

Pinned Comments

TheBill2001 commented on 2024-11-01 21:26 (UTC)

Build configuration could be changed with environment variables.

Or set via KEY=VALUE in ${XDG_CONFIG_HOME}/koboldcpp_build.conf file (default is ~/.config/koboldcpp_build.conf).


KOBOLDCPP_NO_PORTABLE - Disabling portable build (including failsafe and noavx2):

  • '0' - Keep portable build enabled (default)
  • '1' or any other value that isn't '0' - Disable portable build.

TheBill2001 commented on 2024-07-03 14:08 (UTC)

As of 1.69.1-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.

Latest Comments

« First ‹ Previous 1 2

dreieck commented on 2024-03-27 10:29 (UTC)

Please add koboldcpp to the conflicts array.

Regards and thanks for maintaining!

Mr.Smith1974 commented on 2024-03-20 11:09 (UTC)

thank you for adopting the package

TheBill2001 commented on 2024-03-19 09:35 (UTC)

@ilikenwf Install customtkinter will also install tk. It is only needed for the GUI launcher and is not needed for when running in CLI only. I have updated the optional dependency description.

TheBill2001 commented on 2024-03-19 09:30 (UTC)

I have adopted and updated the PKGBUILD to only install necessary files (rather than copying the whole source tree). Let me know if there is any problem.

ilikenwf commented on 2024-02-23 04:13 (UTC) 

# Maintainer: Alpin <alpin 'at' alpindale 'dot' dev>
# Author: LostRuins (concedo)
pkgname=koboldcpp-cuda
pkgver=1.58
pkgrel=1
pkgdesc="Full-featured CUDA build of koboldcpp"
arch=('x86_64')
url="https://github.com/LostRuins/koboldcpp"
license=('AGPL3')
depends=('python' 'cblas' 'openblas' 'clblast' 'cuda' 'tk' 'customtkinter')
optdepends=('customtkinter')
source=("$pkgname-$pkgver.tar.gz::https://github.com/LostRuins/koboldcpp/archive/refs/tags/v$pkgver.tar.gz")
sha256sums=('445df325d5d3d6874758c32e5eb88a9d2b0a75451ff9467b45f9c38d766fb828')

build() {
  cd "$srcdir/koboldcpp-$pkgver"
  make LLAMA_OPENBLAS=1 LLAMA_CLBLAST=1 LLAMA_CUBLAS=1
}

package() {
  cd "$srcdir/koboldcpp-$pkgver"

  install -d "$pkgdir/usr/share/koboldcpp"
  cp -a . "$pkgdir/usr/share/koboldcpp"

  echo '#!/bin/sh' > koboldcpp
  echo 'cd /usr/share/koboldcpp && python koboldcpp.py "$@"' >> koboldcpp

  install -Dm755 koboldcpp "$pkgdir/usr/bin/koboldcpp"
}

ilikenwf commented on 2024-02-23 04:02 (UTC)

Customkinter and libtk need to be required. Also, version 1.58 is current.

Lucki commented on 2024-02-22 17:04 (UTC) (edited on 2024-02-22 17:11 (UTC) by Lucki)

I get the following error when loading a model:

ggml-cuda.cu:3019: ERROR: CUDA kernel vec_dot_q5_K_q8_1_impl_vmmq has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: 520
CUDA error: the function failed to launch on the GPU
  current device: 0, in function ggml_cuda_op_mul_mat_cublas at ggml-cuda.cu:8822
  cublasSgemm_v2(g_cublas_handles[id], CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
GGML_ASSERT: ggml-cuda.cu:241: !"CUDA error"

With the following patch I was able to successfully load a model:

diff --git a/Makefile b/Makefile
index 892f17e7..37d78ee4 100644
--- a/Makefile
+++ b/Makefile
@@ -160,7 +160,7 @@ else
    NVCCFLAGS += -Wno-deprecated-gpu-targets -arch=all
 endif #LLAMA_COLAB
 else
-   NVCCFLAGS += -arch=native
+   NVCCFLAGS += -arch=all
 endif #LLAMA_PORTABLE
 endif # CUDA_DOCKER_ARCH

« First ‹ Previous 1 2