Building the latest llama-cpp on NixOS

Llama-cpp moves pretty fast so you’ll likely find what’s in nixpkgs doesn’t always meet you needs. Getting the latest version built with cuda support and available inside home manager is relatively easy

{ pkgs, ... }:
let
  llama-cpp = pkgs.llama-cpp.overrideAttrs(attrs: rec {
    version = "5869";
    src = pkgs.fetchFromGitHub {
      owner = "ggml-org";
      repo = "llama.cpp";
      tag = "b${version}";
      hash = "sha256-sQK5OHuzRaT5wiz6+6ZBQxpzCLhYjdWy1ZsPcLrvMe4=";
    };
  });
in
{
  home.packages = [
    (llama-cpp.override { cudaSupport = true; })
  ];
}

Just update the version number and hash whenever you want to bump.