install

llama.add builds from source. Arm64/aarch64 is the primary measured target: the published benchmarks were produced on Google Axion (Arm Neoverse-V2) with KleidiAI enabled. x86 builds work for development; it is not the optimized path.

build from source

git clone https://github.com/hardrave/llama.add.git
cd llama.add

# Arm64: enable Arm KleidiAI microkernels (GGML_NATIVE is ON by default,
# which already enables NEON/dotprod/i8mm for the host CPU)
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CPU_KLEIDIAI=ON
cmake --build build -j \
  --target llama-cli llama-server llama-bench \
           llama-add-calibrate llama-perplexity

Binaries land in build/bin/. On x86, omit the KleidiAI flag. Build requirements and backend options (CUDA / Metal / Vulkan / ROCm via the usual -DGGML_* flags) remain aligned with upstream llama.cpp; the published benchmarks cover the CPU-only Arm path.

requirements

git, cmake ≥ 3.14, a C++17 compiler (gcc, clang, or MSVC)
Linux and macOS build with the steps above; Windows builds through cmake/MSVC or WSL

first run

# let the calibrator rank which layers your model can lose
build/bin/llama-add-calibrate -m model.gguf

# the JSON report lands next to the model; apply a named profile from it
build/bin/llama-server -m model.gguf --attention-drop-profile recommended

--attention-drop-profile (minimal/light/recommended/max) is fingerprint-checked against the model, so a report calibrated for a different GGUF is rejected; the manual --attention-drop-layers list still works. Both flags are shared by llama-cli, llama-server, and llama-bench, accepted at startup only: the mask is immutable for the context's lifetime, by design.

what you should see

Calibration writes a JSON sidecar next to the model (model.gguf.llama-add-calibration.json) with the full per-step table, four named profiles, and the model fingerprint. On the next start with --attention-drop-profile recommended, the tools log the resolved mask before inference begins. Real lines from the benchmark captures (Llama 3.1 8B):

llama.add: profile 'recommended' -> layers 19,20,24,25 (held-out dNLL +0.0479)
llama_context: llama.add attention layers enabled = 28/32
llama_kv_cache: size = 7168.00 MiB ( 65536 cells,  28 layers, ...)

The allocator line is the memory saving happening: the baseline run reports 32 layers and 8192.00 MiB at the same context. The tensors are never allocated, not allocated-then-freed.