jpmschweitzerandClaude Opus 4.6
cf8a4e37d8
LlamaModelParams::default() sets n_gpu_layers=0, so even with --features rocm the model ran entirely on CPU at ~19 t/s. Setting n_gpu_layers to a large sentinel value asks llama.cpp to offload every layer the model has; llama.cpp clamps to the real count (27 for Gemma 2 2B). Observed throughput jumps from 19 t/s to 74 t/s on an RX 9070 once the ROCm binary is also compiled for gfx1201 (see tooling commit). Also adds server/sr-voice/.gitignore so locally-built binaries don't sneak into the worktree. Release binaries ship out-of-tree per #850. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 lines
159 B
Plaintext
5 lines
159 B
Plaintext
# Built binaries — platform-specific, rebuilt via distrobox + cargo.
|
|
# Per #850, release binaries will ship as CI artifacts, not in the repo.
|
|
/bin/
|
|
/target/
|