local models · consumer GPUs · real bugs

a little local LLM homelab

I run open-weight models as coding agents against bugs from my own git history: check out the commit before the fix, hand the agent the original report, and score it on the hidden test the real fix made pass. Speed, quality and cost are all measured, and the tasks stay private so they can't leak into anyone's training data. The results are all here, and whatever is running right now is shown below.

Today that means one desktop with two RX 7900 XTX, 48 GB of VRAM between them, and a laptop with an RTX 4090 that runs its own queue, the desktop's old RTX 3060 in an enclosure beside it. It began in July 2026 as two laptops joined by a cable and has been rearranged several times since.

Since	Setup	Model	Generation, tok/s
Jul 2026	Two ThinkPads, a 16 GB NVIDIA card in each, sharing one model over llama.cpp RPC: Wi-Fi briefly, then a Thunderbolt cable	Qwen3.6-35B-A3B UD-Q5_K_XL, Qwen3.6-27B Q4_K_S	no speed worth quoting (16 logged requests)
Aug 2026	One laptop: its RTX 4090 (16 GB) and an RX 7900 XTX (24 GB) in a Thunderbolt 4 enclosure	Qwen3.8-27B UD-Q6_K_XL, 262k context, KV q8_0	20.9 over 1,054 requests
13 Sep 2026	Desktop: the RX 7900 XTX and an RTX 3060 (12 GB)	Qwen3.8-27B UD-Q4_K_XL, 128k, KV f16, split 3:1 over both cards	31.2 over 2,982
29 Sep 2026	Same desktop, the model on the XTX alone	Qwen3.8-27B UD-Q4_K_XL, 128k, KV q8_0	37.1 over 9,029
30 Sep 2026	Same desktop, Strata as a second engine beside llama.cpp	Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS, 128k, KV int8, XTX alone	62.6 over 6,467
6 Oct 2026	The laptop's RTX 4090 as its own queue	Qwen3.8-27B GSQ-RCO IQ2_XS, 128k, KV q4_0	26.8 over 1,449
12 Oct 2026	Desktop: two RX 7900 XTX (48 GB)	Qwen3.8-27B UD-Q4_K_XL, 128k, KV q8_0, tensor split over both cards	61.3 over 216, first night
12 Oct 2026	Laptop: its RTX 4090 and the RTX 3060 in the enclosure	Qwen3.8-27B UD-Q4_K_XL, 128k, KV q8_0, split 2:1	25.4 over 433

Speeds are generated tokens over generation seconds across the logged requests for that model and config, as of 12 October 2026. Rows differ in model, quant and context as well as hardware, so each is what I was getting at the time. Scored runs began in mid-September 2026; the first two setups have no scores.

Most local results on this site were measured on the RX 7900 XTX + RTX 3060 desktop; the newest rows come from the two-XTX desktop and from the laptop.