Kimi-K3 in llama.cpp, Flue 2 Harness, LiteLLM 1.97

llama.cpp lands Kimi-K3 with hybrid KDA+MLA attention and latent MoE, opening a new frontier-class local model for Apple Silicon.

Must read

llama.cpp adds Kimi-K3 text model — Kimi-K3 with hybrid KDA/MLA attention and latent MoE now runs via llama.cpp — worth benchmarking against Qwen3-Coder for your local coding tier.
Flue 2: React-style hooks for agent harnesses — Fred Schott argues agents are defined by their harness; useful framing for your in-house MCP servers and overnight-agent-factory dispatch.
LiteLLM v1.97.0 released — You route through LiteLLM — check the changelog before your next gateway bump.
llama.cpp: —models-dir loads MTP assistant models — Cleaner preset loading for multi-token-prediction draft models; matters if you’re chaining speculative decode locally.

Tools & Frameworks

Flue 2 brings React hooks to agent harnesses

Astro creator Fred Schott ships Flue 2, a meta-harness for agents modelled on React hooks, arguing the harness defines agent behaviour more than the model.

Why this matters: Directly relevant to your MCP + Claude Code harness design.

LiteLLM v1.97.0

New signed release of the model gateway you already run; images signed via cosign with a pinned commit hash for supply-chain verification.

Why this matters: You use LiteLLM in production — verify signatures on upgrade.

Simon Willison ships CORS Chat for local endpoints

Small web UI for exercising OpenAI-Responses-compatible endpoints, tested against LM Studio (—cors) running Qwen3.8 27B on M5 MBP and DGX Spark.

Why this matters: Handy for debugging your local Qwen/Gemma stack against the same API surface as cloud.

Open Models & Local

llama.cpp b10448 adds Kimi-K3

Kimi-K3 support lands with hybrid KDA (linear) + MLA (full) attention, latent MoE, situ activation replacing SwiGLU, MLA output gating, and full-rank KDA gate.

Why this matters: New frontier-class open model to test on Apple Silicon for the local tier of your hybrid stack.

llama.cpp adds MiniMax-Text-01 and MiniMax-M1

Support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM with recurrent-state handling and a logits mask fixing zero-valued embeddings above token 200032.

Why this matters: Another large open model family now runnable locally; watch, don’t switch.

llama.cpp redesigns server yield_to_queue threading

Server rework runs common_speculative_process in a worker and swaps main/worker roles for the threading model, targeting speculative decode throughput.

Why this matters: Matters if you serve local models to multiple concurrent agents overnight.

Sources unavailable today: r/ChatGPTCoding top, r/ClaudeAI top, r/LocalLLaMA top, r/MachineLearning top

Auto-curated daily by Claude Opus 4.7 from Don’t Worry About the Vase (Zvi), GitHub: BerriAI/litellm, GitHub: ggml-org/llama.cpp, Latent Space, Lenny’s Newsletter, SaaStr (Jason Lemkin), Sebastian Raschka, Simon Willison. Source list and editorial profile maintained by Daniel.