Kimi-K3 in llama.cpp, Flue 2 Harness, LiteLLM 1.97
Sonntag, 16. August 2026 - AI News · (letzte 24h)
llama.cpp lands Kimi-K3 with hybrid KDA+MLA attention and latent MoE, opening a new frontier-class local model for Apple Silicon.
Must read
- llama.cpp adds Kimi-K3 text model — Kimi-K3 with hybrid KDA/MLA attention and latent MoE now runs via llama.cpp — worth benchmarking against Qwen3-Coder for your local coding tier.
- Flue 2: React-style hooks for agent harnesses — Fred Schott argues agents are defined by their harness; useful framing for your in-house MCP servers and overnight-agent-factory dispatch.
- LiteLLM v1.97.0 released — You route through LiteLLM — check the changelog before your next gateway bump.
- llama.cpp: —models-dir loads MTP assistant models — Cleaner preset loading for multi-token-prediction draft models; matters if you’re chaining speculative decode locally.
Tools & Frameworks
Flue 2 brings React hooks to agent harnesses
Astro creator Fred Schott ships Flue 2, a meta-harness for agents modelled on React hooks, arguing the harness defines agent behaviour more than the model.
Why this matters: Directly relevant to your MCP + Claude Code harness design.
LiteLLM v1.97.0
New signed release of the model gateway you already run; images signed via cosign with a pinned commit hash for supply-chain verification.
Why this matters: You use LiteLLM in production — verify signatures on upgrade.
Simon Willison ships CORS Chat for local endpoints
Small web UI for exercising OpenAI-Responses-compatible endpoints, tested against LM Studio (—cors) running Qwen3.8 27B on M5 MBP and DGX Spark.
Why this matters: Handy for debugging your local Qwen/Gemma stack against the same API surface as cloud.
Open Models & Local
llama.cpp b10448 adds Kimi-K3
Kimi-K3 support lands with hybrid KDA (linear) + MLA (full) attention, latent MoE, situ activation replacing SwiGLU, MLA output gating, and full-rank KDA gate.
Why this matters: New frontier-class open model to test on Apple Silicon for the local tier of your hybrid stack.
llama.cpp adds MiniMax-Text-01 and MiniMax-M1
Support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM with recurrent-state handling and a logits mask fixing zero-valued embeddings above token 200032.
Why this matters: Another large open model family now runnable locally; watch, don’t switch.
llama.cpp redesigns server yield_to_queue threading
Server rework runs common_speculative_process in a worker and swaps main/worker roles for the threading model, targeting speculative decode throughput.
Why this matters: Matters if you serve local models to multiple concurrent agents overnight.
Sources unavailable today: r/ChatGPTCoding top, r/ClaudeAI top, r/LocalLLaMA top, r/MachineLearning top
Auto-curated daily by Claude Opus 4.7 from Don’t Worry About the Vase (Zvi), GitHub: BerriAI/litellm, GitHub: ggml-org/llama.cpp, Latent Space, Lenny’s Newsletter, SaaStr (Jason Lemkin), Sebastian Raschka, Simon Willison. Source list and editorial profile maintained by Daniel.