Skip to content
trendoviz

Hugging Face releases Inkling 2 Mini for local AI

Hugging Face announced Inkling 2 Mini, a multimodal mixture-of-experts model developed by Thinking Machines Lab and designed for local setups. The announcement describes 26 billion physical parameters, 3 billion active per token, a 1 million-token context, and support for four inference runtimes, but gives no memory, hardware, or throughput measurements.

51 out of 100Emerging7D: Insufficient history ProvisionalPublished

Overview

Hugging Face published “Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup” on November 20, 2026. The post presents Inkling 2 Mini as a multimodal mixture-of-experts transformer with approximately 26 billion physical parameters and 3 billion active parameters per token. It accepts images and text and supports a context size of 1 million tokens [1].

The model is a looped transformer with 24 physical decoder layers. It performs 48 layer visits per token because its 12 middle layers run three times, reusing the same weights on each pass. Thinking Machines Lab ships the model in a mixed-precision format with NVFP4 and MXFP8 expert weights, and a DSpark drafter is included for additional speed with a small memory cost [1].

All substantive details here come from Hugging Face's announcement about a model developed by Thinking Machines Lab. Its benchmark and comparison claims have not been independently verified.

Why it matters

Inkling 2 Mini combines image and text input, a 1 million-token context, and support for four named inference paths: Transformers, Llama App through llama.cpp, MLX, and SGLang. Hugging Face says SGLang support is available from launch day for datacenter deployments [1].

The model is aimed at local use, but the announcement does not specify where the weights are hosted, the checkpoint size, resident memory requirements, minimum hardware, download size, measured local-device throughput, or latency. Those omissions make the model's suitability for a particular local setup difficult to assess from the post alone [1].

Opportunities

Deployment-readiness box

Known from the announcementNot specified in the announcement
Approximately 26 billion physical parameters; 3 billion active per tokenCheckpoint and download size
NVFP4 and MXFP8 expert weightsResident memory use and minimum hardware
Support for Transformers, Llama App through llama.cpp, MLX, and SGLangMeasured throughput, latency, and generation speed

Teams could evaluate the model for local document analysis, image-and-text assistants, or coding workflows, using the runtime that matches their environment. The available evidence does not establish whether the model fits common local hardware [1].

Risks

The 3 billion active parameters per token are lower than the 26 billion physical parameters, but the announcement does not show how that difference translates into memory consumption or generation speed. Any expectation that the sparse routing alone will make the model light or fast on a particular device is a hypothesis, not a measured result [1].

The announcement also omits licensing terms beyond its general open-source framing and does not identify where the weights are hosted [1].

Context

A mixture-of-experts model keeps most parameters inactive for any given token and routes the token to a smaller subset of specialized experts. Inkling 2 Mini has 24 decoder layers and a hidden size of 2,304. At each layer visit, a router selects eight of 128 SwiGLU experts per token; each expert has an intermediate width of 1,152 [1].

The model also uses a looped design rather than stacking 48 distinct layers. Layers 0–5 and 18–23 run once, while layers 6–17 run three times. All three passes through those middle layers reuse the same weights [1].

Known, claimed, inferred

Inkling 2 Mini's looped architecture reuses 12 physical layers across three visits, producing 48 layer visits per token. The first six and last six layers each run once, so the executed sequence is 6 + 3×12 + 6 = 48 [1].

Each looped core layer starts from the same input, the output of the first six layers. Information moves between passes through cross-pass messages: each layer reads a message from the previous pass with its own learned query, covering its query, key, value, and MLP inputs. A small gating controller then combines the layer's current input with that message, deciding how much of the current input to forget and how much of the message to add. Each pass therefore refines the previous one. During the second and third passes, each core layer attends over two residual blocks from the preceding pass, and four rank-64 gates mix separate messages into the query, key, value, and MoE inputs. Although the core weights are shared, each of the 48 visits retains its own key-value and short-convolution state [1].

Attention uses 24 query heads, eight key-value heads, learned relative position logits, a learned sink, and an output gate. Each six-layer block contains five sliding-window layers with a local window of up to 1,024 tokens and one full-attention layer that uses dynamic key and value convolutions [1].

Limitations & uncertainties

Hugging Face reports that Inkling 2 Mini leads 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL, and places first in its class in the Artificial Analysis Intelligence Index v5. On coding, it reports leading FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0, and a result behind Qwen on AutomationBench-AA v1.0.6. These are vendor claims: the post defers the full scores to the model card, and the underlying tables and scores are not present in the announcement text reviewed here, so the comparisons remain unverified [1].

A separate variable-pass research artifact shares the architecture but was trained to accept one, two, or three passes. Hugging Face reports that setting one or two passes instead of three makes inference faster, while generation quality may decline. The publisher states that it tested the variant but did not evaluate it in depth, so it is an experimental component rather than an equally mature release option. No measured speed or quality results are given [1].

The publisher's own “vibe tests” reportedly found Inkling 2 Mini at least comparable with Qwen 3.8 27B, while noting that more validation is needed. This is also a vendor claim [1].

Evidence & sources

The key statements of this analysis, labeled by type, with the sources they rely on. Trendoviz does not run product tests.

  • Primary-source or vendor statementHugging Face announced Inkling 2 Mini on November 20, 2026, describing it as a multimodal mixture-of-experts model developed by Thinking Machines Lab for local AI setups.[1]
  • Primary-source or vendor statementThe model has approximately 26 billion physical parameters and 3 billion active parameters per token, accepts images and text, and supports a 1 million-token context.[1]
  • Primary-source or vendor statementIts 24 physical decoder layers produce 48 executed layer visits per token because the 12 middle layers run three times using shared weights.[1]
  • Primary-source or vendor statementThinking Machines Lab ships the model with NVFP4 and MXFP8 expert weights, and Hugging Face lists support for Transformers, Llama App through llama.cpp, MLX, and SGLang.[1]
  • Primary-source or vendor statementHugging Face claims the model leads 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL, with full scores deferred to the model card and not independently verified here.[1]
  • Primary-source or vendor statementA variable-pass research artifact was tested by Hugging Face but not evaluated in depth; one or two passes reportedly run faster at the possible cost of generation quality.[1]
  • Primary-source or vendor statementThe announcement does not specify checkpoint size, resident memory use, minimum hardware, or measured throughput and latency.[1]

Sources

  1. 1

Updates & corrections

  1. First published · published automatically after passing every check

Found an error? Report it.