Are we learning yet?

A curated guide to
AI & machine learning in Rust

Yes, for inference and infrastructure.
Not yet for research or training at scale.

Rust is the language the AI industry's infrastructure is written in — tokenizers, model formats, dataframes, vector search, GPU runtimes, agent protocols, sandboxes — and it is now a credible choice for inference and for training on a single machine. It is not where you do research, large-scale training, or classical ML if you can use Python.

Rust is what the Python AI stack imports Every AutoTokenizer call in Python runs Rust. tokenizers, safetensors, hf-hub, tiktoken-rs — the plumbing under the whole industry. Retrieval is the least-disputed win Meilisearch, Qdrant, tantivy, LanceDB and Chroma's core are all Rust, and all publish usable Rust clients. The agent protocols are Rust-first rmcp is one of MCP's five Tier 1 official SDKs; the Agent Client Protocol generates its canonical JSON schema from a Rust crate. GPU stopped being a wish NVIDIA shipped two first-party Rust projects in 2026, rustc carries GPU targets, and std::autodiff is on nightly. What is still missing Fine-tuning and LoRA, training at scale, reinforcement learning, evals and guardrails, video generation.

Rust Machine Learning Ecosystem

Each color grades the category, not the crates on the shelf: would you choose Rust for this problem, against any other language?

  • green = comprehensive — covers the category, in production; you would not go wrong choosing Rust
  • yellow = partial — solid pieces to build on, but a sliver of the category; another language covers it better
  • red = too early — experimental or nonexistent; not a default choice yet

LLMs & Foundation Models

Running, serving and adapting pretrained models.

4 shelves: 1 comprehensive · 2 partial · 1 too early LLM inference & serving — partial Structured output & constrained decoding — comprehensive LLM clients & gateways — partial Fine-tuning & post-training — too early

Agents, Tools & Context

Building applications on top of models.

5 shelves: 2 comprehensive · 2 partial · 1 too early Agent frameworks — partial Agent protocols (MCP & ACP) — comprehensive Sandboxing, browser & computer use — comprehensive Memory, state & durable execution — partial Observability & evaluation — too early

Retrieval, Embeddings & Search

Finding the right context.

5 shelves: 3 comprehensive · 2 partial Vector databases & ANN — comprehensive Embeddings & rerankers — comprehensive Full-text & hybrid search — comprehensive RAG pipelines — partial OCR & document AI — partial

Deep Learning & Training

Define a model, differentiate it, train it.

3 shelves: 2 partial · 1 too early Deep learning frameworks — partial Tensors & autodiff — too early Training infrastructure — partial

Model Inference & Runtimes

Run a model that was trained somewhere else.

3 shelves: 2 comprehensive · 1 too early ONNX & general runtimes — comprehensive Tokenizers, formats & model plumbing — comprehensive Edge, embedded & browser — too early

Classical ML & Statistics

The scikit-learn shelf, plus what Rust is actually good at.

8 shelves: 2 comprehensive · 5 partial · 1 too early ML toolkits & linear models — partial Gradient boosting & trees — partial Clustering, dim-reduction & nearest neighbors — comprehensive Statistics — partial Bayesian & probabilistic modeling — partial Time series & anomaly detection — partial Optimization — comprehensive Reinforcement learning — too early

Data & Dataframes

The data layer — mostly consumed from Python.

4 shelves: 2 comprehensive · 2 partial Dataframes & query engines — comprehensive Lakehouse & columnar formats — comprehensive Datasets, formats & loading — partial Pipelines & streaming — partial

Numerical & Scientific Computing

The foundations under machine learning.

5 shelves: 2 comprehensive · 3 partial Arrays & linear algebra — partial Numerical methods & special functions — partial Domain science (bio & geo) — comprehensive Plotting, visualization & notebooks — partial Python interop — comprehensive

GPU & Accelerators

Can I write and run GPU code in Rust?

3 shelves: 1 comprehensive · 1 partial · 1 too early Kernel languages & GPU compilers — partial Vendor runtimes & portable compute — comprehensive Distributed & multi-GPU — too early

Modalities & Domains

Vertical applications — vision, speech, text, robotics.

6 shelves: 2 comprehensive · 3 partial · 1 too early Computer vision — partial Speech & audio — comprehensive Text processing & tokenization — comprehensive Generative media — too early Robotics & embodied AI — partial Safety, privacy & supply chain — partial

Rust ML in the Rust project

Machine learning is now an accepted goal of the Rust project itself. std::autodiff is live on nightly today — #![feature(autodiff)], forward and reverse mode, Enzyme/LLVM-backed. std::offload has its host side upstreamed and its device side in review, waiting on a rustc bump to LLVM 22, with AMD, NVIDIA and now Intel targets in scope. And the accepted 2026 project goal High-Level ML optimizations proposes an MLIR-based backend for rustc that recognizes tensor operations, aiming at “a shiny future where developers are able to develop full ML models in pure Rust.”

Discuss Rust ML

  • Rust-ML Discord — the general-purpose channel for Rust ML. Small and honest about it: 121 members as of 2026-08-31.
  • #wg-autodiff on rust-lang.zulipchat.com — where the compiler-side autodiff and GPU-offload work is discussed.
  • Burn & CubeCL — their Discord, linked from the project site.
  • Scientific Computing in Rust — a free annual online workshop out of UCL with a monthly newsletter and every talk on YouTube. This is where the numerical and GPU work gets presented.

Learn it, watch it, check the dates

How Rust helped us building and deploying Moshi, a state-of-the-art speech model Laurent Mazare, Kyutai · Rust Nation UK 2025

Mazare is the primary author of candle, and Moshi is a real speech-text foundation model his team moved off PyTorch onto a Rust inference stack and then ran at 500,000 sessions on a low-cost GPU cluster. A named engineer, a shipped model, production numbers, and an honest account of what Rust bought them.

Timeline A dated record, 2024–2026: releases, funding, NVIDIA's two Rust projects, the acquisitions — and the projects that died. AI software written in Rust Codex CLI, Zed, Warp, goose, Handy, koharu, OpenShell. Products with Rust cores, kept separate from the crate catalog.