Section 01
What Are Llama 4 and Mistral, and How Do They Differ?
Llama 4 and Mistral are both open-weight foundation model families a developer can download and run on their own hardware, but they come from different companies with different commercial terms attached to the same core idea.
Two Labs, Two Starting Points
Llama 4 is Meta's multimodal model family released April 5, 2025: Scout and Maverick shipped immediately, while Behemoth, a nearly 2 trillion parameter teacher model, remains in training and unreleased as of 2026. Every Llama 4 model is distributed under the custom Llama 4 Community License Agreement, not a standard open-source license.
Mistral is the Paris-based lab whose current lineup spans dense Ministral 3 models (3B, 8B, 14B) up to the flagship Mistral Large 3 and the hybrid-reasoning Mistral Small 4, released March 16, 2026. Every model in this lineup ships under Apache 2.0, a genuine open-source license with no field-of-use restrictions.
Why the License Matters as Much as the Benchmark
For a hosted chatbot, license terms rarely come up. For a companion app that self-hosts specifically to fine-tune personas and avoid a commercial API's content policy, the license is the first filter, not the last — it determines whether a fine-tuned checkpoint can be redistributed, whether the product needs to carry the base model's name, and whether growth past a certain scale requires going back to the vendor for permission. The mechanics of how each model actually generates a reply are covered in the companion piece on how transformer-based language models generate a conversation; this article focuses on what's specific to choosing between these two families.
Section 02
How Does Mixture of Experts Routing Generate a Reply?
Every current flagship in both families uses a Mixture of Experts (MoE) architecture, where a router network activates only a small subset of specialized sub-networks for each token instead of running the entire model.
The Routing Loop, Step by Step
At each token position, a lightweight router scores every available "expert" sub-network and selects a small number of them to actually run — the rest stay idle for that token. Llama 4 Scout routes across 16 experts and Maverick across 128, while Mistral Small 4 routes across 128 experts but activates only 4 per token. The selected experts' outputs are combined into a single hidden state, which is what ultimately gets decoded into the next word of the reply.
This is why total parameter count and active parameter count are different numbers worth tracking separately: Scout and Maverick both activate 17 billion parameters per token despite Maverick carrying roughly four times Scout's total weight (400B vs 109B) — the extra capacity in Maverick sits in experts that simply aren't touched on any given token.
Why MoE Replaced Dense Scaling for 2026 Flagships
A dense transformer's inference cost scales directly with its total parameter count — doubling capacity doubles compute per token. MoE decouples the two: Meta trained Llama 4 on 30+ trillion tokens using FP8 precision specifically to make a model with Behemoth-scale total capacity practical, while Scout still fits on a single NVIDIA H100 GPU with Int4 quantization. Mistral's Large 3 takes the same approach from the other direction, using a "granular" MoE design to reach 675B total parameters while keeping only 41B active.
Example self-hosted inference request
{
"model": "self-hosted-companion-v1",
"temperature": 0.9,
"top_p": 0.95,
"max_tokens": 512,
"context_window": 32768,
"active_experts_per_token": 4,
"system_prompt": ""
}
Section 03
Where Does the Model Sit in a Companion App's Architecture?
Whichever family gets chosen, the language model is only one of five components a real AI companion app needs — swapping Llama 4 for Mistral, or the reverse, should not require rebuilding the other four.
| Component | What It Does | Example Tech | Notes |
|---|---|---|---|
| ⭐ Language Model | Generates the actual reply text from conversation history and persona instructions | Llama 4 Scout/Maverick or Mistral Large 3/Small 4 | Self-hosted, open-weight, swappable behind the same backend |
| Backend / Orchestration | Routes requests, manages sessions, batches inference calls | vLLM, SGLang, or a FastAPI serving layer | Handles MoE expert routing and batching identically for either family |
| Interface | Renders the chat, voice, or avatar experience the user interacts with | Web or native app front end | Model-agnostic; does not care which family sits behind it |
| Memory / Storage | Persists conversation history and persona state between sessions | Vector database plus the model's native context window | Context window size caps how much history the model can use natively |
| Moderation | Screens inputs and outputs against the platform's own content policy | Llama Guard 3 or a custom classifier | Neither family ships default adult-content moderation; this layer is always separate |
The Model Is the Engine, Not the Whole Car
A better language model improves reply quality and persona consistency, but it does not manage memory, render an interface, or moderate content on its own — those four components determine as much of the finished product's quality as the model choice itself.
Section 04
Which Llama 4 and Mistral Models Are Actually Available in 2026?
Four models make up the realistic shortlist for a self-hosted companion backend in 2026 — comparing their active parameter count shows how much inference compute each one actually costs per token, regardless of total size.
Active Parameters Per Token Compared
Scout and Maverick Cost the Same Per Token
Because Scout and Maverick share the same 17B active parameter count, the meaningful difference between them for a companion app isn't inference cost — it's total knowledge capacity and, per Meta's own benchmarks, multimodal quality, where Maverick is positioned as outperforming GPT-4o and Gemini 2.0 Flash in its class.
Mistral Small 4 Is the Budget Option, Large 3 the Flagship
Mistral Small 4's 8B active parameters make it the cheapest of the four to run per reply, at the cost of the smallest total knowledge base — a reasonable trade for a companion app prioritizing low latency over depth. Mistral Large 3, activating 41B parameters, is the most compute-hungry model on this list, positioned by Mistral as reaching parity with the best instruction-tuned open-weight models on LMArena's non-reasoning leaderboard.
Section 05
How Do You Fine-Tune or License Either Model for a Companion App?
Persona customization happens in two layers — a lightweight prompt-level layer that works identically on both families, and a weight-level fine-tuning layer where the license each vendor attaches actually changes what a company can legally do.
Persona Conditioning Without Retraining
A system prompt describing a companion's name, personality, and relationship history is the lightest form of customization and works the same way regardless of which family sits underneath — it happens entirely at inference time and touches no weights, so it carries no licensing implications of its own.
Fine-Tuning and Uncensoring at the Weight Level
Direct Preference Optimization (DPO), the technique both open-weight communities use to align a base model toward a desired behavior without full reinforcement learning, is documented in the original 2023 paper introducing it. A related technique, abliteration, removes a model's default refusal behavior by identifying and ablating the specific direction in its activations responsible for declining a request — commonly applied by the open-source community to both Llama and Mistral checkpoints.
The licensing consequence differs sharply between the two: Apache 2.0 places no restriction on distributing a fine-tuned or abliterated Mistral checkpoint under a company's own brand name. The Llama 4 Community License requires any model trained or fine-tuned using Llama materials to begin its name with "Llama," and requires visible "Built with Llama" attribution on any product built with it — a real branding constraint for a companion app that wants its own distinct name.
Legal Limit, Regardless of License
Neither license nor open weights change what the law allows: fine-tuning or prompting either model to generate sexual content depicting a real, identifiable person without their consent is prohibited under Meta's own Acceptable Use Policy and is a federal offense under the TAKE IT DOWN Act, which criminalizes publishing non-consensual intimate imagery and requires platforms to remove reported content within 48 hours. See the full legal requirements for AI adult platforms under the TAKE IT DOWN Act for the complete compliance checklist.
Section 06
Llama 4 or Mistral: Which License and Deployment Path Fits?
The two families lead on different axes — Llama 4 Scout wins on raw context window, Mistral wins on licensing simplicity and hardware footprint at the small end.
The 700 Million User Clause Most Startups Will Never Hit
The Llama 4 Community License requires any company whose products exceed 700 million monthly active users to request a separate license directly from Meta, granted at Meta's sole discretion. For nearly every companion app on the market this threshold is irrelevant in practice — but it's a real ceiling that Apache 2.0, which imposes no user-count restriction of any kind, simply does not have.
| Model Family | License | Min. Self-Hosting Hardware | Context Window |
|---|---|---|---|
| Llama 4 (Scout / Maverick) | Llama 4 Community License — custom, 700M MAU clause, mandatory "Llama" naming and attribution | Scout: 1x NVIDIA H100 (Int4 quantized) | Scout: 10M tokens / Maverick: ~1M tokens |
| Mistral (Large 3 / Small 4) | Apache 2.0 — fully permissive, no user-count or naming restriction | Small 4: 4x NVIDIA HGX H100 minimum | 256K tokens (both models) |
Key Insight
Pick Llama 4 Scout specifically when persistent, long-running memory is the product's core feature — its 10M-token context window has no equivalent anywhere in Mistral's current lineup. Pick Mistral, particularly Small 4, when the priority is a clean license with zero naming or scale restrictions and the lowest inference cost per reply. Neither pick changes the revenue math of the category itself — that side of the business is covered in why companion apps monetize differently than AI image generators.
