Self-hosting — Glossary

The longer version

Self-hosting means operating the inference software, model weights and hardware environment yourself, whether on machines you own or compute instances you rent. It usually requires a downloadable model and permission under its license to run it for your intended purpose.

The main attractions are control over deployment, data location, software versions and the way requests are processed. For predictable high-volume demand, keeping a GPU busy can sometimes cost less than paying an external API for every token.

The trade-offs include setup work, memory capacity, GPU availability, power or rental costs, upgrades, safety controls, monitoring and failure recovery. A lightly used GPU can be poor value, while an API may scale up and down more conveniently. Self-hosting also does not make a model inherently secure or accurate.

Compare the full cost per successfully completed task, including staffing and downtime, not just the hourly server bill. Test model quality, latency and licenses before migrating important workloads.

Where we use “Self-hosting”

AI Agents & AutomationOpen vs Closed AI Models in 2026: Open Wins Tokens, Closed Wins Dollars
Open-weight models now handle a growing share of AI tokens while closed models keep most of the money. Here is what the capability, price, adoption and security evidence shows, and where it runs out.
Oct 3, 2026Key term
CybersecurityMistral Large 4 Goes Open-Weight: What the Independent Scores Support
Mistral says its open-weight Large 4 leads in defensive security, but the independent Cyber Index puts it third. Here is what the scores, the hardware needs and the dual-use risk actually support.
Nov 1, 2026

The longer version

Where we use “Self-hosting”

Get the Core in your inbox.