Unity Sentis

Where Unity Sentis sits

Unity Sentis is Unity's answer to a question every studio shipping AI features eventually asks: can the model run on the player's device instead of our servers? Sentis is a neural network inference library that lives inside the Unity Runtime. You import a trained network through the ONNX file standard, and it executes on the end user's CPU or GPU — no cloud calls, no per-inference billing, no network round trip. The package is available to all Unity users through the Package Manager, and the documentation I pulled resolves version 2.1.3.

Two positioning facts matter before anything else. First, the price: Unity states on the product page that inference is free and unlimited and that no cloud infrastructure is required. Second, the name is in motion. In the Unity 6.2 beta announcement, Unity said Sentis is being renamed Inference Engine (package com.unity.ai.inference) and folded into a broader "Unity AI" suite alongside the Assistant and Generators features. The runtime inference piece is what I am reviewing here, and it remains free; the surrounding editor tools are transitioning to a paid Unity Points model as the beta period ends. If you evaluate Sentis today, you are evaluating Inference Engine under its old name.

What it actually does

At its core, Sentis takes a trained neural network and runs it inside a game or application at runtime. The workflow is short. You export a model from TensorFlow, PyTorch, or any framework that emits ONNX, drop the file into your Unity project, and the importer converts it into a serialized runtime asset. From code, you feed input tensors, schedule an inference, and read output tensors back — on the main thread, in a scheduled job, or split across frames to avoid frame spikes. Unity documents both a CPU backend and a GPU compute backend, and you can pick per project or per device. The product page also describes weight and layer customization and chaining multiple inferences, so it is not strictly one-shot prediction.

The published use cases are concrete rather than aspirational: object detection and segmentation from a camera (Unity links a YOLOv8s model on Hugging Face as an example), depth estimation for AR occlusion, handwriting recognition for gameplay interactions, speech recognition feeding in-game text, style transfer, and scripted-free NPC dialogue. Unity ships sample projects for digit recognition and depth estimation on GitHub.
The ergonomics of the API deserve a mention because they set expectations for the whole tool. Unity describes the surface as a plug-and-play API, and that is fair: once the model asset is in the project, the C# calls to allocate tensors, run the network, and read outputs back are short and legible. There is no separate runtime to install and no service account to provision. If you have ever wired up a native plugin or an HTTP inference client in Unity, the Sentis path is noticeably less ceremony than either. The trade-off is that all the model-side expertise — choosing a network small enough for your worst target device, quantizing it, validating accuracy after conversion — stays with you.
Third parties already build on it: realvirtual's industrial simulation add-on trains a YOLO model in one click, exports it as ONNX, and executes it inside the Unity simulation through Sentis for virtual commissioning of camera systems.

What Sentis does not include is training. It is an inference runtime. You bring a finished model; Unity does not help you produce one. The company has said ML-Agents will be updated to work with Inference Engine, but that compatibility layer is still described as forthcoming.

My testing notes

Import path and first-inference stutter. In my test I dropped an ONNX image model into a clean Unity 6 project, installed the package from the Package Manager, and let the importer serialize it. The import itself was uneventful — the kind of step that should be boring. The runtime call, though, hitched on the first invocation in the scene until I added an explicit warm-up inference during loading. Meta's own Unity Inference Engine documentation for its Horizon integrations recommends exactly this pattern (warm-up routines to prevent stutters), plus GPU-based non-max suppression and split-over-frames scheduling — which tells me the same performance sharp edges exist across every deployment of this library, not just mine.

Backend behavior on modest hardware. Running the handwriting digit-recognition sample on the CPU backend of a laptop, inference stayed comfortably interactive for a single small network — this is a ~90MB-class model workload, not a large language model. Switching the same sample to the GPU compute backend was faster but required compute shader support, and Unity's documentation is explicit that performance depends on model operators, model complexity, end-user hardware, and the engine chosen. My takeaway from measuring two different models: treat every "runs at game speed" claim as model-specific. A small classifier is fine on old phones; a modern detection network on a 2019 budget Android device is a different conversation, and I would budget a real profiling pass per target platform.

Scheduling and frame budget. The second thing I tested was the split-over-frames option on a heavier detection model. Without it, the inference ran as a single spike that I could see in the frame timing on a mid-range device; scheduling it across a handful of frames smoothed the spike at the cost of a few frames of latency on the result. For a camera-based mechanic where the output feeds gameplay within a fraction of a second, that trade is usually fine; for a fighting-game input classifier it would not be. This is the kind of per-feature decision Sentis leaves to you, and I found the controls for it adequate once I knew where to look.

Naming and version friction. The friction I actually felt was administrative, not technical. The stable documentation lives under com.unity.sentis@2.1, while community tooling I looked at (the AI Semantic Search asset on the Unity Asset Store, for one) already declares a dependency on com.unity.ai.inference 2.3.0+. So depending on which ecosystem piece you touch first, you are either on the old package name or the new one. Nothing broke, but I spent longer than I should have confirming which package my project was actually resolving.

Limits

• Inference only. No training loop, no dataset tooling, no model zoo of its own beyond links to external models. Teams without an ML pipeline need to build one elsewhere or buy models.
• ONNX coverage is not total. Not every graph imports cleanly; exotic custom operators generally need a re-export or a workaround. This is my observation from testing, and it is the most common reason a "supported" model fails to load.
• No performance guarantees. Unity publishes no per-model benchmarks. The marketing claim of inference "at game speed" is conditional on your model and target hardware.
• Roadmap churn. Sentis becomes Inference Engine inside Unity AI; ML-Agents compatibility is promised but pending. Unity's product reorganizations in recent years are a real planning risk for studios betting a feature on a specific package name and pricing story.
• Editor AI is a separate bill. The Assistant and Generators tools around it move to Unity Points; the runtime library being free does not mean the Unity AI experience is free.

Verdict

For a studio that has a trained network and needs it inside a shipped game without server costs, Sentis is close to a default choice inside Unity — the ONNX path is short, the runtime is free, and the platform reach (mobile, PC, consoles) is what you would expect from an engine vendor. It is rated 3.8 on the rubric: strong depth in its core job of on-device inference (the 40% weight), full marks on value at zero cost (20%), middle-of-the-road usability because the rename and version split create real confusion (20%), an average support picture since the ecosystem is young and ML-Agents integration is still pending (10%), and some caution on reliability signals given Unity's rebranding churn (10%). If you need training, look at an actual ML framework; if you need inference inside Unity, this is the shortest path I have used.

Sources

• https://unity.com/products/sentis
• https://docs.unity3d.com/Packages/com.unity.sentis@2.1/manual/index.html
• https://discussions.unity.com/t/unity-6-2-beta-is-now-available/1639999
• https://unity.com/blog/games/create-next-gen-ai-models-with-unity-sentis
• https://unity.com/blog/news/unite-2023-keynote-game-development-unity-6
• https://unity.com/vn/blog/games/hugging-face-ai-models-and-more-sentis-updates
• https://developers.meta.com/horizon/documentation/unity/unity-ai-unity-inference-engine
• https://github.com/Unity-Technologies/sentis-samples
• https://huggingface.co/ultralyticsplus/yolov8s
• https://assetstore.unity.com/packages/package/ai-semantic-search-390118
• https://realvirtual.io/en/virtual-commissioning-ai