Unreal Engine Plugin
Design voices, characters, lines, sound effects, and 3D models inside the Unreal Editor, import them as assets, and let AI agents do the same
Overview
The Tonetta plugin for Unreal Engine 5.8+ puts the whole Tonetta workflow inside the editor and turns the results into assets: USoundWave for audio, and — since plugin 2.3 — skeletal or static meshes with textures, a material instance, a physics asset, and animation clips for 3D models. It talks to tonetta.io over HTTPS with your API key — nothing else to run.
| Piece | What it does |
|---|---|
| Editor panel | Tools ▸ Tonetta: Models, Voices, Characters, Lines, SFX and Music screens with a project selector (New / Edit Project), audition buttons, generation, version history, and Import |
| AI toolset | TonettaService, registered on the engine's native AI toolset system, so agents connected to Unreal's MCP server can go from "design a voice" or "build me a gunship" to an imported asset without you |
| Agent guide command | Tonetta.GenerateAgentConfig writes a Tonetta section into your CLAUDE.md / GEMINI.md / AGENTS.md |
| C++ client | FTonettaApiClient wraps the entire v1 API with async and sync calls |
| Runtime cache | A GameInstance subsystem that serves live endpoint audio in a shipped game with instant fallback and background refresh |
Requirements
- •Unreal Engine 5.8 or newer. The plugin builds on the engine's
ToolsetRegistryplugin, which ships with 5.8 and is enabled automatically. - •A C++-capable project (the plugin compiles from source), Git, and Visual Studio / Xcode as for any UE C++ project.
- •A Tonetta account and an API key for the editor — plus, only if your game fetches live endpoints at runtime, a second read-only key (see below). Generation spends credits.
Install
- 1Clone the plugin into your project's
Pluginsfolder:cd /path/to/YourProject/Plugins git clone https://github.com/kevinpbuckley/tonetta.git Tonetta - 2Build the editor target as usual — right-click the
.uproject→ Generate Visual Studio project files, then buildYourProjectEditor. (Projects using VibeUE can run itsBuildAndLaunchGame.ps1instead.) - 3Open the editor, go to Edit ▸ Plugins ▸ Audio ▸ Tonetta, enable it, and restart when asked.
- 4Later, update with
git pullinsidePlugins/Tonettaand rebuild.
Repository layout
The repo is the plugin: Tonetta.uplugin, Source/ (modules Tonetta, TonettaEditor, TonettaRuntime), Content/, and a README with the same information as this page. Engine modules only, no third-party code.
API Keys: Development vs Shipping
The editor needs a full-scope development key. A shipped game normally needs no key at all — imported USoundWave assets play like any other sound and nothing talks to Tonetta at runtime. The only exception is a game that fetches live endpoints while running: that gets a separate, read-only shipping key, because Config/DefaultGame.ini is packaged inside your game and anything in it can be read out of a shipped build.
| Key | Used by | Scopes | Stored in |
|---|---|---|---|
| Development (required) | The editor: panel, TonettaService, Play In Editor | Everything you want to do from the editor — typically all scopes | Config/DefaultEditor.ini — never staged into packaged builds |
| Shipping (only with live endpoints) | Packaged builds that fetch live endpoints at runtime; leave empty otherwise | Read-only: live_endpoints:read | Config/DefaultGame.ini — ships with the game |
A live_endpoints:read shipping key can fetch cached live-endpoint audio but can never create, delete, or spend credits, so a leaked key costs you nothing. The Public TTS API treats it as cache-only: cached audio is served, generation is never triggered. Warm every live endpoint once from the editor or the dashboard before you ship.
- 1Go to Dashboard ▸ API Keys and create a development key with the scopes you want in the editor.
- 2Only if your game uses live endpoints at runtime: create a second, shipping key with only
live_endpoints:read. - 3Paste each into its slot in Project Settings (next section). Leaving a scope out later shows up as a
missing_scopeerror in the panel — add it to the key here, no plugin change needed.
| Scope | Unlocks in the plugin |
|---|---|
projects:write | New Project / Edit Project, create_project, update_project, delete_project |
characters:write | Characters screen and the character functions |
voices:write | Project (provider) voices |
internal_voices:write | Voice Studio: design, save, delete custom voices |
external_voices:read | Browse providers and their voices |
lines:write / sfx:write | Save, version, delete lines and SFX; generate SFX |
music:write | Save, version, delete music; generate music (music:read reads the model catalog) |
tts:generate | Generate speech and audition characters |
live_endpoints:write | Create and edit live endpoints from the toolset (live_endpoints:read alone is the shipping key) |
models:read / models:write | Models screen: browse and import saved 3D models (read); generate, refine, cancel jobs (write, spends credits) |
Configure
Two sections under Project Settings ▸ Plugins (the panel's Settings button opens the first):
Tonetta (Development Key) — DefaultEditor.ini, never packaged
| Setting | Value |
|---|---|
| Development Api Key | The full-scope key. Used by the panel, the toolset, and PIE. |
Tonetta — DefaultGame.ini, ships with the game
| Setting | Value |
|---|---|
| Api Base Url | https://tonetta.io (default) |
| Shipping Api Key | The read-only key, used by packaged builds. Only needed when the shipped game fetches live endpoints at runtime — leave empty otherwise. |
| Key In Use | Read-only: which key this editor resolved and where it came from. Check here first when the connection looks wrong. |
| Api Key (legacy fallback) | Advanced. The pre-2.2 single key; still honored by both editor and packaged builds when the dedicated keys are empty, and copied into the Development Api Key automatically on first editor start. Clear it once the shipping key is set. |
| Default Project Id | Optional; preselects a project in the panel |
| Import Root Path | Content folder for imported assets — default /Game/Tonetta, with Lines/, SFX/, Music/, and Models/<Name>/ underneath |
| Models: Create Physics Asset / Import Animations / Save Packages | What a model import produces besides the mesh, textures, and material instance (all on by default) |
| Model Master Material | The parent of every imported model's MI_<Name>; defaults to the plugin's M_TonettaModel_Master (BaseColor, Roughness, Metallic, Emissive, Normal, EmissiveStrength). Point it at your own material with the same parameter names to restyle imports. |
| Model Glass Material | The parent of MI_<Name>_Glass, which the importer puts on the slots meta.json lists as transparentSlots (a canopy, a window, a lamp's chimney). Defaults to the plugin's M_TonettaGlass_Master, a translucent, two-sided material with the same texture parameters plus Alpha and the scalar OpacityScale; the plugin creates it on first use when it is missing. |
The same values as ini files, if you prefer editing text:
; Config/DefaultEditor.ini — never packaged
[/Script/Tonetta.TonettaEditorSettings]
DevelopmentApiKey="tk_dev_..."
; Config/DefaultGame.ini — ships with the game
[/Script/Tonetta.TonettaSettings]
ApiBaseUrl="https://tonetta.io"
ShippingApiKey="tk_readonly_..."Per-developer keys
Put DevelopmentApiKey in a gitignored Config/UserEditor.ini instead of DefaultEditor.ini and each developer can use their own key — the credit audit log then shows who generated what.
First Run
Open Tools ▸ Tonetta. The header reads Connected as <key name> plus which key is in use, or an error that says what to fix.
- 1New Project — a project is the workspace voices, characters, and content belong to. A fresh organization has none; create one and it becomes the selected project.
- 2Voices — describe a voice in Voice Studio, Design Voice, listen to the previews, save the one you like (spends credits); or link an external provider voice (ElevenLabs, OpenAI, Google, Resemble) into the project for free.
- 3Characters — bind a voice plus persona/tone instructions; Listen auditions it.
- 4Lines — pick a character, write a line (or a prompt like "warn the player about low oxygen"), optionally Rewrite for free with Character Enhancement and Expression markup, then Generate. Import puts the active version at
<Import Root>/Lines/<name>as aUSoundWave. - 5SFX — describe a sound, set the duration, Generate, Import to
<Import Root>/SFX. - 6Music — describe a track, pick a model, set the length, Generate, Import to
<Import Root>/Music. Billed per generation, not per second, so ask for the length the game needs. - 7Models — describe a hard-surface asset, pick which language model builds it, and Generate. The prompt is the whole specification: the agent decides the asset's name, its rig — static, or one of the rig contracts in force for your organization and project — its style and its weathering from what you wrote. The job runs on Tonetta's servers for a few minutes; the screen lists active jobs with their stage, round, and latest preview render, and a Cancel. When it finishes, Import puts everything under
<Import Root>/Models/<Name>/:SK_<Name>(orSM_for static presets) with its skeleton, the textures,MI_<Name>on the master material, a physics asset, andA_<Name>_<Clip>per clip.
Generated TTS, SFX and music are MP3, ElevenLabs TTS is WAV; the importer sniffs the real format from the bytes, so mislabeled legacy entries still play and import.
What a model import gives your gameplay code
Bone and socket names come from the rig contract the model was built to, so DoesSocketExist / AttachToComponent work without authoring socket assets. Which contracts exist is your call, not ours: built-in ones are replaced, by name, by contracts your organization or one of its projects defines, so a mesh can be built to the skeleton your gameplay code already expects. Read the ones in force from GET /api/v1/models/catalog (or the Rig presets page in the dashboard) rather than assuming a bone list — see 3D Models. Every imported asset is tagged with Tonetta.ContentId and Tonetta.Version metadata, so importing a newer version updates the same assets in place.
Audio in a Shipped Game
- •Imported assets are the normal path: generate and import in the editor, reference the
USoundWavefrom Blueprints or C++ like any other sound. Nothing talks to Tonetta at runtime and the shipped build needs no API key. - •Live endpoints go through
UTonettaSubsystem(GameInstance subsystem):ProcessTonettaRequestreturns the cached line or your fallback sound immediately and refreshes the cache in the background. Blueprint access viaUTonettaBlueprintLibrary. The cache lives inSaved/TonettaCache/. - •This is the one case a shipped build needs a key: packaged builds authenticate with the read-only shipping key, so the server serves cached audio only. Fetch each endpoint once from PIE or the dashboard preview before shipping; an endpoint with no cached audio returns 403 and the subsystem plays the fallback.
FTonettaRequest Request;
Request.LiveEndpointId = TEXT("my-endpoint-id");
Request.DefaultSound = FallbackSound;
Request.DefaultText = FText::FromString(TEXT("Fallback text"));
FTonettaResponse Response =
GetGameInstance()->GetSubsystem<UTonettaSubsystem>()->ProcessTonettaRequest(Request);
// Response.Sound / Response.Text are ready immediately (cache or fallback).AI Agents
Three different links exist; the plugin itself needs only the first.
| Link | Purpose | Needed when |
|---|---|---|
| Plugin → tonetta.io (HTTPS + API key) | Every panel button and every TonettaService call | Always. Not an MCP connection — nothing to install. |
| Agent → editor (Unreal's MCP server: the engine's Unreal MCP plugin, plus VibeUE if you use it) | Lets an agent call TonettaService in the running editor — the only route that can import into the project | You want an agent working in the editor. Tonetta registers as TonettaEditor.TonettaService automatically. |
| Agent → hosted Tonetta MCP (this site's MCP server, same key) | The same operations as remote tools, minus import and the panel | The agent works outside the editor, or the editor is not running. Optional alongside the above. |
Teach your agent
From the editor console:
Tonetta.GenerateAgentConfig ClaudeCodewrites the bundled guide into the right memory file for your agent — CLAUDE.md (Claude Code), GEMINI.md (Gemini), AGENTS.md (Codex / Hermes / Cursor), .github/copilot-instructions.md (Copilot) — or All. The section sits between <!-- BEGIN Tonetta … --> markers, so re-run any time to refresh it without touching your own notes. Agents can run it themselves: unreal.TonettaService.generate_agent_config("ClaudeCode").
The toolset from Python
import unreal, json
svc = unreal.TonettaService
def call(fn, *args):
r = json.loads(fn(*args))
assert r.get("success"), r.get("errors")
return r["result"]
print(call(svc.verify_access)["scopes"])
project_id = (call(svc.list_projects) or [call(svc.create_project, "My Game")])[0]["id"]
d = call(svc.design_voice, "A gravelly old sea captain, weathered and warm")
v = call(svc.save_internal_voice, "Sea Captain", d["requestId"], d["previews"][0]["url"], "A gravelly old sea captain")
c = call(svc.create_character, "Captain Marlow", project_id, v["id"], "Weary but kind. Dry humor.")
rw = call(svc.enhance_text, c["id"], "Dive! Dive! All hands to stations!")
line = call(svc.generate_speech, c["id"], rw["textToSpeech"], "dive-order", "", rw["textToScreen"])
print(call(svc.import_content, line["savedContent"]["id"])["assetPath"]) # /Game/Tonetta/Lines/dive-order
# 3D model: submit, wait (long-polls in 240 s slices), import
job = call(svc.generate_model, "twin-engine attack submarine, angular hull, forward laser cannons",
project_id, "glm-5.3-flash") # the prompt is the whole specification
while True:
j = call(svc.wait_for_model_job, job["jobId"], 240)
if j["done"]: break
assert j["status"] == "SUCCEEDED", j.get("error")
imported = call(svc.import_model, j["contentId"], "", "", True, True)
print(imported["assetPath"], imported["materialPath"], imported["animationPaths"]) # /Game/Tonetta/Models/Harrower/SK_Harrower …Every function returns the v1 envelope as a JSON string. Functions mirror the MCP tool list plus import_content, import_audio_url, show_panel, and generate_agent_config. Calls are synchronous and block the editor while they run.
| Function (Python name) | Does |
|---|---|
verify_access() | Key name, organization, scopes — run it first |
list_projects() · get_project(project_id) · create_project(name, description, default_voice_sample_text) · update_project(id, …) · delete_project(id) | Projects; get_project returns the 3D-model art direction when set |
set_project_modeling(project_id, art_direction) · clear_project_modeling(project_id) | The project's art direction for 3D models: one block of prose, written into every job's prompt verbatim |
get_organization_modeling() · set_organization_modeling(…) · clear_organization_modeling() | The organization-wide art direction every project inherits |
list_characters(project_id) · create_character(…) · update_character(…) · delete_character(id) · sample_character(id, text) | Characters and auditions |
list_voices(project_id) · create_voice(…) · update_voice(…) · delete_voice(id) · sample_voice(id, text) | Project (provider) voices |
list_internal_voices() · design_voice(prompt) · save_internal_voice(…) · delete_internal_voice(id) · sample_internal_voice(id, text) | Voice Studio custom voices |
list_voice_providers() · list_external_voices(provider_id) | Provider connections and their voices |
enhance_text(character_id, text, change_tone, use_ssml) · generate_speech(character_id, text, save_name, settings_json, save_text_to_screen) · generate_sfx(…) · get_music_catalog() · generate_music(prompt, duration_seconds, save_name, project_id, model, negative_prompt) | Text enhancement (free) and audio generation (spends credits) |
list_content(content_type, project_id, character_id) · get_content(id) · update_content(content_id, name, tags_json, character_id) · set_active_content_version(id, version) · delete_content(id) | Saved lines, sound effects, music, 3D models and Canvas images (content_type line | sfx | music | model | image); update_content renames / re-tags / links a model or image to a character (a model's name is its asset name) |
list_live_endpoints(…) · get_live_endpoint(id) · create_live_endpoint(…) · update_live_endpoint(…) · delete_live_endpoint(id) | Live endpoints the runtime cache fetches |
get_model_catalog(project_id) | The operator-curated LLM list with what a job on each typically costs, the effective rig presets, and the fal.ai vendor lists (seed, texture, retopology, image models) — read it before choosing any of them |
generate_model(prompt, project_id, model, reference_images, budget_usd, preset, seed_model, paint_model, retopo_model, concept_model, seed_first, character_id) | Submit a generation job from a prompt; everything after the prompt is optional — preset fixes the rig contract, the *_model keys pick the fal.ai vendors, seed_first=False builds from primitives. Returns the job id and reservation at once |
get_model_job(job_id) · list_model_jobs(status, project_id) | Status, stage, round, preview URLs, cost, art-direction layers used |
wait_for_model_job(job_id, timeout_seconds) | Long-poll (≤ 240 s per call); loop until done |
refine_model(content_id, instructions, model, reference_images, budget_usd, seed_model, paint_model, retopo_model, concept_model) | Start a refine job on a saved model (new version); vendor keys default to the version's own |
cancel_model_job(job_id) | Cancel a queued or running job |
start_model_session(message, content_id, …) · send_model_message(content_id, message, approved_tools) · get_model_session(content_id) · export_model_session(content_id, message) | A conversation with the build agent instead of a job: one step per message, the agent proposes the next steps (vendor steps included, by what they would buy) and asks where to go next; export saves a version |
start_canvas_session(message, content_id, project_id, model, concept_model, edit_model) · send_canvas_message(content_id, message, approved_tools) · get_canvas_session(content_id) · export_canvas_session(content_id, message) | A conversation with the Canvas agent for composite 2D work — sprite sheets, button states, 9-slice panels, exact text, seamless tiles — on a saved image or from a brief; one step per message, the export is a version import_content brings in as a UI texture |
start_sound_session(content_id, message, model) · send_sound_message(content_id, message) · get_sound_session(content_id) · export_sound_session(content_id, message) | A conversation with the Sound agent about a saved line or effect — loudness in LUFS and true peak, comms and room treatments, trims, fades, loops; the export is a new audio version of the same item |
import_model(content_id, asset_name, destination_folder, import_animations, create_physics_asset) · import_model_from_folder(local_folder, …) | Download the active version (or read a local folder) and import mesh, textures, material instance, physics asset, and clips; returns every created asset path |
generate_and_import_model(…) | Submit, wait on an editor ticker, import — in one call |
import_content(content_id, asset_name, destination_folder) · import_audio_url(url, asset_name, destination_folder) | Turn saved audio into USoundWave and a Canvas image into a UI UTexture2D assets (import_content dispatches model ids to the model importer) |
show_panel(screen) · generate_agent_config(client, import) | Editor-only helpers |
The exact signatures come from discover_python_class('unreal.TonettaService') in the editor; the bundled agent guide (Tonetta.GenerateAgentConfig) lists every function with its arguments.
Troubleshooting
| Symptom | Cause / fix |
|---|---|
| Panel says Not connected with a link to the dashboard | No key resolved. Paste your development key into Tonetta (Development Key). |
| Settings show an empty key but the panel is connected | The key comes from a legacy location — Key In Use says which. Since plugin 2.2 the editor copies it into the Development Api Key on first start; if that did not happen, Config/DefaultEditor.ini is not writable. |
missing_scope | The key was created without that permission. Add the scope to the key in the dashboard. |
missing_api_key with a valid key | The base URL is not https://tonetta.io — older domains redirect and drop the Authorization header. |
| Packaged game logs a warning about a legacy key | The build fell back to the single legacy key. Create a read-only key and set Shipping Api Key, then remove the legacy entry. |
Live endpoint returns 403 readOnly in a packaged build | Nothing is cached for that endpoint yet. Fetch it once with a tts:generate key (PIE, dashboard preview, or curl). |
Model import fails with import_failed naming a step | The importer rolled back what it had created. Check the step: mesh — the FBX could not be read by the engine's FBX importer; material — the master material in Settings is missing a parameter; animation — a clip did not match the imported skeleton. Re-run after fixing; nothing is left half-imported. |
Models screen shows missing_scope | The development key needs models:read (browse/import) or models:write (generate/refine). |