All Documentation

Unreal Engine Plugin

Design voices, characters, lines, sound effects, and 3D models inside the Unreal Editor, import them as assets, and let AI agents do the same

Overview

The Tonetta plugin for Unreal Engine 5.8+ puts the whole Tonetta workflow inside the editor and turns the results into assets: USoundWave for audio, and — since plugin 2.3 — skeletal or static meshes with textures, a material instance, a physics asset, and animation clips for 3D models. It talks to tonetta.io over HTTPS with your API key — nothing else to run.

PieceWhat it does
Editor panelTools ▸ Tonetta: Models, Voices, Characters, Lines, SFX and Music screens with a project selector (New / Edit Project), audition buttons, generation, version history, and Import
AI toolsetTonettaService, registered on the engine's native AI toolset system, so agents connected to Unreal's MCP server can go from "design a voice" or "build me a gunship" to an imported asset without you
Agent guide commandTonetta.GenerateAgentConfig writes a Tonetta section into your CLAUDE.md / GEMINI.md / AGENTS.md
C++ clientFTonettaApiClient wraps the entire v1 API with async and sync calls
Runtime cacheA GameInstance subsystem that serves live endpoint audio in a shipped game with instant fallback and background refresh

Requirements

  • •Unreal Engine 5.8 or newer. The plugin builds on the engine's ToolsetRegistry plugin, which ships with 5.8 and is enabled automatically.
  • •A C++-capable project (the plugin compiles from source), Git, and Visual Studio / Xcode as for any UE C++ project.
  • •A Tonetta account and an API key for the editor — plus, only if your game fetches live endpoints at runtime, a second read-only key (see below). Generation spends credits.

Install

  1. 1Clone the plugin into your project's Plugins folder:
    cd /path/to/YourProject/Plugins
    git clone https://github.com/kevinpbuckley/tonetta.git Tonetta
  2. 2Build the editor target as usual — right-click the .uproject → Generate Visual Studio project files, then build YourProjectEditor. (Projects using VibeUE can run its BuildAndLaunchGame.ps1 instead.)
  3. 3Open the editor, go to Edit ▸ Plugins ▸ Audio ▸ Tonetta, enable it, and restart when asked.
  4. 4Later, update with git pull inside Plugins/Tonetta and rebuild.

Repository layout

The repo is the plugin: Tonetta.uplugin, Source/ (modules Tonetta, TonettaEditor, TonettaRuntime), Content/, and a README with the same information as this page. Engine modules only, no third-party code.

API Keys: Development vs Shipping

The editor needs a full-scope development key. A shipped game normally needs no key at all — imported USoundWave assets play like any other sound and nothing talks to Tonetta at runtime. The only exception is a game that fetches live endpoints while running: that gets a separate, read-only shipping key, because Config/DefaultGame.ini is packaged inside your game and anything in it can be read out of a shipped build.

KeyUsed byScopesStored in
Development (required)The editor: panel, TonettaService, Play In EditorEverything you want to do from the editor — typically all scopesConfig/DefaultEditor.ini — never staged into packaged builds
Shipping (only with live endpoints)Packaged builds that fetch live endpoints at runtime; leave empty otherwiseRead-only: live_endpoints:readConfig/DefaultGame.ini — ships with the game

A live_endpoints:read shipping key can fetch cached live-endpoint audio but can never create, delete, or spend credits, so a leaked key costs you nothing. The Public TTS API treats it as cache-only: cached audio is served, generation is never triggered. Warm every live endpoint once from the editor or the dashboard before you ship.

  1. 1Go to Dashboard ▸ API Keys and create a development key with the scopes you want in the editor.
  2. 2Only if your game uses live endpoints at runtime: create a second, shipping key with only live_endpoints:read.
  3. 3Paste each into its slot in Project Settings (next section). Leaving a scope out later shows up as a missing_scope error in the panel — add it to the key here, no plugin change needed.
ScopeUnlocks in the plugin
projects:writeNew Project / Edit Project, create_project, update_project, delete_project
characters:writeCharacters screen and the character functions
voices:writeProject (provider) voices
internal_voices:writeVoice Studio: design, save, delete custom voices
external_voices:readBrowse providers and their voices
lines:write / sfx:writeSave, version, delete lines and SFX; generate SFX
music:writeSave, version, delete music; generate music (music:read reads the model catalog)
tts:generateGenerate speech and audition characters
live_endpoints:writeCreate and edit live endpoints from the toolset (live_endpoints:read alone is the shipping key)
models:read / models:writeModels screen: browse and import saved 3D models (read); generate, refine, cancel jobs (write, spends credits)

Configure

Two sections under Project Settings ▸ Plugins (the panel's Settings button opens the first):

Tonetta (Development Key) — DefaultEditor.ini, never packaged

SettingValue
Development Api KeyThe full-scope key. Used by the panel, the toolset, and PIE.

Tonetta — DefaultGame.ini, ships with the game

SettingValue
Api Base Urlhttps://tonetta.io (default)
Shipping Api KeyThe read-only key, used by packaged builds. Only needed when the shipped game fetches live endpoints at runtime — leave empty otherwise.
Key In UseRead-only: which key this editor resolved and where it came from. Check here first when the connection looks wrong.
Api Key (legacy fallback)Advanced. The pre-2.2 single key; still honored by both editor and packaged builds when the dedicated keys are empty, and copied into the Development Api Key automatically on first editor start. Clear it once the shipping key is set.
Default Project IdOptional; preselects a project in the panel
Import Root PathContent folder for imported assets — default /Game/Tonetta, with Lines/, SFX/, Music/, and Models/<Name>/ underneath
Models: Create Physics Asset / Import Animations / Save PackagesWhat a model import produces besides the mesh, textures, and material instance (all on by default)
Model Master MaterialThe parent of every imported model's MI_<Name>; defaults to the plugin's M_TonettaModel_Master (BaseColor, Roughness, Metallic, Emissive, Normal, EmissiveStrength). Point it at your own material with the same parameter names to restyle imports.
Model Glass MaterialThe parent of MI_<Name>_Glass, which the importer puts on the slots meta.json lists as transparentSlots (a canopy, a window, a lamp's chimney). Defaults to the plugin's M_TonettaGlass_Master, a translucent, two-sided material with the same texture parameters plus Alpha and the scalar OpacityScale; the plugin creates it on first use when it is missing.

The same values as ini files, if you prefer editing text:

; Config/DefaultEditor.ini — never packaged
[/Script/Tonetta.TonettaEditorSettings]
DevelopmentApiKey="tk_dev_..."

; Config/DefaultGame.ini — ships with the game
[/Script/Tonetta.TonettaSettings]
ApiBaseUrl="https://tonetta.io"
ShippingApiKey="tk_readonly_..."

Per-developer keys

Put DevelopmentApiKey in a gitignored Config/UserEditor.ini instead of DefaultEditor.ini and each developer can use their own key — the credit audit log then shows who generated what.

First Run

Open Tools ▸ Tonetta. The header reads Connected as <key name> plus which key is in use, or an error that says what to fix.

  1. 1New Project — a project is the workspace voices, characters, and content belong to. A fresh organization has none; create one and it becomes the selected project.
  2. 2Voices — describe a voice in Voice Studio, Design Voice, listen to the previews, save the one you like (spends credits); or link an external provider voice (ElevenLabs, OpenAI, Google, Resemble) into the project for free.
  3. 3Characters — bind a voice plus persona/tone instructions; Listen auditions it.
  4. 4Lines — pick a character, write a line (or a prompt like "warn the player about low oxygen"), optionally Rewrite for free with Character Enhancement and Expression markup, then Generate. Import puts the active version at <Import Root>/Lines/<name> as a USoundWave.
  5. 5SFX — describe a sound, set the duration, Generate, Import to <Import Root>/SFX.
  6. 6Music — describe a track, pick a model, set the length, Generate, Import to <Import Root>/Music. Billed per generation, not per second, so ask for the length the game needs.
  7. 7Models — describe a hard-surface asset, pick which language model builds it, and Generate. The prompt is the whole specification: the agent decides the asset's name, its rig — static, or one of the rig contracts in force for your organization and project — its style and its weathering from what you wrote. The job runs on Tonetta's servers for a few minutes; the screen lists active jobs with their stage, round, and latest preview render, and a Cancel. When it finishes, Import puts everything under <Import Root>/Models/<Name>/: SK_<Name> (or SM_ for static presets) with its skeleton, the textures, MI_<Name> on the master material, a physics asset, and A_<Name>_<Clip> per clip.

Generated TTS, SFX and music are MP3, ElevenLabs TTS is WAV; the importer sniffs the real format from the bytes, so mislabeled legacy entries still play and import.

What a model import gives your gameplay code

Bone and socket names come from the rig contract the model was built to, so DoesSocketExist / AttachToComponent work without authoring socket assets. Which contracts exist is your call, not ours: built-in ones are replaced, by name, by contracts your organization or one of its projects defines, so a mesh can be built to the skeleton your gameplay code already expects. Read the ones in force from GET /api/v1/models/catalog (or the Rig presets page in the dashboard) rather than assuming a bone list — see 3D Models. Every imported asset is tagged with Tonetta.ContentId and Tonetta.Version metadata, so importing a newer version updates the same assets in place.

Audio in a Shipped Game

  • •Imported assets are the normal path: generate and import in the editor, reference the USoundWave from Blueprints or C++ like any other sound. Nothing talks to Tonetta at runtime and the shipped build needs no API key.
  • •Live endpoints go through UTonettaSubsystem (GameInstance subsystem): ProcessTonettaRequest returns the cached line or your fallback sound immediately and refreshes the cache in the background. Blueprint access via UTonettaBlueprintLibrary. The cache lives in Saved/TonettaCache/.
  • •This is the one case a shipped build needs a key: packaged builds authenticate with the read-only shipping key, so the server serves cached audio only. Fetch each endpoint once from PIE or the dashboard preview before shipping; an endpoint with no cached audio returns 403 and the subsystem plays the fallback.
FTonettaRequest Request;
Request.LiveEndpointId = TEXT("my-endpoint-id");
Request.DefaultSound = FallbackSound;
Request.DefaultText = FText::FromString(TEXT("Fallback text"));

FTonettaResponse Response =
    GetGameInstance()->GetSubsystem<UTonettaSubsystem>()->ProcessTonettaRequest(Request);
// Response.Sound / Response.Text are ready immediately (cache or fallback).

AI Agents

Three different links exist; the plugin itself needs only the first.

LinkPurposeNeeded when
Plugin → tonetta.io (HTTPS + API key)Every panel button and every TonettaService callAlways. Not an MCP connection — nothing to install.
Agent → editor (Unreal's MCP server: the engine's Unreal MCP plugin, plus VibeUE if you use it)Lets an agent call TonettaService in the running editor — the only route that can import into the projectYou want an agent working in the editor. Tonetta registers as TonettaEditor.TonettaService automatically.
Agent → hosted Tonetta MCP (this site's MCP server, same key)The same operations as remote tools, minus import and the panelThe agent works outside the editor, or the editor is not running. Optional alongside the above.

Teach your agent

From the editor console:

Tonetta.GenerateAgentConfig ClaudeCode

writes the bundled guide into the right memory file for your agent — CLAUDE.md (Claude Code), GEMINI.md (Gemini), AGENTS.md (Codex / Hermes / Cursor), .github/copilot-instructions.md (Copilot) — or All. The section sits between <!-- BEGIN Tonetta … --> markers, so re-run any time to refresh it without touching your own notes. Agents can run it themselves: unreal.TonettaService.generate_agent_config("ClaudeCode").

The toolset from Python

import unreal, json
svc = unreal.TonettaService
def call(fn, *args):
    r = json.loads(fn(*args))
    assert r.get("success"), r.get("errors")
    return r["result"]

print(call(svc.verify_access)["scopes"])
project_id = (call(svc.list_projects) or [call(svc.create_project, "My Game")])[0]["id"]
d = call(svc.design_voice, "A gravelly old sea captain, weathered and warm")
v = call(svc.save_internal_voice, "Sea Captain", d["requestId"], d["previews"][0]["url"], "A gravelly old sea captain")
c = call(svc.create_character, "Captain Marlow", project_id, v["id"], "Weary but kind. Dry humor.")
rw = call(svc.enhance_text, c["id"], "Dive! Dive! All hands to stations!")
line = call(svc.generate_speech, c["id"], rw["textToSpeech"], "dive-order", "", rw["textToScreen"])
print(call(svc.import_content, line["savedContent"]["id"])["assetPath"])   # /Game/Tonetta/Lines/dive-order

# 3D model: submit, wait (long-polls in 240 s slices), import
job = call(svc.generate_model, "twin-engine attack submarine, angular hull, forward laser cannons",
           project_id, "glm-5.3-flash")          # the prompt is the whole specification
while True:
    j = call(svc.wait_for_model_job, job["jobId"], 240)
    if j["done"]: break
assert j["status"] == "SUCCEEDED", j.get("error")
imported = call(svc.import_model, j["contentId"], "", "", True, True)
print(imported["assetPath"], imported["materialPath"], imported["animationPaths"])   # /Game/Tonetta/Models/Harrower/SK_Harrower …

Every function returns the v1 envelope as a JSON string. Functions mirror the MCP tool list plus import_content, import_audio_url, show_panel, and generate_agent_config. Calls are synchronous and block the editor while they run.

Function (Python name)Does
verify_access()Key name, organization, scopes — run it first
list_projects() · get_project(project_id) · create_project(name, description, default_voice_sample_text) · update_project(id, …) · delete_project(id)Projects; get_project returns the 3D-model art direction when set
set_project_modeling(project_id, art_direction) · clear_project_modeling(project_id)The project's art direction for 3D models: one block of prose, written into every job's prompt verbatim
get_organization_modeling() · set_organization_modeling(…) · clear_organization_modeling()The organization-wide art direction every project inherits
list_characters(project_id) · create_character(…) · update_character(…) · delete_character(id) · sample_character(id, text)Characters and auditions
list_voices(project_id) · create_voice(…) · update_voice(…) · delete_voice(id) · sample_voice(id, text)Project (provider) voices
list_internal_voices() · design_voice(prompt) · save_internal_voice(…) · delete_internal_voice(id) · sample_internal_voice(id, text)Voice Studio custom voices
list_voice_providers() · list_external_voices(provider_id)Provider connections and their voices
enhance_text(character_id, text, change_tone, use_ssml) · generate_speech(character_id, text, save_name, settings_json, save_text_to_screen) · generate_sfx(…) · get_music_catalog() · generate_music(prompt, duration_seconds, save_name, project_id, model, negative_prompt)Text enhancement (free) and audio generation (spends credits)
list_content(content_type, project_id, character_id) · get_content(id) · update_content(content_id, name, tags_json, character_id) · set_active_content_version(id, version) · delete_content(id)Saved lines, sound effects, music, 3D models and Canvas images (content_type line | sfx | music | model | image); update_content renames / re-tags / links a model or image to a character (a model's name is its asset name)
list_live_endpoints(…) · get_live_endpoint(id) · create_live_endpoint(…) · update_live_endpoint(…) · delete_live_endpoint(id)Live endpoints the runtime cache fetches
get_model_catalog(project_id)The operator-curated LLM list with what a job on each typically costs, the effective rig presets, and the fal.ai vendor lists (seed, texture, retopology, image models) — read it before choosing any of them
generate_model(prompt, project_id, model, reference_images, budget_usd, preset, seed_model, paint_model, retopo_model, concept_model, seed_first, character_id)Submit a generation job from a prompt; everything after the prompt is optional — preset fixes the rig contract, the *_model keys pick the fal.ai vendors, seed_first=False builds from primitives. Returns the job id and reservation at once
get_model_job(job_id) · list_model_jobs(status, project_id)Status, stage, round, preview URLs, cost, art-direction layers used
wait_for_model_job(job_id, timeout_seconds)Long-poll (≤ 240 s per call); loop until done
refine_model(content_id, instructions, model, reference_images, budget_usd, seed_model, paint_model, retopo_model, concept_model)Start a refine job on a saved model (new version); vendor keys default to the version's own
cancel_model_job(job_id)Cancel a queued or running job
start_model_session(message, content_id, …) · send_model_message(content_id, message, approved_tools) · get_model_session(content_id) · export_model_session(content_id, message)A conversation with the build agent instead of a job: one step per message, the agent proposes the next steps (vendor steps included, by what they would buy) and asks where to go next; export saves a version
start_canvas_session(message, content_id, project_id, model, concept_model, edit_model) · send_canvas_message(content_id, message, approved_tools) · get_canvas_session(content_id) · export_canvas_session(content_id, message)A conversation with the Canvas agent for composite 2D work — sprite sheets, button states, 9-slice panels, exact text, seamless tiles — on a saved image or from a brief; one step per message, the export is a version import_content brings in as a UI texture
start_sound_session(content_id, message, model) · send_sound_message(content_id, message) · get_sound_session(content_id) · export_sound_session(content_id, message)A conversation with the Sound agent about a saved line or effect — loudness in LUFS and true peak, comms and room treatments, trims, fades, loops; the export is a new audio version of the same item
import_model(content_id, asset_name, destination_folder, import_animations, create_physics_asset) · import_model_from_folder(local_folder, …)Download the active version (or read a local folder) and import mesh, textures, material instance, physics asset, and clips; returns every created asset path
generate_and_import_model(…)Submit, wait on an editor ticker, import — in one call
import_content(content_id, asset_name, destination_folder) · import_audio_url(url, asset_name, destination_folder)Turn saved audio into USoundWave and a Canvas image into a UI UTexture2D assets (import_content dispatches model ids to the model importer)
show_panel(screen) · generate_agent_config(client, import)Editor-only helpers

The exact signatures come from discover_python_class('unreal.TonettaService') in the editor; the bundled agent guide (Tonetta.GenerateAgentConfig) lists every function with its arguments.

Troubleshooting

SymptomCause / fix
Panel says Not connected with a link to the dashboardNo key resolved. Paste your development key into Tonetta (Development Key).
Settings show an empty key but the panel is connectedThe key comes from a legacy location — Key In Use says which. Since plugin 2.2 the editor copies it into the Development Api Key on first start; if that did not happen, Config/DefaultEditor.ini is not writable.
missing_scopeThe key was created without that permission. Add the scope to the key in the dashboard.
missing_api_key with a valid keyThe base URL is not https://tonetta.io — older domains redirect and drop the Authorization header.
Packaged game logs a warning about a legacy keyThe build fell back to the single legacy key. Create a read-only key and set Shipping Api Key, then remove the legacy entry.
Live endpoint returns 403 readOnly in a packaged buildNothing is cached for that endpoint yet. Fetch it once with a tts:generate key (PIE, dashboard preview, or curl).
Model import fails with import_failed naming a stepThe importer rolled back what it had created. Check the step: mesh — the FBX could not be read by the engine's FBX importer; material — the master material in Settings is missing a parameter; animation — a clip did not match the imported skeleton. Re-run after fixing; nothing is left half-imported.
Models screen shows missing_scopeThe development key needs models:read (browse/import) or models:write (generate/refine).