Skip to main content

Marigold Insulation

RealtimeVoice - End-to-End Realtime Voice Chat plugin for Unreal Engine 5.6–5.8

RealtimeVoice – End-to-End Realtime Voice Chat is a code plugin for Unreal Engine 5.6–5.8 that connects microphone speech, a language model and streaming text-to-speech over one WebSocket connection.

RealtimeVoice – End-to-End Realtime Voice Chat is a plugin for building voice conversations with in-game characters in Unreal Engine. It runs the full loop in real time: the player speaks into the microphone, speech is transcribed with Volcengine Doubao ASR, a language model produces a reply, and that reply is streamed back as speech. The plugin supports Unreal Engine 5.6, 5.7 and 5.8 on Windows. It suits AI assistants, NPC dialogue and conversational game systems, and it is built to be driven entirely from Blueprints.

Specifications

Seller odysseyzjh
Type Plugin
Category Animations
Engines Unreal Engine 5.6–5.8
Distribution Code plugin
Formats Unreal Engine
File types .uasset
Unreal target platforms Windows
C++ classes 8
Network replicated No
Licenses Personal, Professional
Published 2026-09-21

Key features

  • End-to-end realtime voice dialog over a single WebSocket connection
  • Microphone speech recognition, LLM reply and streaming TTS in one loop
  • Native Doubao all-in-one mode or OpenAI-compatible endpoint via HTTP bridge
  • Adjustable model, temperature, max tokens and history turns
  • Barge-in interruption lets players cut off a reply mid-speech
  • One ActorComponent with event delegates for ASR text, assistant text and errors
  • Inheritable UMG chat-widget base class included
  • TTS audio-stream events for driving MetaHuman lip-sync

What the Plugin Includes

The plugin ships as a single Runtime module named RealtimeVoice with 8 C++ classes. Its core is one ActorComponent that exposes event delegates for recognized user speech, assistant text, speech start and end, user interruption and errors. An inheritable UMG chat-widget base class is included for building an on-screen chat interface. TTS audio-stream events are also exposed so you can route the reply audio to other systems.

The plugin contains no Blueprints of its own; all logic is set up in your project's Blueprint graph.

Technical Specifications and Requirements

Microphone input is 16 kHz mono PCM, and TTS output is 24 kHz mono int16. The plugin depends on the engine's AudioCapture plugin, which is enabled automatically. It is distributed as a code plugin and is not network replicated.

The plugin requires a Volcengine (Doubao) account, and you must supply your own AppId and AccessKey. It needs an active internet connection because it is cloud-based and cannot run offline. The Unreal target build platform is Win64.

Language Model Options and Lip-Sync Use

The plugin offers two LLM modes: the native Doubao all-in-one model, or a DeepSeek or OpenAI-compatible endpoint reached through an HTTP bridge. In the second mode you can set the custom model, temperature, max tokens and history turns. Barge-in interruption is supported, so the player can cut off a reply mid-speech.

Because TTS audio-stream events are exposed, they can feed MetaHumanAudioToFaceRuntime for MetaHuman lip-sync.

Use cases

  • AI assistant characters
  • Voice-driven NPC dialogue
  • Conversational game systems
  • MetaHuman lip-sync prototypes

RealtimeVoice - End-to-End Realtime Voice Chat (Volcengine Doubao ASR+LLM+TTS) FAQ

Which Unreal Engine versions does RealtimeVoice support?

RealtimeVoice supports Unreal Engine 5.6, 5.7 and 5.8. The supported target build platform is Win64 on Windows.

Does RealtimeVoice work offline?

No. The plugin is cloud-based and requires an active internet connection. It also requires a Volcengine (Doubao) account with your own AppId and AccessKey.

Is RealtimeVoice built with Blueprints or C++?

The plugin is designed to be driven from Blueprints through one ActorComponent and its event delegates. It contains 8 C++ classes and no Blueprints of its own.

Can RealtimeVoice be used with MetaHuman lip-sync?

Yes. Its TTS output is 24 kHz mono int16 and its audio-stream events can feed MetaHumanAudioToFaceRuntime for lip-sync.

Comments are closed