Hello! now
tms@lcdchip.com
SYSTEM ARCHITECTURE WHITEPAPER · VOICE AI · EDGE COMPUTING · SMART TERMINALS
The most important voice AI decision is not which speech recognition API to use. It is deciding where each part of the voice pipeline should run.
Wake-word detection, acoustic processing, speech recognition, intent classification, large-language-model reasoning, text-to-speech, device control and logging do not need to execute in the same location. A production voice terminal can distribute these workloads between the device and the cloud according to latency, privacy, network availability, model complexity and hardware resources.
This engineering guide presents a system-level architecture for AI kiosks, digital human terminals, smart fitness mirrors, service robots, healthcare terminals, hotel self-service displays, retail assistants and industrial voice HMIs.
1. The Wrong Question: "Should Voice AI Run Locally or in the Cloud?"
This question sounds reasonable but oversimplifies the architecture. A modern voice terminal contains multiple processing stages, and each stage has a different requirement.
Wake-word detection benefits from low latency and local processing. Open-domain reasoning may benefit from a much larger cloud model. Device commands should continue to work when the internet is unavailable. Logs may need centralised fleet analysis. Speech synthesis may be local for basic prompts but cloud-generated for dynamic conversations.
2. Decompose the Voice AI Pipeline Before Choosing the Architecture
Microphones, codec, PDM/I²S, gain, acoustic enclosure and capture path.
VAD, AEC, noise suppression, beamforming and gain control.
Keyword detection, button-to-talk or other activation logic.
Audio-to-text using local, cloud or hybrid ASR.
Local command parser, NLU, edge model or cloud LLM.
Device control, API request, database operation or business workflow.
Local or cloud TTS, screen UI, avatar animation and device feedback.
Latency, failure reason, model version, network state and anonymised telemetry.
Breaking the system into these stages makes it possible to design a routing policy instead of treating voice interaction as one cloud API call.
3. Three Voice AI Architectures
Cloud-Centric Voice AI
The device captures audio and performs basic acoustic processing, while speech recognition and most intelligence run on remote infrastructure.
Strong when
- The application requires open-domain conversation.
- Models change frequently.
- Knowledge must remain current.
- Multiple languages must be added quickly.
- Local compute cost must remain low.
Main engineering risks
- Network-dependent interaction
- Variable end-to-end latency
- Cloud service outages
- Operating cost per request
- Data-governance complexity
On-Device Voice AI
Wake word, recognition, intent processing and most device actions execute locally. Cloud connectivity may be optional.
Strong when
- Internet access is unreliable.
- Commands are limited and predictable.
- Low latency is critical.
- Privacy requirements favour local processing.
- The device must remain functional offline.
Main engineering risks
- Model memory footprint
- Compute and thermal requirements
- Multilingual model storage
- Limited open-domain intelligence
- More complex local model lifecycle
Hybrid Voice AI
Local processing handles activation, acoustic processing, core commands and failure recovery, while cloud services provide advanced recognition, knowledge retrieval and large-model reasoning.
Strong when
- The product is a commercial smart terminal.
- Basic commands must work offline.
- Open conversation is also required.
- Privacy boundaries must be clearly defined.
- The product requires graceful network failure behaviour.
Main engineering risks
- Routing-policy complexity
- Model/version synchronisation
- Local/cloud result conflicts
- Fallback-state design
- More extensive validation
4. Architecture Decision Matrix
| Requirement | Cloud-Centric | On-Device | Hybrid |
|---|---|---|---|
| Wake word | Possible but inefficient as a cloud-first function | Strong fit | Strong fit |
| Fixed device commands | Works but introduces unnecessary dependency | Strong fit | Strong fit |
| Open-domain conversation | Strong fit | Hardware/model dependent | Strong fit |
| Offline operation | Weak | Strong | Strong for defined functions |
| Lowest network dependency | Weak | Strong | Medium to strong |
| Rapid language expansion | Strong | Requires model distribution | Strong |
| Privacy-sensitive commands | Requires careful governance | Strong fit | Strong when routing is controlled |
| Large AI models | Strong | Hardware constrained | Strong |
| Fleet-wide analytics | Strong | Requires telemetry design | Strong |
| Network outage resilience | Weak | Strong | Strong when fallback is designed correctly |
5. Latency Budget: Measure the Pipeline, Not Only the Final Response
Teams frequently measure voice performance with one number: "How long did the assistant take to answer?"
That number is useful for the user experience but weak for engineering. The system should record latency at each processing boundary.
This allows engineers to identify whether slow interaction comes from acoustic capture, speech recognition, network transport, model inference, business API calls or text-to-speech.
Recommended latency telemetry
- Wake detection latency
- Speech endpoint detection latency
- ASR processing latency
- Network round-trip contribution
- LLM or intent-engine latency
- External API latency
- TTS first-audio latency
- Total speech-to-response latency
- P50, P95 and worst-case observations
6. Build a Routing Policy Instead of Hard-Coding "Local" or "Cloud"
Hybrid architecture requires a routing layer that decides where each request should execute. The decision can depend on intent type, network state, privacy classification, local model availability, language and product configuration.
Route Locally
- Wake word
- Volume adjustment
- Brightness control
- Start / stop
- Screen navigation
- Emergency commands
- Basic machine functions
Route to Cloud
- Open-domain questions
- Knowledge retrieval
- Complex customer support
- Live business information
- Long multilingual dialogue
- Large-model reasoning
- Dynamic content generation
Route Dynamically
- Language-dependent recognition
- Optional cloud enhancement
- Confidence-based ASR retry
- Privacy-sensitive workflows
- Network-quality-based routing
- Local model unavailable
- Regional service restrictions
Do not let network failure decide the user experience accidentally. Define the fallback path before the product ships.
7. Design the Privacy Boundary as Part of the Architecture
Privacy cannot be added after the voice pipeline is finished. The engineering team should define exactly when audio exists, where it is processed, when it leaves the device and what is retained.
Always-On Wake Detection
Short audio windows processed locally for activation logic.
Acoustic Processing
AEC, noise suppression, VAD and other audio preparation.
Active Speech
The system decides whether recognised speech remains local or may be transmitted.
Cloud ASR / AI
Only data required by the selected cloud workflow should cross the boundary.
Logs and Analytics
Store only defined telemetry and retention fields required for product operation.
Privacy design questions
Raw audio, transcript, intent, anonymised metric or nothing?
After wake word, push-to-talk, touch confirmation or explicit user action?
Audio clips, transcripts, command type, diagnostic metadata or aggregate statistics?
Define retention by data category rather than keeping everything indefinitely.
Provide microphone mute or clearly defined voice-disable behaviour where appropriate.
Use screen state, LED, icon, tone or avatar feedback to make active listening visible.
8. Offline Fallback Is a Product Feature, Not an Error Message
Many cloud-connected terminals display "Network Error" when internet access disappears. For voice interaction, this is poor architecture.
Instead, divide functionality into capability tiers.
No Network
- Wake word
- Basic navigation
- Volume control
- Brightness control
- Local emergency commands
- Touch fallback
Limited Network
- Local ASR where available
- Cached content
- Local device control
- Queued telemetry
- Retry logic
- Clear network-state UI
Full Cloud Connectivity
- Cloud ASR
- Open dialogue
- Knowledge search
- Cloud business APIs
- Dynamic TTS
- Fleet services
The user should experience graceful capability reduction rather than complete product failure.
9. Audio Front-End: Cloud AI Cannot Fix Poor Acoustic Hardware
A larger language model cannot recover information that was never captured correctly. Voice AI quality therefore begins with acoustic engineering.
Microphone Topology
Define microphone count, geometry, spacing, interface, directionality and expected speaking distance.
Acoustic Echo Cancellation
Required when the terminal must hear the user while its own speaker or TTS output is active.
Noise Suppression
Validate against both steady noise such as ventilation and dynamic noise such as nearby conversations.
Beamforming
Consider when user position, interaction distance and environmental noise make single-channel capture insufficient.
Mechanical Acoustics
Microphone holes, seals, waterproof membranes, glass, speaker position and enclosure cavities alter real performance.
Noise Coupling
Fans, switching regulators, displays, amplifiers, USB peripherals and mechanical vibration can enter the audio path.
10. Edge Hardware Sizing: Plan for Concurrent Workloads
Do not size a voice AI terminal by running each subsystem separately. The real product may run display rendering, camera capture, local inference, voice processing, networking and storage operations simultaneously.
11. Multilingual Voice AI Is a Model-Lifecycle Problem
Adding languages is not simply adding another menu option. Each language may affect wake-word models, ASR packages, pronunciation, domain vocabulary, TTS, storage capacity and validation workload.
Wake Layer
Wake-word pronunciation, false activation and accent variation.
Recognition Layer
Language model availability, offline package size and cloud provider coverage.
Intent Layer
Command synonyms, domain terms and ambiguous phrasing.
Response Layer
TTS voice, pronunciation, speed and regional localisation.
UI Layer
Transcription display, error messaging and language-selection flow.
Validation Layer
Native speakers, accents, environmental noise and application-specific vocabulary.
12. Model Lifecycle: Voice Models Must Be Versioned Like Firmware
A production voice terminal may contain several independently changing models: wake word, VAD, noise processing, ASR, intent recognition, LLM routing, TTS and vision AI.
If these assets are updated independently without compatibility control, an OTA release can improve one metric while breaking another.
A production release should record this complete compatibility set, not only the Android application version.
13. Failure Containment: Design What Happens When Each Layer Breaks
| Failure | Expected Product Behaviour | What Must Not Happen |
|---|---|---|
| Wake model unavailable | Offer touch or button activation and report diagnostic state | Silent loss of all interaction |
| Microphone failure | Show voice unavailable state and preserve screen controls | Repeated cloud requests with empty audio |
| Internet unavailable | Switch to supported local commands and show network state | Freeze UI waiting indefinitely |
| Cloud ASR timeout | Retry according to policy or use local fallback | Duplicate user actions |
| LLM service unavailable | Use controlled fallback message and retain device commands | Disable unrelated local functions |
| TTS unavailable | Display response on screen | Discard successful AI result |
| OTA introduces regression | Rollback to validated release | Leave fleet permanently degraded |
| Storage near capacity | Rotate logs and protect critical system partitions | Allow logging to break boot or updates |
14. Observability: A Voice Terminal Must Explain Why It Failed
Field debugging becomes extremely expensive when the only customer report is: "Sometimes it doesn't understand me."
A production system should collect structured diagnostic events without automatically retaining unnecessary voice content.
Audio State
Microphone available, AFE loaded, gain state and speaker activity.
Wake State
Wake attempt, accepted/rejected state and model version.
ASR State
Local/cloud path, language, completion state and processing time.
Routing State
Why a request used local or cloud processing.
Network State
Connectivity, timeout class and reconnect behaviour.
System State
CPU load, memory pressure, temperature and storage condition.
This makes field failures reproducible without turning every deployment into a raw-audio surveillance system.
15. Validation KPIs for a Production Voice AI Terminal
"Works" is not a useful acceptance criterion. Voice interaction should be evaluated with measurable KPIs.
Wake Success Rate
Percentage of intended wake commands successfully detected under defined test conditions.
False Wake Rate
Unintended activations caused by music, conversation, similar phrases or environmental sound.
False Rejection Rate
Valid activation attempts that the wake engine rejects.
Word Error Rate
Useful ASR metric for controlled transcription test sets.
Command Success Rate
Percentage of spoken commands resulting in the intended product action.
End-to-End Latency
Measure distribution rather than a single best-case demo result.
Offline Completion Rate
Percentage of designated offline workflows successfully completed without cloud connectivity.
Network Recovery Rate
Ability to restore cloud functions after connection loss without reboot.
Thermal Stability
Confirm performance remains stable during sustained voice, display, camera and AI workload.
16. Validation Matrix: Lab Testing Is Not Enough
Quiet Lab
Baseline wake, ASR and latency measurements.
Background Conversation
Test competing human speech and false activation.
Music / TV Audio
Validate wake engine and AEC against media playback.
Device Speaker Active
Test user interruption while TTS or media is playing.
Real Installation
Test reflections, distance, crowd noise and enclosure acoustics.
Network Degradation
Test latency, timeout, fallback and reconnect behaviour.
Long-Run Workload
Voice + camera + display + AI + network active simultaneously.
OTA Regression
Re-run voice KPIs after production software updates.
Factory Production
Mic test, speaker test, software version, device identity and functional validation.
17. Voice + Vision AI: The Next Smart Terminal Architecture
The strongest commercial terminals increasingly combine multiple interaction channels. Voice identifies what the user wants. Vision provides environmental or user context. The screen provides confirmation and control.
Voice
Wake word, command, conversation and hands-free control.
Vision
Presence detection, body analysis, object recognition, QR capture or user context.
Display
Listening indicator, transcription, results, avatar and fallback interface.
Edge AI
Local wake, vision model, routing and privacy-sensitive processing.
Cloud AI
Knowledge, complex reasoning, dynamic content and fleet services.
Physical I/O
Sensors, scanners, printers, relays, motors and business peripherals.
This architecture is particularly relevant to digital humans, fitness mirrors, healthcare kiosks, service robots and intelligent retail terminals.
18. Hardware Platform Requirements for Multimodal Voice AI
A suitable edge terminal platform should be reviewed for the complete workload rather than one AI benchmark.
AI Accelerator
Local voice, vision or lightweight multimodal inference where required.
Audio Interfaces
Digital/analog microphone support, I²S/PDM/PCM/TDM and speaker path.
Camera Interfaces
MIPI CSI or USB camera resources for multimodal interaction.
Display Interfaces
Support the target screen resolution, refresh rate and embedded panel topology.
Network
Ethernet, Wi-Fi or cellular paths for cloud AI and fleet management.
Industrial I/O
RS232, RS485, CAN, GPIO or other product-specific interfaces.
Memory
Enough headroom for OS, application, audio models, vision models and buffers.
Storage
Models, language packs, logs, offline assets and A/B update images.
Thermal Headroom
Validate sustained concurrent workloads inside the final enclosure.
19. Which Architecture Fits Common Smart-Terminal Applications?
| Application | Recommended Starting Architecture | Reason |
|---|---|---|
| AI Digital Human | Hybrid | Local activation and fallback plus cloud dialogue and dynamic knowledge. |
| Smart Fitness Mirror | Hybrid | Local device commands and vision processing with optional cloud coaching. |
| Industrial Voice HMI | On-device or Hybrid | Core machine functions should not depend completely on internet connectivity. |
| Hotel AI Concierge | Hybrid | Needs multilingual conversation and live service information but should retain local fallback. |
| Healthcare Kiosk | Hybrid with strict privacy boundary | Local activation and controlled routing are important for sensitive workflows. |
| Service Robot | Hybrid | Local control must remain available while cloud AI adds richer conversation. |
| Simple Voice Controller | On-device | Fixed commands may not justify cloud dependency. |
20. Production RFQ Pack for Voice AI Hardware Projects
A high-quality voice AI RFQ should describe the architecture, not only ask for an "AI motherboard."
- Final application and installation environment
- Required languages and regional accents
- Wake word or activation method
- Expected speaking distance and direction
- Microphone quantity and preferred microphone interface
- Background noise environment
- Speaker quantity, power and expected playback level
- AEC / NS / beamforming requirement
- Offline command list
- Cloud conversation requirement
- Speech recognition architecture: local, cloud or hybrid
- AI model and LLM integration requirement
- Privacy and data-routing policy
- Network: Ethernet, Wi-Fi, 4G/5G or offline mode
- Display resolution and touch interface
- Camera or vision-AI requirement
- OS requirement: Android, Linux or custom BSP
- OTA and rollback requirement
- Logging and remote diagnostic requirement
- Power input and enclosure design
- Operating-temperature and thermal constraints
- Validation KPI requirements
- Factory microphone and speaker test requirements
- Prototype, pilot and mass-production quantity
- Target product lifecycle and deployment regions
21. From Voice AI Architecture to Production Hardware
For a commercial voice terminal, motherboard selection should happen after the voice architecture is defined. The hardware must support the required microphone path, speaker system, edge AI workload, display, camera, networking, storage, OTA architecture and production test strategy.
Platforms such as RK3576-class AI terminal boards can be evaluated when a project requires voice + vision + display processing, local AI acceleration, Android or Linux, multiple high-speed interfaces and commercial smart-terminal deployment.
Standard AI Terminal Board
Best for early voice/vision application evaluation and proof of concept.
Customized Platform
Best when audio routing, microphone interface, firmware or peripheral configuration needs adaptation.
Custom Voice AI PCBA
Best when microphone topology, connector layout, enclosure, thermal path and production volume are fixed.
22. Recommended Voice AI Content Cluster
FAQ: Cloud vs On-Device Voice AI
Should wake-word detection run locally?
For many smart terminals, local wake-word detection is a strong architectural choice because it reduces network dependency and provides a clear activation boundary before cloud processing.
Is on-device speech recognition completely offline?
It depends on the recognition engine and platform implementation. Product teams should explicitly verify that the deployed recognizer and required language models operate locally instead of assuming that all speech APIs are offline.
What is hybrid voice AI?
Hybrid voice AI distributes processing between the device and the cloud. Local processing typically handles activation, basic commands and fallback, while cloud systems handle complex recognition, knowledge retrieval or large-model reasoning.
What should happen when internet connectivity fails?
The system should switch to a predefined offline capability tier, preserve essential device controls and clearly communicate which cloud functions are temporarily unavailable.
Why does voice AI need edge hardware if a cloud model is used?
The edge device still handles audio capture, acoustic processing, activation, UI, networking, local commands, failure recovery and potentially camera or vision workloads.
How should voice AI latency be measured?
Measure individual pipeline stages as well as total user-perceived latency. Useful checkpoints include wake detection, ASR completion, AI reasoning, external API execution and TTS first-audio time.
What metrics should be tested before mass production?
Typical validation metrics include wake success rate, false wake rate, false rejection rate, ASR word error rate, command success rate, end-to-end latency, offline completion rate, reconnect behaviour and thermal stability.
