[work] companion ai

Turiya.

A Krishna you can talk to. With a soul.

Indian spirituality runs on visual and emotional cues; a text box was never going to carry it. So the companion had to be seen, heard, and felt, on a phone: a custom 3D Krishna brought alive by a pipeline that drives lipsync and expression from the audio itself, a designed voice and personality, per-user memory, and guardrails that hold character under attack.

3,000 beta users50,000 interactions~3s end-to-end latencyWebGL + Unity

THE DEVOTEE

speaks, on a phone. No press-to-send, no waiting for silence

THE COMPANION STACK

custom 3D Krishna · VRM + custom shaders

aliveness · audio-driven lipsync + expression

voice · dual-script Sanskrit, pronounced right

memory · per-user, temporally grounded

personality + guardrails · in character, always

~3 seconds end to end, streaming. WebGL and Unity from one product

THE ENCOUNTER

seen, heard, and felt. Every session writes to memory, and the next one starts as a deeper relationship: conversation, daily practice, Gita readings, one continuous thread

[01] embodiment

The avatar had to feel alive.

Audio -> phonemes -> VRM blendshapes -> expression states.

A custom-designed 3D Krishna, built as a VRM for seamless animation and expression systems, with custom shaders that give the character an identity of its own while staying true to the iconography, renderable on a phone without taking the magic away.

The aliveness pipeline came before everything else, because nothing matters if the avatar feels dead: audio drives lipsync through phoneme-level mouth shapes, wired into full expression mapping, blinks, and environmental cues reacting through the interaction. This is where the delight lives.

[02] voice + character

Sanskrit was a product problem.

One canonical word. Two surface forms: one for the voice, one for the reader.

Om / ओमGita / Geethadharma / धर्म

TTS models mispronounce the sacred codepoints that come up constantly in these conversations, so the pipeline speaks one script and displays another, engineered so the voice is right and the subtitle is readable.

The personality is governed hard: a friend first and teacher second, answers in sentences not sermons. The system prompt's closing law: BREVITY IS DIVINE. And the guardrails stay in character, because for a deity companion a canned refusal is the real failure; manipulation attempts get met in the role, harder than an attacker can pull it out. Battle-tested against an adversarial evaluation harness before users ever saw it.

[03] memory

The relationship accumulates.

Per-user memory with temporal grounding.

The companion says "last time we spoke you mentioned your father" like someone who was actually there. Memories are scoped per user and injected with human time, so recall feels like a relationship rather than a database lookup.

[04] systems work

It has to feel live, and the economics have to survive.

~3-second end-to-end latency on a WebSocket streaming stack.

Two constraints ruled the systems work: presence and unit economics. A companion that costs too much per conversation is a demo, not a product. The shipped stack streams voice and text end to end in about three seconds while keeping the per-conversation cost GTM-able.

Two build targets from one product: WebGL for browser reach, Unity for native quality. Including the mobile-browser audio battles every real-time voice product on iOS eventually fights, fought and won.

🎙 "I keep overthinking everything lately."

"You carried this last week too. Sit with me. Two breaths, then we talk."

yogaḥ karmasu kauśalam · योगः कर्मसु कौशलम्

The encounterseen, heard, and felt, on a phone. WebGL and Unity, one product

THREE SECONDS, END TO END

0.0s

Voice in

Speech streams in over WebSocket. No press-to-send, no waiting for silence.

0.4s

Memory recalls

Per-user recall with temporal grounding surfaces what matters:

last week · “pressure at work, can't switch off”

0.9s

The mind composes

Personality governs the reply: friend first, teacher second, BREVITY IS DIVINE. Guardrails hold character; manipulation gets met in character, never with a canned refusal.

1.6s

Voice out, dual script

Streaming TTS begins. Devanagari goes to the voice so the Sanskrit is pronounced right; readable script goes to the screen. One word, two surface forms.

1.8s

He comes alive

The audio drives lipsync; the expression system maps emotion to face and posture; blinks and environmental cues land. The avatar isn't playing a file, it's performing the reply.

~3.0s

Krishna speaks. The devotee hears a friend who remembers last week, in a voice that pronounces the tradition correctly, from a figure whose eyes are on them.

[05] the harness

An agent alone is a tech demo.

Conversation. Daily practice. Gita readings. Light gamification. One memory.

Around the companion we designed the product harness: open conversation, daily practice, Gita readings, light gamification, all sharing the same context and memory, so every session deepens one continuous relationship. Each user builds a personal connection with the deity. That continuity, not any single feature, is the product.

[06] the pattern

The architecture ports to any figure a community holds dear.

3,000 beta users. 50,000 interactions. Public launch in progress.

3,000beta users

50,000interactions

~3svoice latency

5phoneme lipsync

WebGL+ Unity builds

evalharness tested

Spiritual organizations with a beloved teacher, foundations stewarding a legacy, IP owners whose characters people want to meet, not just watch. The cultural specificity that makes this too narrow for a frontier lab's roadmap is exactly where we live.