Suraksha — an agentic voice companion that keeps aging parents safe at home

Suraksha is a voice companion on a Galaxy Watch and a tablet for parents who live alone. It talks with her in Hindi or English and acts on what she asks (WhatsApp her son, set a reminder, remember where the spare key is), reminds her about her medicines, and when the watch detects a fall it asks her first, then gets the family member who can actually reach her. Runs end to end on Vultr.

Live demoGitHub0

Overview

The editor won't turn markdown into formatting, so this is plain text. After pasting, select each heading line ("The problem", "What it does", "Architecture" and so on) and click H2. The bullet lines can be turned into a list with the list button.

Suraksha (सुरक्षा, "protection") — technical overview

Live demo: https://139-84-209-205.sslip.io

Code: https://github.com/paccmann218/suraksha

demo video : https://youtu.be/63YKbm9hJvY

Team Huang Gang: Harshendra M and Naomi

The problem

Millions of Indian parents grow old at home while their children work in other cities. The options today are all bad in their own way: cameras take away her privacy, panic buttons only help if she can reach one and is willing to "make a fuss", pill-box alarms ring and forget, and the family WhatsApp group depends on someone noticing she has gone quiet. Suraksha is a calm middle ground: it asks her first, and brings in family only when something is actually wrong.

What it does

* Agentic voice companion. She talks to the watch or tablet in Hindi, English or Hinglish. The companion agent has 10 real tools: send a message to a family member (WhatsApp + in-app), create reminders, add medicines and log doses, remember and recall facts, schedule its own follow-ups, set privacy rules, and look back over recent days. What she hears back is built from what the tools actually did, not from what the model claims, so a confirmation always matches the database.

* Fall detection that asks her first. The Wear OS app watches the accelerometer all day for a free fall followed by an impact, or a hard impact after stillness. It then asks her out loud and listens. "I'm okay" closes it. Two minutes of silence, or the Help button, starts a critical family escalation.

* Medicines. At each dose time Suraksha asks her on the watch and tablet; she answers by voice ("haan, le li") or by tapping Taken. No answer means one follow-up at 20 minutes; at 60 minutes the dose is marked missed and only the children she allows to hear about her health are told.

* Long-term memory. Facts she asks it to remember, facts the companion notices, and a nightly summary of her day are stored and embedded. Before every reply the companion retrieves the memories most relevant to what she just said. She can see and delete every memory; her family cannot see them at all.

* Proactive conversations. A morning check-in on her first movement, medicine asks, follow-ups on things she mentioned ("how is your knee now?"), and a check-in if she is not up by her usual time. Openers are written with her memories in context.

* Consent she controls. Spoken privacy rules ("don't tell Priya about my health, except in an emergency") become Cedar policies built from fixed templates. Every tool call by every agent is authorised by Cedar first, and every allow and deny is written to an audit log she can read in Hindi.

* Family app and WhatsApp. A calm dashboard ("Boring is good."), Ask Suraksha (answered only from records, otherwise "not sure"), notes she hears read aloud, today's doses and adherence, and alerts on WhatsApp and web push. Family can reply YES, NO or ARRIVED on WhatsApp to answer an escalation.

Architecture

* Devices: Galaxy Watch (Kotlin + Jetpack Compose for Wear OS), Amma's tablet and family phones (Next.js 15 web app), a neighbour handset page, WhatsApp (Meta Cloud API or Twilio).

* Front door on Vultr Cloud Compute: Caddy with automatic HTTPS in front of the API and the web app; Suraksha's own sign-in (PBKDF2 password hashes + signed JWT) and a device key for the watch.

* Runtime: a FastAPI service hosting the agents (built with Strands Agents), plus a sweep worker that runs every minute for medicines, reminders, follow-ups, fall and wake timeouts, the escalation workflow and the nightly memory summary.

* AI and voice on Vultr Serverless Inference: chat completions with tool calling (qwen3.8-flash-next, failing over to glm-5.3-flash), Whisper speech-to-text, Chatterbox text-to-speech, and the Vector Store for memory.

* State on Vultr Managed PostgreSQL; voice clips on Vultr Object Storage, served through presigned URLs.

The agents

* Companion: her only voice. Tool-using agent with her memories, today's medicines and recent days in context; Hindi replies are written in Devanagari in a warm, feminine voice.

* Family Q&A: answers a child's question only from records its tools returned in that turn (for example local wake times against her usual window, or today's dose schedule). The asking child is the Cedar principal, so it cannot reveal what Amma keeps private.

* Deviation judge: runs behind a cheap deterministic gate and decides whether a fall, an unwell report or silence needs the family. The judge informs; code and policy decide.

* Follow-up and memory writers: write proactive openers and the nightly summary as typed (Pydantic) outputs.

Design decisions

* The model chooses the words; code decides the actions. Whether family are contacted, in what order, with what timeouts, is deterministic.

* Cedar fails closed: unknown tools and evaluation errors are denials, and they are audited.

* No silent fallbacks: if both models fail, the app says the companion is unavailable instead of inventing an answer.

* No invented data: the watch sends only real sensor readings. History before today is simulated and labelled so the routine chart has a baseline; today is live.

* A durable escalation workflow: Rahul is asked first, then Priya after a timeout, then (only if Amma allows it) Sunita aunty next door, then everyone. Each wait stores a one-time token and a deadline in PostgreSQL; replies over the app, push or WhatsApp record the answer once; the worker turns expired deadlines into timeouts; an advance holds a short lease so two workers never run the same step. A crash or redeploy loses nothing.

* Exactly once where it matters: conditional writes under row locks make "ask once", "claim once" and "answer once" hold across the API and the worker; the one-minute sweep runs under a PostgreSQL advisory lock.

* A "before you go in" hand-off note for whoever accepts: what happened, last heard, medicines, with every line filtered by Cedar for that reader.

Data model

One PostgreSQL table using a single-table design (pk, sk and a sparse gsi1 index for due work) with JSONB items: profile, family roster, consent rules and audit, conversation turns, day summaries, medicines and every dose, memories, reminders, follow-ups, alerts, escalations and their workflow state, sign-in accounts and WhatsApp number links.

Deployment and operations

* deploy/vultr/provision.py provisions everything through the Vultr API v2, uploads the code and starts docker compose (Caddy, API, worker, web). Re-running it redeploys to the same URL.

* /health/deep and the Live stack panel on /command call every dependency and show which Vultr service answered.

* 75 offline tests (no network) cover falls, escalation, consent, medicines, memory, WhatsApp, the watch channel and model outages.

What's next

Multi-family tenancy for NGOs and care programmes, approved WhatsApp templates in more Indian languages, and an on-watch fall model trained on labelled data alongside today's rule-based detector.

How it uses Vultr

Suraksha runs end to end on Vultr, and one script (deploy/vultr/provision.py) sets it all up through the Vultr API v2. * Serverless Inference, chat + tool calling: the brain of every agent. The companion (10 tools: WhatsApp family, reminders, medicines, memory, follow-ups, privacy rules), the family Q&A agent, the "is this unusual?" judge, check-ins and nightly memory summaries. Called through Strands Agents' OpenAI-compatible model: qwen3.8-flash-next, with the whole turn retried on glm-5.3-flash on a throttle, 5xx or timeout. * Serverless Inference, speech-to-text: whisper-large-v3-turbo hears her in Hindi, English or Hinglish from the watch and tablet. Silent clips are dropped before the call. * Serverless Inference, text-to-speech: chatterbox is Suraksha's voice in Hindi and English. * Serverless Inference, Vector Store: her long-term memory. One collection per parent; items added when she says "remember…", searched before every reply, deleted when she says "forget". * Managed PostgreSQL: all state in one JSONB single-table design. Conditional writes under row locks keep "ask once / answer once" exact; an advisory lock keeps the one-minute sweep single-runner. The durable escalation workflow lives here, so a restart loses nothing. * Object Storage: every spoken clip is cached as content-addressed MP3 and handed to the watch as a 15-minute presigned URL. * Cloud Compute (Bangalore): docker compose runs Caddy (automatic HTTPS), the FastAPI agents, the sweep worker and the Next.js app. * Vultr API v2: creates the inference subscription, database, bucket and instance, then locks the database to the instance's IP. Re-running it redeploys to the same URL. The /command page has a live panel, and /health/deep calls every Vultr dependency and reports which one answered.

No comments yet

Be the first to comment on this project.