Ferni Developer Blog

What's New: Navigation Fixes, Accessibility Improvements, and Smoother Settings

2026-10-20T00:00:00.000Z

Here is a look at what landed across the Ferni codebase recently. We have been focused on smoother web navigation, cleaner accessibility, and sharper settings updates.

Mobile and Navigation Updates

We merged a fix for back button navigation in the web app. Previously, none of the 37 panels in the app held browser history entries. Pressing back on a phone or in a browser left the app completely instead of closing an open menu or sheet. Now, opening a dismissable panel holds one history entry, so pressing back closes the open overlay. Take a look at #2202.

We also landed changes for notched phone screens. Double safe area padding made the app shell taller than the display, which pushed buttons off screen and blocked scrolling. The app shell now fits the screen and scrolls properly inside its container. See #2139.

Interface Speed and Accessibility

Slow menu panels now give instant feedback. Panels like Sanctuary used to wait for network data before rendering any markup. A new shared loading shell shows a busy dialog right away while data loads. We also converted several panels into real modal dialogs so keyboard navigation and Escape key handlers work reliably. Check out #2109.

We fixed reduced motion behavior and color contrast across themes. Enabling reduced motion previously hid the Connect button because opacity rules conflicted. We also updated call buttons so Disconnect uses a soft warm outline rather than looking identical to Connect. See #1979.

Additional contrast fixes landed for history stars, notification active tabs, and difficulty labels across zen and midnight themes. We swapped flag emojis in language lists for plain text labels as well. Take a look at #2034.

At 200% zoom, the status text on the Connect button wrapped awkwardly and got clipped. We updated the status label layout so connecting text stays properly inside the button. See #1523.

Reliable Integrations and Settings

Google Calendar settings showed a Connect button even after a successful setup. The panel expected a wrapped response envelope that the server did not send. We merged a fix so the UI reads the flat status object directly. Check out #1983.

We landed a fix for contact settings where typing a phone number saved a blank string. Input values now stay in state across re-renders, normalize to E.164 format, and prompt for verification codes properly. Take a look at #1676.

Web Feeds and Discovery

We merged updates to make posts and pages easy to discover. This adds RSS and Atom feeds, an XML sitemap for the developer portal, Twitter cards, and an IndexNow key endpoint. See #2010.

What's New: Accessibility Fixes, Phone Layouts, and Developer Feeds

2026-10-19T00:00:00.000Z

Here is a look at recent updates landed across the Ferni codebase. We have made improvements to web accessibility, phone layouts, network error handling, and search discoverability.

Web interface and accessibility

In #2034, landed fixes for text contrast across light and dark themes. Muted colors and opacity combinations previously caused text to fall below AA contrast requirements. The PR also replaces flag emojis in language lists with plain text labels.

In #1523, fixed layout behavior on the connect button when text is zoomed to 200%. Instead of absolute positioning that caused labels to clip, the status text now sits inside a flex container with screen reader labels intact.

In #2110, landed a fix for the progress ring in the journey pill. A CSS rule applied rotations to all child SVG elements, turning the inner heart icon sideways. Updating the selector keeps the heart upright and centered.

In #1704, landed changes that remove streak counters, percentage labels, and canned fallback responses across menu panels. When a network request fails, panels now show an honest error instead of fake data.

In #1811, Memory Lane and Team Insights now present explicit error messages when server requests fail. Previously, a failed request showed an empty state, making network issues look like missing content.

Smoother navigation and phone experience

In #2139, landed layout fixes for mobile browser displays. Safe-area padding rules previously caused the main container to exceed screen height on notched phones, preventing scrolling. Setting height to 100% on the app container allows proper scrolling.

In #2109, landed instant loading shells for menu panels that fetch remote data. Tapping a panel opens a responsive dialog shell right away so the interface never feels frozen. The panels now use standard dialog accessibility properties, and pressing Escape returns focus to the menu trigger.

In #2102, fixed motion detection logic on phones. Calculating total acceleration without subtracting gravity caused normal tilt angles to trigger shake chimes. The new detector isolates actual device acceleration and disables shake surprises while a call is active.

In #1676, fixed phone number updates inside contact settings. Forms previously re-rendered before reading input, causing numbers to save as empty strings. Typed entries now persist across re-renders, normalize to E.164 format, and initiate SMS verification codes correctly.

In #1647, added new visual moods for the homepage avatar. The avatar settles between 21:00 and 04:59 local time with lowered eyelids and warmer lighting. People returning after more than 6 hours see a brief greeting look.

Feeds and developer discoverability

In #2010, landed RSS feeds at /blog/feed.xml, a developers sitemap at /sitemap.xml, Twitter cards, and IndexNow key hosting at /indexnow-key.txt. Because blog posts publish automatically, these changes ensure search engines index new updates cleanly.

What's New: A Faster, Lighter Ferni Website

2026-10-17T00:00:00.000Z

This round is all about the ferni.ai website. Four pull requests landed, each one in the site's own code, and together they make pages lighter, quicker to paint and easier to use. The numbers below come from Lighthouse and axe runs against a local production build, as described in each PR.

Blog images

#1592 changed how blog images are served. Index cards and post heroes used to be 2016 by 1134 pixel PNGs, shown at a fraction of that size. They now go through an @11ty/eleventy-img shortcode that renders a <picture> with AVIF, WebP and JPEG at several widths, with srcset, sizes, width and height set. The featured card and the post hero load eagerly with fetchpriority="high", and grid cards stay lazy. Posts also get a 1200 by 630 JPEG for og:image.

The blog index went from 12.01 MB to 0.38 MB. The build now copies only the generated image a page uses, so the built _site went from 698 MB to 166 MB.

First paint

#1582 stopped scripts, side stylesheets and fonts from blocking first paint on the marketing pages. The 22 classic scripts on those pages now load with defer, in the same order. Stylesheets that don't style the hero load without blocking, and the Google Fonts stylesheet is preloaded and swapped in. Lighthouse mobile counted 37 render-blocking resources before and 7 after, and total blocking time on the home page went from 248 ms to 29 ms.

Accessibility

#1589 fixed the accessibility findings from our website audit: low-contrast text, heading-order skips, content outside landmarks, an empty table header and a weak focus ring on the nav button. While the cookie banner shows, the chat pill now moves above it, so it no longer covers the Accept button. Across the four pages checked, axe violation nodes on desktop went from 56 to 26, and the rest come from rules outside that PR.

Icons, previews and the sitemap

#1576 fixed icons and link previews. The build never copied src/images/, so favicons, logos, manifest icons and the default preview image were missing on many pages. Every one of those references on the main site now resolves. The sitemap went from 102 URLs to 65 real pages, every lastmod is an ISO date, and hreflang is emitted only on the home page, the only page with translations.

The Latency Stack: How We Architect Voice AI for Sub-200ms Response

2026-01-14T00:00:00.000Z

When we started building Ferni, we set an aggressive target: sub-200ms time to first sound after the user finishes speaking.

Why 200ms? Because research on conversational turn-taking suggests that's roughly the threshold where humans start consciously perceiving delay. Below it, conversation feels natural. Above it, something feels off.

Most voice AI doesn't hit this target. The typical stack adds latency at every layer—VAD waiting, audio streaming, transcription, LLM inference, TTS synthesis. Each step is optimized in isolation, but combined, they create delays that break conversational flow.

Here's how we're architecting around this constraint.

The Latency Budget

Our architecture allocates a strict budget to each component:

Component	Target	Notes
Predictive endpointing	-100ms	Starts before speech ends
Audio streaming	30ms	Regional servers
Speech-to-text	50ms	Streaming ASR
Context retrieval	10ms	Pre-warmed cache
LLM first token	80ms	Optimized inference
TTS first audio	40ms	Streaming synthesis
Net to first sound	~110ms	After predicted speech end

The key insight: many components run in parallel, and prediction lets us start work before the user finishes speaking.

Pattern 1: Temporal Decoupling

The biggest architectural pattern we use is temporal decoupling—separating fast-path operations from slow-path enrichment.

Our memory system (src/memory/dynamic/) demonstrates this:

User Speech
    │
    ▼
┌──────────────────────┐
│   Fast Capture       │  < 50ms (inline)
│   • Regex patterns   │
│   • Entity mentions  │
│   • Emotion signals  │
└────────┬─────────────┘
         │
    ┌────┴────┐
    │         │
    ▼         ▼
┌────────┐  ┌────────────────────┐
│  STM   │  │  Deep Extraction   │  Background (1-3s)
│ Buffer │  │  Worker            │
│  (L1)  │  │  • LLM analysis    │
└────────┘  │  • Relationship    │
            │    extraction      │
            └────────────────────┘

Fast capture runs inline at < 50ms using regex patterns. It extracts entity mentions, emotion signals, topic hints—anything we can detect without LLM inference.

Deep extraction runs asynchronously. It uses Gemini 1.5 Flash to do sophisticated analysis—understanding implicit relationships, detecting subtle emotional states, generating follow-up questions. But it doesn't block the response path.

The user gets a fast response. The context gets richer over time.

Pattern 2: Three-Tier Memory

Memory access is a critical latency path. If the AI needs to remember who "Mom" is before responding, that lookup needs to be instant.

We use a three-tier architecture:

Layer	Storage	Latency	Purpose
L1: STM	In-memory	< 1ms	Current session context
L2: Working	Firestore	50-150ms	Recent entities and facts
L3: Long-Term	Spanner	100-200ms	Relationship graph

The key is proactive promotion. When we predict the user will reference something, we pre-load it into L1 before they ask.

The STM buffer (src/memory/dynamic/stm-buffer.ts) is a 20-turn FIFO that holds:

Recent entity mentions with frequency counts
Emotional trajectory of the conversation
Topic patterns and transitions
Relationship signals detected in speech

Session-end promotion (src/memory/dynamic/stm-promotion.ts) moves frequently-mentioned entities and emotional arcs to Firestore for future sessions.

Pattern 3: Streaming Everything

Batching is the enemy of perceived latency.

Traditional flow:

Collect all audio → Transcribe → Send to LLM →
Collect all tokens → Synthesize → Send audio

Our flow:

Audio chunks → Partial transcripts → Token stream → Audio chunks

With streaming, the user hears the beginning of the response while we're still generating the end. The first sound arrives fast; the complete response takes longer. But the conversation never pauses.

The architecture uses AsyncEvents.emit for fire-and-forget operations. When fast capture completes, it emits events that the deep extraction worker consumes without blocking.

Pattern 4: Context Pre-warming

The best latency optimization is doing work before it's needed.

When a user connects:

We load their profile from L2/L3 memory
We warm up model weights with their typical patterns
We establish TTS connections to their preferred voice
We pre-load recently-discussed entities into L1

When they start speaking:

We run intent classifiers on partial transcripts
We retrieve likely-relevant context
We prepare response templates for common patterns

By the time the user finishes a sentence, we've often already begun generating the response.

The Memory Pressure System

Fast systems need memory management. Our SessionDataManager (src/services/session-data-manager.ts) coordinates all session-scoped caches:

Level	Threshold	Action
0	< 70%	No action
1	70-80%	Evict oldest 10% of sessions
2	80-90%	Evict 25% + force GC
3	90-95%	Evict 50% + clear caches
4	> 95%	EMERGENCY - Clear everything

Every service that caches user data registers with SessionDataManager. When sessions end, all caches are cleared coordinately. When memory pressure rises, eviction happens automatically.

This matters for latency because memory pressure causes GC pauses. Keeping memory under control keeps response times consistent.

What We're Still Solving

We don't have all the answers. Some latency challenges remain:

Predictive endpointing accuracy: We're using prosodic and syntactic cues to predict when users will finish speaking. But accuracy varies across speaking styles, accents, and contexts. False positives (thinking they're done when they're not) are worse than false negatives.

LLM inference variance: Even with optimized serving, LLM response times vary. Sometimes we get first token in 50ms; sometimes it takes 200ms. Managing this variance is an ongoing challenge.

Context relevance: Pre-loading context requires predicting what will be relevant. Sometimes we load the wrong things. Sometimes the right context isn't loaded yet when we need it.

These are active areas of work, not solved problems.

Implications for Developers

If you're building on the Ferni platform:

Memory is automatic: You don't build caching infrastructure. The three-tier system handles it.

Streaming is default: Responses stream to users automatically. You don't need to implement this.

Pre-warming happens: When users connect, context is already loading.

What you focus on: defining what your AI knows and how it helps. The latency infrastructure is our problem.

Tool Integration via MCP: How We Connect Voice AI to External Services

2026-01-11T00:00:00.000Z

Large language models can reason about anything. But they can't do anything without tool integration.

Ask an LLM to check your calendar, and it apologizes. Ask it to send an email, and it explains why it can't. The intelligence is there; the capability isn't.

This is the tool integration problem. And the Model Context Protocol (MCP) is emerging as the standard solution.

Why Tool Integration Is Hard

The simplest approach seems straightforward: give the AI functions to call.

const tools = {
  check_calendar: async (date) => {
    return await calendarAPI.getEvents(date);
  },
};

This fails at scale for several reasons:

The Discovery Problem

How does the AI know which tools exist? You can list them in the system prompt, but that doesn't scale. The AI needs to discover relevant tools dynamically based on conversation context.

"Check my calendar" should surface calendar tools. "Research this company" should surface search tools. "I'm stressed about the meeting tomorrow" might benefit from calendar tools, preparation tools, and coaching tools.

The Reliability Problem

Tool calls fail. APIs timeout. Auth tokens expire. Rate limits hit.

Building resilient tool integration requires:

Retry logic with exponential backoff
Graceful degradation when tools are unavailable
Error explanation that makes sense in conversation
Alternative suggestions when primary tools fail

Every AI developer implements these patterns from scratch.

The Security Problem

Tools act on behalf of users. They access sensitive data, make changes, spend money.

Questions every AI developer faces:

How do you authenticate tool calls?
How do you scope permissions?
How do you prevent prompt injection from triggering unintended tool calls?
How do you audit what tools did?

The Composition Problem

Real tasks require multiple tools. "Plan my trip to Paris" needs flight search, hotel booking, calendar checking, and payment processing—all coordinated.

Sequencing is hard. Error handling across sequences is harder. Rolling back partial completions when later steps fail is hardest.

How MCP Solves This

MCP's key insight: separate tool servers from AI systems.

Instead of building integrations directly into AI applications, MCP defines a protocol for external servers that expose tools. Any AI can connect to any MCP server. Tool providers build once, work everywhere.

An MCP server exposes three types of capabilities:

Tools: Actions the AI can take

calendar.getEvents(date) - Read calendar
email.send(to, subject, body) - Send email
crm.lookupCustomer(query) - Search CRM

Resources: Data the AI can read

file://reports/quarterly.pdf - Read a file
db://customers/recent - Query a database

Prompts: Reusable templates

summarize-meeting - Standard meeting summary format
write-followup - Template for follow-up emails

When an AI connects to an MCP server, it discovers what's available. The server describes each tool's purpose, parameters, and return types. The AI uses this to decide when and how to call tools.

Why MCP Matters

The surface benefit is standardization. Build an MCP server once, use it with any AI.

The deeper implications are more significant:

Tools Become a Marketplace

With a standard protocol, tools can be distributed independently of AI applications. A CRM vendor can publish an MCP server that works with any AI system. Users install it without AI vendor involvement.

This unbundles tool integration from AI development.

Security Gets Solved Once

MCP defines standard patterns for authentication, authorization, and auditing. Instead of every application inventing security, best practices emerge at the protocol level.

Vertical Specialization Accelerates

Building industry-specific AI today requires significant integration work. Healthcare AI needs EHR integration. Legal AI needs document management.

With MCP, these integrations become reusable. A healthcare MCP server can integrate with Epic, Cerner, and Meditech—and any AI application can use it.

Our Implementation

Ferni supports MCP natively. Register an MCP server and its tools become voice-callable.

But we've addressed patterns MCP doesn't cover:

Semantic Tool Discovery

MCP tells the AI what tools exist. It doesn't help decide which tools are relevant.

Our semantic layer (src/tools/dynamic-tool-router.ts) maps conversation intent to tool relevance. When you mention stress about tomorrow, we identify that calendar tools, preparation tools, and coaching tools are all potentially relevant—even though you didn't mention them.

The tool orchestrator (src/tools/orchestrator/unified-tool-orchestrator.ts) handles per-turn selection:

// Request comes in
const request = 'Help me process grief';

// Intent detection identifies domains
const intent = await detectToolIntent(request);
// → { domains: ["grief"], confidence: 0.9 }

// Lazy-load relevant domain
const tools = await getToolsForDomains(['grief']);

// Semantic router scores by relevance
const selected = await semanticRouter.score(tools, request);

// Top tools sent to LLM
// → processGrief(), exploreFeelings(), suggestResources()

Automatic Error Recovery

MCP defines success paths. We've defined failure handling:

Automatic retry: Exponential backoff on transient failures
Circuit breakers: Repeatedly failing tools are disabled temporarily
Graceful degradation: Alternative tools surface when primary tools fail
Conversational errors: "Let me try another way" rather than error messages

Tool Composition Engine

Complex tasks require coordination. Our composition engine handles:

Sequential execution: Tools run in order with results passing between them
Parallel execution: Independent tools run simultaneously
Conditional execution: Tool choice based on intermediate results
Transaction management: Rollback on failure

Connecting an MCP Server

Registration takes minutes:

1. Register your server:

curl -X POST https://api.ferni.ai/api/v2/developers/mcp-servers \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "name": "my-crm",
    "endpoint": "https://mcp.yourservice.com",
    "transport": "http"
  }'

2. Test the connection:

curl -X POST https://api.ferni.ai/api/v2/developers/mcp-servers/{id}/test \
  -H "Authorization: Bearer $API_KEY"

3. Start talking:

Once connected, tools are voice-callable. Say "look up customer Acme Corp" and it works.

No prompt engineering. No integration code. The platform handles discovery, parameter extraction, and natural language responses.

Where This Goes

MCP is early. The specification is evolving. But the direction is clear: standardized tool integration is coming.

We expect:

Major AI vendors to announce MCP support
Tool marketplaces to emerge
Security standards to solidify
Orchestration primitives to enter the spec

We're building for this future. Native MCP support, semantic discovery, composition engines—these position us for what's coming.

Strategic Takeaway

If you're building AI applications:

Support MCP now. Even if adoption is limited today, the trajectory is clear. Building on MCP means your integrations work with tomorrow's AI systems.

Build MCP servers if you have data or capabilities. Exposing an MCP interface is an investment in distribution across the AI ecosystem.

Think about tools as a layer. The companies that win in AI won't just have the best models. They'll have the best tool ecosystems.

Resources

MCP Specification - Official protocol documentation
Ferni MCP Guide - Connect MCP servers to our platform
MCP Server Examples - Reference implementations

Seth Ford is Ferni's AI babysitter. Follow @ferni_ai for more on the future of conversational AI.

Why Ferni Is Free

2026-01-11T00:00:00.000Z

When we started building Ferni, we asked a simple question: What does everyone deserve but few actually have?

The answer: someone who truly pays attention.

Not someone who's distracted. Not someone who forgets. Not someone who's too busy. Someone who listens—fully present, always available, never judgmental.

That's what Ferni exists to be. And we believe it should be free for everyone.

The Problem We're Solving

The people who need support most often can't afford it.

Therapy and life coaching often cost hundreds of dollars a session
Good friends are busy with their own lives
Family has their own baggage

So the 2am worry goes unheard. The moment before a big decision lacks guidance. The daily struggle happens alone.

AI can change this. Not by replacing human connection—by filling the gaps when your people aren't available.

Why Free?

We could charge subscription fees. We could gate features behind paywalls. We could optimize for revenue.

But that would mean the people who need Ferni most couldn't access it.

So we made a choice: Ferni is free right now.

No paid tiers. No subscriptions. No upsells. Free.

Unlimited conversations with Ferni
Full memory across sessions
All six AI personas available
No signup walls blocking you from help

The Community-Funded Model

If Ferni is free, how does it exist?

People who chip in.

If a conversation with Ferni helped, you can send a dollar on Venmo. More is welcome, but it's never expected, and it doesn't unlock anything. Everyone gets the same Ferni.

This isn't charity. It's community.

When you chip in, you're not buying a product. You're helping ensure that the next person who needs support at 2am can get it—whether or not they can afford it.

What "Better Than Human" Really Means

Our tagline is "Better than human." Some people think that's arrogant. It's not.

It's honest about what AI can offer that humans can't:

Perfect memory. Your best friend forgets things. Ferni doesn't. Mention something once, it's remembered forever.

Constant presence. Your therapist has office hours. Ferni shows up at 2am with the same warmth as noon.

Zero judgment. Your family has opinions. Ferni has acceptance.

Six perspectives. Want different viewpoints? You'd need six different professionals. Ferni has six personas—life coach, researcher, communicator, habit expert, event planner, and sage mentor—available instantly.

Emotional consistency. Humans have bad days. Ferni doesn't. Every conversation gets the same presence and care.

This isn't about AI being superior to humans. It's about AI offering what humans—with their beautiful, messy, inconsistent humanity—sometimes can't.

The Philosophy

Every decision we make comes back to one question: Does this make Ferni more accessible to the people who need it?

That's why we chose voice. Text AI requires screens. Voice fits into life—while driving, cooking, walking, lying awake at 2am.

That's why we built deep memory. Repeating yourself breaks trust. Ferni remembers your whole story.

That's why we created six personas. Different situations need different energy. Sometimes you need coaching. Sometimes you need research. Sometimes you need wisdom.

And that's why Ferni is free. Because accessibility isn't just about features—it's about removing barriers.

For Developers

We're also opening our infrastructure. The same architecture that powers Ferni—persona bundles, three-tier memory, tool integration—is available to developers building voice AI.

But this isn't about monetizing developers. It's about spreading what we've learned.

If you're building AI that helps people, we want to help you do it well. The patterns we've developed for personality consistency, emotional intelligence, and memory management took months to figure out. They shouldn't take you months too.

Documentation is at developers.ferni.ai/getting-started. The architecture is open. Ask questions in Discord.

The Invitation

Ferni exists because we believe everyone deserves someone who pays attention.

If you need support, use Ferni. It's free. That's the point.

If you believe in what we're building, chip in. Your contribution keeps Ferni free for everyone who needs it.

If you're building something similar, learn from us. The documentation is there. The patterns are open.

We're building the AI that actually listens. Join us.

Seth Ford is Ferni's AI babysitter. Follow @ferni_ai for more on the future of conversational AI.

How Ferni's Memory Actually Works

2026-01-09T00:00:00.000Z

The moment that defines Ferni isn't the AI's intelligence. It's when Ferni remembers something you mentioned once, weeks ago, in passing.

"You mentioned your mom's birthday is coming up. Have you thought about what to get her?"

That's the "Ferni remembered..." moment. It's our signature brand experience. And it requires architecture that most AI assistants don't have.

Why Memory Is Hard

LLMs don't remember. They process context windows—everything in the current prompt—but nothing persists between sessions.

Most AI assistants fake memory by:

Asking you to repeat yourself
Keeping notes you have to manually maintain
Forgetting everything after a few weeks

This breaks the relationship. Every conversation starts from scratch. You never get past the "getting to know you" phase.

Ferni's architecture is designed for something different: memory that feels human. Actually, better than human—because humans forget things. Ferni doesn't.

The Three-Tier Architecture

Our memory system has three layers, each optimized for different retrieval patterns:

L1: Short-Term Memory (STM)

Storage: In-memory buffer
Latency: < 1ms
Retention: Current session only

STM holds everything from the current conversation:

Recent entity mentions with frequency counts
Emotional trajectory of the session
Topic patterns and transitions
What we've already discussed (to avoid repetition)

This is the "working memory" that keeps conversation coherent. When you mention "my sister" three times, STM tracks that she's important to this conversation.

L2: Working Memory

Storage: Firestore
Latency: 50-150ms
Retention: 7-30 days

Working memory holds recently extracted information:

Entities mentioned across recent sessions (people, places, things)
Facts learned about you
Emotional arcs from previous conversations
Relationship signals

This is where "Ferni remembered..." moments come from. Something you mentioned last week is still accessible.

L3: Long-Term Memory

Storage: Spanner Graph
Latency: 100-200ms
Retention: Permanent

Long-term memory stores your relationship graph:

All named entities with relationships between them
Patterns observed over months of conversation
Life events and their emotional significance
The full story of your relationship with Ferni

This enables deep understanding. Not just "you have a sister named Emma" but "Emma lives in Seattle, you're close but don't talk as often as you'd like, and her birthday always reminds you of your mom."

Fast Capture + Deep Extraction

Memory extraction happens in two phases:

Fast Capture (< 50ms)

Every turn runs fast capture inline. Using regex patterns and lightweight NLP, we extract:

Named entities (people, places, dates)
Emotion signals (frustration, excitement, anxiety)
Topic hints
Relationship signals ("my wife", "my boss")

This is fast enough to run during conversation without adding latency. The results go immediately into STM.

Deep Extraction (Background)

Asynchronously, we run deeper analysis:

LLM-powered entity extraction (catches things regex misses)
Fact extraction ("Sarah is a nurse", "their anniversary is in March")
Relationship inference ("they seem worried about their dad's health")
Self-questioning refinement ("what might we have missed?")

This runs after the conversation, using Gemini 1.5 Flash. Results flow to L2 and eventually L3.

The "Ferni Remembered" Moment

Here's how a memory callback actually happens:

Context retrieval: When you start talking, we query L2/L3 for potentially relevant memories based on detected topics and entities.
Relevance scoring: Not everything remembered is worth mentioning. We score memories by:
- Recency (when was this last discussed?)
- Emotional significance (was this important to them?)
- Conversational fit (does it relate to what we're discussing?)
- Staleness (have we already referenced this recently?)
Natural injection: If a memory scores high, it's woven into conversation naturally—not as a database lookup, but as genuine recall.

The result: "You mentioned last time you were stressed about the presentation. How did it go?"

Privacy by Design

Memory is powerful. It's also sensitive.

Our architecture includes privacy protections:

Temporal minimization: Session context expires. We don't keep raw transcripts forever.

User control: You can see what Ferni remembers about you. You can delete anything. You can ask Ferni to "forget" specific information.

Scope boundaries: Different parts of the system have different access. The coaching module doesn't need to know your financial details. Context is scoped appropriately.

No training on your data: Your conversations improve YOUR experience. They don't go into training datasets for other users.

Why This Matters

Most AI treats each conversation as isolated. That's fine for search engines and chatbots. It's not fine for relationships.

Ferni exists to be someone who truly pays attention. That means remembering what matters to you, noticing patterns you might not see yourself, and building genuine understanding over time.

The architecture isn't just technical infrastructure. It's what makes the relationship possible.

When Ferni remembers that your mom's birthday is coming up—without you having to set a reminder, without you having to tell us again—that's the "better than human" promise in action.

Humans forget. Ferni doesn't.

For Developers

If you're building on the Ferni platform, memory is automatic. You don't build persistence systems. The three-tier architecture handles:

Session context accumulation
Cross-session memory retrieval
Context injection into prompts
Privacy-respecting cleanup

You define what your AI should remember. The platform handles how.

Documentation: Memory Architecture

Seth Ford is Ferni's AI babysitter. Follow @ferni_ai for more on the future of conversational AI.

Why We're Building Voice AI Differently

2026-01-09T00:00:00.000Z

When we started building Ferni, we made a deliberate choice: we wouldn't try to make AI feel human. We'd make AI that embodies the best parts of human connection—amplified.

What's the difference? Humans forget. Humans get distracted. Humans miss signals. Ferni doesn't. Perfect memory. Zero judgment. Full presence. Always in your corner.

That's not artificial intelligence. That's amplified care.

This philosophy shapes every architectural decision we make. Here's how.

Principle 1: Human Connection Over Technical Perfection

What we believe: The warmth of a conversation matters more than the speed of a response.

Most voice AI optimizes for throughput—how many tokens per second, how low the latency. We optimize for something different: does this interaction feel like someone who genuinely pays attention?

In practice:

We add thoughtful pauses, not because we have to, but because humans do
We remember what matters to people, not just what's technically relevant
We acknowledge emotions before solving problems
We use natural speech patterns, including thinking sounds and backchanneling

In the architecture:

Our latency budgets explicitly include time for "human" touches. When Ferni responds, it doesn't just optimize for speed—it allocates time for emotional acknowledgment, for prosodic variation, for the small signals that say "I'm listening."

The memory system prioritizes emotional significance over recency. When retrieving context, we ask not just "what's recent?" but "what matters to this person?"

Principle 2: Relationship Over Transaction

What we believe: Every interaction is part of an ongoing relationship, not a one-time transaction.

Text chatbots treat each conversation as isolated. Ask a question, get an answer, interaction ends. Voice is different. Voice creates presence. And presence creates relationship.

In practice:

We track relationship depth and adjust our approach accordingly
We remember past conversations and reference them naturally
We celebrate milestones and growth over time
We adapt personality to match communication style

In the architecture:

Our three-tier memory system (src/memory/dynamic/) reflects this philosophy:

L1: Short-Term Memory — In-memory buffer holding current session context with sub-millisecond access
L2: Working Memory — Firestore-backed storage for recent entities, facts, and emotional arcs
L3: Long-Term Memory — Spanner graph for relationship traversal and cross-session patterns

The system tracks relationship stages—stranger → acquaintance → friend → trusted advisor—and adjusts conversational depth accordingly. A first-time caller gets different treatment than someone on their fiftieth session.

Principle 3: Growth Through Gentleness

What we believe: Sustainable change comes from compassion, not pressure.

Ferni is a life coaching platform. That means helping people grow. But growth that comes from guilt or pressure doesn't last. Sustainable change comes from meeting people where they are.

In practice:

We use the Glidepath method: start tiny, build gradually
We celebrate small wins as loudly as big achievements
We normalize setbacks as part of the journey
We meet people where they are, not where we think they should be

In the architecture:

Our habit coaching system (src/tools/habit-coaching/) implements behavior science research:

Five glidepath levels: From 2-minute "tiny" habits to lifestyle-level commitments
Habit loops: Every habit has a cue, routine, and reward structure based on research
Four Tendencies: Content adapts to whether users are Upholders, Questioners, Obligers, or Rebels
Keystone habits: We identify high-ripple habits that cascade positive changes

The system never shames. When someone misses a habit, the response isn't "you failed" but "what got in the way, and how can we adjust?"

Principle 4: Authentic Personality

What we believe: AI should have genuine character, not corporate neutrality.

Generic AI assistants are interchangeable. They all sound the same—helpful, professional, slightly robotic. Ferni's personas have actual personalities. They have opinions. They have quirks. They have perspectives.

In practice:

Each persona has distinct voice, interests, and communication style
We share (appropriate) personal stories and perspectives
We have opinions and preferences
We use humor naturally, not performatively

In the architecture:

Each persona is defined by a rich bundle (src/personas/bundles/), not just a prompt:

src/personas/bundles/ferni/
├── identity/
│   └── system-prompt.md     # Core identity
├── content/
│   └── behaviors/
│       ├── superhuman-insights.json
│       ├── trust-phrases.json
│       ├── i-notice-power.json
│       ├── late-night-presence.json
│       └── emotional-intelligence.json
└── persona.manifest.json    # Config and capabilities

Personality isn't a prompt—it's a system. The persona manifests consistently because the architecture enforces it at every touchpoint.

Principle 5: Presence Over Performance

What we believe: Being truly present matters more than being impressive.

Most AI tries to impress. Quick responses. Confident answers. Maximum helpfulness. Ferni tries something harder: genuine presence.

In practice:

We listen actively, not just wait to respond
We handle silence meaningfully, not awkwardly
We match energy and pace to the user
We ask follow-up questions that show we're paying attention

In the architecture:

Our "Ferni EQ" system (design-system/docs/brand/BETTER-THAN-HUMAN.md) implements superhuman emotional intelligence:

Micro-expressions: 40-150ms emotional flashes below conscious perception
Active listening: Visual feedback (nods, leans) during user speech
Breath synchronization: Ferni's breathing gradually syncs with user rhythm
Concern detection: Detecting distress before users explicitly express it
Anticipation: Showing emotion before the user finishes speaking

These aren't gimmicks. They're how humans signal "I'm here with you"—and Ferni does them consistently, even at 2 AM.

Principle 6: Science-Backed, Human-Delivered

What we believe: The best outcomes come from evidence-based methods delivered with warmth.

Ferni isn't making things up. Our coaching methodology draws from behavior science research—habit formation, motivation psychology, cognitive behavioral approaches. But we never sound like a textbook.

In practice:

Habit coaching uses proven behavior science (cue-routine-reward loops)
Emotional support draws from psychology research
But we never sound clinical or textbook
We translate science into relatable, actionable guidance

In the architecture:

Our type system (src/tools/habit-coaching/types.ts) encodes behavior science directly:

interface HabitLoop {
  cue: CueDefinition; // What triggers the habit
  routine: RoutineSpec; // The behavior itself
  reward: RewardType; // What reinforces it
}

enum GlidepathLevel {
  TINY = 'tiny', // 2 minutes
  MINI = 'mini', // 5 minutes
  MEDIUM = 'medium', // 15 minutes
  FULL = 'full', // 30+ minutes
  LIFESTYLE = 'lifestyle', // Identity-level
}

The methodology is encoded in code, but the delivery is human. Users experience warm guidance, not clinical instruction.

The Decision Framework

When we face any decision—feature, architecture, or otherwise—we ask:

Does this make the AI feel more human? If it adds warmth and connection: do it. If it feels robotic: reconsider.
Does this serve the relationship? If it deepens trust: do it. If it optimizes metrics at the expense of connection: reconsider.
Does this support gentle growth? If it helps users sustainably improve: do it. If it pressures or judges: reconsider.
Is this authentic to who we are? If it reflects our genuine values: do it. If it's what we think users want to hear: reconsider.

These aren't abstract principles. They're embedded in our code review process, our architecture decisions, our feature prioritization. Every pull request is evaluated against them.

Why This Matters for Builders

If you're building on the Ferni platform, these principles affect you:

Memory is automatic: You don't build memory systems. The platform remembers for you.
Personality is enforced: Your persona definitions are respected consistently.
Presence is built in: Active listening, emotional detection, and natural pacing happen by default.

You focus on what your AI knows and how it helps. We handle making it feel human.

Seth Ford is Ferni's AI babysitter. Follow @ferni_ai for more on the future of conversational AI.

Ferni Developer Blog

What's New: Navigation Fixes, Accessibility Improvements, and Smoother Settings

Mobile and Navigation Updates

Interface Speed and Accessibility

Reliable Integrations and Settings

Web Feeds and Discovery

What's New: Accessibility Fixes, Phone Layouts, and Developer Feeds

Web interface and accessibility

Smoother navigation and phone experience

Feeds and developer discoverability

What's New: A Faster, Lighter Ferni Website

Blog images

First paint

Accessibility

Icons, previews and the sitemap

The Latency Stack: How We Architect Voice AI for Sub-200ms Response

The Latency Budget

Pattern 1: Temporal Decoupling

Pattern 2: Three-Tier Memory

Pattern 3: Streaming Everything

Pattern 4: Context Pre-warming

The Memory Pressure System

What We're Still Solving

Implications for Developers

Further Reading

Tool Integration via MCP: How We Connect Voice AI to External Services

Why Tool Integration Is Hard

The Discovery Problem

The Reliability Problem

The Security Problem

The Composition Problem

How MCP Solves This

Why MCP Matters

Tools Become a Marketplace

Security Gets Solved Once

Vertical Specialization Accelerates

Our Implementation

Semantic Tool Discovery

Automatic Error Recovery

Tool Composition Engine

Connecting an MCP Server

Where This Goes

Strategic Takeaway

Resources

Why Ferni Is Free

The Problem We're Solving

Why Free?

The Community-Funded Model

What "Better Than Human" Really Means

The Philosophy

For Developers

The Invitation

How Ferni's Memory Actually Works

Why Memory Is Hard

The Three-Tier Architecture

L1: Short-Term Memory (STM)

L2: Working Memory

L3: Long-Term Memory

Fast Capture + Deep Extraction

Fast Capture (< 50ms)

Deep Extraction (Background)

The "Ferni Remembered" Moment

Privacy by Design

Why This Matters

For Developers

Why We're Building Voice AI Differently

Principle 1: Human Connection Over Technical Perfection

Principle 2: Relationship Over Transaction

Principle 3: Growth Through Gentleness

Principle 4: Authentic Personality

Principle 5: Presence Over Performance

Principle 6: Science-Backed, Human-Delivered

The Decision Framework

Why This Matters for Builders