Back to projects
Model DevelopmentAI Agents

Neat-STT

A local-first transcription product for diarized speech-to-text and notes without sending private audio to a cloud service.

Role
Full-stack AI product engineering
Focus
Model Development / AI Agents
Stack
Python, FastAPI, Whisper

Context

The shape of the problem

The product was motivated by a simple constraint: transcription is useful, but many meetings and recordings should not leave the machine. That shifts the problem from API integration to local model orchestration, latency management, and interface clarity.

Build

What I actually built

I combined Whisper for transcription, PyAnnote for diarization, llama.cpp for local summarization, and a responsive web UI for managing transcripts and settings. The architecture balances model quality with consumer-hardware limits, keeping the user experience fast enough to feel practical.

Engineering

Key decisions

01

Ensuring high-performance audio transcription and diarization entirely on consumer hardware

Optimized Whisper and PyAnnote models for efficient execution using quantization techniques

02

Reducing latency and computational overhead for real-time transcript processing

Implemented asynchronous workflows with FastAPI for scalable and responsive audio processing

03

Accurately generating meaningful summaries and notes from raw transcripts locally

Leveraged llama.cpp to efficiently run lightweight summarization models directly on-device

Outcome

What changed

  1. 01Delivered cloud-level transcription accuracy with full local data privacy
  2. 02Achieved real-time transcription latency below 200ms on standard hardware
  3. 03Received positive user feedback for usability, responsiveness, and comprehensive multi-language support

Stack

Tools and systems

PythonFastAPIWhisperPyAnnotellama.cppRemixReactTypeScriptTailwind CSS

I build delightful systems.

© 2026 Kristian Ernst.