I built a Hebrew transcription pipeline on the ivrit.ai models — an open Hebrew speech-model project — in roughly two prompts to Claude Code, speaker identification included. The hard half was not the transcription but the editor, where a human listens to, corrects and approves every line against the audio. A legal document cannot come out of ASR output nobody checked.
Here is how it started. My wife is a lawyer, and every so often she hands me a concrete problem. This time it was transcribing recorded interviews: you get an audio file, and you need to produce a transcript document in a standard legal format.
The pipeline is the easy half
The chain is unremarkable: ffmpeg converts the uploaded file (m4a, mp3, wav, ogg or opus) to 16 kHz mono wav, faster-whisper transcribes it, pyannote does speaker diarization, and a merge function stitches the two together. The models come from ivrit.ai — whisper-large-v3-turbo-ct2 and pyannote-speaker-diarization-3.1.
Assigning a speaker to a line is nothing but time overlap: for each Whisper segment, pick the speaker whose diarization turn overlaps it most. Speakers holding less than two percent of total speaking time are discarded as noise, and a segment with no overlap inherits the speaker of the segment before it, or, if there is no previous segment, the dominant speaker in the recording. Not clever logic, and that is fine — the human in the editor fixes whatever it got wrong.
Incidentally, that assignment function appears twice in the codebase — once in the tool that generates my dev fixtures, once in the worker — copied word for word, on purpose. The point is that an uploaded transcript merges speakers identically to the fixtures I developed against, so what I saw in the editor in development is what happens in production.
The one non-obvious piece is the confidence signal. faster-whisper returns an avg_logprob per segment, and I map it to a 0-to-1 value with exp. That number is what later flags suspicious lines in the editor.
One more small thing that can eat an evening if you do not know about it: whisper.cpp’s output carries invisible bidirectional control characters — RTL and LTR marks — and the parser that converts it into my dev fixtures strips them. Otherwise they leak quietly into the final document.
The editor is the product
The pipeline is a few hundred lines. The editor is everything else, and it is what makes this usable by a lawyer rather than by me:
- Clicking a line’s timestamp seeks the audio to that moment and plays. That is the whole idea.
- Playback speed control (0.75× to 2×) and five-second skips backward and forward.
- Every line is editable in place, and every line has a speaker selector — you can correct the diarization, which is the thing the algorithm most often gets wrong.
- Merge a line into the one above it, or split a line at the cursor. The split derives the cut timestamp from the ratio of the two text lengths.
- Lines with confidence below 0.75 are visually marked. That does not mean they are wrong; it means listen there first.
Two design decisions there only surface once you actually use it. First, auto-scroll follows the currently playing line — but if focus is inside the segment list, meaning you are mid-typing, it does not yank the screen out from under you. Second, saving is automatic after two seconds of quiet, and if the same transcript was edited in another window in the meantime the server answers 409 and the UI shows a “this recording was edited elsewhere — reload” banner instead of overwriting. The export button is disabled while any change is unsaved, so you cannot download a document that is missing your last correction.
The standard format is its own problem
Export is docx via python-docx or PDF via WeasyPrint, with checkboxes for timestamps and speaker names. Consecutive segments from the same speaker fold into one paragraph, because nobody wants to read a document where every sentence is its own line.
Hebrew in Word is a genre of pain. The rules I converged on: w:bidi on the paragraph, w:rtl only on runs that are actually Hebrew, w:cs and w:szCs on every run, David as the font, and never pre-shape the text with python-bidi. Text is split into runs by script before it enters the document, so a number or an English term inside a Hebrew sentence does not come out reversed.
What it costs to run
Apart from the GPU, almost nothing. The backend is a Lambda that scales to zero when idle, and S3 plus CloudFront are pennies at this volume. The worker is a SageMaker async endpoint on ml.g5.xlarge configured to scale down to zero instances, so you pay only while a job is running — the price being a several-minute cold start.
There are also quotas, because an unmetered GPU button is a problem waiting to happen. The default is two active recordings per user, checked in the backend before an upload URL is issued, and two hours per recording, enforced with ffprobe against the original file before anything is transcoded or transcribed. No point paying to transcribe a file that will be rejected anyway.
One thing worth getting right there: the worker is an endpoint you can invoke directly, without going through the backend. So it re-validates the job body itself — username, recording id, file extension — and additionally requires the audio path to be exactly the path a legitimate job for those values would have produced. Without that check, a forged job with a perfectly valid username could point at someone else’s file.
What the repo does not contain
No sample audio, no transcripts — the .gitignore says so explicitly. All development and the demo ran on mock data, not real interviews, so there is no privacy exposure here. The live deployment sits behind HTTP Basic Auth at the CloudFront edge, one credential per user with a private S3 prefix each. That is not a real user-account system, and that is a known limitation.
The idea I have not acted on
This works as a SaaS, and I can see why someone would build that business. But there is a more interesting version. The users of this tool are already required to listen to the entire recording and fix every wrong word — that is the job, independent of anyone’s model-training plans. Every correction is a verified audio-and-text pair, which is precisely the material you need to train a Hebrew speech model.
So what if ivrit.ai gave this to Israel’s Ministry of Justice at cost price, and took the corrections back as training data? Free annotators, from work that has to happen anyway.
What I took from it
When a model solves the problem, it is easy to think the product is finished. It is not. In any domain with legal or regulatory liability, the product is the interface where a human checks the output — how fast one line can be verified, how easy the correction is, and how hard it is to accidentally export something unverified. The pipeline was two prompts; the editor was the project.
This is about the cleanest example I have of how I approach an AI adoption project inside an organization: first find where a human has to stay in the loop, then build that person a tool. Here is what that looks like in practice.