lessonRecorder

Dev log

What changed, why, and what is still open. Newest first.

2026-09-27 — processing on our side

featurefix

Post-save processing pipeline. Every saved recording is now processed on the server (one job at a time): real ASR over the recording's audio → transcript + captions; stills sampled across the recording → described by a vision model; then a summary of the spoken script and the frames. The share page and /s/<token>/summary carry the result, so anyone opening the link gets everything with no work.
System / computer audio can now be transcribed. It never could before: Chrome's SpeechRecognition only hears the microphone. Anything played through the machine — a video, a call, a slide narration — was silently absent from the live transcript. Server-side ASR works on the mixed audio, so it is included now.
Chinese is markedly better. Re-transcription uses a real ASR model instead of the browser recogniser: on the "Liu Huan" recording the live transcript was 65 characters for 4 minutes; the rebuilt one is 2,512 characters of coherent 繁體中文.
Provider fallback. HKBU GenAI is the primary model, but its shared quota does run out (HTTP 429). A second provider (OpenRouter moonshotai/kimi-k2.6) is now tried automatically, and a quota error no longer discards the transcript — the job reports partial with the reason.
Media repair. PUT /api/recordings/:id/media plus tools/retranscribe.mjs --repair rewrite a recording's container so it carries a duration header — Chrome's MediaRecorder writes none, which is why seeking was unreliable.
Frame-stall watchdog. A display track can stay "live" while delivering no frames at all (normally when the shared tab/window is hidden), which made a 13-minute screen recording audio-only with all its video in the last 20 seconds. The recorder now warns while recording.

2026-09-15 — hub, summaries, agent API

Mission-control hub at /hub — an index of the modules, the admin notes doc, and link/status checks.
AI summary + consultant. Summaries cover the script and sampled frames; the consultant answers questions grounded in one recording, for the owner and (rate-limited) for anyone with the share link.
Keyless agent API — /s/<token>/meta, /transcript.txt, /captions.vtt, /summary, /media, and POST /s/<token>/ask, documented at /s/<token>/api.
Transcript repair. The live transcript was emitting runs of 1–2 character lines: every recogniser restart committed the in-progress partial as a finished line. Partials are now carried into the next utterance, the restart is slower, and a hint appears when a session is producing almost nothing (usually a language mismatch — 廣東話 recorded as 中文).
tools/retranscribe.mjs rebuilds a transcript + captions from the audio (chunked, so long recordings don't make the model drift).
Team accounts for the recorder were provisioned.

Earlier

System-audio capture, one-click new session, resilient captions, screen-share recovery (ed643b7).
Public agent API guide + chunked media fetch (6710df1).
Import existing audio/video files into the library (6c3dac5).

This is almost always a one-time OS permission, not the app. On a shared/classroom machine it is often still denied because nobody granted it (or the browser wasn't restarted afterwards).

macOS (classroom iMacs, MacBooks)

Open System Settings (or System Preferences) → Privacy & Security.
Choose Screen Recording.
Enable the browser you use — Google Chrome (or Safari/Edge). If it is not listed, click + and add it from Applications.
Quit the browser completely (⌘Q, not just close the window) and open it again. macOS only applies the change on a fresh launch.
Reload this page and press ▶ Start; in Chrome's picker choose Entire Screen.
Managed/lab Macs: the setting may be locked by policy — ask IT to allow Screen Recording for the browser.

Windows

In the picker choose what to share; no OS permission is usually required.
If sharing fails, check Settings → Privacy & security → Camera/Microphone and the browser's own site permissions (the 🔒 icon in the address bar).

Always

Pick Entire Screen, not a single tab or window. A tab/window that gets hidden stops producing frames — the recording then looks blank even though it is still running.
Keep this recorder tab visible (or in a second window) while recording.
Microphone: allow it when asked, and pick the right input in the sidebar.
Still stuck? Use Camera + mic or Voice only to keep the session, and tell us what the browser said.

⚠️ Known issues / in progress

HKBU GenAI quota (429). The shared university quota can be exhausted, which now falls back to a second provider. Worth raising the limit, or making the fallback the default for automatic jobs.
Classroom screen sharing — see the section above; the app now links here from the setup card and from the failure message.
Live transcript quality remains browser-limited; post-save processing corrects it, so the live text should be treated as a preview.
Old recordings can be brought up to date with tools/retranscribe.mjs <id> and --repair; a library-wide sweep is not automated yet.
Starting a new session while an earlier one is unfinished is being looked at — a partial session auto-uploads on the next load, and there is currently no explicit "discard and start fresh".