127 lines
3.9 KiB
Markdown
127 lines
3.9 KiB
Markdown
|
# Clio - Architecture Overview
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
Multi-device audio recording system with end-to-end encryption. Records continuously on phone and portable devices, encrypts immediately, syncs to central server, transcribes with GPU, and provides a web interface for search and highlights.
|
||
|
|
|
||
|
|
## Components
|
||
|
|
|
||
|
|
| Repository | Purpose | Deployment Target |
|
||
|
|
|------------|---------|-------------------|
|
||
|
|
| clio-api | FastAPI server, recordings DB, highlights | Server (10.0.0.45) |
|
||
|
|
| clio-android | Android recording app | Phone (manual APK) |
|
||
|
|
| clio-ui | React viewer frontend | Server (10.0.0.45) |
|
||
|
|
| clio-ai | Whisper transcription, wake word detection | Desktop (10.0.0.130) |
|
||
|
|
| clio-infra | Deployment configs, docs, NixOS modules | All |
|
||
|
|
|
||
|
|
## Security Model
|
||
|
|
|
||
|
|
```
|
||
|
|
Phone (has: server public key) Server (has: private key)
|
||
|
|
───────────────────────────── ────────────────────────
|
||
|
|
Record audio → temp.ogg
|
||
|
|
Encrypt with public key →
|
||
|
|
recording_*.ogg.enc
|
||
|
|
│
|
||
|
|
└──── HTTP POST ────────→ Receive encrypted file
|
||
|
|
Decrypt with private key
|
||
|
|
Store as .ogg in recordings/
|
||
|
|
```
|
||
|
|
|
||
|
|
- **Sealed box encryption**: X25519 key exchange + XSalsa20-Poly1305
|
||
|
|
- **Forward secrecy**: Each encryption uses ephemeral keys
|
||
|
|
- **Phone never stores unencrypted**: Temp file deleted immediately after encryption
|
||
|
|
- **If no public key configured**: Recordings are deleted (security-first design)
|
||
|
|
|
||
|
|
## File Format (SREC)
|
||
|
|
|
||
|
|
Encrypted files use a chunked format for streaming large files:
|
||
|
|
|
||
|
|
```
|
||
|
|
[4 bytes: "SREC" magic]
|
||
|
|
[1 byte: version (1)]
|
||
|
|
[4 bytes: chunk count, big-endian]
|
||
|
|
For each chunk (64KB max):
|
||
|
|
[4 bytes: encrypted length, big-endian]
|
||
|
|
[N bytes: sealed box ciphertext]
|
||
|
|
```
|
||
|
|
|
||
|
|
## Data Flow
|
||
|
|
|
||
|
|
```
|
||
|
|
1. Recording
|
||
|
|
Phone/Pi → temp.ogg → encrypt → recording_*.ogg.enc
|
||
|
|
|
||
|
|
2. Sync
|
||
|
|
Phone/Pi → HTTP POST → Server decrypts → recordings/YYYY/MM/DD/*.ogg
|
||
|
|
|
||
|
|
3. Transcription (async, on desktop)
|
||
|
|
Desktop pulls recordings → Whisper → JSON segments → POST to server
|
||
|
|
|
||
|
|
4. Viewing
|
||
|
|
Browser → clio-ui → clio-api → recordings + transcripts + highlights
|
||
|
|
```
|
||
|
|
|
||
|
|
## API Endpoints (clio-api)
|
||
|
|
|
||
|
|
| Endpoint | Method | Description |
|
||
|
|
|----------|--------|-------------|
|
||
|
|
| `/health` | GET | Health check |
|
||
|
|
| `/api/v1/public-key` | GET | Server public key (Base64) |
|
||
|
|
| `/api/v1/recordings/upload` | POST | Upload encrypted file |
|
||
|
|
| `/api/v1/recordings` | GET | List recordings with transcripts |
|
||
|
|
| `/api/v1/recordings/{id}` | GET | Recording details |
|
||
|
|
| `/api/v1/recordings/{id}/audio` | GET | Stream audio file |
|
||
|
|
| `/api/v1/import/transcription` | POST | Import transcription JSON |
|
||
|
|
| `/api/v1/highlights` | GET/POST | Manage highlights |
|
||
|
|
| `/api/v1/sources` | GET | List recording sources |
|
||
|
|
|
||
|
|
## Database Schema
|
||
|
|
|
||
|
|
**recordings**: id, source_name, audio_path, duration, language, recorded_at, transcribed_at
|
||
|
|
**segments**: id, recording_id, start_time, end_time, text, confidence
|
||
|
|
**highlights**: id, recording_id, start_seconds, end_seconds, tag, note, created_at
|
||
|
|
|
||
|
|
## Sync Modes (Android)
|
||
|
|
|
||
|
|
| Mode | Behavior |
|
||
|
|
|------|----------|
|
||
|
|
| Never | No sync, files stay on phone |
|
||
|
|
| LAN Only | Sync only when IP matches subnet (e.g., `10.0.0.`) |
|
||
|
|
| WiFi Only | Sync on any WiFi network |
|
||
|
|
| Any Network | Sync on WiFi or mobile data |
|
||
|
|
|
||
|
|
## Hosts & Paths
|
||
|
|
|
||
|
|
### Server (10.0.0.45)
|
||
|
|
- Code: `/work/hosting/recordings`
|
||
|
|
- Audio: `/media/external/Hosting/Recordings`
|
||
|
|
- Database: `/work/hosting/recordings/data/recordings.db`
|
||
|
|
- Frontend: `/var/www/recordings-viewer`
|
||
|
|
- Domain: `recordings.hallocks.xyz`
|
||
|
|
|
||
|
|
### Desktop (10.0.0.130)
|
||
|
|
- Transcription scripts
|
||
|
|
- Real-time wake word detection
|
||
|
|
- GPU: RTX 3090
|
||
|
|
|
||
|
|
## Dependencies
|
||
|
|
|
||
|
|
### clio-api (Python)
|
||
|
|
- fastapi, uvicorn
|
||
|
|
- pynacl (libsodium)
|
||
|
|
- aiofiles
|
||
|
|
- sqlite3
|
||
|
|
|
||
|
|
### clio-android (Kotlin)
|
||
|
|
- lazysodium-android
|
||
|
|
- androidx.work
|
||
|
|
- okhttp3
|
||
|
|
|
||
|
|
### clio-ai (Python)
|
||
|
|
- faster-whisper
|
||
|
|
- torch, torchaudio
|
||
|
|
- sounddevice
|
||
|
|
- webrtcvad
|
||
|
|
- scikit-learn
|