154 lines
7 KiB
Markdown
154 lines
7 KiB
Markdown
|
# Clio - Consolidated TODO
|
||
|
|
|
||
|
|
## Architecture Overview
|
||
|
|
|
||
|
|
```
|
||
|
|
┌─────────────────────────────────────────────────────────────────────────┐
|
||
|
|
│ TARGET ARCHITECTURE │
|
||
|
|
├─────────────────────────────────────────────────────────────────────────┤
|
||
|
|
│ │
|
||
|
|
│ ┌──────────┐ ┌──────────────────────────────────────────────┐ │
|
||
|
|
│ │ Phone │───────▶│ Server (10.0.0.45) │ │
|
||
|
|
│ │ (Android)│ push │ - Storage hub (recordings) │ │
|
||
|
|
│ └──────────┘ │ - clio-api (FastAPI) │ │
|
||
|
|
│ │ - clio-ui (React frontend) │ │
|
||
|
|
│ ┌──────────┐ │ - Database (highlights, transcripts) │ │
|
||
|
|
│ │ Pi │───────▶│ - Forgejo (code.hallocks.xyz) │ │
|
||
|
|
│ │(portable)│ push └──────────────────────────────────────────────┘ │
|
||
|
|
│ └──────────┘ │ │
|
||
|
|
│ │ pulls recordings │
|
||
|
|
│ ▼ │
|
||
|
|
│ ┌──────────────────────────────────────────────┐ │
|
||
|
|
│ │ Desktop (10.0.0.130) │ │
|
||
|
|
│ │ - GPU: RTX 3090 │ │
|
||
|
|
│ │ - clio-ai/transcription (Whisper) │ │
|
||
|
|
│ │ - clio-ai/real_time (wake word detection) │ │
|
||
|
|
│ │ - Off at night (non-critical) │ │
|
||
|
|
│ └──────────────────────────────────────────────┘ │
|
||
|
|
│ │
|
||
|
|
└─────────────────────────────────────────────────────────────────────────┘
|
||
|
|
```
|
||
|
|
|
||
|
|
## Immediate Tasks
|
||
|
|
|
||
|
|
### Infrastructure Setup
|
||
|
|
- [x] Import Pi recorder code (systemd setup) → clio-pi/
|
||
|
|
- [ ] Migrate ~100s GB of recordings from Pi to server storage
|
||
|
|
- [ ] Pi: Add sync script to push recordings to server
|
||
|
|
- [ ] Pi: Auto-detect audio device (currently hardcoded to hw:1)
|
||
|
|
- [ ] Set up Forgejo repos for all clio-* projects
|
||
|
|
- [ ] Configure Forgejo Actions for server deployments (10.0.0.45)
|
||
|
|
- [ ] Create backup strategy for viewer, highlights, and database
|
||
|
|
|
||
|
|
### Deployment Pipeline
|
||
|
|
- [ ] clio-api: Forgejo Actions → deploy to 10.0.0.45
|
||
|
|
- [ ] clio-ui: Forgejo Actions → deploy to 10.0.0.45:/var/www/recordings-viewer
|
||
|
|
- [ ] clio-ai: Manual deploy to desktop (10.0.0.130) - not via Actions
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Feature Backlog
|
||
|
|
|
||
|
|
### High Priority
|
||
|
|
- [ ] QR code for key import (avoid email/typing)
|
||
|
|
- [ ] Server URL configuration in app UI
|
||
|
|
- [ ] Manual sync trigger button
|
||
|
|
- [ ] View sync history/status in app
|
||
|
|
- [ ] Display last successful sync time on phone
|
||
|
|
- [ ] Server status page/dashboard:
|
||
|
|
- When phone/pi last synced
|
||
|
|
- Total recordings count
|
||
|
|
- Recent uploads
|
||
|
|
- Disk space usage
|
||
|
|
|
||
|
|
### Wake Word / Voice Recognition (clio-ai)
|
||
|
|
- [ ] Manage wake words in UI (currently hardcoded)
|
||
|
|
- [ ] Centralize voice recognition training data
|
||
|
|
- [ ] Real-time detection improvements (currently sucks even on desktop)
|
||
|
|
- [ ] Future: Real-time detection on phone/pi (low priority)
|
||
|
|
|
||
|
|
### Audio Analysis & Intelligence
|
||
|
|
- [ ] Speech detection bar in recording detail view
|
||
|
|
- [ ] Sleeping/silence detection for overnight recordings
|
||
|
|
- [ ] Auto voice recognition / speaker identification
|
||
|
|
- [ ] Auto highlight generation (audio-based, not text search)
|
||
|
|
|
||
|
|
### Viewer UX
|
||
|
|
- [ ] Background audio playback (mini player across views)
|
||
|
|
- [ ] Highlight duration (currently point-only, add end time)
|
||
|
|
- [ ] Auto-scroll timeline to recordings
|
||
|
|
|
||
|
|
### Medium Priority
|
||
|
|
- [ ] Configurable recording quality (bitrate, sample rate)
|
||
|
|
- [ ] Configurable segment duration (currently 2 hours)
|
||
|
|
- [ ] Delete synced files automatically (with retention)
|
||
|
|
- [ ] Server authentication for upload endpoint
|
||
|
|
- [ ] Server HTTPS with Let's Encrypt (partially done via Caddy)
|
||
|
|
|
||
|
|
### Low Priority / Nice to Have
|
||
|
|
- [ ] Multiple server destinations
|
||
|
|
- [ ] Compression before encryption
|
||
|
|
- [ ] Resume interrupted uploads
|
||
|
|
- [ ] Docker container for server
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Questions to Resolve
|
||
|
|
|
||
|
|
### Key Rotation
|
||
|
|
- What happens when server keypair changes?
|
||
|
|
- Options:
|
||
|
|
1. Keep old private key archived
|
||
|
|
2. Sync all files before rotating
|
||
|
|
3. Store key version in encrypted file header
|
||
|
|
|
||
|
|
### Manual Decryption
|
||
|
|
- Document steps to decrypt files copied via USB
|
||
|
|
- CLI tool needed
|
||
|
|
|
||
|
|
### Desktop Worker Architecture
|
||
|
|
- How does desktop pull recordings from server?
|
||
|
|
- How does it push transcriptions back?
|
||
|
|
- Queue system needed?
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Known Limitations
|
||
|
|
|
||
|
|
1. **No streaming encryption**: Entire temp file must complete before encryption
|
||
|
|
2. **No partial sync**: If upload fails, whole file retries
|
||
|
|
3. **Single server**: Can only sync to one destination at a time
|
||
|
|
4. **No authentication**: Relies on encryption for confidentiality
|
||
|
|
5. **Desktop off at night**: Real-time detection unavailable
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Repository Structure
|
||
|
|
|
||
|
|
```
|
||
|
|
/work/projects/clio/
|
||
|
|
├── clio-api/ # FastAPI server (recordings, highlights, transcripts)
|
||
|
|
├── clio-android/ # Android recording app
|
||
|
|
├── clio-ui/ # React frontend (viewer)
|
||
|
|
├── clio-ai/ # ML/AI components
|
||
|
|
│ ├── real_time/ # Wake word detection, voice classifier
|
||
|
|
│ └── transcription/ # Whisper transcription scripts
|
||
|
|
├── clio-pi/ # Raspberry Pi recorder
|
||
|
|
│ ├── bin/ # Recording scripts
|
||
|
|
│ ├── systemd/ # Service units
|
||
|
|
│ └── setup.sh # Pi setup script
|
||
|
|
└── clio-infra/ # Deployment, configs, docs
|
||
|
|
├── nixos/ # NixOS modules
|
||
|
|
├── scripts/ # Deploy scripts
|
||
|
|
└── TODO.md # This file
|
||
|
|
```
|
||
|
|
|
||
|
|
## Hosts
|
||
|
|
|
||
|
|
| Host | IP | Role |
|
||
|
|
|------|-----|------|
|
||
|
|
| Server | 10.0.0.45 | Storage, API, UI, Forgejo |
|
||
|
|
| Desktop | 10.0.0.130 | GPU transcription, real-time detection |
|
||
|
|
| Phone | (mobile) | Recording source |
|
||
|
|
| Pi | (TBD) | Portable recording source |
|