docs: replace long dashes, document icons, emoji and avatar rules
Update the status of the client UX defects handled by the interface work.
This commit is contained in:
parent
220af2d6a4
commit
ca3465daa5
19 changed files with 311 additions and 303 deletions
|
|
@ -1,4 +1,4 @@
|
|||
# Admin Panel — Design
|
||||
# Admin Panel - Design
|
||||
|
||||
> Target: Phase 1.1 (core) → Phase 1.2 (guilds, roles) → Phase 2 (plugins, bot permissions)
|
||||
|
||||
|
|
@ -6,11 +6,11 @@
|
|||
|
||||
| Layer | Config file (TOML) | Admin panel (GUI) |
|
||||
|-------|-------------------|-------------------|
|
||||
| **Infrastructure** | `bind`, `ports`, `data_dir`, TLS certs | — |
|
||||
| **Infrastructure** | `bind`, `ports`, `data_dir`, TLS certs | - |
|
||||
| **Node identity** | `name`, `address` | Display only |
|
||||
| **Storage** | `backend`, `sqlite_path`, `retention` | — |
|
||||
| **Storage** | `backend`, `sqlite_path`, `retention` | - |
|
||||
| **Security** | federation gate, rate limits, session timeout | Ban list, mute, kick |
|
||||
| **Runtime state** | — | Channels, roles, members, invites, audit log |
|
||||
| **Runtime state** | - | Channels, roles, members, invites, audit log |
|
||||
| **Plugins** | Plugin enable/disable | Permission keys, commands |
|
||||
|
||||
**Rule of thumb:** If it needs a server restart → TOML. If it can apply live → GUI.
|
||||
|
|
@ -24,22 +24,22 @@ Designed for both built-in actions and plugin/bot commands (inspired by LuckPerm
|
|||
### Built-in permission keys
|
||||
|
||||
```
|
||||
vnox.admin.manage_channels — create, edit, delete channels
|
||||
vnox.admin.manage_roles — create, edit, delete roles
|
||||
vnox.admin.manage_members — kick, ban, mute, timeout
|
||||
vnox.admin.manage_invites — create, delete invites
|
||||
vnox.admin.manage_guild — edit guild name, icon, settings
|
||||
vnox.admin.view_audit_log — read audit log
|
||||
vnox.admin.manage_channels - create, edit, delete channels
|
||||
vnox.admin.manage_roles - create, edit, delete roles
|
||||
vnox.admin.manage_members - kick, ban, mute, timeout
|
||||
vnox.admin.manage_invites - create, delete invites
|
||||
vnox.admin.manage_guild - edit guild name, icon, settings
|
||||
vnox.admin.view_audit_log - read audit log
|
||||
|
||||
vnox.channel.read — view channel
|
||||
vnox.channel.send — send messages
|
||||
vnox.channel.voice_connect — join voice channel
|
||||
vnox.channel.voice_speak — speak in voice (push-to-talk)
|
||||
vnox.channel.voice_mute_members — server-mute others
|
||||
vnox.channel.read - view channel
|
||||
vnox.channel.send - send messages
|
||||
vnox.channel.voice_connect - join voice channel
|
||||
vnox.channel.voice_speak - speak in voice (push-to-talk)
|
||||
vnox.channel.voice_mute_members - server-mute others
|
||||
|
||||
vnox.dm.send — send direct messages
|
||||
vnox.friend.request — send friend requests
|
||||
vnox.presence.view — see online status
|
||||
vnox.dm.send - send direct messages
|
||||
vnox.friend.request - send friend requests
|
||||
vnox.presence.view - see online status
|
||||
```
|
||||
|
||||
### Custom permission keys (plugin/bot commands)
|
||||
|
|
@ -47,24 +47,24 @@ vnox.presence.view — see online status
|
|||
Plugins register their permissions at load time via the RPC API:
|
||||
|
||||
```
|
||||
mybot.command.greet — /greet command
|
||||
mybot.command.play — /play music
|
||||
mybot.command.ban_override — override sub-command
|
||||
mybot.command.greet - /greet command
|
||||
mybot.command.play - /play music
|
||||
mybot.command.ban_override - override sub-command
|
||||
```
|
||||
|
||||
**Wildcard support:**
|
||||
```
|
||||
vnox.admin.* — all admin permissions
|
||||
vnox.channel.* — all channel permissions
|
||||
mybot.* — all commands from mybot
|
||||
* — everything (owner only)
|
||||
vnox.admin.* - all admin permissions
|
||||
vnox.channel.* - all channel permissions
|
||||
mybot.* - all commands from mybot
|
||||
* - everything (owner only)
|
||||
```
|
||||
|
||||
### How it works
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Role Editor — "Moderator" │
|
||||
│ Role Editor - "Moderator" │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ Permission keys: │
|
||||
│ │
|
||||
|
|
@ -91,14 +91,14 @@ mybot.* — all commands from mybot
|
|||
```
|
||||
|
||||
**Resolution order (highest priority wins):**
|
||||
1. Owner — overrides everything
|
||||
2. Channel override (user-specific) — allow/deny
|
||||
3. Channel override (role-specific) — allow/deny
|
||||
1. Owner - overrides everything
|
||||
2. Channel override (user-specific) - allow/deny
|
||||
3. Channel override (role-specific) - allow/deny
|
||||
4. Role hierarchy (top role has higher priority)
|
||||
5. @everyone default
|
||||
6. Server-wide default
|
||||
|
||||
Each override is `(allow_mask, deny_mask)` — deny always beats allow at the same level.
|
||||
Each override is `(allow_mask, deny_mask)` - deny always beats allow at the same level.
|
||||
|
||||
### Database
|
||||
|
||||
|
|
@ -184,10 +184,10 @@ Server Settings → Channels → [+ Add]
|
|||
Double-click channel or right-click → "Edit Channel".
|
||||
|
||||
Same modal pre-filled, plus:
|
||||
- **Name change** — updates slug
|
||||
- **Category change** — moves channel
|
||||
- **Drag position** — reorder inside category
|
||||
- **Delete** — with confirmation (soft delete in DB)
|
||||
- **Name change** - updates slug
|
||||
- **Category change** - moves channel
|
||||
- **Drag position** - reorder inside category
|
||||
- **Delete** - with confirmation (soft delete in DB)
|
||||
|
||||
### Drag & Drop
|
||||
|
||||
|
|
@ -363,7 +363,7 @@ PID 0x00A2 GUILD_TRANSFER client → gateway (owner only)
|
|||
}
|
||||
```
|
||||
|
||||
**ADMIN_UPDATE_CHANNEL (0x0072) — includes position changes via drag & drop:**
|
||||
**ADMIN_UPDATE_CHANNEL (0x0072) - includes position changes via drag & drop:**
|
||||
```json
|
||||
{
|
||||
"channel_id": "uuid",
|
||||
|
|
@ -390,7 +390,7 @@ Gateway checks:
|
|||
2. Is user in guild? → 403 if no
|
||||
3. Does user have vnox.admin.manage_channels?
|
||||
- Resolve role hierarchy
|
||||
- Check channel_overrides (none — new channel)
|
||||
- Check channel_overrides (none - new channel)
|
||||
- Check wildcards (vnox.admin.*)
|
||||
- Owner override → 403 if no
|
||||
│
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Central Hub — опциональный координационный сервер
|
||||
# Central Hub - опциональный координационный сервер
|
||||
|
||||
## Мотивация
|
||||
|
||||
|
|
@ -6,12 +6,12 @@ VNOX спроектирован как self-hosted платформа: кажд
|
|||
Но для adoption нужна точка входа для тех, у кого нет своего сервера, и механизм
|
||||
обнаружения публичных нодов.
|
||||
|
||||
Central Hub — опциональный сервер, который **не заменяет** собственный нод
|
||||
Central Hub - опциональный сервер, который **не заменяет** собственный нод
|
||||
и **не хостит** голос/чат. Он только координирует.
|
||||
|
||||
## Что делает Central Hub
|
||||
|
||||
### 1. Discovery / Registry — реестр публичных нодов
|
||||
### 1. Discovery / Registry - реестр публичных нодов
|
||||
|
||||
Владельцы нодов могут опубликовать свой сервер в реестре:
|
||||
- адрес, название, описание, регион
|
||||
|
|
@ -19,19 +19,19 @@ Central Hub — опциональный сервер, который **не з
|
|||
- мета: онлайне, кол-во пользователей
|
||||
|
||||
Клиент может найти сервер по тегам/названию и подключиться.
|
||||
Без Central Hub — только прямой ввод адреса или закладки.
|
||||
Без Central Hub - только прямой ввод адреса или закладки.
|
||||
|
||||
### 2. Global identity bridge — аккаунт без своего нода
|
||||
### 2. Global identity bridge - аккаунт без своего нода
|
||||
|
||||
Пользователь регистрирует ключ на Central Hub:
|
||||
- `user@hub` — глобальный идентификатор
|
||||
- `user@hub` - глобальный идентификатор
|
||||
- никнейм, аватар (опционально)
|
||||
- привязка к одному или нескольким "гостевым" нодам
|
||||
|
||||
Позволяет зайти в VNOX без поднятия собственной инфраструктуры.
|
||||
При этом Central Hub не хранит сообщения, не релеит голос, не управляет каналами.
|
||||
|
||||
### 3. Push notification relay — поддержка мобильных
|
||||
### 3. Push notification relay - поддержка мобильных
|
||||
|
||||
Мобильные клиенты не могут держать постоянное TCP/UDP соединение.
|
||||
Central Hub принимает пуш-уведомления от нодов:
|
||||
|
|
@ -41,7 +41,7 @@ Central Hub принимает пуш-уведомления от нодов:
|
|||
|
||||
Без этого мобильный клиент будет получать сообщения только с задержками.
|
||||
|
||||
### 4. User directory / friend graph — глобальная социальная сеть
|
||||
### 4. User directory / friend graph - глобальная социальная сеть
|
||||
|
||||
Список друзей и поиск пользователей не привязан к одному ноду:
|
||||
- `@user` ищется по Central Hub
|
||||
|
|
@ -53,13 +53,13 @@ Central Hub принимает пуш-уведомления от нодов:
|
|||
- ❌ Не хостит голосовые каналы
|
||||
- ❌ Не хостит текстовые каналы
|
||||
- ❌ Не хранит историю сообщений
|
||||
- ❌ Не управляет сессиями — только направляет
|
||||
- ❌ Не заменяет self-hosted нод — только дополняет
|
||||
- ❌ Не управляет сессиями - только направляет
|
||||
- ❌ Не заменяет self-hosted нод - только дополняет
|
||||
|
||||
Hub — это телефонная книга и коммутатор, а не сам телефон.
|
||||
Hub - это телефонная книга и коммутатор, а не сам телефон.
|
||||
|
||||
## Деплой
|
||||
|
||||
Central Hub — отдельный бинарник (например, `vnox-hub`).
|
||||
Central Hub - отдельный бинарник (например, `vnox-hub`).
|
||||
Работает на HTTP(S) / WebSocket, минимальные требования.
|
||||
Может быть поднят кем угодно, не обязательно авторами VNOX.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ Three related but distinct features:
|
|||
| **Stickers** | 512×512 max | PNG, GIF, WebP, Lottie† | Per-guild + per-user | Server filesystem |
|
||||
| **GIF Picker** | External | GIF (Tenor/Giphy API) | Per-server config | Proxy via server |
|
||||
|
||||
† Lottie — future, not Phase 2.
|
||||
† Lottie - future, not Phase 2.
|
||||
|
||||
### Difference from unicode reactions
|
||||
|
||||
|
|
@ -171,7 +171,7 @@ Server sends on guild join. Client caches locally.
|
|||
}
|
||||
```
|
||||
|
||||
Image bytes are NOT included — client lazy-loads via `EMOJI_DATA` + `EMOJI_DATA_RESP`.
|
||||
Image bytes are NOT included - client lazy-loads via `EMOJI_DATA` + `EMOJI_DATA_RESP`.
|
||||
|
||||
### EMOJI_ADD (0x0071)
|
||||
|
||||
|
|
@ -205,7 +205,7 @@ Server removes file + DB row, broadcasts `EMOJI_UPDATE` with `deleted: true`.
|
|||
|
||||
### EMOJI_DATA / EMOJI_DATA_RESP (0x0078/0x0079)
|
||||
|
||||
Lazy-load pattern — client requests image bytes when first rendering an emoji.
|
||||
Lazy-load pattern - client requests image bytes when first rendering an emoji.
|
||||
|
||||
Request:
|
||||
```json
|
||||
|
|
@ -408,36 +408,36 @@ When two nodes federate:
|
|||
### GIF proxy
|
||||
|
||||
GIF search always goes through the user's home node.
|
||||
The server proxies to Tenor/Giphy — no federation needed for GIF results (they're just URLs).
|
||||
The server proxies to Tenor/Giphy - no federation needed for GIF results (they're just URLs).
|
||||
|
||||
---
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Phase 1.3 — Unicode reactions only
|
||||
### Phase 1.3 - Unicode reactions only
|
||||
- `message_reactions` table (already designed)
|
||||
- Unicode emoji picker in client
|
||||
- No custom emoji, no stickers, no GIF
|
||||
|
||||
### Phase 2 — Custom emoji
|
||||
### Phase 2 - Custom emoji
|
||||
- Add `guild_emojis` table
|
||||
- Add packet types 0x0070–0x0073, 0x0078–0x0079
|
||||
- Client emoji picker with custom tab
|
||||
- Admin upload/delete UI
|
||||
- No federation support yet
|
||||
|
||||
### Phase 2 — Stickers
|
||||
### Phase 2 - Stickers
|
||||
- Add `guild_stickers` + `user_stickers` tables
|
||||
- Add packet types 0x0074–0x0077
|
||||
- Sticker panel in client
|
||||
- New message content_type: "sticker"
|
||||
|
||||
### Phase 2 — GIF picker
|
||||
### Phase 2 - GIF picker
|
||||
- Add Tenor/Giphy config to gateway
|
||||
- Add packet types 0x0080–0x0081
|
||||
- GIF picker widget in client
|
||||
- Server-side proxy (API key protection)
|
||||
|
||||
### Phase 3 — Federation
|
||||
### Phase 3 - Federation
|
||||
- Cross-node emoji/sticker sync
|
||||
- `@node` suffix in references
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Direct Messages — Phase 1.1
|
||||
# Direct Messages - Phase 1.1
|
||||
|
||||
## Goal
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Private Mode — Phase 1.1
|
||||
# Private Mode - Phase 1.1
|
||||
|
||||
## Goal
|
||||
|
||||
|
|
@ -81,4 +81,4 @@ if !config.federation.enabled {
|
|||
| `private` | Fully isolated. Like OwnCord. Safe for personal LAN/ home server. |
|
||||
| `federated` | Can discover, peer, and sync with other VNOX nodes. Like Matrix. |
|
||||
|
||||
Default is `private` — opt-in to federation.
|
||||
Default is `private` - opt-in to federation.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Slint UI Migration — Phase 1.1
|
||||
# Slint UI Migration - Phase 1.1
|
||||
|
||||
## Current state
|
||||
|
||||
|
|
@ -18,23 +18,23 @@ Desktop client uses **egui** (immediate-mode GUI). Functional but:
|
|||
| Chat app fit | ❌ feel | ✅ modern | ✅ modern |
|
||||
| Learning curve | Low | Medium | Medium |
|
||||
|
||||
**Recommendation:** Slint — declarative DSL makes UI easier to iterate,
|
||||
**Recommendation:** Slint - declarative DSL makes UI easier to iterate,
|
||||
built-in design system produces professional look out of the box.
|
||||
|
||||
## Migration strategy
|
||||
|
||||
### Stage 1 — Side-by-side (week 1)
|
||||
### Stage 1 - Side-by-side (week 1)
|
||||
- Embed Slint canvas alongside existing egui panels
|
||||
- Rewrite sidebar (server list + channel list) in Slint first
|
||||
- Keep chat area in egui
|
||||
- Verify data flow works between both frameworks
|
||||
|
||||
### Stage 2 — Chat panel (week 2)
|
||||
### Stage 2 - Chat panel (week 2)
|
||||
- Rewrite channel chat (text messages + input bar) in Slint
|
||||
- Rewrite voice panel in Slint
|
||||
- Remove egui chat dependency
|
||||
|
||||
### Stage 3 — Settings & polish (week 3)
|
||||
### Stage 3 - Settings & polish (week 3)
|
||||
- Rewrite settings window in Slint
|
||||
- Add dark/light theme toggle
|
||||
- Add accent color picker
|
||||
|
|
@ -102,7 +102,7 @@ client/src/ui/
|
|||
|
||||
## Open questions
|
||||
|
||||
- Slint's rendering backend (gl/winit/software) — test on all target platforms
|
||||
- Slint's rendering backend (gl/winit/software) - test on all target platforms
|
||||
- Slint's text input / rich text support for chat messages (emoji, markdown?)
|
||||
- Whether to remove egui entirely or keep for specific panels (e.g. overlay)
|
||||
- Licensing: Slint is LGPL (compatible with GPL-3.0 for now, but check for future)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Video, Camera & Screen Share — Design Audit
|
||||
# Video, Camera & Screen Share - Design Audit
|
||||
|
||||
> Audits `docs/05-features/video.md` against the current codebase and roadmap.
|
||||
> Each gap has: problem, recommendation, priority, target file.
|
||||
|
|
@ -12,19 +12,19 @@ But critical details are missing for actual implementation. This audit fills tho
|
|||
|
||||
---
|
||||
|
||||
## 1. Video-node — Integration with existing architecture
|
||||
## 1. Video-node - Integration with existing architecture
|
||||
|
||||
### Gap 1.1 — Video-node lifecycle
|
||||
### Gap 1.1 - Video-node lifecycle
|
||||
|
||||
**Problem:** `video.md` says "separate binary or embedded in voice-node" but doesn't decide. Voice-node has just been refactored with jitter buffer — video needs the same relay pattern.
|
||||
**Problem:** `video.md` says "separate binary or embedded in voice-node" but doesn't decide. Voice-node has just been refactored with jitter buffer - video needs the same relay pattern.
|
||||
|
||||
**Recommendation:** Separate binary (`vnox-video-node`). Video frames are fundamentally different from voice packets (keyframes, fragmentation, higher bandwidth). Keeping them separate avoids coupling and allows independent scaling.
|
||||
|
||||
**Priority:** P0 — must decide before writing a line of code.
|
||||
**Priority:** P0 - must decide before writing a line of code.
|
||||
|
||||
**Target:** `video.md` — add "Deployment" section.
|
||||
**Target:** `video.md` - add "Deployment" section.
|
||||
|
||||
### Gap 1.2 — Channel membership signalling
|
||||
### Gap 1.2 - Channel membership signalling
|
||||
|
||||
**Problem:** Video-node needs to know which users are in which channel. Currently this is managed by gateway. How does video-node learn membership?
|
||||
|
||||
|
|
@ -40,27 +40,27 @@ But critical details are missing for actual implementation. This audit fills tho
|
|||
}
|
||||
```
|
||||
|
||||
**Priority:** P0 — video relay can't work without knowing recipients.
|
||||
**Priority:** P0 - video relay can't work without knowing recipients.
|
||||
|
||||
**Target:** `video.md` — add "Gateway integration" section.
|
||||
**Target:** `video.md` - add "Gateway integration" section.
|
||||
|
||||
### Gap 1.3 — No encryption spec
|
||||
### Gap 1.3 - No encryption spec
|
||||
|
||||
**Problem:** Voice packets are encrypted with ChaCha20-Poly1305. Video packets are not addressed — are they encrypted? With what keys?
|
||||
**Problem:** Voice packets are encrypted with ChaCha20-Poly1305. Video packets are not addressed - are they encrypted? With what keys?
|
||||
|
||||
**Recommendation:** Same ChaCha20-Poly1305 AEAD as voice. Each video session gets ephemeral keys derived via the same X25519 ECDH + HKDF-SHA256 path used for voice. Video-node does NOT decrypt (transparent relay, same as voice-node).
|
||||
|
||||
The session key exchange happens during `HELLO` handshake — add a `video_key` field alongside the existing `voice_key`.
|
||||
The session key exchange happens during `HELLO` handshake - add a `video_key` field alongside the existing `voice_key`.
|
||||
|
||||
**Priority:** P0 — security regression if video is plaintext.
|
||||
**Priority:** P0 - security regression if video is plaintext.
|
||||
|
||||
**Target:** `video.md` — add "Encryption" section; `docs/02-protocol/security.md` — add video encryption note.
|
||||
**Target:** `video.md` - add "Encryption" section; `docs/02-protocol/security.md` - add video encryption note.
|
||||
|
||||
---
|
||||
|
||||
## 2. Camera Capture — Missing details
|
||||
## 2. Camera Capture - Missing details
|
||||
|
||||
### Gap 2.1 — nokhwa API surface and fallback
|
||||
### Gap 2.1 - nokhwa API surface and fallback
|
||||
|
||||
**Problem:** `video.md` mentions `nokhwa` but not the concrete API or what happens on platforms where it doesn't work (Linux v4l2 permissions, Wayland).
|
||||
|
||||
|
|
@ -86,11 +86,11 @@ pub struct StreamConfig {
|
|||
|
||||
Make `nokhwa` a feature gate (same pattern as `rnnoise` for voice).
|
||||
|
||||
**Priority:** P0 — blocks camera implementation.
|
||||
**Priority:** P0 - blocks camera implementation.
|
||||
|
||||
**Target:** New file: `client/src/video/capture.rs` (spec in `video.md`).
|
||||
|
||||
### Gap 2.2 — Device hotplug / switch
|
||||
### Gap 2.2 - Device hotplug / switch
|
||||
|
||||
**Problem:** No mention of what happens when camera is unplugged/replugged or user switches camera mid-call.
|
||||
|
||||
|
|
@ -106,11 +106,11 @@ enum CaptureEvent {
|
|||
|
||||
Client handles `DeviceLost` by showing placeholder tile. User can select new device in settings without leaving call.
|
||||
|
||||
**Priority:** P1 — quality of life, not MVP blocker.
|
||||
**Priority:** P1 - quality of life, not MVP blocker.
|
||||
|
||||
**Target:** `video.md` — add "Device lifecycle" to capture section.
|
||||
**Target:** `video.md` - add "Device lifecycle" to capture section.
|
||||
|
||||
### Gap 2.3 — Resolution / FPS negotiation
|
||||
### Gap 2.3 - Resolution / FPS negotiation
|
||||
|
||||
**Problem:** `video.md` says "720p default, 30 fps" but doesn't say how clients agree on parameters. Different clients have different cameras.
|
||||
|
||||
|
|
@ -124,15 +124,15 @@ max_fps = 30
|
|||
max_bitrate_kbps = 2500
|
||||
```
|
||||
|
||||
**Priority:** P1 — needed before multi-user video testing.
|
||||
**Priority:** P1 - needed before multi-user video testing.
|
||||
|
||||
**Target:** `video.md` — add "Capability negotiation" section.
|
||||
**Target:** `video.md` - add "Capability negotiation" section.
|
||||
|
||||
---
|
||||
|
||||
## 3. Screen Share — Almost entirely missing
|
||||
## 3. Screen Share - Almost entirely missing
|
||||
|
||||
### Gap 3.1 — Screen capture crate
|
||||
### Gap 3.1 - Screen capture crate
|
||||
|
||||
**Problem:** `video.md` mentions "Screen sharing (desktop capture)" as one bullet in Phase 2, no detail.
|
||||
|
||||
|
|
@ -140,18 +140,18 @@ max_bitrate_kbps = 2500
|
|||
|
||||
| Platform | Crate | Notes |
|
||||
|----------|-------|-------|
|
||||
| Windows | `windows-capture` or DXGI via `winapi` | DXGI Desktop Duplication API — best perf |
|
||||
| Windows | `windows-capture` or DXGI via `winapi` | DXGI Desktop Duplication API - best perf |
|
||||
| Linux X11 | `x11cap` or raw XSHM | X11 only, no Wayland |
|
||||
| Linux Wayland | `pipewire` via `pw-video` or xdg-desktop-portal | Portal is the standard path |
|
||||
| macOS | `screencapturekit` via objc bindings or `CGDisplay` | ScreenCaptureKit requires macOS 13+ |
|
||||
|
||||
Abstract behind same `VideoSource` trait as camera. Two implementations: `CameraSource`, `ScreenSource`.
|
||||
|
||||
**Priority:** P1 — needed for Phase 2 screen share.
|
||||
**Priority:** P1 - needed for Phase 2 screen share.
|
||||
|
||||
**Target:** New doc: `docs/05-features/screen-share.md` or extend `video.md` Phase 2.
|
||||
|
||||
### Gap 3.2 — Window/display selection UI
|
||||
### Gap 3.2 - Window/display selection UI
|
||||
|
||||
**Problem:** User needs to pick which screen or window to share. No UI spec.
|
||||
|
||||
|
|
@ -176,13 +176,13 @@ Abstract behind same `VideoSource` trait as camera. Two implementations: `Camera
|
|||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Priority:** P1 — UX blocker for screen share.
|
||||
**Priority:** P1 - UX blocker for screen share.
|
||||
|
||||
**Target:** `video.md` — add "Screen share UI" section.
|
||||
**Target:** `video.md` - add "Screen share UI" section.
|
||||
|
||||
### Gap 3.3 — System audio capture with screen
|
||||
### Gap 3.3 - System audio capture with screen
|
||||
|
||||
**Problem:** "Share system audio" checkbox in the mockup — how? No mention in video.md.
|
||||
**Problem:** "Share system audio" checkbox in the mockup - how? No mention in video.md.
|
||||
|
||||
**Recommendation:**
|
||||
- Windows: WASAPI loopback (`cpal` supports it with `loopback` config)
|
||||
|
|
@ -191,11 +191,11 @@ Abstract behind same `VideoSource` trait as camera. Two implementations: `Camera
|
|||
|
||||
Feature-gate it: `--features screen-audio`. Not all platforms support it cleanly.
|
||||
|
||||
**Priority:** P2 — nice to have, not MVP.
|
||||
**Priority:** P2 - nice to have, not MVP.
|
||||
|
||||
**Target:** `video.md` Phase 2 — add "System audio" bullet.
|
||||
**Target:** `video.md` Phase 2 - add "System audio" bullet.
|
||||
|
||||
### Gap 3.4 — Screen share FPS strategy
|
||||
### Gap 3.4 - Screen share FPS strategy
|
||||
|
||||
**Problem:** Screen share has different FPS needs than camera. Coding: 15fps is fine. Gaming: 60fps needed.
|
||||
|
||||
|
|
@ -203,15 +203,15 @@ Feature-gate it: `--features screen-audio`. Not all platforms support it cleanly
|
|||
|
||||
Default: 15fps for screen share, 30fps for camera.
|
||||
|
||||
**Priority:** P2 — optimization, not MVP.
|
||||
**Priority:** P2 - optimization, not MVP.
|
||||
|
||||
**Target:** `video.md` — add to Phase 2.
|
||||
**Target:** `video.md` - add to Phase 2.
|
||||
|
||||
---
|
||||
|
||||
## 4. Interaction Model — Video + Voice
|
||||
## 4. Interaction Model - Video + Voice
|
||||
|
||||
### Gap 4.1 — Video tied to voice channel
|
||||
### Gap 4.1 - Video tied to voice channel
|
||||
|
||||
**Problem:** `video.md` implies video is in the same channel as voice. What if users want video-only (no voice) or voice-only (no video)?
|
||||
|
||||
|
|
@ -226,25 +226,25 @@ Channel membership = voice channel membership. Video is additive.
|
|||
|
||||
New voice state: `VOICE_STATE` gets a `video: bool` field.
|
||||
|
||||
**Priority:** P0 — architectural decision, affects protocol design.
|
||||
**Priority:** P0 - architectural decision, affects protocol design.
|
||||
|
||||
**Target:** `video.md` — add "Interaction model" section; `docs/02-protocol/packets.md` — extend VOICE_STATE.
|
||||
**Target:** `video.md` - add "Interaction model" section; `docs/02-protocol/packets.md` - extend VOICE_STATE.
|
||||
|
||||
### Gap 4.2 — Video without voice
|
||||
### Gap 4.2 - Video without voice
|
||||
|
||||
**Problem:** Some users may want to watch a stream without transmitting audio. Current model requires voice channel join.
|
||||
|
||||
**Recommendation:** Phase 1: allow joining voice channel muted+deafened to watch video. Phase 3: separate "watch-only" mode (VIEW permission).
|
||||
|
||||
**Priority:** P2 — Phase 1 workaround is acceptable.
|
||||
**Priority:** P2 - Phase 1 workaround is acceptable.
|
||||
|
||||
**Target:** `video.md` — add "Watch-only mode" to Phase 3.
|
||||
**Target:** `video.md` - add "Watch-only mode" to Phase 3.
|
||||
|
||||
---
|
||||
|
||||
## 5. Performance & Quality
|
||||
|
||||
### Gap 5.1 — Bitrate adaptation
|
||||
### Gap 5.1 - Bitrate adaptation
|
||||
|
||||
**Problem:** No mechanism to adjust video bitrate based on network conditions. Voice has jitter buffer adaptation, video has nothing.
|
||||
|
||||
|
|
@ -265,11 +265,11 @@ Video-node sends periodic `VIDEO_STATS` with per-receiver loss rates:
|
|||
}
|
||||
```
|
||||
|
||||
**Priority:** P1 — needed for real-world use.
|
||||
**Priority:** P1 - needed for real-world use.
|
||||
|
||||
**Target:** `video.md` — add "Adaptive bitrate" section.
|
||||
**Target:** `video.md` - add "Adaptive bitrate" section.
|
||||
|
||||
### Gap 5.2 — Simulcast design
|
||||
### Gap 5.2 - Simulcast design
|
||||
|
||||
**Problem:** "Simulcast" is listed as Phase 3 with no design. Different receivers have different bandwidth.
|
||||
|
||||
|
|
@ -283,13 +283,13 @@ Sender → encode: 1080p (4Mbps) + 720p (1.5Mbps) + 360p (500kbps)
|
|||
1080p→A 720p→B 360p→C (based on per-receiver stats)
|
||||
```
|
||||
|
||||
Requires VP9 SVC or H.264 simulcast. Complex — Phase 3 is correct.
|
||||
Requires VP9 SVC or H.264 simulcast. Complex - Phase 3 is correct.
|
||||
|
||||
**Priority:** P2 — Phase 3, but design it now to avoid architecture lock-in.
|
||||
**Priority:** P2 - Phase 3, but design it now to avoid architecture lock-in.
|
||||
|
||||
**Target:** `video.md` Phase 3 — expand "Simulcast" bullet.
|
||||
**Target:** `video.md` Phase 3 - expand "Simulcast" bullet.
|
||||
|
||||
### Gap 5.3 — Keyframe interval
|
||||
### Gap 5.3 - Keyframe interval
|
||||
|
||||
**Problem:** New viewers joining mid-stream need a keyframe to start decoding. Video.md doesn't mention keyframe strategy.
|
||||
|
||||
|
|
@ -301,15 +301,15 @@ Requires VP9 SVC or H.264 simulcast. Complex — Phase 3 is correct.
|
|||
|
||||
Add `REQUEST_KEYFRAME` packet for explicit PLI (Picture Loss Indication).
|
||||
|
||||
**Priority:** P1 — new viewers can't see video without keyframe.
|
||||
**Priority:** P1 - new viewers can't see video without keyframe.
|
||||
|
||||
**Target:** `video.md` — add "Keyframe handling" section.
|
||||
**Target:** `video.md` - add "Keyframe handling" section.
|
||||
|
||||
---
|
||||
|
||||
## 6. Missing Protocol Details
|
||||
|
||||
### Gap 6.1 — No packet fragmentation spec
|
||||
### Gap 6.1 - No packet fragmentation spec
|
||||
|
||||
**Problem:** Video frames can be 50-100KB. UDP MTU is ~1400 bytes. Video.md says "Packetize into MTU-friendly chunks" but doesn't define the fragmentation protocol.
|
||||
|
||||
|
|
@ -325,27 +325,27 @@ Each video frame:
|
|||
3. Last chunk sets `LAST_FRAG` flag
|
||||
4. Receiver reassembles before decode
|
||||
|
||||
New packet type: `VIDEO_FRAME_FRAGMENT (0x0012)` — separate from unfragmented `VIDEO_FRAME (0x0011)`.
|
||||
New packet type: `VIDEO_FRAME_FRAGMENT (0x0012)` - separate from unfragmented `VIDEO_FRAME (0x0011)`.
|
||||
|
||||
Wait — `VIDEO_FRAME` ID isn't assigned in the packet registry yet. Register:
|
||||
Wait - `VIDEO_FRAME` ID isn't assigned in the packet registry yet. Register:
|
||||
|
||||
```
|
||||
0x0011 VIDEO_FRAME single (small) video frame, no fragmentation
|
||||
0x0012 VIDEO_FRAME_FRAG fragment of a larger video frame
|
||||
```
|
||||
|
||||
**Priority:** P0 — video can't work over UDP without fragmentation.
|
||||
**Priority:** P0 - video can't work over UDP without fragmentation.
|
||||
|
||||
**Target:** `video.md` — replace packet format section; `docs/02-protocol/packets.md` — add 0x0011/0x0012 to registry.
|
||||
**Target:** `video.md` - replace packet format section; `docs/02-protocol/packets.md` - add 0x0011/0x0012 to registry.
|
||||
|
||||
### Gap 6.2 — No RTCP-like receiver reports
|
||||
### Gap 6.2 - No RTCP-like receiver reports
|
||||
|
||||
**Problem:** Sender has no feedback about what receivers are experiencing. Voice has implicit feedback (jitter buffer stats), video needs explicit.
|
||||
|
||||
**Recommendation:** Minimal receiver report packet:
|
||||
|
||||
```json
|
||||
// 0x0013 VIDEO_RECEIVER_REPORT — client → video-node → sender
|
||||
// 0x0013 VIDEO_RECEIVER_REPORT - client → video-node → sender
|
||||
{
|
||||
"frame_seq": 1042,
|
||||
"loss_cumulative": 15,
|
||||
|
|
@ -357,21 +357,21 @@ Wait — `VIDEO_FRAME` ID isn't assigned in the packet registry yet. Register:
|
|||
|
||||
Client sends every 1 second. Video-node aggregates and forwards to sender.
|
||||
|
||||
**Priority:** P1 — needed for ABR and quality adaptation.
|
||||
**Priority:** P1 - needed for ABR and quality adaptation.
|
||||
|
||||
**Target:** `video.md` — add "Receiver reports" section.
|
||||
**Target:** `video.md` - add "Receiver reports" section.
|
||||
|
||||
---
|
||||
|
||||
## 7. Guild Integration (Phase 1.2)
|
||||
|
||||
### Gap 7.1 — Video permissions already defined
|
||||
### Gap 7.1 - Video permissions already defined
|
||||
|
||||
**Status:** ✅ `STREAM` (bit 18) permission is already in the community model (`docs/07-community-model.md`). Covers both camera and screen share.
|
||||
|
||||
No gap here — just implement the check in gateway when processing `VIDEO_SESSION_JOIN`.
|
||||
No gap here - just implement the check in gateway when processing `VIDEO_SESSION_JOIN`.
|
||||
|
||||
### Gap 7.2 — Channel-level video settings
|
||||
### Gap 7.2 - Channel-level video settings
|
||||
|
||||
**Problem:** Guild admins may want to disable video in certain channels (text-only channels, AFK channel).
|
||||
|
||||
|
|
@ -383,37 +383,37 @@ ALTER TABLE channels ADD COLUMN video_allowed BOOLEAN NOT NULL DEFAULT 1;
|
|||
|
||||
Gateway rejects `VIDEO_SESSION_JOIN` if `video_allowed = false`.
|
||||
|
||||
**Priority:** P1 — admin control expected by guild owners.
|
||||
**Priority:** P1 - admin control expected by guild owners.
|
||||
|
||||
**Target:** `docs/10-database.md` — add column to channels table; `docs/07-community-model.md` — add `video_allowed` to channel settings.
|
||||
**Target:** `docs/10-database.md` - add column to channels table; `docs/07-community-model.md` - add `video_allowed` to channel settings.
|
||||
|
||||
---
|
||||
|
||||
## 8. Mobile Considerations (Phase 3)
|
||||
|
||||
### Gap 8.1 — Front/back camera switch
|
||||
### Gap 8.1 - Front/back camera switch
|
||||
|
||||
**Problem:** Mobile video.md doesn't mention camera switching at all.
|
||||
|
||||
**Recommendation:** Mobile client sends `CAMERA_SWITCH` event to video-node — no protocol change needed, just local capture change. New frame stream with `camera: front|back` metadata.
|
||||
**Recommendation:** Mobile client sends `CAMERA_SWITCH` event to video-node - no protocol change needed, just local capture change. New frame stream with `camera: front|back` metadata.
|
||||
|
||||
**Priority:** P2 — Phase 3.
|
||||
**Priority:** P2 - Phase 3.
|
||||
|
||||
**Target:** `docs/04-clients/mobile.md` (doesn't exist yet — create in Phase 3).
|
||||
**Target:** `docs/04-clients/mobile.md` (doesn't exist yet - create in Phase 3).
|
||||
|
||||
### Gap 8.2 — Self preview
|
||||
### Gap 8.2 - Self preview
|
||||
|
||||
**Problem:** Desktop video grid has "Self-view (small, picture-in-picture corner)". On mobile the self-view is more important (front camera framing).
|
||||
|
||||
**Recommendation:** Mobile layout: self-view full-width at top, other participants in scrollable grid below. Toggle button to swap.
|
||||
|
||||
**Priority:** P2 — Phase 3.
|
||||
**Priority:** P2 - Phase 3.
|
||||
|
||||
**Target:** `docs/04-clients/mobile.md`.
|
||||
|
||||
---
|
||||
|
||||
## Gap Summary — What to Add Before Implementation
|
||||
## Gap Summary - What to Add Before Implementation
|
||||
|
||||
| # | Gap | Priority | Target file |
|
||||
|---|-----|----------|-------------|
|
||||
|
|
@ -441,13 +441,13 @@ Gateway rejects `VIDEO_SESSION_JOIN` if `video_allowed = false`.
|
|||
|
||||
## Recommended Implementation Order
|
||||
|
||||
1. **Update `video.md`** — incorporate P0 and P1 gaps from this audit into the design doc
|
||||
2. **Register packet types** — 0x0011–0x0013 in `packets.md`
|
||||
3. **Video-node prototype** — separate binary, UDP relay, no encoding/decoding (transparent relay like voice)
|
||||
4. **Gateway ↔ video-node signalling** — internal TCP control channel for membership
|
||||
5. **Client camera capture** — `VideoSource` trait + `NokhwaSource` behind feature gate
|
||||
6. **Fragmentation** — implement FRAGMENTED/LAST_FRAG for video frames
|
||||
7. **Grid UI** — basic 2×2 grid in client
|
||||
8. **Encryption** — ChaCha20-Poly1305 for video frames
|
||||
9. **ABR + receiver reports** — iterate toward production quality
|
||||
10. **Screen share** — `ScreenSource` implementation + picker UI
|
||||
1. **Update `video.md`** - incorporate P0 and P1 gaps from this audit into the design doc
|
||||
2. **Register packet types** - 0x0011–0x0013 in `packets.md`
|
||||
3. **Video-node prototype** - separate binary, UDP relay, no encoding/decoding (transparent relay like voice)
|
||||
4. **Gateway ↔ video-node signalling** - internal TCP control channel for membership
|
||||
5. **Client camera capture** - `VideoSource` trait + `NokhwaSource` behind feature gate
|
||||
6. **Fragmentation** - implement FRAGMENTED/LAST_FRAG for video frames
|
||||
7. **Grid UI** - basic 2×2 grid in client
|
||||
8. **Encryption** - ChaCha20-Poly1305 for video frames
|
||||
9. **ABR + receiver reports** - iterate toward production quality
|
||||
10. **Screen share** - `ScreenSource` implementation + picker UI
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Video Chat — Architecture
|
||||
# Video Chat - Architecture
|
||||
|
||||
## Goal
|
||||
|
||||
|
|
@ -14,9 +14,9 @@ Current (voice only):
|
|||
└─ Client: mic → Opus → UDP → voice-node → UDP → client → playback
|
||||
|
||||
With video:
|
||||
├─ Gateway: TCP (signaling) — UNCHANGED
|
||||
├─ Voice-node: UDP (Opus voice) — UNCHANGED
|
||||
├─ Video-node: UDP (H.264/VP9 relay) — NEW
|
||||
├─ Gateway: TCP (signaling) - UNCHANGED
|
||||
├─ Voice-node: UDP (Opus voice) - UNCHANGED
|
||||
├─ Video-node: UDP (H.264/VP9 relay) - NEW
|
||||
│ └─ Receives encoded frames, relays to channel members
|
||||
└─ Client: mic → Opus → UDP → voice-node
|
||||
webcam → H.264 → UDP → video-node ← UDP → decode → display grid
|
||||
|
|
@ -29,7 +29,7 @@ New optional component, separate binary or embedded in voice-node.
|
|||
### Responsibilities
|
||||
- Receive encoded video frames over UDP
|
||||
- Relay to all other clients in the same voice channel
|
||||
- No transcoding (relay only — CPU efficient)
|
||||
- No transcoding (relay only - CPU efficient)
|
||||
- Max resolution / bitrate per channel configurable
|
||||
|
||||
### Packet format
|
||||
|
|
@ -60,10 +60,10 @@ New file: `client/src/video/capture.rs`
|
|||
|
||||
### Dependencies
|
||||
|
||||
- `nokhwa` — cross-platform camera capture (Rust)
|
||||
- `ffmpeg-next` or `rav1e` — H.264/VP9 encoding
|
||||
- `nokhwa` - cross-platform camera capture (Rust)
|
||||
- `ffmpeg-next` or `rav1e` - H.264/VP9 encoding
|
||||
|
||||
## Client UI — Video grid
|
||||
## Client UI - Video grid
|
||||
|
||||
New component: `client/src/ui/video.rs`
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue