add client docs, README, CHANGELOG
This commit is contained in:
parent
98ad2c4d75
commit
c465f53307
GPG key ID:
253067914055423B
16 changed files with 2933 additions and 0 deletions
405
docs/05-features/admin-panel.md
Normal file
405
docs/05-features/admin-panel.md
Normal file
|
|
@ -0,0 +1,405 @@
|
|||
# Admin Panel — Design
|
||||
|
||||
> Target: Phase 1.1 (core) → Phase 1.2 (guilds, roles) → Phase 2 (plugins, bot permissions)
|
||||
|
||||
## Separation of concerns
|
||||
|
||||
| Layer | Config file (TOML) | Admin panel (GUI) |
|
||||
|-------|-------------------|-------------------|
|
||||
| **Infrastructure** | `bind`, `ports`, `data_dir`, TLS certs | — |
|
||||
| **Node identity** | `name`, `address` | Display only |
|
||||
| **Storage** | `backend`, `sqlite_path`, `retention` | — |
|
||||
| **Security** | federation gate, rate limits, session timeout | Ban list, mute, kick |
|
||||
| **Runtime state** | — | Channels, roles, members, invites, audit log |
|
||||
| **Plugins** | Plugin enable/disable | Permission keys, commands |
|
||||
|
||||
**Rule of thumb:** If it needs a server restart → TOML. If it can apply live → GUI.
|
||||
|
||||
---
|
||||
|
||||
## Permission System
|
||||
|
||||
Designed for both built-in actions and plugin/bot commands (inspired by LuckPerms).
|
||||
|
||||
### Built-in permission keys
|
||||
|
||||
```
|
||||
vnox.admin.manage_channels — create, edit, delete channels
|
||||
vnox.admin.manage_roles — create, edit, delete roles
|
||||
vnox.admin.manage_members — kick, ban, mute, timeout
|
||||
vnox.admin.manage_invites — create, delete invites
|
||||
vnox.admin.manage_guild — edit guild name, icon, settings
|
||||
vnox.admin.view_audit_log — read audit log
|
||||
|
||||
vnox.channel.read — view channel
|
||||
vnox.channel.send — send messages
|
||||
vnox.channel.voice_connect — join voice channel
|
||||
vnox.channel.voice_speak — speak in voice (push-to-talk)
|
||||
vnox.channel.voice_mute_members — server-mute others
|
||||
|
||||
vnox.dm.send — send direct messages
|
||||
vnox.friend.request — send friend requests
|
||||
vnox.presence.view — see online status
|
||||
```
|
||||
|
||||
### Custom permission keys (plugin/bot commands)
|
||||
|
||||
Plugins register their permissions at load time via the RPC API:
|
||||
|
||||
```
|
||||
mybot.command.greet — /greet command
|
||||
mybot.command.play — /play music
|
||||
mybot.command.ban_override — override sub-command
|
||||
```
|
||||
|
||||
**Wildcard support:**
|
||||
```
|
||||
vnox.admin.* — all admin permissions
|
||||
vnox.channel.* — all channel permissions
|
||||
mybot.* — all commands from mybot
|
||||
* — everything (owner only)
|
||||
```
|
||||
|
||||
### How it works
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Role Editor — "Moderator" │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ Permission keys: │
|
||||
│ │
|
||||
│ ┌─ Built-in ──────────────────────────────┐ │
|
||||
│ │ ☑ vnox.admin.manage_channels │ │
|
||||
│ │ ☑ vnox.admin.manage_members │ │
|
||||
│ │ ☐ vnox.admin.manage_roles │ │
|
||||
│ │ ☑ vnox.admin.view_audit_log │ │
|
||||
│ │ ☑ vnox.channel.* │ │
|
||||
│ └──────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Plugin permissions ────────────────────┐ │
|
||||
│ │ ☑ mybot.command.greet │ │
|
||||
│ │ ☐ mybot.command.play │ │
|
||||
│ │ ☑ mybot.command.* │ │
|
||||
│ └──────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Custom key ────────────────────────────┐ │
|
||||
│ │ [ Type a permission key... ] [ +Add ] │ │
|
||||
│ └──────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Cancel] [Save] │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Resolution order (highest priority wins):**
|
||||
1. Owner — overrides everything
|
||||
2. Channel override (user-specific) — allow/deny
|
||||
3. Channel override (role-specific) — allow/deny
|
||||
4. Role hierarchy (top role has higher priority)
|
||||
5. @everyone default
|
||||
6. Server-wide default
|
||||
|
||||
Each override is `(allow_mask, deny_mask)` — deny always beats allow at the same level.
|
||||
|
||||
### Database
|
||||
|
||||
Already specified in `docs/10-database.md`:
|
||||
|
||||
```
|
||||
roles(id, guild_id, name, color, permissions BITMASK, position)
|
||||
channel_overrides(id, channel_id, target_id, target_type, allow_mask, deny_mask)
|
||||
member_roles(member_guild_id, member_user_id, role_id)
|
||||
```
|
||||
|
||||
**Custom permission keys** are stored separately:
|
||||
|
||||
```sql
|
||||
CREATE TABLE permission_keys (
|
||||
id TEXT PRIMARY KEY,
|
||||
guild_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL, -- e.g. "mybot.command.greet"
|
||||
description TEXT, -- e.g. "Allow using /greet command"
|
||||
plugin_id TEXT, -- which plugin registered it, null = built-in
|
||||
created_at TIMESTAMP NOT NULL
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX idx_perm_keys_guild_name ON permission_keys(guild_id, name);
|
||||
```
|
||||
|
||||
Role grants use the same `roles.permissions` bitmask for built-in keys (first 128 bits).
|
||||
For custom/plugin keys beyond 128, they're stored in a separate table:
|
||||
|
||||
```sql
|
||||
CREATE TABLE role_perm_grants (
|
||||
role_id TEXT NOT NULL,
|
||||
permission_key TEXT NOT NULL, -- "mybot.command.greet"
|
||||
granted INTEGER NOT NULL, -- 1 = allow, 0 = deny
|
||||
PRIMARY KEY (role_id, permission_key),
|
||||
FOREIGN KEY(role_id) REFERENCES roles(id)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Channel Management
|
||||
|
||||
### Create Channel
|
||||
|
||||
```
|
||||
Right-click channel list → "Create Channel"
|
||||
or
|
||||
Server Settings → Channels → [+ Add]
|
||||
```
|
||||
|
||||
**Modal:**
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
│ Create Channel │
|
||||
├──────────────────────────────────────┤
|
||||
│ │
|
||||
│ Name [_________________] │
|
||||
│ │
|
||||
│ Type ○ Text ● Voice │
|
||||
│ │
|
||||
│ Category [General ▼] [None] │
|
||||
│ │
|
||||
│ Topic [_________________] │
|
||||
│ │
|
||||
│ ┌─ Permissions ▾ ─────────────┐ │
|
||||
│ │ @everyone │ │
|
||||
│ │ ☑ View Channel │ │
|
||||
│ │ ☑ Send Messages │ │
|
||||
│ │ ☐ Voice Connect │ │
|
||||
│ │ │ │
|
||||
│ │ @admin │ │
|
||||
│ │ ☑ Everything │ │
|
||||
│ └──────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Cancel] [Create] │
|
||||
└──────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Edit Channel
|
||||
|
||||
Double-click channel or right-click → "Edit Channel".
|
||||
|
||||
Same modal pre-filled, plus:
|
||||
- **Name change** — updates slug
|
||||
- **Category change** — moves channel
|
||||
- **Drag position** — reorder inside category
|
||||
- **Delete** — with confirmation (soft delete in DB)
|
||||
|
||||
### Drag & Drop
|
||||
|
||||
```
|
||||
Channels Server: My Server
|
||||
─────────────────────────────────────────────
|
||||
▶ TEXT CHANNELS [+]
|
||||
○ general
|
||||
○ announcements
|
||||
▸ VOICE CHANNELS [+]
|
||||
○ lobby ← drag handles ≡
|
||||
○ afk ← can drag to reorder inside category
|
||||
← can drag to another category
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
- Drag handle `≡` on hover
|
||||
- Drop zone highlight between channels and around categories
|
||||
- Hold `Ctrl` to copy permission overrides when moving to another category
|
||||
- Position stored in `channels.position` column (integer, step by 10 for gaps)
|
||||
- Changes broadcast via `CHANNEL_UPDATED` packet to all guild members in real-time
|
||||
|
||||
---
|
||||
|
||||
## Server Settings UI Layout
|
||||
|
||||
```
|
||||
Server Settings: "My Server"
|
||||
─────────────────────────────────────────────
|
||||
│ Navigation │ Content Panel │
|
||||
│ │ │
|
||||
│ Overview │ ┌──────────────┐ │
|
||||
│ Channels │ │ │ │
|
||||
│ Roles │ │ (dynamic) │ │
|
||||
│ Members │ │ │ │
|
||||
│ Invites │ └──────────────┘ │
|
||||
│ Bans │ │
|
||||
│ Audit Log │ │
|
||||
│ Plugins │ │
|
||||
│ ─────────────────── │ │
|
||||
│ Danger Zone │ │
|
||||
│ Delete Server │ │
|
||||
│ Transfer Owner │ │
|
||||
│ │ │
|
||||
└───────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Overview tab
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────┐
|
||||
│ Server Overview │
|
||||
│ │
|
||||
│ Name [My Server________________] │
|
||||
│ Description [_______________________] │
|
||||
│ │
|
||||
│ Icon [🔵 Upload image] drag & drop │
|
||||
│ │
|
||||
│ Owner: loki5512344 │
|
||||
│ Region: Warsaw (default) │
|
||||
│ Created: 2026-06-06 │
|
||||
│ │
|
||||
│ ████████████████████░░ Manage Channels │
|
||||
│ ████████████░░░░░░░░░░ Manage Roles │
|
||||
│ │
|
||||
│ [Save Changes] │
|
||||
└──────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Plugins tab
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────┐
|
||||
│ Plugins │
|
||||
│ │
|
||||
│ ┌─ Installed ─────────────────────┐ │
|
||||
│ │ 🎵 MusicBot v1.2 [Config] │ │
|
||||
│ │ [Disable] [Permissions ▸] │ │
|
||||
│ │ │ │
|
||||
│ │ 🤖 ModBot v0.8 [Config] │ │
|
||||
│ │ [Disable] [Permissions ▸] │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Install from URL...] │
|
||||
│ │
|
||||
│ ┌─ Permission Keys ──────────────────┐ │
|
||||
│ │ Search: [___________________] 🔍 │ │
|
||||
│ │ │ │
|
||||
│ │ Key │ Granted │ │
|
||||
│ │ ──────────────────────────────── │ │
|
||||
│ │ musicbot.command.play │ admin │ │
|
||||
│ │ musicbot.command.skip │ @everyone│ │
|
||||
│ │ modbot.warn │ mod │ │
|
||||
│ │ modbot.ban │ admin │ │
|
||||
│ └──────────────────────────────────────┘
|
||||
└──────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## LNEx Protocol Packets
|
||||
|
||||
New packets for admin operations:
|
||||
|
||||
```
|
||||
PID 0x0070 ADMIN_LIST_CHANNELS client → gateway
|
||||
PID 0x0071 ADMIN_CREATE_CHANNEL client → gateway
|
||||
PID 0x0072 ADMIN_UPDATE_CHANNEL client → gateway
|
||||
PID 0x0073 ADMIN_DELETE_CHANNEL client → gateway
|
||||
PID 0x0074 CHANNEL_CREATED gateway → broadcast
|
||||
PID 0x0075 CHANNEL_UPDATED gateway → broadcast
|
||||
PID 0x0076 CHANNEL_DELETED gateway → broadcast
|
||||
|
||||
PID 0x0080 ADMIN_LIST_ROLES client → gateway
|
||||
PID 0x0081 ADMIN_CREATE_ROLE client → gateway
|
||||
PID 0x0082 ADMIN_UPDATE_ROLE client → gateway
|
||||
PID 0x0083 ADMIN_DELETE_ROLE client → gateway
|
||||
PID 0x0084 ADMIN_ADD_ROLE_MEMBER client → gateway
|
||||
PID 0x0085 ADMIN_REMOVE_ROLE_MEMBER client → gateway
|
||||
|
||||
PID 0x0090 ADMIN_KICK_MEMBER client → gateway
|
||||
PID 0x0091 ADMIN_BAN_MEMBER client → gateway
|
||||
PID 0x0092 ADMIN_UNBAN_MEMBER client → gateway
|
||||
PID 0x0093 ADMIN_MUTE_MEMBER client → gateway
|
||||
|
||||
PID 0x00A0 GUILD_UPDATE client → gateway
|
||||
PID 0x00A1 GUILD_DELETE client → gateway (owner only)
|
||||
PID 0x00A2 GUILD_TRANSFER client → gateway (owner only)
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
|
||||
**ADMIN_CREATE_CHANNEL (0x0071):**
|
||||
```json
|
||||
{
|
||||
"guild_id": "uuid",
|
||||
"category_id": null,
|
||||
"type": "voice",
|
||||
"name": "lobby",
|
||||
"topic": "General voice"
|
||||
}
|
||||
```
|
||||
|
||||
**CHANNEL_CREATED broadcast (0x0074):**
|
||||
```json
|
||||
{
|
||||
"channel": {
|
||||
"id": "uuid",
|
||||
"guild_id": "uuid",
|
||||
"category_id": null,
|
||||
"type": "voice",
|
||||
"name": "lobby",
|
||||
"topic": "General voice",
|
||||
"position": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**ADMIN_UPDATE_ROLE (0x0082):**
|
||||
```json
|
||||
{
|
||||
"role_id": "uuid",
|
||||
"name": "Moderator",
|
||||
"color": 16744960,
|
||||
"permissions": {
|
||||
"vnox.admin.manage_channels": true,
|
||||
"vnox.admin.manage_members": true,
|
||||
"mybot.command.greet": true,
|
||||
"mybot.command.warn": true
|
||||
},
|
||||
"hoist": true,
|
||||
"mentionable": true
|
||||
}
|
||||
```
|
||||
|
||||
**ADMIN_UPDATE_CHANNEL (0x0072) — includes position changes via drag & drop:**
|
||||
```json
|
||||
{
|
||||
"channel_id": "uuid",
|
||||
"name": "general",
|
||||
"category_id": "uuid",
|
||||
"position": 5,
|
||||
"topic": "General discussion"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Permission checks flow
|
||||
|
||||
```
|
||||
Client click "Create Channel"
|
||||
│
|
||||
▼
|
||||
Client sends ADMIN_CREATE_CHANNEL → Gateway
|
||||
│
|
||||
▼
|
||||
Gateway checks:
|
||||
1. Is user authenticated? → 403 if no
|
||||
2. Is user in guild? → 403 if no
|
||||
3. Does user have vnox.admin.manage_channels?
|
||||
- Resolve role hierarchy
|
||||
- Check channel_overrides (none — new channel)
|
||||
- Check wildcards (vnox.admin.*)
|
||||
- Owner override → 403 if no
|
||||
│
|
||||
▼
|
||||
Gateway: INSERT into channels table
|
||||
Gateway: INSERT into channel_overrides if specified
|
||||
Gateway: Broadcast CHANNEL_CREATED to all guild members
|
||||
Gateway: Log to audit_logs
|
||||
│
|
||||
▼
|
||||
All clients in guild see new channel in channel list
|
||||
```
|
||||
65
docs/05-features/central-hub.md
Normal file
65
docs/05-features/central-hub.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# Central Hub — опциональный координационный сервер
|
||||
|
||||
## Мотивация
|
||||
|
||||
VNOX спроектирован как self-hosted платформа: каждый сам поднимает свой нод.
|
||||
Но для adoption нужна точка входа для тех, у кого нет своего сервера, и механизм
|
||||
обнаружения публичных нодов.
|
||||
|
||||
Central Hub — опциональный сервер, который **не заменяет** собственный нод
|
||||
и **не хостит** голос/чат. Он только координирует.
|
||||
|
||||
## Что делает Central Hub
|
||||
|
||||
### 1. Discovery / Registry — реестр публичных нодов
|
||||
|
||||
Владельцы нодов могут опубликовать свой сервер в реестре:
|
||||
- адрес, название, описание, регион
|
||||
- теги (`gaming`, `dev`, `ru`, `cs2`, ...)
|
||||
- мета: онлайне, кол-во пользователей
|
||||
|
||||
Клиент может найти сервер по тегам/названию и подключиться.
|
||||
Без Central Hub — только прямой ввод адреса или закладки.
|
||||
|
||||
### 2. Global identity bridge — аккаунт без своего нода
|
||||
|
||||
Пользователь регистрирует ключ на Central Hub:
|
||||
- `user@hub` — глобальный идентификатор
|
||||
- никнейм, аватар (опционально)
|
||||
- привязка к одному или нескольким "гостевым" нодам
|
||||
|
||||
Позволяет зайти в VNOX без поднятия собственной инфраструктуры.
|
||||
При этом Central Hub не хранит сообщения, не релеит голос, не управляет каналами.
|
||||
|
||||
### 3. Push notification relay — поддержка мобильных
|
||||
|
||||
Мобильные клиенты не могут держать постоянное TCP/UDP соединение.
|
||||
Central Hub принимает пуш-уведомления от нодов:
|
||||
- входящее DM
|
||||
- упоминание в чате
|
||||
- звонок в войс
|
||||
|
||||
Без этого мобильный клиент будет получать сообщения только с задержками.
|
||||
|
||||
### 4. User directory / friend graph — глобальная социальная сеть
|
||||
|
||||
Список друзей и поиск пользователей не привязан к одному ноду:
|
||||
- `@user` ищется по Central Hub
|
||||
- запрос в друзья через хаб, а не через конкретный сервер
|
||||
- cross-node DM: хаб знает, на каком ноде сейчас пользователь
|
||||
|
||||
## Границы (чего Central Hub НЕ делает)
|
||||
|
||||
- ❌ Не хостит голосовые каналы
|
||||
- ❌ Не хостит текстовые каналы
|
||||
- ❌ Не хранит историю сообщений
|
||||
- ❌ Не управляет сессиями — только направляет
|
||||
- ❌ Не заменяет self-hosted нод — только дополняет
|
||||
|
||||
Hub — это телефонная книга и коммутатор, а не сам телефон.
|
||||
|
||||
## Деплой
|
||||
|
||||
Central Hub — отдельный бинарник (например, `vnox-hub`).
|
||||
Работает на HTTP(S) / WebSocket, минимальные требования.
|
||||
Может быть поднят кем угодно, не обязательно авторами VNOX.
|
||||
443
docs/05-features/custom-emoji-stickers.md
Normal file
443
docs/05-features/custom-emoji-stickers.md
Normal file
|
|
@ -0,0 +1,443 @@
|
|||
# Custom Emoji, Stickers & GIF
|
||||
|
||||
> Target phase: 1.3 (emoji picker) / 2 (custom emoji, stickers, GIF)
|
||||
> Depends on: Phase 1.2 (guild system, permissions)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Three related but distinct features:
|
||||
|
||||
| Feature | Size | Format | Scope | Storage |
|
||||
|---------|------|--------|-------|---------|
|
||||
| **Custom Emoji** | 128×128 max | PNG, GIF, WebP | Per-guild | Server filesystem |
|
||||
| **Stickers** | 512×512 max | PNG, GIF, WebP, Lottie† | Per-guild + per-user | Server filesystem |
|
||||
| **GIF Picker** | External | GIF (Tenor/Giphy API) | Per-server config | Proxy via server |
|
||||
|
||||
† Lottie — future, not Phase 2.
|
||||
|
||||
### Difference from unicode reactions
|
||||
|
||||
Unicode emoji reactions (`message_reactions` table, Phase 1.3) use single-codepoint emoji (👍, 🔥, 😂).
|
||||
Custom emoji extend this with server-hosted images referenced as `:emoji_name:` in messages.
|
||||
|
||||
---
|
||||
|
||||
## Storage & Schema
|
||||
|
||||
### Filesystem layout
|
||||
|
||||
```
|
||||
data/
|
||||
├── guilds/
|
||||
│ └── {guild_id}/
|
||||
│ ├── emojis/
|
||||
│ │ ├── kappa.png
|
||||
│ │ ├── pogchamp.gif
|
||||
│ │ └── ...
|
||||
│ └── stickers/
|
||||
│ ├── wave.png
|
||||
│ └── ...
|
||||
├── users/
|
||||
│ └── {user_id}/
|
||||
│ └── stickers/ # personal stickers
|
||||
│ └── ...
|
||||
└── emoji_cache/ # client-side cache (per client, not server)
|
||||
```
|
||||
|
||||
### Database tables
|
||||
|
||||
#### guild_emojis
|
||||
|
||||
```sql
|
||||
CREATE TABLE guild_emojis (
|
||||
id TEXT PRIMARY KEY, -- UUID
|
||||
guild_id TEXT NOT NULL, -- FK guilds
|
||||
name TEXT NOT NULL, -- :name: reference (lowercase, alphanumeric + underscore)
|
||||
filename TEXT NOT NULL, -- on-disk filename (id.png)
|
||||
content_type TEXT NOT NULL, -- image/png, image/gif, image/webp
|
||||
file_size INTEGER NOT NULL, -- bytes
|
||||
width INTEGER NOT NULL, -- px
|
||||
height INTEGER NOT NULL, -- px
|
||||
is_animated BOOLEAN NOT NULL, -- true for GIF
|
||||
uploaded_by TEXT NOT NULL, -- FK users
|
||||
created_at TIMESTAMP NOT NULL,
|
||||
UNIQUE(guild_id, name)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_guild_emojis_guild ON guild_emojis(guild_id);
|
||||
```
|
||||
|
||||
#### guild_stickers
|
||||
|
||||
```sql
|
||||
CREATE TABLE guild_stickers (
|
||||
id TEXT PRIMARY KEY,
|
||||
guild_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
description TEXT,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
file_size INTEGER NOT NULL,
|
||||
width INTEGER NOT NULL,
|
||||
height INTEGER NOT NULL,
|
||||
uploaded_by TEXT NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL,
|
||||
UNIQUE(guild_id, name)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_guild_stickers_guild ON guild_stickers(guild_id);
|
||||
```
|
||||
|
||||
#### user_stickers
|
||||
|
||||
```sql
|
||||
CREATE TABLE user_stickers (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL,
|
||||
file_size INTEGER NOT NULL,
|
||||
width INTEGER NOT NULL,
|
||||
height INTEGER NOT NULL,
|
||||
created_at TIMESTAMP NOT NULL,
|
||||
UNIQUE(user_id, name)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_user_stickers_user ON user_stickers(user_id);
|
||||
```
|
||||
|
||||
### Limits (configurable in server config)
|
||||
|
||||
```toml
|
||||
[limits.emojis]
|
||||
max_per_guild = 150 # default: 150 static + animated combined
|
||||
max_static_per_guild = 100
|
||||
max_animated_per_guild = 50
|
||||
max_file_size_kb = 256 # per emoji
|
||||
max_dimension = 128 # px, both width and height
|
||||
|
||||
[limits.stickers]
|
||||
max_per_guild = 50
|
||||
max_per_user = 25
|
||||
max_file_size_kb = 512 # per sticker
|
||||
max_dimension = 512
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Protocol
|
||||
|
||||
### New packet types
|
||||
|
||||
```
|
||||
0x0070 EMOJI_SYNC server → client, full emoji list for guild
|
||||
0x0071 EMOJI_ADD client → server, admin upload
|
||||
0x0072 EMOJI_DELETE client → server, admin delete
|
||||
0x0073 EMOJI_UPDATE server → client, delta update (single emoji added/removed)
|
||||
|
||||
0x0074 STICKER_SYNC server → client, full sticker list
|
||||
0x0075 STICKER_SEND client → server, send sticker in channel
|
||||
0x0076 STICKER_UPLOAD client → server, upload sticker
|
||||
0x0077 STICKER_DELETE client → server, delete sticker
|
||||
|
||||
0x0078 EMOJI_DATA client → server, request emoji image bytes
|
||||
0x0079 EMOJI_DATA_RESP server → client, image bytes (lazy-load, not in sync)
|
||||
|
||||
0x0080 GIF_SEARCH client → server, proxy search to Tenor/Giphy
|
||||
0x0081 GIF_TRENDING client → server, trending GIFs
|
||||
```
|
||||
|
||||
### EMOJI_SYNC (0x0070)
|
||||
|
||||
Server sends on guild join. Client caches locally.
|
||||
|
||||
```json
|
||||
{
|
||||
"guild_id": "uuid",
|
||||
"emojis": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "kappa",
|
||||
"content_type": "image/png",
|
||||
"is_animated": false,
|
||||
"width": 128,
|
||||
"height": 128,
|
||||
"file_size": 4096
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Image bytes are NOT included — client lazy-loads via `EMOJI_DATA` + `EMOJI_DATA_RESP`.
|
||||
|
||||
### EMOJI_ADD (0x0071)
|
||||
|
||||
Admin uploads new emoji. Requires `MANAGE_EMOJIS` permission.
|
||||
|
||||
```json
|
||||
{
|
||||
"guild_id": "uuid",
|
||||
"name": "pogchamp",
|
||||
"data": "<base64>"
|
||||
}
|
||||
```
|
||||
|
||||
Server validates:
|
||||
- Name: lowercase alphanumeric + underscore, 2-32 chars
|
||||
- Image: PNG/GIF/WebP, max 256KB, max 128×128
|
||||
- Count: under guild limit
|
||||
|
||||
Response: new emoji object (same shape as in EMOJI_SYNC), server broadcasts `EMOJI_UPDATE` to guild.
|
||||
|
||||
### EMOJI_DELETE (0x0072)
|
||||
|
||||
```json
|
||||
{
|
||||
"guild_id": "uuid",
|
||||
"emoji_id": "uuid"
|
||||
}
|
||||
```
|
||||
|
||||
Server removes file + DB row, broadcasts `EMOJI_UPDATE` with `deleted: true`.
|
||||
|
||||
### EMOJI_DATA / EMOJI_DATA_RESP (0x0078/0x0079)
|
||||
|
||||
Lazy-load pattern — client requests image bytes when first rendering an emoji.
|
||||
|
||||
Request:
|
||||
```json
|
||||
{
|
||||
"emoji_id": "uuid"
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"emoji_id": "uuid",
|
||||
"content_type": "image/png",
|
||||
"data": "<base64>"
|
||||
}
|
||||
```
|
||||
|
||||
Client caches image bytes on disk in `emoji_cache/{emoji_id}.{ext}`.
|
||||
|
||||
### STICKER_SEND (0x0075)
|
||||
|
||||
Stickers are sent as a separate message type (not inline). The sticker reference is embedded in a chat message.
|
||||
|
||||
```json
|
||||
{
|
||||
"channel_id": "uuid",
|
||||
"sticker_id": "uuid",
|
||||
"sticker_type": "guild" // "guild" | "user"
|
||||
}
|
||||
```
|
||||
|
||||
Gateway creates a message with `content_type: "sticker"` and `content: {sticker_id, sticker_type}`.
|
||||
|
||||
### GIF_SEARCH (0x0080)
|
||||
|
||||
Client requests GIF search via server (server proxies to Tenor/Giphy so API key stays server-side).
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "cat dancing",
|
||||
"limit": 20,
|
||||
"offset": 0
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"id": "tenor_12345",
|
||||
"url": "https://media.tenor.com/...",
|
||||
"preview_url": "https://media.tenor.com/.../tiny.gif",
|
||||
"width": 498,
|
||||
"height": 280,
|
||||
"title": "Cat Dancing"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
GIF config in server `config.toml`:
|
||||
|
||||
```toml
|
||||
[gif]
|
||||
enabled = true
|
||||
provider = "tenor" # "tenor" | "giphy"
|
||||
api_key = "TENOR_API_KEY"
|
||||
max_results = 50
|
||||
```
|
||||
|
||||
When `enabled = false` the GIF picker is hidden in all clients.
|
||||
|
||||
---
|
||||
|
||||
## Client Integration
|
||||
|
||||
### Message format with custom emoji
|
||||
|
||||
Messages reference custom emoji as `<:emoji_name:emoji_id>` inline in text content:
|
||||
|
||||
```
|
||||
User: check out this <:kappa:abc123> and <:pog:def456>
|
||||
```
|
||||
|
||||
Client renders by looking up emoji_id in local cache, falling back to lazy-load via `EMOJI_DATA_RESP`.
|
||||
|
||||
Unicode emoji (`👍`) render as-is via system font.
|
||||
|
||||
### Emoji picker
|
||||
|
||||
New widget: `client/src/ui/chat/emoji_picker.rs`
|
||||
|
||||
```
|
||||
┌─────────────────────────────┐
|
||||
│ [Search emoji... 🔍] │
|
||||
│─────────────────────────────│
|
||||
│ Favorites │ Custom │ GIF │ ← tabs
|
||||
│────────────┴─────────┴──────│
|
||||
│ 👍 😂 🔥 💯 🙏 😎 │
|
||||
│ 🎉 ✨ 😭 💀 👀 🫡 │
|
||||
│ 😈 🐱 💪 🚀 🫶 🔒 │
|
||||
│─────────────────────────────│
|
||||
│ ┌───┐ ┌───┐ ┌───┐ ┌───┐ │
|
||||
│ │Kappa│ │Pog │ │Hmm │ │Clap│ │ ← custom (rendered as images)
|
||||
│ └───┘ └───┘ └───┘ └───┘ │
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
### Sticker panel
|
||||
|
||||
New widget: `client/src/ui/chat/sticker_panel.rs`
|
||||
|
||||
- Grid of sticker thumbnails
|
||||
- Tabs: Guild stickers / My stickers
|
||||
- Click to send as sticker message (not inline)
|
||||
- Send button shows sticker preview before sending
|
||||
|
||||
### GIF picker
|
||||
|
||||
New widget: `client/src/ui/chat/gif_picker.rs`
|
||||
|
||||
- Search bar at top
|
||||
- Trending grid when empty query
|
||||
- Results grid with animated previews
|
||||
- Click to send GIF as embedded image in chat message
|
||||
|
||||
### Local cache
|
||||
|
||||
```
|
||||
%APPDATA%/vnox/cache/
|
||||
├── emojis/{emoji_id}.{ext} # custom emoji images
|
||||
├── stickers/{sticker_id}.{ext} # sticker images
|
||||
└── gifs/{gif_id}.gif # recently used GIFs (LRU, max 50)
|
||||
```
|
||||
|
||||
Cache invalidation: client re-requests on version mismatch (server increments `emoji_version` per guild).
|
||||
|
||||
---
|
||||
|
||||
## Admin UX
|
||||
|
||||
### Permissions
|
||||
|
||||
| Permission | Bit | Description |
|
||||
| ----------------- | --- | --------------------------- |
|
||||
| MANAGE_EMOJIS | 12 | Upload/delete guild emojis |
|
||||
| MANAGE_STICKERS | 13 | Upload/delete guild stickers|
|
||||
|
||||
These fit into the existing u128 permission bitmask (bits 12-13 are available).
|
||||
|
||||
### Emoji management panel
|
||||
|
||||
Accessible from guild settings (admin only):
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Guild Emojis [Upload] │
|
||||
│──────────────────────────────────────────────────│
|
||||
│ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌────────┐│
|
||||
│ │Kappa │ │ Pog │ │ Hmm │ │ Clap │ │ (free) ││
|
||||
│ │ 4KB │ │ 12KB │ │ 8KB │ │ 16KB │ │ ││
|
||||
│ │ [✕] │ │ [✕] │ │ [✕] │ │ [✕] │ │ ││
|
||||
│ └──────┘ └──────┘ └──────┘ └──────┘ └────────┘│
|
||||
│──────────────────────────────────────────────────│
|
||||
│ 4/150 emojis used [Save order]│
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Upload flow:
|
||||
1. Admin clicks Upload → file dialog (PNG/GIF/WebP)
|
||||
2. Client validates: ≤256KB, ≤128×128, valid format
|
||||
3. Client sends `EMOJI_ADD` with base64 data
|
||||
4. Server validates + stores + broadcasts `EMOJI_UPDATE`
|
||||
|
||||
### Sticker management
|
||||
|
||||
Same pattern as emoji management, separate tab in guild settings.
|
||||
|
||||
Personal stickers managed in user settings (no admin required).
|
||||
|
||||
---
|
||||
|
||||
## Federation Considerations (Phase 3)
|
||||
|
||||
### Emoji visibility across nodes
|
||||
|
||||
When two nodes federate:
|
||||
1. Each node maintains its own emoji set
|
||||
2. Guild emoji metadata synced via federation protocol
|
||||
3. Image bytes fetched on-demand (lazy, same as client)
|
||||
4. Reference format includes origin node: `<:emoji_name:emoji_id@node>`
|
||||
|
||||
### Cross-node sticker send
|
||||
|
||||
1. Sticker reference sent as `{sticker_id, origin_node}`
|
||||
2. Receiving node fetches sticker metadata + image from origin node
|
||||
3. Caches locally for subsequent renders
|
||||
|
||||
### GIF proxy
|
||||
|
||||
GIF search always goes through the user's home node.
|
||||
The server proxies to Tenor/Giphy — no federation needed for GIF results (they're just URLs).
|
||||
|
||||
---
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Phase 1.3 — Unicode reactions only
|
||||
- `message_reactions` table (already designed)
|
||||
- Unicode emoji picker in client
|
||||
- No custom emoji, no stickers, no GIF
|
||||
|
||||
### Phase 2 — Custom emoji
|
||||
- Add `guild_emojis` table
|
||||
- Add packet types 0x0070–0x0073, 0x0078–0x0079
|
||||
- Client emoji picker with custom tab
|
||||
- Admin upload/delete UI
|
||||
- No federation support yet
|
||||
|
||||
### Phase 2 — Stickers
|
||||
- Add `guild_stickers` + `user_stickers` tables
|
||||
- Add packet types 0x0074–0x0077
|
||||
- Sticker panel in client
|
||||
- New message content_type: "sticker"
|
||||
|
||||
### Phase 2 — GIF picker
|
||||
- Add Tenor/Giphy config to gateway
|
||||
- Add packet types 0x0080–0x0081
|
||||
- GIF picker widget in client
|
||||
- Server-side proxy (API key protection)
|
||||
|
||||
### Phase 3 — Federation
|
||||
- Cross-node emoji/sticker sync
|
||||
- `@node` suffix in references
|
||||
131
docs/05-features/direct-messages.md
Normal file
131
docs/05-features/direct-messages.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# Direct Messages — Phase 1.1
|
||||
|
||||
## Goal
|
||||
|
||||
Allow users to send private messages to each other outside of channels.
|
||||
DM conversations appear in a dedicated section of the sidebar (like Discord).
|
||||
|
||||
## Wire protocol
|
||||
|
||||
New packet type in LNEx:
|
||||
|
||||
```
|
||||
DM_START → client→gateway: "open DM with user X"
|
||||
DM_MESSAGE → client→gateway: "send message to DM"
|
||||
gateway→client: "new message in DM"
|
||||
DM_HISTORY → client→gateway: "get DM history"
|
||||
gateway→client: DM message list
|
||||
```
|
||||
|
||||
### DM_START payload
|
||||
|
||||
```json
|
||||
{
|
||||
"packet_id": "DM_START",
|
||||
"target_user_id": "<pubkey of recipient>"
|
||||
}
|
||||
```
|
||||
|
||||
Gateway response:
|
||||
|
||||
```json
|
||||
{
|
||||
"packet_id": "DM_START",
|
||||
"dm_id": "dm_<uid1>_<uid2>",
|
||||
"other_user": {
|
||||
"user_id": "<pubkey>",
|
||||
"nickname": "alice"
|
||||
},
|
||||
"messages": []
|
||||
}
|
||||
```
|
||||
|
||||
### DM_MESSAGE payload
|
||||
|
||||
```json
|
||||
{
|
||||
"packet_id": "DM_MESSAGE",
|
||||
"dm_id": "dm_<uid1>_<uid2>",
|
||||
"sender_id": "<pubkey>",
|
||||
"content": "hello!",
|
||||
"timestamp": 1715000000
|
||||
}
|
||||
```
|
||||
|
||||
## DM ID format
|
||||
|
||||
`dm_<user1_hex>_<user2_hex>` where user1 < user2 lexicographically.
|
||||
This ensures the same DM has the same ID on both sides.
|
||||
|
||||
## Database schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE direct_messages (
|
||||
id TEXT PRIMARY KEY, -- "dm_<uid1>_<uid2>"
|
||||
user1_id TEXT NOT NULL, -- lexicographically smaller
|
||||
user2_id TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
last_message_at INTEGER,
|
||||
UNIQUE(user1_id, user2_id)
|
||||
);
|
||||
|
||||
CREATE TABLE dm_messages (
|
||||
id TEXT PRIMARY KEY,
|
||||
dm_id TEXT NOT NULL REFERENCES direct_messages(id),
|
||||
sender_id TEXT NOT NULL,
|
||||
body TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_dm_messages_dm_id ON dm_messages(dm_id, created_at);
|
||||
CREATE INDEX idx_dm_participant ON direct_messages(user1_id, user2_id);
|
||||
```
|
||||
|
||||
## Gateway handler
|
||||
|
||||
`gateway/src/handler/direct_message.rs` (new file):
|
||||
|
||||
1. `handle_dm_start(session, target_user)`:
|
||||
- Find or create DM record (lexicographic user ID ordering)
|
||||
- Return DM history
|
||||
|
||||
2. `handle_dm_send(session, dm_id, body)`:
|
||||
- Validate session is participant of this DM
|
||||
- Save to `dm_messages` table
|
||||
- If recipient is connected → forward `DM_MESSAGE` to their session
|
||||
- If recipient is offline → stored for delivery on reconnect
|
||||
|
||||
3. `handle_dm_history(session, dm_id)`:
|
||||
- Return last N messages (N = 50 default)
|
||||
|
||||
## Client UI
|
||||
|
||||
- New section in sidebar: "Direct Messages" with user list
|
||||
- Click a user → opens DM chat panel (same layout as channel chat)
|
||||
- Button on user context menu: "Message"
|
||||
- Unread count badge on DM entries
|
||||
- New messages trigger notification dot
|
||||
|
||||
## Client state
|
||||
|
||||
Add to `client/src/ui/state/types.rs`:
|
||||
|
||||
```rust
|
||||
pub struct DmConversation {
|
||||
pub dm_id: String,
|
||||
pub other_user_id: String,
|
||||
pub other_nickname: String,
|
||||
pub messages: Vec<ChatMessage>,
|
||||
pub unread_count: u32,
|
||||
}
|
||||
|
||||
// Add to UiState:
|
||||
pub dms: Vec<DmConversation>,
|
||||
pub active_dm: Option<String>,
|
||||
```
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should DMs support group conversations (3+ users)? → Phase 2
|
||||
- Should DMs be encrypted (E2EE)? → Phase 2
|
||||
- Should users be able to block DMs from specific users? → Phase 2
|
||||
78
docs/05-features/encryption.md
Normal file
78
docs/05-features/encryption.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# Encryption — Phase 1.1
|
||||
|
||||
> **Phase 1.1 status:** Fully implemented. ChaCha20-Poly1305 AEAD with X25519 ECDH + HKDF-SHA256 key exchange. Both TCP (control) and UDP (voice) paths are now encrypted. Forward secrecy via ephemeral session keys.
|
||||
|
||||
## Goal
|
||||
|
||||
Replace all plaintext traffic (TCP + UDP) with ChaCha20-Poly1305 AEAD encryption.
|
||||
X25519 ECDH ephemeral key exchange during handshake for forward secrecy.
|
||||
|
||||
✅ **COMPLETED in Phase 1.1**
|
||||
|
||||
## Key exchange (during HELLO handshake)
|
||||
|
||||
1. Server generates ephemeral X25519 keypair per session
|
||||
2. Server sends public key in `HELLO` packet
|
||||
3. Client generates own ephemeral X25519 keypair
|
||||
4. Both compute shared secret via X25519 ECDH
|
||||
5. Shared secret → HKDF-SHA256 → two keys:
|
||||
- `client→server` encryption key
|
||||
- `server→client` encryption key
|
||||
6. Ephemeral keys discarded after session (forward secrecy)
|
||||
|
||||
## Packet encryption
|
||||
|
||||
- Algorithm: **ChaCha20-Poly1305** (AEAD)
|
||||
- 96-bit nonce: `session_id || packet_sequence`
|
||||
- 16-byte Poly1305 authentication tag per packet
|
||||
- Applied to ALL packets on both TCP and UDP
|
||||
|
||||
## Why ChaCha20-Poly1305 over AES-GCM
|
||||
|
||||
- Constant-time on all platforms (no hardware AES requirement)
|
||||
- Faster in software on ARM (common for mobile — Phase 3)
|
||||
- Simpler nonce management (no IV collision risk)
|
||||
|
||||
## Implementation plan
|
||||
|
||||
### ✅ Client (DONE)
|
||||
- `client/src/net/crypto.rs` — SessionCrypto with ChaCha20-Poly1305
|
||||
- `client/src/net/voice.rs` — `build_packet()` encrypts voice payloads with c2s_key
|
||||
- `client/src/net/voice.rs` — `spawn_recv()` decrypts incoming voice with s2c_key
|
||||
- `client/src/net/session/session_loop.rs` — voice_seq counter for encryption nonces
|
||||
|
||||
### ✅ Gateway (DONE)
|
||||
- `gateway/src/proto/crypto.rs` — X25519 ECDH, HKDF-SHA256, ChaCha20-Poly1305
|
||||
- `gateway/src/net/handshake.rs` — ephemeral key exchange in HELLO/AUTH
|
||||
- `gateway/src/net/io.rs` — TCP framing with encryption
|
||||
|
||||
### ✅ Voice
|
||||
- `client/src/net/voice.rs` — encrypted voice packets on UDP (plaintext at rest in voice-node, encrypted on wire)
|
||||
- Voice-node treats packets as opaque bytes (transparent relay)
|
||||
- End-to-end encryption: client A → encrypted → voice-node → encrypted → client B
|
||||
|
||||
## Nonce management
|
||||
|
||||
Each session has a monotonic sequence counter:
|
||||
- Start at 0 on session establishment
|
||||
- Increment per packet (both directions independently)
|
||||
- Nonce = `session_id (8 bytes) || sequence (4 bytes)`
|
||||
- 12-byte nonce fits ChaCha20-Poly1305 standard
|
||||
|
||||
## Key dependencies
|
||||
|
||||
Already in `Cargo.toml` (workspace):
|
||||
- `chacha20poly1305 = "0.10"`
|
||||
- `x25519-dalek = { version = "2", features = ["static_secrets"] }`
|
||||
|
||||
Need to add:
|
||||
- `hkdf = "0.12"` for key derivation
|
||||
- `sha2 = "0.10"` (HKDF dependency, likely already transitive)
|
||||
|
||||
## Testing
|
||||
|
||||
- ✅ Unit test: encrypt → decrypt round-trip with known keys (`client/src/net/crypto.rs`)
|
||||
- ✅ Unit test: tampered ciphertext fails authentication
|
||||
- ✅ Unit test: build_packet creates valid encrypted packets
|
||||
- Manual test: Run client + gateway + voice-node, join voice channel, verify packets encrypted
|
||||
- Wireshark: Capture UDP traffic, confirm it's not readable plaintext
|
||||
84
docs/05-features/private-mode.md
Normal file
84
docs/05-features/private-mode.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
# Private Mode — Phase 1.1
|
||||
|
||||
## Goal
|
||||
|
||||
Allow running a VNOX server in fully isolated mode.
|
||||
When private, the server does not discover, connect to, or federate with any other node.
|
||||
|
||||
This makes VNOX safe for personal/family use (OwnCord-style),
|
||||
while keeping the option to go federated later.
|
||||
|
||||
## Config
|
||||
|
||||
In `dev/config.toml`:
|
||||
|
||||
```toml
|
||||
[federation]
|
||||
enabled = false # ← PRIVATE: server is isolated
|
||||
# enabled = true # ← FEDERATED: can peer with other nodes
|
||||
```
|
||||
|
||||
Or with explicit mode:
|
||||
|
||||
```toml
|
||||
[server]
|
||||
mode = "private" # isolated, no federation
|
||||
# mode = "federated" # can discover and peer
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
### Gateway config
|
||||
|
||||
```rust
|
||||
// gateway/src/domain/config.rs
|
||||
pub struct Config {
|
||||
pub server: ServerConfig,
|
||||
pub federation: FederationConfig,
|
||||
}
|
||||
|
||||
pub struct ServerConfig {
|
||||
pub mode: ServerMode, // Private | Federated
|
||||
}
|
||||
|
||||
pub enum ServerMode {
|
||||
Private,
|
||||
Federated,
|
||||
}
|
||||
|
||||
pub struct FederationConfig {
|
||||
pub enabled: bool,
|
||||
pub known_nodes: Vec<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Behavior when private
|
||||
|
||||
- Gateway does not send any federation packets
|
||||
- Gateway does not accept inbound federation connections
|
||||
- No node discovery (DNS SRV queries skipped)
|
||||
- No federation port needs to be open
|
||||
- All features (channels, voice, DMs) work normally, just local-only
|
||||
|
||||
### Gate check
|
||||
|
||||
```rust
|
||||
// gateway/src/handler/federation.rs
|
||||
pub async fn should_sync_to_remotes(config: &Config) -> bool {
|
||||
config.federation.enabled
|
||||
}
|
||||
|
||||
// All federation entry points start with:
|
||||
if !config.federation.enabled {
|
||||
return Ok(()); // silently skip
|
||||
}
|
||||
```
|
||||
|
||||
## Result
|
||||
|
||||
| Mode | Behaviour |
|
||||
|------|-----------|
|
||||
| `private` | Fully isolated. Like OwnCord. Safe for personal LAN/ home server. |
|
||||
| `federated` | Can discover, peer, and sync with other VNOX nodes. Like Matrix. |
|
||||
|
||||
Default is `private` — opt-in to federation.
|
||||
108
docs/05-features/slint-migration.md
Normal file
108
docs/05-features/slint-migration.md
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
# Slint UI Migration — Phase 1.1
|
||||
|
||||
## Current state
|
||||
|
||||
Desktop client uses **egui** (immediate-mode GUI). Functional but:
|
||||
- Grey, utilitarian aesthetic
|
||||
- Limited design system
|
||||
- Text-heavy, no modern chat app feel
|
||||
- Difficult to style consistently
|
||||
|
||||
## Why Slint
|
||||
|
||||
| Criteria | egui | Slint | Iced |
|
||||
|----------|------|-------|------|
|
||||
| UI language | Rust code | Declarative DSL (`.slint`) | Elm-style |
|
||||
| Design quality | Utilitarian | Design-oriented | Good |
|
||||
| Performance | Fast | Fast | Moderate |
|
||||
| Chat app fit | ❌ feel | ✅ modern | ✅ modern |
|
||||
| Learning curve | Low | Medium | Medium |
|
||||
|
||||
**Recommendation:** Slint — declarative DSL makes UI easier to iterate,
|
||||
built-in design system produces professional look out of the box.
|
||||
|
||||
## Migration strategy
|
||||
|
||||
### Stage 1 — Side-by-side (week 1)
|
||||
- Embed Slint canvas alongside existing egui panels
|
||||
- Rewrite sidebar (server list + channel list) in Slint first
|
||||
- Keep chat area in egui
|
||||
- Verify data flow works between both frameworks
|
||||
|
||||
### Stage 2 — Chat panel (week 2)
|
||||
- Rewrite channel chat (text messages + input bar) in Slint
|
||||
- Rewrite voice panel in Slint
|
||||
- Remove egui chat dependency
|
||||
|
||||
### Stage 3 — Settings & polish (week 3)
|
||||
- Rewrite settings window in Slint
|
||||
- Add dark/light theme toggle
|
||||
- Add accent color picker
|
||||
- Apply consistent spacing/typography
|
||||
|
||||
## Example Slint structure
|
||||
|
||||
```slint
|
||||
// client/src/ui/main.slint
|
||||
export component MainWindow inherits Window {
|
||||
in-out property <[Channel]> channels;
|
||||
in-out property <[Message]> messages;
|
||||
in-out property <string> active-channel;
|
||||
|
||||
HorizontalLayout {
|
||||
ServerSidebar {}
|
||||
ChatPanel {}
|
||||
VoicePanel {}
|
||||
}
|
||||
}
|
||||
|
||||
component ServerSidebar {
|
||||
VerticalLayout {
|
||||
Text { text: "VNOX"; }
|
||||
ListView {
|
||||
for ch in channels: ChannelRow {
|
||||
text: ch.name;
|
||||
icon: ch.kind == "voice" ? "~" : "#";
|
||||
}
|
||||
}
|
||||
UserBar {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Data binding
|
||||
|
||||
Use Slint's `in-out property` bindings to sync with Rust state:
|
||||
|
||||
```rust
|
||||
// client/src/ui/app.rs
|
||||
use slint::ComponentHandle;
|
||||
|
||||
let ui = MainWindow::new()?;
|
||||
ui.set_channels(slint::ModelRc::from(models));
|
||||
ui.on_send_message(|content| {
|
||||
// forward to net layer
|
||||
});
|
||||
ui.run()?;
|
||||
```
|
||||
|
||||
## File structure after migration
|
||||
|
||||
```
|
||||
client/src/ui/
|
||||
├── app.rs # entry, wiring
|
||||
├── main.slint # main layout
|
||||
├── sidebar.slint # server sidebar + channel list
|
||||
├── chat.slint # message list + input bar
|
||||
├── voice.slint # voice panel
|
||||
├── settings.slint # settings window
|
||||
├── theme.slint # color/font definitions
|
||||
└── state/ # UiState (stays in Rust)
|
||||
```
|
||||
|
||||
## Open questions
|
||||
|
||||
- Slint's rendering backend (gl/winit/software) — test on all target platforms
|
||||
- Slint's text input / rich text support for chat messages (emoji, markdown?)
|
||||
- Whether to remove egui entirely or keep for specific panels (e.g. overlay)
|
||||
- Licensing: Slint is LGPL (compatible with GPL-3.0 for now, but check for future)
|
||||
453
docs/05-features/video-screen-audit.md
Normal file
453
docs/05-features/video-screen-audit.md
Normal file
|
|
@ -0,0 +1,453 @@
|
|||
# Video, Camera & Screen Share — Design Audit
|
||||
|
||||
> Audits `docs/05-features/video.md` against the current codebase and roadmap.
|
||||
> Each gap has: problem, recommendation, priority, target file.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
`video.md` provides a solid high-level sketch (video-node, H.264 UDP relay, nokhwa capture, grid UI).
|
||||
But critical details are missing for actual implementation. This audit fills those gaps.
|
||||
|
||||
---
|
||||
|
||||
## 1. Video-node — Integration with existing architecture
|
||||
|
||||
### Gap 1.1 — Video-node lifecycle
|
||||
|
||||
**Problem:** `video.md` says "separate binary or embedded in voice-node" but doesn't decide. Voice-node has just been refactored with jitter buffer — video needs the same relay pattern.
|
||||
|
||||
**Recommendation:** Separate binary (`vnox-video-node`). Video frames are fundamentally different from voice packets (keyframes, fragmentation, higher bandwidth). Keeping them separate avoids coupling and allows independent scaling.
|
||||
|
||||
**Priority:** P0 — must decide before writing a line of code.
|
||||
|
||||
**Target:** `video.md` — add "Deployment" section.
|
||||
|
||||
### Gap 1.2 — Channel membership signalling
|
||||
|
||||
**Problem:** Video-node needs to know which users are in which channel. Currently this is managed by gateway. How does video-node learn membership?
|
||||
|
||||
**Recommendation:** Gateway sends `VIDEO_SESSION_JOIN` / `VIDEO_SESSION_LEAVE` events to video-node over TCP (internal control channel), same pattern as planned for gateway→voice-node membership signalling (Phase 2 in roadmap).
|
||||
|
||||
```json
|
||||
// Gateway → Video-node (internal TCP)
|
||||
{
|
||||
"event": "VIDEO_SESSION_JOIN",
|
||||
"channel_id": 12345,
|
||||
"user_id": "<pubkey>",
|
||||
"socket_addr": "192.168.1.5:7701"
|
||||
}
|
||||
```
|
||||
|
||||
**Priority:** P0 — video relay can't work without knowing recipients.
|
||||
|
||||
**Target:** `video.md` — add "Gateway integration" section.
|
||||
|
||||
### Gap 1.3 — No encryption spec
|
||||
|
||||
**Problem:** Voice packets are encrypted with ChaCha20-Poly1305. Video packets are not addressed — are they encrypted? With what keys?
|
||||
|
||||
**Recommendation:** Same ChaCha20-Poly1305 AEAD as voice. Each video session gets ephemeral keys derived via the same X25519 ECDH + HKDF-SHA256 path used for voice. Video-node does NOT decrypt (transparent relay, same as voice-node).
|
||||
|
||||
The session key exchange happens during `HELLO` handshake — add a `video_key` field alongside the existing `voice_key`.
|
||||
|
||||
**Priority:** P0 — security regression if video is plaintext.
|
||||
|
||||
**Target:** `video.md` — add "Encryption" section; `docs/02-protocol/security.md` — add video encryption note.
|
||||
|
||||
---
|
||||
|
||||
## 2. Camera Capture — Missing details
|
||||
|
||||
### Gap 2.1 — nokhwa API surface and fallback
|
||||
|
||||
**Problem:** `video.md` mentions `nokhwa` but not the concrete API or what happens on platforms where it doesn't work (Linux v4l2 permissions, Wayland).
|
||||
|
||||
**Recommendation:** Abstract behind a `VideoSource` trait:
|
||||
|
||||
```rust
|
||||
// client/src/video/capture.rs
|
||||
pub trait VideoSource {
|
||||
fn devices() -> Vec<DeviceInfo>;
|
||||
fn start(config: StreamConfig) -> Result<FrameStream>;
|
||||
}
|
||||
|
||||
struct NokhwaSource { ... }
|
||||
struct StubSource { ... } // fallback
|
||||
|
||||
pub struct StreamConfig {
|
||||
pub device_id: String,
|
||||
pub resolution: Resolution, // 640x480, 1280x720, 1920x1080
|
||||
pub fps: u32, // 15, 30
|
||||
pub format: PixelFormat, // NV12, I420, BGRA
|
||||
}
|
||||
```
|
||||
|
||||
Make `nokhwa` a feature gate (same pattern as `rnnoise` for voice).
|
||||
|
||||
**Priority:** P0 — blocks camera implementation.
|
||||
|
||||
**Target:** New file: `client/src/video/capture.rs` (spec in `video.md`).
|
||||
|
||||
### Gap 2.2 — Device hotplug / switch
|
||||
|
||||
**Problem:** No mention of what happens when camera is unplugged/replugged or user switches camera mid-call.
|
||||
|
||||
**Recommendation:** `VideoSource` emits events:
|
||||
```rust
|
||||
enum CaptureEvent {
|
||||
Frame(VideoFrame),
|
||||
DeviceLost(String),
|
||||
DeviceReconnected(String),
|
||||
Error(String),
|
||||
}
|
||||
```
|
||||
|
||||
Client handles `DeviceLost` by showing placeholder tile. User can select new device in settings without leaving call.
|
||||
|
||||
**Priority:** P1 — quality of life, not MVP blocker.
|
||||
|
||||
**Target:** `video.md` — add "Device lifecycle" to capture section.
|
||||
|
||||
### Gap 2.3 — Resolution / FPS negotiation
|
||||
|
||||
**Problem:** `video.md` says "720p default, 30 fps" but doesn't say how clients agree on parameters. Different clients have different cameras.
|
||||
|
||||
**Recommendation:** Server advertises max resolution per channel. Client sends its capability in `VIDEO_SESSION_JOIN`. Server picks min(max_server, max_client) per sender.
|
||||
|
||||
```toml
|
||||
# server config
|
||||
[video.channel_defaults]
|
||||
max_resolution = "1280x720"
|
||||
max_fps = 30
|
||||
max_bitrate_kbps = 2500
|
||||
```
|
||||
|
||||
**Priority:** P1 — needed before multi-user video testing.
|
||||
|
||||
**Target:** `video.md` — add "Capability negotiation" section.
|
||||
|
||||
---
|
||||
|
||||
## 3. Screen Share — Almost entirely missing
|
||||
|
||||
### Gap 3.1 — Screen capture crate
|
||||
|
||||
**Problem:** `video.md` mentions "Screen sharing (desktop capture)" as one bullet in Phase 2, no detail.
|
||||
|
||||
**Recommendation:**
|
||||
|
||||
| Platform | Crate | Notes |
|
||||
|----------|-------|-------|
|
||||
| Windows | `windows-capture` or DXGI via `winapi` | DXGI Desktop Duplication API — best perf |
|
||||
| Linux X11 | `x11cap` or raw XSHM | X11 only, no Wayland |
|
||||
| Linux Wayland | `pipewire` via `pw-video` or xdg-desktop-portal | Portal is the standard path |
|
||||
| macOS | `screencapturekit` via objc bindings or `CGDisplay` | ScreenCaptureKit requires macOS 13+ |
|
||||
|
||||
Abstract behind same `VideoSource` trait as camera. Two implementations: `CameraSource`, `ScreenSource`.
|
||||
|
||||
**Priority:** P1 — needed for Phase 2 screen share.
|
||||
|
||||
**Target:** New doc: `docs/05-features/screen-share.md` or extend `video.md` Phase 2.
|
||||
|
||||
### Gap 3.2 — Window/display selection UI
|
||||
|
||||
**Problem:** User needs to pick which screen or window to share. No UI spec.
|
||||
|
||||
**Recommendation:** Modal dialog when user clicks "Screen Share":
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Share Your Screen │
|
||||
│─────────────────────────────────────────│
|
||||
│ [Screens] [Windows] │ ← tabs
|
||||
│─────────────────────────────────────────│
|
||||
│ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Display 1│ │ Display 2│ │
|
||||
│ │1920×1080 │ │2560×1440 │ │
|
||||
│ │ [✓] │ │ │ │
|
||||
│ └──────────┘ └──────────┘ │
|
||||
│─────────────────────────────────────────│
|
||||
│ ☐ Share system audio │
|
||||
│ ☐ Optimize for video (60fps) │
|
||||
│─────────────────────────────────────────│
|
||||
│ [Cancel] [Share] │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Priority:** P1 — UX blocker for screen share.
|
||||
|
||||
**Target:** `video.md` — add "Screen share UI" section.
|
||||
|
||||
### Gap 3.3 — System audio capture with screen
|
||||
|
||||
**Problem:** "Share system audio" checkbox in the mockup — how? No mention in video.md.
|
||||
|
||||
**Recommendation:**
|
||||
- Windows: WASAPI loopback (`cpal` supports it with `loopback` config)
|
||||
- Linux: PulseAudio monitor or PipeWire
|
||||
- macOS: BlackHole or ScreenCaptureKit (built-in on 13+)
|
||||
|
||||
Feature-gate it: `--features screen-audio`. Not all platforms support it cleanly.
|
||||
|
||||
**Priority:** P2 — nice to have, not MVP.
|
||||
|
||||
**Target:** `video.md` Phase 2 — add "System audio" bullet.
|
||||
|
||||
### Gap 3.4 — Screen share FPS strategy
|
||||
|
||||
**Problem:** Screen share has different FPS needs than camera. Coding: 15fps is fine. Gaming: 60fps needed.
|
||||
|
||||
**Recommendation:** Client detects content type (static → low FPS, motion → high FPS) and adapts. User can override with "Optimize for video" checkbox.
|
||||
|
||||
Default: 15fps for screen share, 30fps for camera.
|
||||
|
||||
**Priority:** P2 — optimization, not MVP.
|
||||
|
||||
**Target:** `video.md` — add to Phase 2.
|
||||
|
||||
---
|
||||
|
||||
## 4. Interaction Model — Video + Voice
|
||||
|
||||
### Gap 4.1 — Video tied to voice channel
|
||||
|
||||
**Problem:** `video.md` implies video is in the same channel as voice. What if users want video-only (no voice) or voice-only (no video)?
|
||||
|
||||
**Recommendation:** Video is a capability toggle within a voice channel, not a separate channel type:
|
||||
|
||||
1. User joins voice channel (as today)
|
||||
2. User clicks "Enable Camera" → client starts sending video frames to video-node
|
||||
3. User clicks "Disable Camera" → stops sending, voice continues
|
||||
4. Same for "Share Screen"
|
||||
|
||||
Channel membership = voice channel membership. Video is additive.
|
||||
|
||||
New voice state: `VOICE_STATE` gets a `video: bool` field.
|
||||
|
||||
**Priority:** P0 — architectural decision, affects protocol design.
|
||||
|
||||
**Target:** `video.md` — add "Interaction model" section; `docs/02-protocol/packets.md` — extend VOICE_STATE.
|
||||
|
||||
### Gap 4.2 — Video without voice
|
||||
|
||||
**Problem:** Some users may want to watch a stream without transmitting audio. Current model requires voice channel join.
|
||||
|
||||
**Recommendation:** Phase 1: allow joining voice channel muted+deafened to watch video. Phase 3: separate "watch-only" mode (VIEW permission).
|
||||
|
||||
**Priority:** P2 — Phase 1 workaround is acceptable.
|
||||
|
||||
**Target:** `video.md` — add "Watch-only mode" to Phase 3.
|
||||
|
||||
---
|
||||
|
||||
## 5. Performance & Quality
|
||||
|
||||
### Gap 5.1 — Bitrate adaptation
|
||||
|
||||
**Problem:** No mechanism to adjust video bitrate based on network conditions. Voice has jitter buffer adaptation, video has nothing.
|
||||
|
||||
**Recommendation:** Client-side adaptive bitrate (ABR):
|
||||
- Measure packet loss and RTT from video-node ACKs
|
||||
- Adjust encoder bitrate up/down based on available bandwidth
|
||||
- 3 tiers: low (500kbps), medium (1500kbps), high (4000kbps)
|
||||
|
||||
Video-node sends periodic `VIDEO_STATS` with per-receiver loss rates:
|
||||
|
||||
```json
|
||||
{
|
||||
"channel_id": 12345,
|
||||
"receivers": {
|
||||
"user_a": { "loss_pct": 0.5, "rtt_ms": 12 },
|
||||
"user_b": { "loss_pct": 8.0, "rtt_ms": 150 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Priority:** P1 — needed for real-world use.
|
||||
|
||||
**Target:** `video.md` — add "Adaptive bitrate" section.
|
||||
|
||||
### Gap 5.2 — Simulcast design
|
||||
|
||||
**Problem:** "Simulcast" is listed as Phase 3 with no design. Different receivers have different bandwidth.
|
||||
|
||||
**Recommendation:** Sender encodes 2-3 quality layers. Video-node forwards appropriate layer to each receiver based on their reported bandwidth.
|
||||
|
||||
```
|
||||
Sender → encode: 1080p (4Mbps) + 720p (1.5Mbps) + 360p (500kbps)
|
||||
│
|
||||
Video-node
|
||||
╱ │ ╲
|
||||
1080p→A 720p→B 360p→C (based on per-receiver stats)
|
||||
```
|
||||
|
||||
Requires VP9 SVC or H.264 simulcast. Complex — Phase 3 is correct.
|
||||
|
||||
**Priority:** P2 — Phase 3, but design it now to avoid architecture lock-in.
|
||||
|
||||
**Target:** `video.md` Phase 3 — expand "Simulcast" bullet.
|
||||
|
||||
### Gap 5.3 — Keyframe interval
|
||||
|
||||
**Problem:** New viewers joining mid-stream need a keyframe to start decoding. Video.md doesn't mention keyframe strategy.
|
||||
|
||||
**Recommendation:**
|
||||
- Sender inserts keyframe every 2 seconds (configurable)
|
||||
- Video-node caches last keyframe per sender
|
||||
- On new viewer join: video-node sends cached keyframe immediately, then regular frames
|
||||
- `VIDEO_SESSION_JOIN` triggers keyframe push
|
||||
|
||||
Add `REQUEST_KEYFRAME` packet for explicit PLI (Picture Loss Indication).
|
||||
|
||||
**Priority:** P1 — new viewers can't see video without keyframe.
|
||||
|
||||
**Target:** `video.md` — add "Keyframe handling" section.
|
||||
|
||||
---
|
||||
|
||||
## 6. Missing Protocol Details
|
||||
|
||||
### Gap 6.1 — No packet fragmentation spec
|
||||
|
||||
**Problem:** Video frames can be 50-100KB. UDP MTU is ~1400 bytes. Video.md says "Packetize into MTU-friendly chunks" but doesn't define the fragmentation protocol.
|
||||
|
||||
**Recommendation:** Reuse the LNEx fragmentation flags from the base packet header (bits 2-3):
|
||||
|
||||
```
|
||||
FRAGMENTED (bit 2) + LAST_FRAG (bit 3)
|
||||
```
|
||||
|
||||
Each video frame:
|
||||
1. Split into 1400-byte chunks
|
||||
2. Each chunk gets same `frame_seq`, incrementing `fragment_seq`
|
||||
3. Last chunk sets `LAST_FRAG` flag
|
||||
4. Receiver reassembles before decode
|
||||
|
||||
New packet type: `VIDEO_FRAME_FRAGMENT (0x0012)` — separate from unfragmented `VIDEO_FRAME (0x0011)`.
|
||||
|
||||
Wait — `VIDEO_FRAME` ID isn't assigned in the packet registry yet. Register:
|
||||
|
||||
```
|
||||
0x0011 VIDEO_FRAME single (small) video frame, no fragmentation
|
||||
0x0012 VIDEO_FRAME_FRAG fragment of a larger video frame
|
||||
```
|
||||
|
||||
**Priority:** P0 — video can't work over UDP without fragmentation.
|
||||
|
||||
**Target:** `video.md` — replace packet format section; `docs/02-protocol/packets.md` — add 0x0011/0x0012 to registry.
|
||||
|
||||
### Gap 6.2 — No RTCP-like receiver reports
|
||||
|
||||
**Problem:** Sender has no feedback about what receivers are experiencing. Voice has implicit feedback (jitter buffer stats), video needs explicit.
|
||||
|
||||
**Recommendation:** Minimal receiver report packet:
|
||||
|
||||
```json
|
||||
// 0x0013 VIDEO_RECEIVER_REPORT — client → video-node → sender
|
||||
{
|
||||
"frame_seq": 1042,
|
||||
"loss_cumulative": 15,
|
||||
"loss_fraction": 2, // percent of last N packets
|
||||
"jitter_ms": 8,
|
||||
"rtt_ms": 35
|
||||
}
|
||||
```
|
||||
|
||||
Client sends every 1 second. Video-node aggregates and forwards to sender.
|
||||
|
||||
**Priority:** P1 — needed for ABR and quality adaptation.
|
||||
|
||||
**Target:** `video.md` — add "Receiver reports" section.
|
||||
|
||||
---
|
||||
|
||||
## 7. Guild Integration (Phase 1.2)
|
||||
|
||||
### Gap 7.1 — Video permissions already defined
|
||||
|
||||
**Status:** ✅ `STREAM` (bit 18) permission is already in the community model (`docs/07-community-model.md`). Covers both camera and screen share.
|
||||
|
||||
No gap here — just implement the check in gateway when processing `VIDEO_SESSION_JOIN`.
|
||||
|
||||
### Gap 7.2 — Channel-level video settings
|
||||
|
||||
**Problem:** Guild admins may want to disable video in certain channels (text-only channels, AFK channel).
|
||||
|
||||
**Recommendation:** Add `video_allowed: bool` to channel config. Default: `true` for voice channels, `false` for text channels.
|
||||
|
||||
```sql
|
||||
ALTER TABLE channels ADD COLUMN video_allowed BOOLEAN NOT NULL DEFAULT 1;
|
||||
```
|
||||
|
||||
Gateway rejects `VIDEO_SESSION_JOIN` if `video_allowed = false`.
|
||||
|
||||
**Priority:** P1 — admin control expected by guild owners.
|
||||
|
||||
**Target:** `docs/10-database.md` — add column to channels table; `docs/07-community-model.md` — add `video_allowed` to channel settings.
|
||||
|
||||
---
|
||||
|
||||
## 8. Mobile Considerations (Phase 3)
|
||||
|
||||
### Gap 8.1 — Front/back camera switch
|
||||
|
||||
**Problem:** Mobile video.md doesn't mention camera switching at all.
|
||||
|
||||
**Recommendation:** Mobile client sends `CAMERA_SWITCH` event to video-node — no protocol change needed, just local capture change. New frame stream with `camera: front|back` metadata.
|
||||
|
||||
**Priority:** P2 — Phase 3.
|
||||
|
||||
**Target:** `docs/04-clients/mobile.md` (doesn't exist yet — create in Phase 3).
|
||||
|
||||
### Gap 8.2 — Self preview
|
||||
|
||||
**Problem:** Desktop video grid has "Self-view (small, picture-in-picture corner)". On mobile the self-view is more important (front camera framing).
|
||||
|
||||
**Recommendation:** Mobile layout: self-view full-width at top, other participants in scrollable grid below. Toggle button to swap.
|
||||
|
||||
**Priority:** P2 — Phase 3.
|
||||
|
||||
**Target:** `docs/04-clients/mobile.md`.
|
||||
|
||||
---
|
||||
|
||||
## Gap Summary — What to Add Before Implementation
|
||||
|
||||
| # | Gap | Priority | Target file |
|
||||
|---|-----|----------|-------------|
|
||||
| 1 | Video-node binary decision + deployment | P0 | `video.md` |
|
||||
| 2 | Gateway membership signalling to video-node | P0 | `video.md` |
|
||||
| 3 | Video encryption (ChaCha20-Poly1305) | P0 | `video.md`, `security.md` |
|
||||
| 4 | Video frame fragmentation protocol | P0 | `video.md`, `packets.md` |
|
||||
| 5 | Video ↔ voice interaction model | P0 | `video.md`, `packets.md` |
|
||||
| 6 | VideoSource trait + feature-gated nokhwa | P0 | `video.md` |
|
||||
| 7 | Keyframe caching + PLI request | P1 | `video.md` |
|
||||
| 8 | Receiver reports (RTCP-like) | P1 | `video.md` |
|
||||
| 9 | Adaptive bitrate design | P1 | `video.md` |
|
||||
| 10 | Screen share crate selection + ScreenSource | P1 | `video.md` |
|
||||
| 11 | Window/display selection UI | P1 | `video.md` |
|
||||
| 12 | Resolution/FPS negotiation | P1 | `video.md` |
|
||||
| 13 | Channel-level video_allowed setting | P1 | `database.md`, `community-model.md` |
|
||||
| 14 | Camera hotplug/switch handling | P1 | `video.md` |
|
||||
| 15 | Video-only (watch) mode | P2 | `video.md` |
|
||||
| 16 | System audio with screen share | P2 | `video.md` |
|
||||
| 17 | Screen share FPS strategy | P2 | `video.md` |
|
||||
| 18 | Simulcast design | P2 | `video.md` |
|
||||
| 19 | Mobile front/back camera + self preview | P2 | `mobile.md` |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Implementation Order
|
||||
|
||||
1. **Update `video.md`** — incorporate P0 and P1 gaps from this audit into the design doc
|
||||
2. **Register packet types** — 0x0011–0x0013 in `packets.md`
|
||||
3. **Video-node prototype** — separate binary, UDP relay, no encoding/decoding (transparent relay like voice)
|
||||
4. **Gateway ↔ video-node signalling** — internal TCP control channel for membership
|
||||
5. **Client camera capture** — `VideoSource` trait + `NokhwaSource` behind feature gate
|
||||
6. **Fragmentation** — implement FRAGMENTED/LAST_FRAG for video frames
|
||||
7. **Grid UI** — basic 2×2 grid in client
|
||||
8. **Encryption** — ChaCha20-Poly1305 for video frames
|
||||
9. **ABR + receiver reports** — iterate toward production quality
|
||||
10. **Screen share** — `ScreenSource` implementation + picker UI
|
||||
99
docs/05-features/video.md
Normal file
99
docs/05-features/video.md
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
# Video Chat — Architecture
|
||||
|
||||
## Goal
|
||||
|
||||
Add real-time video to voice channels.
|
||||
Users can share their webcam feed alongside voice.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Current (voice only):
|
||||
├─ Gateway: TCP (signaling, chat)
|
||||
├─ Voice-node: UDP (Opus voice relay)
|
||||
└─ Client: mic → Opus → UDP → voice-node → UDP → client → playback
|
||||
|
||||
With video:
|
||||
├─ Gateway: TCP (signaling) — UNCHANGED
|
||||
├─ Voice-node: UDP (Opus voice) — UNCHANGED
|
||||
├─ Video-node: UDP (H.264/VP9 relay) — NEW
|
||||
│ └─ Receives encoded frames, relays to channel members
|
||||
└─ Client: mic → Opus → UDP → voice-node
|
||||
webcam → H.264 → UDP → video-node ← UDP → decode → display grid
|
||||
```
|
||||
|
||||
## Video-node
|
||||
|
||||
New optional component, separate binary or embedded in voice-node.
|
||||
|
||||
### Responsibilities
|
||||
- Receive encoded video frames over UDP
|
||||
- Relay to all other clients in the same voice channel
|
||||
- No transcoding (relay only — CPU efficient)
|
||||
- Max resolution / bitrate per channel configurable
|
||||
|
||||
### Packet format
|
||||
|
||||
```json
|
||||
{
|
||||
"packet_id": "VIDEO_FRAME",
|
||||
"channel_id": 12345,
|
||||
"sender_id": "<pubkey>",
|
||||
"frame_seq": 42,
|
||||
"codec": "h264", // or "vp9"
|
||||
"keyframe": false,
|
||||
"data": "<base64 encoded frame>"
|
||||
}
|
||||
```
|
||||
|
||||
Initially JSON (matching Phase 1 convention), binary framing in Phase 2.
|
||||
|
||||
## Client capture pipeline
|
||||
|
||||
New file: `client/src/video/capture.rs`
|
||||
|
||||
1. Enumerate webcam devices via `nokhwa` or `video4linux`
|
||||
2. Capture frames at configurable resolution (720p default)
|
||||
3. Encode to H.264 via `ffmpeg-next` or hardware encoder
|
||||
4. Packetize into MTU-friendly chunks
|
||||
5. Send over UDP to video-node
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `nokhwa` — cross-platform camera capture (Rust)
|
||||
- `ffmpeg-next` or `rav1e` — H.264/VP9 encoding
|
||||
|
||||
## Client UI — Video grid
|
||||
|
||||
New component: `client/src/ui/video.rs`
|
||||
|
||||
- Grid layout (max 4×4 = 16 participants visible)
|
||||
- Active speaker highlight (green border)
|
||||
- Self-view (small, picture-in-picture corner)
|
||||
- Mute video button per participant
|
||||
- Resolution/quality indicator per stream
|
||||
|
||||
Layout modes:
|
||||
- 1 participant → full width
|
||||
- 2-4 → 2×2 grid
|
||||
- 5-9 → 3×3 grid
|
||||
- 10-16 → 4×4 grid with scrolling
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1 (MVP)
|
||||
- H.264 encoding with `nokhwa` + `ffmpeg-next`
|
||||
- Single video-node binary
|
||||
- 2×2 grid in UI
|
||||
- 720p max resolution, 30 fps
|
||||
|
||||
### Phase 2
|
||||
- VP9 support (better quality/bitrate)
|
||||
- Adaptive resolution (auto downscale on packet loss)
|
||||
- Picture-in-picture self-view
|
||||
- Screen sharing (desktop capture)
|
||||
|
||||
### Phase 3
|
||||
- Hardware encoding (NVENC/VAAPI)
|
||||
- Simulcast (different resolution per stream)
|
||||
- Recording support
|
||||
Loading…
Add table
Add a link
Reference in a new issue