feat(client): protocol v2 wire layer, screen sharing, channel webhooks and secure voice
Verified against the real gateway via tests/gateway_interop.rs.
This commit is contained in:
parent
8460fb28fd
commit
3d7dd5413b
65 changed files with 6048 additions and 1156 deletions
|
|
@ -24,26 +24,45 @@ before capture. `target-object`, `node.dont-fallback`, `node.dont-reconnect`,
|
|||
stream, microphone or default output. The UI displays actual captured audio
|
||||
level. Session teardown stops both pipelines before closing the portal FD.
|
||||
|
||||
**This is local preview, not broadcasting.** The existing client/server media
|
||||
protocol does not yet negotiate separate video/application-audio tracks, codec
|
||||
capabilities or recipient subscriptions. No frames are sent to other members.
|
||||
The next integration must use established WebRTC transport with authenticated
|
||||
signalling, independent tracks, backpressure and participant authorization in the
|
||||
combined server daemon. Do not send video through custom UDP fragments or reuse
|
||||
the microphone encryption/packet format without a documented protocol change.
|
||||
Windows process-loopback and macOS adapters are not implemented; unsupported
|
||||
builds report that capability rather than recording a different source.
|
||||
## Broadcast and watch
|
||||
|
||||
After the local preview is visible, **Start broadcast** explicitly announces it
|
||||
in the authenticated voice channel. The server checks current voice/guild
|
||||
membership and voice-speaking permissions. Other members choose **Watch
|
||||
broadcast**; receiving never starts automatically. A separate native viewer
|
||||
renders actual decoded frames and can mute only the broadcast. Closing the
|
||||
viewer revokes its subscription and clears queued samples. Broadcast audio uses
|
||||
the same selected CPAL output device and deafen/volume controls as voice; it does
|
||||
not silently open the default output. Stereo application audio stays stereo.
|
||||
|
||||
GStreamer `webrtcbin` negotiates ICE + DTLS-SRTP, VP8 video and independent Opus
|
||||
tracks. Authenticated LNEx protobuf carries only bounded SDP/ICE signalling.
|
||||
One publisher has at most four viewers; packets never use custom crypto or video
|
||||
fragments. Existing voice remains an independent microphone lane by default,
|
||||
respecting mute/PTT/VAD; broadcasting does not duplicate it. The transport also
|
||||
supports an explicitly enabled separate microphone track through the gated,
|
||||
bounded existing-capture tap for integrations that need it.
|
||||
|
||||
The default uses host ICE candidates, suitable for reachable LAN/public-address
|
||||
peers. Administrators may explicitly set `VNOX_STUN_SERVER=stun://host:port` and
|
||||
`VNOX_TURN_SERVER=turns://user:password@host:port` in the client environment.
|
||||
Some NATs require TURN; no public STUN server, third-party relay or system-audio
|
||||
fallback is silently selected. ICE failure times out visibly. TURN credentials
|
||||
are configuration secrets and must not be committed. Windows process-loopback
|
||||
and macOS capture adapters remain unsupported; those builds report capability
|
||||
rather than recording a different source.
|
||||
|
||||
## Dependencies and verification
|
||||
|
||||
`screen-capture` is enabled by default. A build without it remains available with
|
||||
`cargo run --no-default-features`. Linux needs GStreamer core/base development
|
||||
packages and the PipeWire GStreamer plugin (`gst-plugin-pipewire` on Arch;
|
||||
packages the GStreamer WebRTC, libnice, VP8/Opus plugins and the PipeWire GStreamer plugin (`gst-plugin-pipewire` on Arch;
|
||||
`gstreamer1.0-pipewire` on Debian/Ubuntu), a ScreenCast-capable portal backend,
|
||||
PipeWire and `pw-dump`. Installation must not rewrite user audio configuration.
|
||||
|
||||
```sh
|
||||
cargo test --lib media:: # Synthetic video/audio; no desktop recording
|
||||
cargo test --lib media:: # Includes actual two-peer VP8 + separate Opus delivery
|
||||
VNOX_GATEWAY_BIN=/path/to/vnox-serverd cargo test --lib media::gateway_tests -- --ignored
|
||||
python3 scripts/check-selected-audio.py # Temporary null sink + synthetic app, checks disconnect
|
||||
cargo run --bin screen_probe -- --list-audio # Metadata only
|
||||
cargo run --bin screen_probe -- --choose # Explicit portal consent, 15s local capture, no files
|
||||
|
|
@ -52,7 +71,9 @@ cargo run --bin screen_probe -- --choose # Explicit portal consent, 15s loc
|
|||
The Python integration check requires `pactl`, `gst-launch-1.0`, the GStreamer
|
||||
`level` element and a running session manager. It creates and removes only its
|
||||
own temporary null sink; it does not change the default device. Manual consent
|
||||
and real compositor preview still require an interactive desktop check.
|
||||
and real compositor preview still require an interactive desktop check. The
|
||||
two-peer transport fixture uses only synthetic video and distinct 440/880 Hz
|
||||
audio, decodes both independently, and never opens microphone/output hardware.
|
||||
|
||||
Primary contracts: [ScreenCast portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html),
|
||||
[PipeWire properties](https://docs.pipewire.org/group__pw__keys.html),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue