feat(client): add RU/EN localization and native capture preview
This commit is contained in:
parent
8e5ab56611
commit
8460fb28fd
43 changed files with 4103 additions and 275 deletions
59
docs/04-clients/screen-capture.md
Normal file
59
docs/04-clients/screen-capture.md
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
# Local screen capture and separate audio
|
||||
|
||||
The voice view's **Screen preview** opens a native window. Choose an optional
|
||||
playback application or output device, then **Choose screen or window**. The
|
||||
operating system's ScreenCast portal asks which source may be recorded. **Stop**,
|
||||
closing the preview, and cancelling selection revoke capture. Preview pixels are
|
||||
held only in memory, with one latest RGBA frame; they do not enter chat history,
|
||||
preferences, files, or logs. Microphone capture remains entirely in the existing
|
||||
voice pipeline and is never enabled by preview.
|
||||
|
||||
## Current capability
|
||||
|
||||
Linux Wayland capture uses xdg-desktop-portal + PipeWire + GStreamer. Video is
|
||||
limited to 1920×1080/30fps, copied out of PipeWire buffers, and displayed natively.
|
||||
The application selector lists active `Stream/Output/Audio` playback streams;
|
||||
applications with several playback streams may have several entries. It does not
|
||||
list microphones or VNOX's own streams. The system-output choice is explicit and
|
||||
may include every application and notification playing on that output.
|
||||
|
||||
Selected audio uses a **separate local PipeWire connection**, never the portal's
|
||||
restricted video FD. Its stable `object.serial` is checked against a fresh graph
|
||||
before capture. `target-object`, `node.dont-fallback`, `node.dont-reconnect`,
|
||||
`node.dont-move` and `on-disconnect=error` prevent silently selecting another
|
||||
stream, microphone or default output. The UI displays actual captured audio
|
||||
level. Session teardown stops both pipelines before closing the portal FD.
|
||||
|
||||
**This is local preview, not broadcasting.** The existing client/server media
|
||||
protocol does not yet negotiate separate video/application-audio tracks, codec
|
||||
capabilities or recipient subscriptions. No frames are sent to other members.
|
||||
The next integration must use established WebRTC transport with authenticated
|
||||
signalling, independent tracks, backpressure and participant authorization in the
|
||||
combined server daemon. Do not send video through custom UDP fragments or reuse
|
||||
the microphone encryption/packet format without a documented protocol change.
|
||||
Windows process-loopback and macOS adapters are not implemented; unsupported
|
||||
builds report that capability rather than recording a different source.
|
||||
|
||||
## Dependencies and verification
|
||||
|
||||
`screen-capture` is enabled by default. A build without it remains available with
|
||||
`cargo run --no-default-features`. Linux needs GStreamer core/base development
|
||||
packages and the PipeWire GStreamer plugin (`gst-plugin-pipewire` on Arch;
|
||||
`gstreamer1.0-pipewire` on Debian/Ubuntu), a ScreenCast-capable portal backend,
|
||||
PipeWire and `pw-dump`. Installation must not rewrite user audio configuration.
|
||||
|
||||
```sh
|
||||
cargo test --lib media:: # Synthetic video/audio; no desktop recording
|
||||
python3 scripts/check-selected-audio.py # Temporary null sink + synthetic app, checks disconnect
|
||||
cargo run --bin screen_probe -- --list-audio # Metadata only
|
||||
cargo run --bin screen_probe -- --choose # Explicit portal consent, 15s local capture, no files
|
||||
```
|
||||
|
||||
The Python integration check requires `pactl`, `gst-launch-1.0`, the GStreamer
|
||||
`level` element and a running session manager. It creates and removes only its
|
||||
own temporary null sink; it does not change the default device. Manual consent
|
||||
and real compositor preview still require an interactive desktop check.
|
||||
|
||||
Primary contracts: [ScreenCast portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html),
|
||||
[PipeWire properties](https://docs.pipewire.org/group__pw__keys.html),
|
||||
[GStreamer AppSink](https://docs.rs/gstreamer-app/0.25.2/gstreamer_app/struct.AppSink.html).
|
||||
Loading…
Add table
Add a link
Reference in a new issue