guard/tests/clickhouse_writer.rs
loki5512344 c1a076a4a3
fix(core): close auth bypass, hangs and memory leaks found by review
Security:
- An empty API_PASSWORD passed validation (only the literal "changeme" was
  rejected) and verify_password("", "") returned true, so a login with an
  empty password minted an admin token. Require >= 16 characters and >= 32
  non-whitespace bytes for JWT_SECRET; verify rejects an empty stored secret.
- syn_challenge_enabled required a real challenge_secret_hex: the public
  placeholder from config.h is now rejected in every spelling.
- docker-compose: Redis/ClickHouse/Grafana are loopback-only and every
  password uses ${VAR:?} instead of a default. An exposed Redis was a way to
  lift bans and publish unbans to the whole fleet with one redis-cli.
- Empty CORS_ORIGIN no longer means "allow any origin".
- ban_cidr / userspace blacklist validate and mask prefixes; CIDR bans are
  stored as ranges, so a /24 or /64 ban actually covers the subnet (before,
  only the network address was blocked).

Availability:
- ClickHouse client has connect/request timeouts and no mutex held across
  await: a hung analytics backend can no longer freeze the ban path. Events
  go through a bounded buffer with a drop counter.
- Redis uses a ConnectionManager with per-command timeouts; pubsub and
  reconcile reset the backoff after success; publish cannot block shutdown.
- Ban durations from pubsub are clamped and use checked_add; the sync task
  survives panics instead of silently dying.
- Blacklist TOCTOU (remove_if) and replace_redis no longer empties the table
  during a Redis reconnect.
- RateLimiter and reputation have hard caps with incremental eviction instead
  of an unbounded DashMap and a stop-the-world retain.
- The attack detector scores before updating its baseline and needs several
  quiet windows to leave UnderAttack, so a sustained flood keeps banning new
  offenders instead of ending after one tick.
- accept() backs off on EMFILE/ENFILE, shutdown Err is treated as shutdown,
  a global semaphore bounds in-flight connections, rate limiting runs before
  PoW, whitelisted IPs are never banned, and connect/handshake/relay have
  timeouts with an idle budget.
- PoW uses the configured difficulty as a floor, the clamp is enforced, and
  nonce framing survives split segments and coalesced handshake bytes.
- Subnet tracker memory is capped (prefixes and per-prefix sources) and the
  monitor bans with the configured TTL.

CI and docs:
- CI runs the test suite with --no-default-features --features protocol-http;
  make ebpf fails on errors; the secrets gate catches weak compose defaults.
- doctor skips protected endpoints without a token and exits non-zero on
  failure; a drain timeout returns exit code 1.
- TODO §2b records the review findings, what is fixed and what is deferred;
  configuration/testing docs describe the new XDP and secret semantics.
2026-11-16 12:06:23 +01:00

81 lines
2.6 KiB
Rust

//! Integration tests for the ClickHouse sink: buffering, bounded flush and
//! retry on failure.
use std::time::{Duration, Instant};
use rampart::store::clickhouse::{ClickHouseEvent, ClickHouseWriter, MAX_BUFFER_EVENTS};
fn event(seq: usize) -> ClickHouseEvent {
ClickHouseEvent {
timestamp: chrono::Utc::now(),
event_type: "test".to_string(),
ip: format!("192.0.2.{}", seq % 250),
data_float: 0.0,
data_int: 0,
data_string: String::new(),
}
}
#[tokio::test]
async fn push_only_buffers_events() {
let mut writer = ClickHouseWriter::new("http://127.0.0.1:1");
writer.push(event(0)).await.expect("push must succeed");
assert_eq!(writer.pending(), 1, "push must not flush eagerly");
assert_eq!(writer.dropped(), 0);
}
#[tokio::test]
async fn empty_flush_is_a_noop() {
let mut writer = ClickHouseWriter::new("http://127.0.0.1:1");
writer.flush().await.expect("empty flush must not touch the network");
}
#[tokio::test]
async fn failed_flush_requeues_the_batch() {
let mut writer = ClickHouseWriter::new("http://127.0.0.1:1");
writer.push(event(1)).await.expect("push must succeed");
let result = writer.flush().await;
assert!(result.is_err(), "connection refused must surface as an error");
assert_eq!(writer.pending(), 1, "failed batch must be re-queued");
assert_eq!(writer.dropped(), 0, "a single failed batch is not a drop");
}
#[tokio::test]
async fn stalled_server_flush_is_bounded() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.expect("bind");
let addr = listener.local_addr().expect("local addr");
let server = tokio::spawn(async move {
let (_stream, _) = listener.accept().await.expect("accept");
// Keep the connection open without ever answering.
std::future::pending::<()>().await;
});
let mut writer = ClickHouseWriter::new(&format!("http://{addr}/"));
writer.push(event(2)).await.expect("push must succeed");
let started = Instant::now();
let result = writer.flush().await;
let elapsed = started.elapsed();
server.abort();
assert!(result.is_err(), "a stalled server must fail the flush");
assert!(
elapsed < Duration::from_secs(5),
"flush must be bounded, took {elapsed:?}"
);
assert_eq!(writer.pending(), 1, "timed-out batch must be re-queued");
}
#[tokio::test]
async fn buffer_is_bounded_and_overflow_is_counted() {
let mut writer = ClickHouseWriter::new("http://127.0.0.1:1");
for seq in 0..MAX_BUFFER_EVENTS + 7 {
writer.push(event(seq)).await.expect("push must succeed");
}
assert_eq!(writer.pending(), MAX_BUFFER_EVENTS, "buffer must stay bounded");
assert_eq!(writer.dropped(), 7, "overflow must be counted");
}