Skip to main content

AFS_AETHER_JOURNAL_SYNC_BACKLOG_BYTES

Deep-dive reference for:

  • AFS_AETHER_JOURNAL_SYNC_BACKLOG_BYTES

Status​

  • active

Audience​

  • production/user

Type and Default​

  • Type: integer bytes (u64).
  • Default: 2097152 bytes (2 MiB).

What It Does​

Defines an early WAL flush trigger for journal sync when pending appended bytes reach the configured backlog threshold.

This is a journal durability cadence control, not a transport backpressure threshold.

Runtime Behavior​

In interval sync policy, each append increases a pending-byte counter.

When pending bytes reach threshold:

  • pending_bytes >= journal_sync_backlog_bytes and journal_sync_backlog_bytes > 0

the sync controller triggers an immediate WAL flush with reason backlog, resets pending bytes on success, and schedules the next interval deadline.

Policy interactions:

  • interval: backlog threshold is active.
  • manual: backlog threshold is ignored; flushes happen only via explicit manual flush.
  • immediate: background append tracking is skipped, so backlog threshold does not apply.

Input Surfaces​

  • Env: AFS_AETHER_JOURNAL_SYNC_BACKLOG_BYTES=<bytes>
  • Config file (aether.toml, top-level key):
    • journal_sync_backlog_bytes = <bytes>
  • Programmatic override:
    • ConfigOverrides.journal_sync_backlog_bytes

No dedicated CLI flag currently exists for this specific WAL sync threshold.

Resolution and Override Order​

  • Defaults < aether.toml < environment < overrides.

Value Rules​

  • Must parse as unsigned integer bytes (u64).
  • Unit suffixes are not accepted (2MiB is invalid; use raw bytes).
  • 0 disables backlog-triggered flushes.
    • In interval policy, timer-based flushing can still occur.

Important Interaction Notes​

  • Tune alongside:
    • AFS_AETHER_JOURNAL_SYNC_POLICY
    • AFS_AETHER_JOURNAL_SYNC_INTERVAL_MS
  • Lower values flush more often (lower pending-byte peaks, higher flush frequency).
  • Higher values flush less often (larger pending-byte bursts before forced flush).
  • Config reload can update this threshold at runtime; observers receive the changed sync config and update journal sync policy state.

Observability​

  • aether_journal_wal_flush_total with labels:
    • reason=backlog|interval|manual|shutdown
    • outcome=success|error
  • aether_journal_sync_count_total with label:
    • policy=backlog|interval|manual|shutdown

These counters confirm whether backlog-triggered flushes are actually firing.

Example​

export AFS_AETHER_JOURNAL_SYNC_POLICY=interval
export AFS_AETHER_JOURNAL_SYNC_INTERVAL_MS=5
export AFS_AETHER_JOURNAL_SYNC_BACKLOG_BYTES=1048576

This keeps interval mode active, but forces an early WAL flush whenever pending append bytes reach 1 MiB before the timer fires.

Primary Implementation Sources​

  • packages/aether/src/config/types/model_defaults.rs
  • packages/aether/src/config/types/file_config_loader.rs
  • packages/aether/src/config/types/config_resolution.rs
  • packages/aether/src/config/types/apply_env.rs
  • packages/aether/src/runtime/journal/sync.rs
  • packages/aether/src/config/reload.rs
  • packages/aether/src/metrics/names.rs