Performance tuning parameters

Performance settings should be changed as a group and tested against the target workload. The examples below are YAML for brewfs mount --config; equivalent BREWFS_* environment variables are used by the benchmark runners.

Start with the safe baseline

This is the recommended starting point for Redis metadata and an S3-compatible data backend. It keeps the default upload_before_commit write path: after a successful fsync or close, both object data and metadata are durable and visible.

data:
  backend: s3
  s3:
    bucket: brewfs-data
    endpoint: http://127.0.0.1:9000
    region: us-east-1
    force_path_style: true
    disable_payload_checksum: true
    part_size: 16777216
    max_concurrency: 32

meta:
  backend: redis
  redis:
    url: redis://127.0.0.1:6379/0
  open_file_cache_ttl_ms: 30000
  open_file_cache_capacity: 65536
  allow_write_open_cache: false

cache:
  root: /var/cache/brewfs
  writeback_mode: upload_before_commit

fuse:
  workers: 1
  max_background: 512

Set cache.root to fast local SSD storage, not a network filesystem. The cache must have sufficient free space for the configured read and write SSD budgets.

Capacity and parallelism controls

chunk_size defaults to 64 MiB and block_size to 4 MiB. Keep block size aligned with the workload and object-store behavior; changing either affects object layout and should be evaluated on a fresh test dataset.

Read-heavy and mixed random I/O

For throughput-oriented reads, the benchmark profile increases cache capacity and FUSE concurrency. read_direct_io: true prevents large buffered FUSE reads from being split, but it also disables mmap for the affected read handles.

cache:
  root: /var/lib/brewfs/cache
  read_memory_bytes: 4294967296
  read_ssd_bytes: 4294967296
  write_memory_bytes: 4294967296
  write_ssd_bytes: 4294967296
  memory_budget_bytes: 12884901888
  prefetch_enabled: true
  prefetch_max_bytes: 67108864
  prefetch_concurrency: 64

fuse:
  workers: 6
  max_background: 512
  read_direct_io: true

For low-latency random reads, test with cache.range_background_prefetch: false as well. Background full-block fetches can help sequential access, but may compete with foreground reads on an object store. Compare p95/p99 latency, not only throughput, before leaving it enabled.

Large writes and S3 writeback

The high-throughput writeback profile commits metadata before asynchronous uploads. It is only valid with data.backend: s3; it is intentionally not a safe default.

cache:
  writeback_mode: commit_before_upload
  writeback_persist_sync: false
  writeback_require_stage_before_commit: true
  writeback_upload_concurrency: 6
  upload_concurrency: 32
  dirty_slice_target_size: 67108864
  dirty_slice_max_age_ms: 2000
  writeback_recent_pending_soft_bytes: 2147483648
  writeback_recent_pending_hard_bytes: 3221225472
  compression: none
  verify_cache_checksum: full

fuse:
  workers: 6
  max_background: 512

commit_before_upload can improve write throughput by making the local staging cache the immediate writeback target. If the process exits unexpectedly or that cache disk fails before queued objects reach S3, data already published in metadata can be missing; another client can temporarily observe the missing object. Use it only when this durability window is acceptable and monitor pending uploads. Keep upload_before_commit for the normal production durability contract.

compression: none removes compression CPU from benchmark-style large writes. It trades more network and object-store space for CPU throughput; lz4 remains the general-purpose default, while zstd favors storage reduction over CPU cost.

Metadata-intensive workloads

The open-file cache avoids repeated attribute and slice lookups. A conservative configuration caches read handles only:

meta:
  open_file_cache_ttl_ms: 30000
  open_file_cache_capacity: 65536
  allow_write_open_cache: false

For a single-client benchmark or a workload that explicitly accepts weaker cross-client close-to-open freshness, use this short-lived profile:

meta:
  open_file_cache_ttl_ms: 1000
  open_file_cache_capacity: 65536
  allow_write_open_cache: true

Do not enable allow_write_open_cache: true as a blanket multi-client production optimization. It allows non-append write handles to reuse cached attributes, which can delay observation of another client's changes.

Read-ahead, cache integrity, and bandwidth

Validate every profile

Use a fresh benchmark dataset, drain writeback before cold-read tests, and record the complete configuration alongside the result. Inspect BrewFS statistics for cache hits, background prefetches, pending writeback bytes, and metadata open-cache hit rates. The Benchmark methodology documents the fio profiles and reproducibility rules used for the published numbers.