Skip to main content

Streaming and binary flows

The public gRPC API is most useful where request and response bodies are large, continuous, or both.

Server-streaming RPCs​

These RPCs stream data from the server back to the client:

  • FileSystemService.ReadFile
  • FileSystemService.StreamFileRange
  • FileSystemService.DownloadSessionAsTarball
  • MessageBusService.Subscribe

Use them when you need:

  • large file reads without buffering the entire file in one response
  • range reads for partial hydration or preview workflows
  • tarball export without materializing the whole archive in memory client-side
  • a long-lived event or bus subscription

Client-streaming RPCs​

These RPCs stream data from the client to the server:

  • FileSystemService.WriteFile
  • FileSystemService.SyncChangesFromTarball
  • UnderlaySourceService.UploadSourceArchive

Use them when you need:

  • large file uploads
  • atomic tarball-based session sync
  • uploaded bundle or snapshot ingestion into managed sources

Why these flows fit gRPC better than HTTP​

The HTTP surface is strong for request and response control operations, but gRPC is the better fit when the interaction needs:

  • backpressure-aware streaming
  • fewer framing conversions for binary payloads
  • lower overhead for long-lived subscriptions
  • generated clients that understand typed streams

Binary payload conventions​

You will encounter bytes fields in several public messages. In practical terms:

  • grpcurl expects bytes values as base64 in JSON mode
  • generated clients let you send raw byte arrays or stream chunks directly
  • message-bus payload format is your own application contract
  • test reports and uploaded source archives should be treated as binary-safe payloads, not plain text assumptions

Design guidance for client authors​

  • Keep streamed file I/O separate from metadata RPCs.
  • Resume long-lived subscriptions with application-level cursors where the API offers them.
  • Use paginated manifest APIs for structure discovery, then stream file contents only when needed.
  • Use tarball sync or archive upload for bulk transfer, not repeated small mutation RPCs when you already have an archive representation.
  • Expect normal gRPC retry and reconnect behavior around transient transport failures, but make sure your own workflow is idempotent where needed.

RPCs to pay attention to first​

If you are building a serious gRPC client, the highest-value streaming RPCs to model early are:

  • FileSystemService.ReadFile
  • FileSystemService.WriteFile
  • FileSystemService.StreamFileRange
  • MessageBusService.Subscribe
  • UnderlaySourceService.UploadSourceArchive