Fuzzball Documentation
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Volume Operations

Suppose you have raw data in an AWS S3 bucket. You create a workflow with several jobs that process and analyze the data in multiple steps. You need to perform the following:

  • create (or locate) a volume for use within the job
  • copy the data from your S3 bucket to the volume
  • mount the volume in the first job, which performs preprocessing, then mount the same volume in the second job, which analyzes the preprocessed data
  • copy the analysis results back to your S3 bucket at workflow completion

These operations are called volume creation, ingress, mounting, and egress respectively. The following sections describe each one in turn:

  • Creation – naming rules, permissions, and creating a persistent volume before a workflow uses it
  • Ingress – the source schemes Fuzzball accepts, and pulling data in before the jobs run
  • Mounts – attaching a volume to one or more jobs
  • Egress – copying results out before an ephemeral volume is destroyed