Fuzzball Documentation
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Adding Dependent Jobs and Volumes

This page expands the workflow from A First Basic Workflow by adding a second job that runs after the first. The second job takes the fortune generated by the fortune command in the first job and reformats it with the cowsay command.

The two jobs need a shared place to write and read the fortune. So alongside the second job you will create an ephemeral volume, which shares data between jobs within a workflow.

This example requires that you are able to create ephemeral volumes. If you cannot do so, you may need to contact your administrator to gain the appropriate permissions.

Reopen the workflow from the last example in the Workflow Editor. If it is not already open:

  1. Select Workflows in the left-hand menu, then open the Runs tab.
  2. Find the workflow containing the fortune command and open it.
  3. Select Open in Editor in the upper right.

Workflow status page with the OPEN IN EDITOR button highlighted

First, we need to save the output from our fortune command somewhere. You can start by clicking on the job so that it is selected and the job configuration panel opens on the right hand side. You can edit the job so that standard output is redirected to a file called /tmp/fortune.txt.

The command should be this:

fortune >/tmp/fortune.txt

new command sending fortune standard output to text file

This is a good start, but /tmp is specific to this container and will not persist between jobs. So we need to set up a volume that the jobs (when we have more than one) can share.

Drag a volume from the toolbox at the top of the canvas onto the grid, as you did with the job:

A volume dragged onto the canvas with its Properties panel

Now define a new ephemeral volume for the jobs in this workflow to use. This example assumes a storage provisioner already configured by an administrator. The volume here has the reference label test-volume, which is the key that job mounts and other parts of the workflow use to refer to it. Set Mode to Ephemeral and give the volume a size: Fuzzball then creates it when the workflow starts and removes it when the workflow completes. That choice is what produces the use: ephemeral line in the generated Fuzzfile.

volume configuration for fortune and cowsay workflow

The Workflow Editor saves the volume configuration automatically.

Now select the fortune job card again and configure it to use the new volume.

Under Environment, select Add mount to attach a mounted volume.

The ADD MOUNT button on the job’s Environment tab

Select test-volume in the Volume drop-down and set Location to /tmp.

job configuration showing how to mount an ephemeral volume

The job fortune will now write its output to an ephemeral volume (that can be used by other jobs in the same workflow) instead of writing to /tmp within the container.

Now add a second dependent job to process the data the fortune job wrote. Drag a new job into place from the toolbox at the top of the canvas and name it cowsay. Then draw a line from the fortune job’s attachment point to the new cowsay job to indicate that cowsay depends on fortune.

workflow grid now shows a second job dependent on the first

Now configure the cowsay job with the correct command to read the data from /tmp/fortune.txt and pipe the output into the Cowsay program. The following should do the trick:

cat /tmp/fortune.txt | cowsay

The cowsay job’s Script field with the cat and cowsay command

As before, open the Environment tab. Set this job to run in the same container, since the lolcow container includes both fortune and cowsay, and mount test-volume at /tmp.

cowsay job environment configuration

To finish the workflow, you can allocate resources for the cowsay job. One core and 1 GB of memory are enough.

Once you have finished setting up the cowsay job you can hover your pointer over the test-volume card on the grid. The Workflow Editor highlights the jobs that use this volume with a glow effect.

jobs using volume highlighted in the workflow grid

Now submit the workflow as before with the Start button at the top right of the editor, optionally naming it. After it completes you should see something like this.

finished workflow showing logs from cowsay

This example breaks trivial operations into multiple jobs for illustrative purposes. In practice it is usually better to carry out more than one operation in the same job if it is easy to do so.

As in the last section, you can see the Fuzzfile at any time from the Workflow Editor by selecting the YAML tab in the panel on the right. Now that the workflow has a second job and an ephemeral volume to share, the Fuzzfile the Workflow Editor generates is longer than it was in the last section.

version: v4

volumes:
  test-volume:
    use: ephemeral
    size: 1GB

jobs:
  fortune:
    mounts:
      /tmp: test-volume
    image:
      uri: docker://wresch/lolcow
    script: |
      #!/bin/sh
      fortune >/tmp/fortune.txt
    resource:
      cpu:
        cores: 1
      memory:
        size: 1GB
  cowsay:
    depends-on:
      - name: fortune
        status: FINISHED
    mounts:
      /tmp: test-volume
    image:
      uri: docker://wresch/lolcow
    script: |
      #!/bin/sh
      cat /tmp/fortune.txt | cowsay
    resource:
      cpu:
        cores: 1
      memory:
        size: 1GB
Preview workflow in web UI

Because the example above mounts the same volume in every job, you can shorten the Fuzzfile by declaring default mounts that apply to all jobs in the workflow:

version: v4
volumes:
  test-volume:
    use: ephemeral
    size: 1GB

defaults:
  job:
    mounts:
      /tmp: test-volume

jobs:
  fortune:
    image:
      uri: docker://wresch/lolcow
    script: |
      #!/bin/sh
      fortune >/tmp/fortune.txt
    resource:
      cpu:
        cores: 1
      memory:
        size: 1GB
  cowsay:
    depends-on:
      - name: fortune
        status: FINISHED
    image:
      uri: docker://wresch/lolcow
    script: |
      #!/bin/sh
      cat /tmp/fortune.txt | cowsay
    resource:
      cpu:
        cores: 1
      memory:
        size: 1GB
Preview workflow in web UI

In the next section we will add a few more jobs that can run in parallel and show how data ingress and egress allow you to import and export data to your workflow.