Data Ingress using an Ephemeral Volume
This example demonstrates how volume creation,
data ingress, and
mounts work. We will create an ephemeral
volume (called eph_vol_1), accomplish
data ingress from a public GitHub repo, mount the volume
into a job
container, and access the data using the
program cat.
You can run this example unchanged, because it uses no secrets.
version: v4
volumes:
eph_vol_1:
use: ephemeral
size: 1GB
ingress:
- source:
uri: https://raw.githubusercontent.com/octocat/Hello-World/master/README
destination:
uri: file://README
jobs:
cat-file:
image:
uri: docker://alpine
mounts:
/data: eph_vol_1
script: |
#!/bin/sh
cat /data/README
resource:
cpu:
cores: 2
affinity: NUMA
memory:
size: 2GB
The job outputs the following text in the logs.
Hello World!
This pattern suits workflows that read publicly available data – a public git repository, an HTTPS URL, the HuggingFace Hub – and produce results small enough to read from the job logs. When you need to keep the output, use egress or a persistent volume instead.