Getting Node Details
Get detailed information on a specific compute node in your Fuzzball cluster.
From the Nodes page (see Listing Nodes), click any row to open a Node details dialog showing:
- Node Information: ID, hostname, CPU, and memory, plus cluster when the node reports a cluster ID, and the substrate version when the node reports it (older substrate versions omit it).
- Status: what Fuzzball has observed about the node, and when it last reported.
- Holds: what has been asked of the node – each hold’s effect, the reason recorded on it, who placed it, when it was placed, and when it was last refreshed if that was later. The strongest effect is listed first. Shown only when the node carries any.
- Attributed job failures: how many jobs stopped responding on this node, and when the last one did. Shown only when there are any.
- Resource Summary: total, allocated, and available cores and memory.
- Device Summary: per-device counts (total, requested, and available) when consumable devices are present on the node.
- Annotations: key/value annotations set on the node (for example,
nvidia.com/gpu.model), useful for identifying what to target with a workflow’s resource requirements. Shown only when the node has any. - Running Jobs: one row per job, labelled with the job name (or its ID when the job is unnamed), with a View workflow link that opens the job’s workflow at that stage.
- Events: the node’s recorded history – status changes, holds placed and released,
allocations added and removed, evictions, and attributed job failures – with the time and a
description of each. Recent shows the latest 50 events, newest first. Full history pages
through everything retained, oldest first, the same order as
fuzzball node events. Use the type selector to show one kind of event. Events are retained for 90 days.
Above the node’s status, the details offer the hold actions that apply to the node:
- Cordon: places a cordon hold, so no new work is placed on the node. Work already running is left alone. Shown until you have cordoned the node yourself.
- Uncordon: releases the cordon holds an administrator placed. Holds placed by monitoring or the provisioner, and evict or deprovision holds, stay in place, and the success message names any holds that remain. Shown only when an administrator cordon exists.
- Evict: places an evict hold, which takes running work off the node. A step whose
policy.retry.onincludesnodeis requeued on another node; anything else is failed. The node stays out of service until the evict hold is released. Shown until the node carries an evict hold.
Each action asks for confirmation first. Cordon and evict also take an optional reason, which
is recorded on the hold. If the server leaves the node unchanged, the error message gives its
reason. These actions do the same thing as fuzzball node cordon, uncordon, and evict. Only
a cluster administrator can use them; other users see them disabled, with a note saying why.
Releasing any other hold is done from the CLI. See
Node Cordoning and Uncordoning
and Evicting and Deprovisioning Nodes.
Get detailed information about a specific node by providing its node ID:
$ fuzzball node show <node-id>Example:
$ fuzzball node show 172.21.0.6/7333
ID: 172.21.0.6/7333
Hostname: substrate-localnode-1
CPU: cpu/arm64
Substrate: v2.7.0
Cluster: local-dev
Status: Ready
Reported: 2026-09-17T14:05:00Z
Holds: 1
7f3c1a90-2b11-4e6d-9a7f-2f0c5d8e1b42 cordon admin "failing disk, replacing Friday"
since 2026-09-14T14:02:11Z
Device plugins: 2
nvidia ready nvidia.com/gpu
amd failed "no AMD GPU found"
Resources:
Cores: 20 total (1 allocated, 19 available)
Memory: 127.9 GB total (1.0 GB allocated, 126.9 GB available)
nvidia.com/gpu: 1 total (0 requested, 1 available)
JOB ID | JOB NAME | REQUESTED CORES | REQUESTED MEMORY (GB) | REQUESTED DEVICES
fab5985c-3157-5e6c-b60b-f2fd8e17fefd data-processing 2 4.0 nvidia.com/gpu:1
e3211b77-117a-54ce-a2d9-420b4c673416 Hidden Job 1 1.0 NoneThe output includes:
- ID: Node identifier
- Hostname: Network hostname
- CPU: CPU model name or architecture
- Substrate: The
fuzzball-substrateversion running on the node, as reported by substrate itself. Only shown when the node reports it (older substrate versions omit it). - Cluster: Resolved name of the Fuzzball cluster this node belongs to, or the raw
cluster_idif the name cannot be resolved. Only shown whencluster_idis present — most visible when connected to a Federate cluster where nodes span multiple underlying Orchestrate clusters.
Each hold records something that has been asked of the node: its id, the effect
(cordon, evict or deprovision), who placed it, an optional reason, and
when it was placed. The section is omitted when the node carries no holds.
Holds are separate from status because they answer a different question. Status
reports what Fuzzball has observed about the node; a hold reports what somebody
decided about it. A node reading Ready with a cordon hold is not a
contradiction – the hardware is fine and no new work will be placed there.
| Effect | Means |
|---|---|
cordon | Place no new work here |
evict | Place no new work here, and take off what is already running |
deprovision | Place no new work here, and tear the node down once it has no allocations |
Only an administrator can place or release a hold. See Node Cordoning and Uncordoning.
Each device plugin fuzzball-substrate loaded or tried to load on the node:
its name, its state (ready or failed), the device kinds it serves, and the
error for one that failed. A failed plugin leaves its device kind out of the
node’s resources, so the scheduler passes the node over for every job
requesting that kind while the node stays Ready for work that needs no
device. The fix is on the node: repair the plugin and restart
fuzzball-substrate.
The section is omitted when the node reports no plugins, which is also what a substrate older than v2.7.1 sends. See Device Plugins.
- Status: What Fuzzball has observed about the node –
Ready,Not Ready,Offline, orUnknownbefore its first report. A cordon is a hold, not a status; see Holds - Reported: When the node last sent a health report. A node that has never reported says so rather than showing a zero timestamp.
- Attributed job failures: How many jobs stopped responding on this node, and when the last one did. Omitted when there are none.
For what each status means and how a silent node is detected, see Node Health Monitoring.
- Total: Capacity of the node
- Allocated: Currently in use by running jobs
- Available: Free capacity for new jobs
Fuzzball tracks consumable devices per device type:
- Total Count: Number of devices installed on the node
- Requested Count: Devices currently requested by jobs
- Available Count: Devices remaining that are not requested by jobs
For each job that is currently running on the node:
- JOB ID: Unique identifier for the job
- JOB NAME: Job name from the workflow definition
- REQUESTED CORES: CPU cores requested by the job
- REQUESTED MEMORY (GB): Memory requested by the job
- REQUESTED DEVICES: Consumable devices requested by the job
The entries shown are the workflow stages currently executing on this node. A
stage is any unit of work in a workflow, so this
list can include compute jobs, services, and the internal data-movement, image, and volume stages
that support them. When you do not have permission to view a stage’s workflow, it appears as
Hidden Job with only its ID visible.
By default the command prints the human-readable summary shown above. For
programmatic access or scripting, request structured output with -o json or
-o yaml:
$ fuzzball node show <node-id> -o jsonThe structured output includes additional low-level fields not shown in the default view, suitable
for parsing with tools like jq or yq.