Fuzzball Documentation
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Agent Models

The Fuzzball agent is the conversational assistant you hold a session with: you ask it for something and it answers. Acting on Fuzzball on your behalf – running workflows, reading resources – arrives when the hosted tool surface is wired up; in this release a session converses. It answers using an OpenAI-compatible inference endpoint. Which endpoint it uses is not configured on the server: it is a model registration you create with fuzzball model, so different users, groups and organizations can point the agent at different providers on the same cluster.

A session that does not name a model uses the registration called default, so register your main model under that name.

A registration is five things:

FieldMeaning
NameWhat you refer to the model by
EndpointThe OpenAI-compatible API root, e.g. https://api.openai.com/v1
Model idThe model to request from that endpoint, e.g. gpt-4o
SecretThe secret holding the credential, if the endpoint needs one
Context windowHow many tokens the model accepts in one request (--context-window), optional

The name and the model id are separate on purpose. A registration called fast can request gpt-4o-mini, and moving fast to another provider later does not change what anyone types.

Throughout this page a turn is one exchange in a session: your message, the model’s reply, and any tool calls in between. A registration is used at turn time, so a turn is also where a bad registration shows up.

Registering a model

Hosted providers need a credential. Put it in a secret of type openai, which holds the token and the base URLs that token may be sent to:

$ fuzzball secret create secret://user/openai --type openai --edit
$ fuzzball model add default \
    --base-url https://api.openai.com/v1 \
    --model-id gpt-4o \
    --secret secret://user/openai
Registered user model "default" -> https://api.openai.com/v1 (gpt-4o)

The --secret value is a reference, not a credential: secret://SCOPE/NAME, where SCOPE is user, group, organization or cluster. (account is also accepted as a deprecated alias for group, and means exactly the same thing; write group in anything new.) Beyond the scope and the name, nothing is accepted — no userinfo, no port, no query, no fragment, no ./.. segments, no percent-encoding in the name, and no leading or trailing whitespace.

Anything else in the reference is refused when you register or update the model, not stored. The reference is returned to every reader of the registration, which at group, organization or cluster scope is not only you, so an extra part – a token pasted into a query, say – would otherwise be disclosed to all of them.

Only a cluster-scoped model can use a secret://cluster/... secret. A cluster administrator can register one credential for everyone:

$ fuzzball model add default --scope cluster \
    --base-url https://api.openai.com/v1 \
    --model-id gpt-4o \
    --secret secret://cluster/openai

Any user who can use that model can then use the credential through it, but cannot read it, and the secret’s allowed base URLs still bound where it is sent. A user, group or organization model that references a cluster secret is refused when it is registered or updated. Otherwise anyone could send the cluster’s credential to an endpoint of their own choosing. Other secrets are read with the caller’s own access, so nobody can spend a secret they could not read themselves.

A model running inside the cluster, reached at its in-cluster address, usually needs no credential:

$ fuzzball model add house \
    --base-url http://vllm.fuzzball.svc:8000/v1 \
    --model-id llama-3.1-8b
Registered user model "house" -> http://vllm.fuzzball.svc:8000/v1 (llama-3.1-8b)
No credential attached. If the base URL is an endpoint on this Fuzzball cluster, each session calls it as its own user, so the endpoint's scope decides who can use it; any other endpoint must accept unauthenticated requests.

A plaintext http:// endpoint registers successfully but is not called unless the server allows it. Orchestrate requires https:// for inference by default, because a model that does carry a credential would send it in the clear on every request — and the check runs before anything knows whether this model has one, so it refuses unauthenticated endpoints too.

So the registration above succeeds and then every turn on it fails, saying why:

model "house" uses a plaintext http endpoint, and plaintext inference is disabled
on this cluster: register the model with an https base URL, or ask an
administrator to set spec.fuzzball.agent.model.allowInsecureInference (or
CONVERSE_ALLOW_INSECURE_INFERENCE=true on Orchestrate)

To use a plaintext endpoint, a cluster administrator sets it on the Fuzzball custom resource:

spec:
  fuzzball:
    agent:
      model:
        allowInsecureInference: true

Orchestrate reads it at startup, so it is an operator task, not something you can set per registration. Outside an operator deployment, setting CONVERSE_ALLOW_INSECURE_INFERENCE=true in Orchestrate’s environment does the same.

The setting applies to every registration on the cluster at once, and Orchestrate warns on every startup while it is on. Prefer an https:// in-cluster endpoint where you have one; if the endpoint is one you started yourself, Serving an AI Model gives you an https:// base URL to register.

A model served by a Fuzzball workflow

A model you serve as a Fuzzball workflow, for example from the catalog’s vLLM entry, needs no secret. Register its endpoint URL, with the path its API lives under:

$ fuzzball model add vllm \
    --base-url https://endpoints.fuzzball.example.com/endpoints/endpoint-<id>/v1 \
    --model-id Qwen/Qwen3-0.6B

When the base URL is one of this cluster’s endpoints, Orchestrate calls it as the session’s user. For every inference request it mints a short-lived endpoint token, exactly as fuzzball endpoint generate-token would for you, bound to that one endpoint and expiring within the inference timeout. It sends that token in FB-Authorization. The service proxy checks it, removes it, and passes the workload a signed identity for the caller instead, which the catalog’s LiteLLM proxy accepts in place of its master key. Your own Fuzzball access token is never sent to a model.

So the endpoint’s scope is the access control, as it is for a browser or curl:

Endpoint scopeWho can use the model
userOnly the user who started the workflow
groupMembers of the workflow’s group
organizationMembers of the workflow’s organization
publicAnyone. Members of the workflow’s organization are called with an endpoint token; anyone else, or anyone when the cluster cannot mint them a token, is called with no Fuzzball credential, as their own curl would be

Anyone else gets a refusal naming the model, without any detail about whose it is. When the workflow has stopped, the turn reports the model as unavailable and names the workflow that last served it, rather than timing out:

model "vllm" is unavailable: its endpoint is gone; it was last served by workflow vllm-run (1b2c...),
which is canceled; a new run of the workflow serves at a new endpoint URL unless it attaches to a
persistent endpoint, so update the model's base URL after restarting it

The workflow is named only to the user who started it. Everyone else, and these cases, get the same message without the name:

  • the model stopped before any turn reached it: Orchestrate records which workflow is serving a model’s endpoint each time a token is minted for it;
  • the model’s base URL was changed since it last answered.

An ordinary endpoint’s URL is derived from its workflow, so a new run of the workflow serves the model at a new URL; update the registration with fuzzball model update vllm --base-url .... To keep one URL across restarts, serve the model on a persistent endpoint. Between runs, a persistent endpoint has no workflow behind it, and a turn reports:

model "vllm" is unavailable: its Fuzzball endpoint is not serving right now: no workflow is running behind it, it is still starting, or the endpoint is briefly unavailable

If a long turn outlasts your own sign-in, the next call reports that your sign-in expired; sign in again and resend.

If the model also needs a key of its own, register it with --secret as usual. The key goes in Authorization next to the endpoint token, unless it is itself shaped like a Fuzzball token, which the service proxy removes before the request reaches the model.

The endpoint has to be reached at its endpoint URL, through the service proxy: an address inside the cluster network gets no endpoint token. The certificate the endpoint URL presents must be one Orchestrate trusts. Operator deployments already trust the CA that signs the cluster’s own endpoints, including a private or custom one. Elsewhere, add that CA to the trust store Orchestrate runs with; a call to an endpoint it does not trust fails with a certificate error.

If you omit --model-id, it defaults to the name.

--base-url must be http or https and must name a host. A URL of any other scheme is refused when you register it, rather than accepted and then failing on your first turn. So is one that no secret’s allowed base URLs could ever match: whitespace at either end, a host that is not an IP address or a hostname of letters, digits, - and _ (write an internationalized host in its punycode xn-- form; an IPv6 zone is not supported), or a path with an encoded slash (%2F), *, an empty segment (//), a . or .. segment, a backslash, a byte outside printable ASCII, or anything still percent-encoded after one decode.

A base URL is not private. fuzzball model list and fuzzball model show return it to everyone who can see the registration, and at organization or cluster scope that is not only the person who created it. Query values are hidden when a model is read back (?api-version=REDACTED), but the rest of the URL, including the query’s parameter names, is visible to all of them.

A username or password in the URL is refused outright at registration, because inference authenticates with the secret’s token and never with URL credentials, so there is no legitimate reason for one. A fragment – anything from a # onwards – is refused for the same reason: no HTTP request ever sends one, so it cannot affect which endpoint is called, and it would only sit in the registration and come back out of every read.

A query string is permitted, because real endpoints need one: Azure OpenAI wants ?api-version=2024-02-01. The stored URL keeps its query values and inference sends them; only reads hide them. Still, do not put a credential in the query. It belongs in the openai secret, which is never returned by any read path.

Because reads hide query values, a base URL with a query, copied from model show, cannot be registered or set with model update --base-url: it is refused with “re-enter the full base URL”. Updating any other field leaves the stored URL as it is.

--context-window is optional, but set it. It records how many tokens the model accepts in one request, and fuzzball model show displays it. The agent sizes each request to fit it (see Long conversations). Left unset, the window is stored as 0, which means “not set” (model show prints not set), never a limit of zero tokens, and the agent assumes 32,000 tokens. That guess is kept small on purpose, because a guess that is too large risks a provider silently cutting off the prompt, but it is too small for the agent’s own tools. So fuzzball model add warns when the flag is left out, and fuzzball model update --context-window 0 warns when it clears the window. Each warning ends with the model update command that sets the window. After model add, that command names the new model by id when its scope is wider than user, because a name reaches only the most specific registration, and also when the name itself looks like an id, since model update tries such an argument as an id first. After model update, it repeats the name or id you typed, which has just reached that model:

warning: no --context-window given; the model's context window is recorded as 0 (not set), and the agent assumes 32,000 tokens, too few for its tools. Set the model's real window: fuzzball model update --context-window N -- default

Long conversations and the context window

Before every call to the model, the agent builds a view of the conversation sized to that model’s window, keeping 35% of the window free for the reply. The stored conversation is never changed: switching a session to a model with a smaller window only changes what is sent, and switching back sends the earlier turns again.

When the conversation does not fit, the agent reduces the view in this order:

  1. It drops the output of older tool calls, keeping the record of each call. An evicted output is marked as evicted.
  2. It replaces older turns with a summary, written by the same model and reused on later turns. Anything the summary does not cover, including when a summary cannot be written, is marked as left out.
  3. It keeps the last three turns as they are. As a last resort it drops the output of tool calls inside them, oldest first, so a turn that reads more than the window holds can continue. Your messages and the agent’s replies are never dropped.

A turn the model cannot take fails with an error that says why, naming the model:

Error saysWhat to do
The agent’s instructions and tools alone need more than the model hasIf no window is registered, register the real one: fuzzball model update --context-window N -- NAME. Otherwise use a model with a larger window, or grant fewer permissions, which offers fewer tools
This message is too large, or does not fit beside the recent turnsSend a shorter message or less context; it is not recorded
The most recent turns need more than the model hasSwitch to a model with a larger window, or start a new session
The model rejected the request as too largeCheck the window registered for the model, or a request-size limit on a proxy in front of it

Token counts are estimated, not exact, and the estimate is deliberately generous, because different models count tokens differently. Data such as base64, hashes, numbers and log lines is counted at about one token per byte. Because of that, the agent’s own tools are counted at about 35,000 to 47,000 tokens, depending on the permissions the session is granted. The smallest context window that works is therefore about 54,000 tokens with the fewest permissions and about 72,000 with all of them; a window below that for the session’s permissions refuses every turn.

Scopes, and how they shadow

A registration lives at one of four scopes:

ScopeOwned byRegistered with
userYou--scope user (the default)
groupYour currently selected group (not a personal account)--scope group
organizationYour organization--scope organization
clusterThe cluster--scope cluster

A personal account – the one every user gets, yours or anyone else’s – cannot hold a group-scoped model. Asking for --scope group while one is selected is refused: select a real group to publish to it, or register the model for yourself at user scope where the cluster allows that (see allowAddModel below). The refusal suggests whichever of the two the cluster permits.

A user-scoped model is visible only to you, whichever group you have selected. That holds for an organization owner too: selecting a member’s personal account does not show them the member’s own models.

Unlike secrets, model scopes shadow rather than accumulate. A name always resolves to exactly one model: the most specific definition visible to you. So a cluster administrator can publish default for everybody, an organization can point the same name at its own endpoint, and you can override it again with your own — and every session that says default gets the nearest one.

fuzzball model list shows the models you would actually get, one row per name:

$ fuzzball model list
NAME    | SCOPE | SOURCE                      | MODEL         | ENDPOINT
default | user  | registered                  | gpt-4o        | https://api.openai.com/v1
house   | user  | registered                  | llama-3.1-8b  | http://vllm.fuzzball.svc:8000/v1
qwen    | group | discovered (6b1f0c2e-...)   | Qwen/Qwen3-8B | https://endpoint-3f2a....endpoints.fuzzball.example.com/v1

The secret reference and the context window are not in the listing — they are long, and fuzzball model show NAME prints them along with everything else about one registration.

model list fetches every page itself, so there is no cursor to carry. Use --page-size to change the server batch size and --max-pages to cap a runaway listing; a capped result says so on stderr, so it cannot be mistaken for a complete one when piped.

Add --all to see the definitions those are hiding:

$ fuzzball model list --all
ID                                   | NAME    | SCOPE        | MODEL         | SHADOWED
7c9e6679-7425-40de-944b-e07fc1f90ae7 | default | user         | gpt-4o        |
1f2d3c44-9a10-4bb2-8c31-5de0a1b2c3d4 | default | organization | gpt-4o-mini   | hidden
9b81c0aa-2f44-4c6e-9a77-30cf2e5b7a10 | house   | user         | llama-3.1-8b  |
discovered (6b1f0c2e-...)            | qwen    | group        | Qwen/Qwen3-8B |

The id column appears only with --all, in place of the endpoint. A discovered model has no id, so its cell names the workflow serving it instead (with -o json or -o yaml its id is empty, and source and workflow_id are fields of their own). The id is there because that is the view whose job is to answer “why am I not getting the model my administrator set up” — and the answer usually ends in inspecting the definition that is being hidden:

$ fuzzball model show 1f2d3c44-9a10-4bb2-8c31-5de0a1b2c3d4
Name:      default
Id:        1f2d3c44-9a10-4bb2-8c31-5de0a1b2c3d4
Scope:     organization
Endpoint:  https://api.openai.com/v1
Model:     gpt-4o-mini
Secret:    secret://organization/openai
Context:   128000 tokens

This definition is hidden by a more specific one of the same name.
Created:   2026-08-19 10:14:02AM
Updated:   2026-08-19 10:14:02AM

show, update and remove all take either a name or an id, and act on registrations. A name resolves to your most specific registration of that name, so model show default describes the registration a session would get rather than one it shadows — reach a hidden registration by its id instead. When a model discovered on a running workflow answers the name (see Discovered models), a name still picks that registration, which the discovered model is hiding.

Nothing stops you naming a registration so that it looks like an id. If you do, the name still works: an id-shaped argument is looked up as an id first and falls back to the name search when no such registration exists. A genuine failure — not authenticated, cluster unreachable — is reported as itself rather than reworded as “no model named …”, so the fallback cannot hide a real problem.

Publishing at a scope above your own is an administrative act. Registering, updating or removing a group or organization model requires being an owner of that group or organization (an owner synced from a federation does not count, just as they cannot add a secret there), and cluster scope requires a cluster administrator.

When a cluster restricts who may register a model

A cluster can set a floor: the narrowest scope a registration may be created at. It is an operator setting on the Fuzzball custom resource, not something a user can change:

spec:
  fuzzball:
    agent:
      model:
        allowAddModel: organization

The floor admits its own scope and every broader one. organization therefore allows organization and cluster registrations and refuses user and group ones; cluster allows only cluster registrations, and so in practice only cluster administrators, since that is the authority cluster scope already requires. Omitting the field, or leaving it empty, means user — every scope is allowed, which is how a cluster behaves if it says nothing.

The floor applies when a registration is created. It does not restrict updating or removing one that already exists, so raising it later does not strand the registrations people already have. Registering below the floor is refused, naming both the floor and the scope that was asked for:

$ fuzzball model add house --base-url https://api.example.com/v1 --model-id llama-3.1-8b
this cluster only registers models at organization scope or broader; user scope is not permitted
A value the cluster cannot parse stops the agent (fuzzball-agent) at startup rather than being ignored. A misspelled floor that silently fell back to “allow everything” would be the one failure mode worth avoiding here.

Sessions stay in the group they started in

A session records the group, organization and cluster you had selected when you created it, and every turn resolves its model against those. A turn is refused if you have since selected a different group, organization or cluster:

session "NAME" was created under a different group, organization or cluster than the one you have selected; switch back to it, or start a new session here

The model comes from the session’s group while the credential is read as you, in the group you have selected now; refusing keeps the two from coming from different groups. Switch back, or start a new session where you are.

Changing and removing a registration

Only the fields you name are changed:

$ fuzzball model update default --model-id gpt-4o-mini
Updated user model "default" -> https://api.openai.com/v1 (gpt-4o-mini)

A model’s name and scope are not editable. Together they are what sessions resolve against, so changing either would silently re-point every session that named it. Remove and re-add instead.

Two flags can be cleared, and two cannot:

FlagWhat clearing it means
--secret ""Detach the credential, for an endpoint that no longer needs one
--context-window 0Record the window as 0 (not set); the agent assumes 32,000 tokens (see above), and the CLI warns
--base-url ""Refused. A registration with no endpoint cannot answer a turn
--model-id ""Refused. Same reason

update applies the same rules to a field as add does. An update that accepted what a create rejects would be a way to reach a state the API says is impossible, and the failure would surface much later — at the point a turn tries to call the endpoint, rather than at the command that broke it.

Removing a registration removes only the registration:

$ fuzzball model remove default
Removed user model "default"

The endpoint keeps running and the secret is untouched — the agent just stops being able to reach it by that name.

Where a credential may be sent

An openai secret carries a list of allowed base URLs, and that list is not optional. The credential is spent by the server on your behalf, so the allow-list is what bounds a mis-registered model: if a model’s endpoint is not covered by its secret’s allow-list, the turn is refused rather than the token being sent anyway.

The allow-list is checked each time the model is used, not when it is registered. Registering a model whose endpoint its secret does not allow succeeds, and every turn on it is then refused, naming the model, the endpoint and the secret. It also means tightening a secret’s allow-list takes effect on the next turn.

If the endpoint itself rejects the credential with HTTP 401 or 403, the turn fails with a message naming the model, the secret and the HTTP status, plus the provider’s error type and code when they are known ones, for example:

model "gpt": its endpoint refused the credential from secret "secret://user/openai"
(inference: the endpoint returned HTTP 401 (invalid_api_key, invalid_request_error)):
check that the secret's key is valid for this endpoint

A 403 can mean the key is invalid, or that it is not allowed to use this model or endpoint, for example by a region, organization or network rule; the provider’s code, when shown, says which. A model with no secret gets a message saying so. The rest of the provider’s response is not shown, because it may echo the key.

Only a cluster-scoped model may use a cluster-scoped secret. Every other credential is read as whoever is taking the turn, so that nobody can spend a secret they could not read themselves. A cluster secret is not readable that way, so for a cluster model Fuzzball reads it with its own service token and logs the user whose turn spent it. A user, group or organization model that names a secret://cluster/... reference is refused when it is registered.

A registered endpoint is called directly. If it answers with a redirect, Fuzzball does not follow the redirect and the turn fails: a redirect could otherwise carry the credential to a host the allow-list never covered, or downgrade the connection to plaintext.

Discovered models

A model served by a running workflow is available to the agent without registering it, when the workflow’s endpoint says what it serves. The endpoint annotations are the ones the catalog’s vLLM, LiteLLM, opencode and hermes-agent entries already use:

AnnotationsWhat is discovered
ciq.com/api: openai and ciq.com/model: NAMEOne model, called NAME
ciq.com/api: openai-gatewayEvery model the endpoint’s /v1/models lists
      endpoints:
        - name: openai
          port-name: openai-api
          protocol: http
          type: subdomain
          scope: group
          annotations:
            ciq.com/api: openai
            ciq.com/model: qwen

Discovery runs as you, so you discover only endpoints whose scope admits you, and each discovered model takes its endpoint’s scope: user, group, organization, or, for a public endpoint, the cluster-wide level. It is called exactly as a registration of that endpoint would be (see above), so no secret is needed. The replicas of a pool are one model, and a model you can reach directly is not listed again through a gateway in the same workflow that re-serves it, which is how the catalog puts LiteLLM in front of vLLM.

Discovered models appear in fuzzball model list with source discovered and the workflow that serves them, and they join the same shadowing order as registrations, with one rule on top: a model on someone else’s endpoint, or listed through any gateway, never shadows a registration, or a model you serve yourself, at any scope. Registering at group, organization or cluster scope is reserved to that scope’s owners, and starting a workflow is not a way around it, so an administrator’s registrations, default included, stay put. A gateway counts as someone else’s even when it is yours, because a gateway can re-serve other people’s endpoints – the catalog’s LiteLLM serves every model its owner can reach – so a name it lists answers only when nobody has registered it and you serve no direct endpoint of it. A model on a direct endpoint of your own (ciq.com/api: openai) shadows like a registration of yours at its scope, and at the same scope a registration still wins. The catalog’s vLLM entry has such an endpoint only with Proxy=false. By default (Proxy=true) it exposes only its LiteLLM gateway, so its model answers a name only when nobody has registered it and you serve no direct endpoint of it; start it with Proxy=false to make the model yours. When several workflows serve one name at one scope, the one that has been running longest is used, and the others are listed as shadowed with --all. A session records the name, not the workflow. A turn tries one: when the workflow it would use has stopped, the turn reports it and does not move the conversation anywhere else,

model "qwen" is unavailable: workflow 6b1f0c2e-..., which served it, has stopped; the next turn looks again

and the next turn uses whatever serves the name then – the other workflow, or one you started in its place. If nothing does, it reports the model as unavailable, names the workflow that last served it, and says how to start or register one; the next turns say the same until something serves the name. If the endpoint refuses you, or cannot be reached, the turn reports that. When discovery cannot see everything (the endpoint listing fails, a gateway’s models cannot be listed, or a workflow’s start time cannot be read), and your last turn on that name in that group went to a discovered workflow that might be among what is missing, the turn reports:

model "qwen" is unavailable: the workflows serving it cannot be listed right now; try again shortly

The conversation is not moved to another workflow, or to a registration that may be an external provider, in the meantime. Otherwise the turn uses what discovery did see, and the next turn looks again. Which workflow – and which endpoint, for a persistent one – you last used is kept in the model database, per user, group and name, so it holds across a restart and on every replica. Only your own turns read or change it. Keeping it is best effort, not a guarantee: if recording a turn fails, or two of your turns on the same name run at once on different servers, the record can be missing or out of date, and a later turn that also cannot see everything may then be moved.

On a federate cluster, models are discovered from the endpoints of the orchestrate clusters it federates, and their workflows’ start times are read from the copies replicated to it. A workflow submitted straight to one of those clusters has no copy on the federate, so its start is never known; the same goes for any workflow whose copy cannot be read at the moment. Such a workflow is not waited for: a new session ranks it as if it had just started – among definitions that otherwise tie, after every workflow whose start is known, and among such workflows by workflow id – while a session already using it is moved only by a definition that would win whatever its start turned out to be. So on a federate a turn is not held back for a workflow it cannot read. A session leaves such a workflow once its endpoint is gone: a turn that the proxy answers with “Endpoint not found” reports

model "qwen" is unavailable: its Fuzzball endpoint is gone; the next turn looks again

and the next turn chooses afresh (with nothing left serving the name, it says there is no model). A gateway whose model list the proxy answers with “Endpoint not found” is gone too, and serves nothing. This is best-effort, on four counts:

  • If the cluster where such a workflow ran fails to tell the federate that its endpoint is gone, the federate keeps listing it, and while it still ranks first – your own endpoint, one at a narrower scope, or the only one serving the name – each turn says the same.
  • If that cluster fails to tell the federate that a persistent endpoint was unbound, the endpoint answers that it is not serving, and a session on it stays there.
  • In federations whose members run Fuzzball older than 4.4.0, a member answered “Endpoint not found” when it could not look an endpoint up at all, so a passing fault there can read as the endpoint being gone.
  • While the federate’s own workflow service is failing, a workflow that has since ended is not dropped (its start, and its end, are unknown). A session it still ranks first for – one already on it, or one where it is your own endpoint, at a narrower scope, or the only one serving the name – is told each turn that the workflow has stopped, until the service answers again.

A discovery that is missing something is not waited on again by every turn: for up to 30 seconds the server answers from what it last saw, holding back the turns that need holding back, while it looks again in the background. Discovery holds at most 10,000 models; past that, the gateways that do not fit count as ones that could not be listed. A gateway that answers with an empty model list is taken at its word, even one that is still loading its models. A model name longer than 256 characters, or one holding control characters, is not offered.

The server keeps what it found for up to 30 seconds. After starting a workflow, run fuzzball model discover to look now (it reuses any look less than 5 seconds old, a turn’s included, or joins one already under way). If some workflows could not be listed it says so, rather than that nothing serves a model.

A discovered default model

To make an in-cluster model the agent’s default for everyone its endpoint’s scope admits, with no registration, serve it under the name default: set ciq.com/model: default on its endpoint, and have the server answer to default as a model id (for vLLM, --served-model-name default). A session that names no model then uses it, unless a registration called default exists: a registration always wins over someone else’s endpoint, so to pin the default for everyone, register it. Until then, any member’s endpoint can be the default for everyone it admits – an organization-scope or public endpoint for the whole organization – when nothing more specific claims the name.

The catalog’s vLLM entry does both with its ServedModelName value:

$ fuzzball workflow catalog start vLLM --values Model=hf://openai/gpt-oss-20b,ServedModelName=default,Scope=group