Local models

Models that run on this computer: the engine, installed models, Hugging Face downloads and what fits. Paths are relative to /api/v1.

GET /runtime

The local model runtimes: whether each one is observed on this computer, and the models it has.

Answers The JSON envelope.

Errors1

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.

Any operation can also answer the common errors.

GET /runtime/bootstrap

The runtime at start-up, as the backend reports it (talos.harness-ui.runtime-bootstrap.v1): the one source the window trusts for what is running.

Answers The JSON envelope.

Errors1

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.

Any operation can also answer the common errors.

GET /local-models

The GGUF models installed on this computer.

Answers The JSON envelope.

Errors1

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.

Any operation can also answer the common errors.

GET /local-models/fit-estimate

Estimate whether a model of a given size would fit this machine, before downloading it (nothing is loaded).

Query

bytes

The size of the model file, in bytes.

contextTokens

The context length to plan for, in tokens (optional).

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

GET /huggingface/search

Search GGUF models on Hugging Face.

Query

query

The words to search for.

limit

How many results (default 20).

cursor

The cursor of the next page, from the previous answer.

sort

Order: downloads by default.

direction

-1 (descending, default) or 1.

author

Only models by this author.

filter

Hub filters, repeatable (gguf is always applied).

Answers The JSON envelope.

Errors1

Code	Status	Meaning
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

GET /huggingface/repo

One Hugging Face repository: its description and GGUF files, pinned to an exact commit.

Query

reporequired

the repository (owner/name).

revision

A branch or commit (default main), resolved to a commit before the files are listed.

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

GET /huggingface/image

An image of a Hugging Face model card, served through the local server.

Query

urlrequired

the address of the image.

Answers The bytes themselves (a file or an image), not JSON.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

GET /huggingface/downloads

The model downloads in progress or finished, with their state.

Answers The JSON envelope.

Errors1

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.

Any operation can also answer the common errors.

GET /model-lab/capacity

The capacity of this machine for local models: memory, graphics and what they allow.

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
REPORT_UNAVAILABLE	404	The report or service asked for is not available on this server yet.

Any operation can also answer the common errors.

POST /runtime/load

Load a model into a local runtime.

Body

runtimeIdrequired

The runtime.

modelIdrequired

The model to load.

contextLength

The context length to load it with, in tokens.

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /runtime/unload

Free the memory of a local runtime: stops it, and with it the model it holds.

Body

runtimeIdrequired

The runtime.

modelId

Optional and ignored: unloading stops the runtime.

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /local-models/import

Import a GGUF file from this computer into the model library.

The body is the file itself (application/octet-stream). Headers: x-talos-model-id (the id to give it), x-talos-model-filename, x-talos-model-bytes (the exact size) and, optionally, x-talos-model-name.

Body

None: the body is empty.

Headers

x-talos-model-bytesx-talos-model-filenamex-talos-model-idx-talos-model-name

Answers The JSON envelope.

Errors3

Code	Status	Meaning
LOCAL_IMPORT_INVALID	422	Check the GGUF file chosen and try again.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /huggingface/download

Download a GGUF model from Hugging Face, pinned to an exact commit and checked file by file.

Body

idrequired

The id the model gets in the library.

reporequired

The repository (owner/name).

revisionrequired

The commit, 40 to 64 hexadecimal digits.

filesrequired

The files, each { path, bytes, sha256 }.

bytesrequired

The total size in bytes.

pathrequired

The main file.

licenserequired

The license of the model.

Answers The JSON envelope.

Errors2

Code	Status	Meaning
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

GET /local-models/{modelId}/fit

Whether an installed model fits this machine, and with which settings: measured from the file, the machine and a probe of the runtime, never a real load.

Path

modelId

The installed model.

Query

profile

The profile to measure for.

contextTokens

The context length to plan for, in tokens (a positive integer).

Answers The JSON envelope.

Errors3

Code	Status	Meaning
FIT_INVALID	400	The parameters of the check are not valid.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /local-models/{modelId}/rename

Rename an installed model.

Path

modelId

The installed model.

Body

namerequired

The new name.

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /local-models/{modelId}/copy-path

The path of an installed model’s file, to copy it.

Path

modelId

The installed model.

Body

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /local-models/{modelId}/delete

Delete an installed model.

Path

modelId

The installed model.

Body

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /local-models/{modelId}/qualify

Qualify an installed model: run it for one real turn to measure how it behaves on this machine (it loads the model, so it needs your consent).

Path

modelId

The installed model.

Body

consent

true: you agree to load and run the model.

profile

The profile to measure for.

contextTokens

The context length, in tokens (a positive integer).

Answers The JSON envelope.

Errors3

Code	Status	Meaning
FIT_INVALID	400	The parameters of the check are not valid.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /huggingface/downloads/{downloadId}/pause

Pause a model download.

Path

downloadId

The download.

Body

None: the body is empty.

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /huggingface/downloads/{downloadId}/resume

Resume a paused model download.

Path

downloadId

The download.

Body

None: the body is empty.

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.

POST /huggingface/downloads/{downloadId}/cancel

Cancel a model download.

Path

downloadId

The download.

Body

None: the body is empty.

Answers The JSON envelope.

Errors3

Code	Status	Meaning
NOT_FOUND	404	Nothing answers at this address, or the resource it names does not exist.
QUERY_INVALID	400	The request is not valid: a query parameter or a body field is missing, unknown or malformed.
RUNTIME_NOT_AVAILABLE	503	The local runtime is not available.

Any operation can also answer the common errors.