Models

Discover how to manage your local models. OpenAI compliant.

Note that the model services handle only managed models. Models that you download yourself are ignored. To learn about manually downloaded models check Documentation » Models and Documentation » Configuration.

GEThttp://localhost:33322/v1/models

list models

Lists all local models.

Response attributes

  • Name
    object
    Type
    string
    Description

    The type of this object, always "list".

  • Name
    page
    Type
    integer
    Description

    Current page.

  • Name
    per_page
    Type
    integer
    Description

    Number of model descriptors per page.

  • Name
    data
    Type
    ModelDesc[]
    Description

    The model descriptors for all local models.

  • Name
    id
    Type
    string
    Description

    Unique identifier of this model.

  • Name
    created
    Type
    integer
    Description

    Unix Timestamp of the creation time of this model.

  • Name
    object
    Type
    string
    Description

    The type of this item, always "model".

  • Name
    owned_by
    Type
    string
    Description

    Owner of the model repository.

Request

GET
/v1/models
curl http://localhost:33322/v1/models \
  -H "Authorization: Bearer no-key-required"

Response

{
    "object":"list","data":[
        {"id":"TheBloke/neural-chat-7B-v3-3-GGUF","created":1708333035,"object":"model","owned_by":"TheBloke"},
        {"id":"TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF","created":1708333060,"object":"model","owned_by":"TheBloke"},
        {"id":"TheBloke/phi-2-GGUF","created":1708333062,"object":"model","owned_by":"TheBloke"},
        {"id":"distil-whisper/distil-small.en","created":1708333030,"object":"model","owned_by":"distil-whisper"}
    ],
    "page":1,
    "per_page":4
}
GEThttp://localhost:33322/v1/models{model}

retrieve model

Retrieve model information.

Response attributes

  • Name
    id
    Type
    string
    Description

    Unique identifier of this model.

  • Name
    created
    Type
    integer
    Description

    Unix Timestamp of the creation time of this model.

  • Name
    object
    Type
    string
    Description

    The type of this item, always "model".

  • Name
    owned_by
    Type
    string
    Description

    Owner of the model repository.

Request

GET
/v1/models
curl http://localhost:33322/v1/models/TheBloke%2fneural-chat-7B-v3-3-GGUF \
  -H "Authorization: Bearer no-key-required"

Response

{
     "id":"TheBloke/neural-chat-7B-v3-3-GGUF","created":1708333035,"object":"model","owned_by":"TheBloke"
}
DELETEhttp://localhost:33322/v1/models{model}

delete model

Delete local model.

Response attributes

  • Name
    id
    Type
    string
    Description

    Unique identifier of this model.

  • Name
    object
    Type
    string
    Description

    The type of this item, always "model".

  • Name
    deleted
    Type
    bool
    Description

    Whether the model was actually deleted.

Request

DELETE
/v1/models
curl -X DELETE http://localhost:33322/v1/models/TheBloke%2fneural-chat-7B-v3-3-GGUF \
  -H "Authorization: Bearer no-key-required"

Response

{
     "id":"TheBloke/neural-chat-7B-v3-3-GGUF","object":"model","deleted":true
}