Chat

Generate text from text.


POSThttp://localhost:33322/v1/chat/completions

Create chat completion

Given a list of messages belonging to a chat history, generate a response.

Required attributes

  • Name
    messages
    Type
    array
    Description

    A list of messages representing a chat history. It is essentially the context used by the model to generate a response.

  • Name
    model
    Type
    string
    Description

    The model used for chat completions. WARNING: currently, this attribute is ignored and the default model is used.

  • Default
  • Streaming

Request

POST
/v1/chat/completions
curl http://localhost:33322/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key-required" \
-d '{
  "model": "default",
  "messages": [
    {
      "role": "system",
      "content": "You are EdgenChat, a helpful AI assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}'

Response

{"id":"f403d6f4-4826-40b1-8798-77e4837e5041","choices":[{"message":{"role":"assistant","content":"Hello! How can I help you today?","name":null,"tool_calls":null},"finish_reason":null,"index":0}],"created":1708958149,"model":"main","system_fingerprint":"edgen-0.1.3","object":"text_completion","usage":{"completion_tokens":0,"prompt_tokens":0,"total_tokens":0}}

GEThttp://localhost:33322/v1/chat/completions/status

Chat completion status

Shows the current status of the chat completions endpoint (e.g. downloads).

Response attributes

  • Name
    active_model
    Type
    string
    Description

    The model that is currently active for this endpoint.

  • Name
    donwload_ongoing
    Type
    bool
    Description

    The model for this endpoint is currently being downloaded.

  • Name
    donwload_progress
    Type
    number
    Description

    The progress of the ongoing model download in percent.

  • Name
    last_errors
    Type
    string[]
    Description

    Errors that occurred recently on this endpoint.

Request

GET
/v1/chat/completions/status
curl http://localhost:33322/v1/chat/completions/status \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key-required"

Response

{"active_model":"neural-chat-7b-v3-3.Q4_K_M.gguf","download_ongoing":false,"download_progress":100,"last_errors":["Custom { kind: PermissionDenied, error: \"verboten\" }]}