# Adding a New LLM Model

**Step 1: Navigate to Models Section**

1. From the SoikaStack dashboard, locate and click on the **Models** section in the left sidebar.

![](https://soika-labs.gitbook.io/soika-mockingjay/~gitbook/image?url=https%3A%2F%2F2184813754-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FBSwkgyDGOD52nsc2FrK2%252Fuploads%252FRzZK7q5gSWbgcUB31Zz0%252Fimage.png%3Falt%3Dmedia%26token%3D784ddd31-96eb-49fb-aa0c-c9d0bb767807\&width=768\&dpr=4\&quality=100\&sign=e5baf1e9\&sv=2)

1. Click on **Add Deploy Model** button typically located in the top-right corner of the page and select catalog.

**Step 2: Select Model Type**

1. In the model selection screen, you'll see various model categories.
2. Select the **Large Language Models (LLM)** category.

![](https://soika-labs.gitbook.io/soika-mockingjay/~gitbook/image?url=https%3A%2F%2F2184813754-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FBSwkgyDGOD52nsc2FrK2%252Fuploads%252FGuRVpYBt5KptxAGSr7QL%252Fimage.png%3Falt%3Dmedia%26token%3D67f18a76-7dc8-4e30-aaa3-a767d6a53112\&width=768\&dpr=4\&quality=100\&sign=322c3c13\&sv=2)

**Step 3: Choose LLM Model**

1. Browse the available LLM models. You'll typically see options like:
   * Llama 2
   * Mistral
   * Falcon
   * Other open-source models
2. Click on your desired model to view its details.

**Step 4: Configure Model Settings**

1. Set the model configuration parameters:
   * **Name**: Give your model instance a descriptive name (e.g., "Llama-2-7B-Production")
   * **Description**: (Optional) Add details about the model's purpose
   * **GPU Selection**: Choose which GPU(s) to allocate for this model
   * **Memory Allocation**: Set the amount of VRAM to dedicate to this model
   * **Quantization**: Select the quantization level (e.g., 4-bit, 8-bit) if applicable
   * **Context Length**: Set the maximum token context window
   * **Advanced Parameters**: Configure model-specific parameters like temperature, top\_p, etc.

![](https://soika-labs.gitbook.io/soika-mockingjay/~gitbook/image?url=https%3A%2F%2F2184813754-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FBSwkgyDGOD52nsc2FrK2%252Fuploads%252FZkRjTsxUxsny8FvloYYl%252Fimage.png%3Falt%3Dmedia%26token%3D18ee8752-4868-4e86-b63c-b81ff472bea2\&width=768\&dpr=4\&quality=100\&sign=f10e921b\&sv=2)

**Step 5: Deploy the Model**

1. Review all settings once more.
2. Click on **Deploy Model** button.
3. The system will download and initialize the model, which may take several minutes depending on the model size and your internet connection.
4. You'll see a progress indicator during deployment.

**Step 6: Test the Model**

1. Once deployment is complete, navigate to the **Models** section.
2. Find your newly deployed model in the list.
3. Click on **Test** to open the testing interface.
4. Enter a prompt to verify the model is working correctly.
5. Click **Submit** to generate a response.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://soika-labs.gitbook.io/soika-mockingjay/soikastack-adding-llm-models-and-generating-api-keys/adding-a-new-llm-model.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.