Skip to content

Loaded Files

Every text generation related file you can load

Loaded Files tab in Launcher UI

  • Text Model:

    • This is the primary LLM you are attempting to run. It should generally be a GGUF file, but GGML are still supported in part.

  • Text Lora:

    • This is a low rank adapter for your LLM, making minor changes to it. generally you will never use this.

  • Lora Base:

    • The base model the LoRA was trained on, rather than the model it will be applied to.

  • Vision mmproj:

    • For models that support vision, this is what converts the image into tokens. it should be a GGUF file, and can often be found on the GGUF page for the base model of the model you are using.
  • Vision Force CPU:
    • Runs the mmproj entirely on CPU, allowing you to load more layers of the text model on the gpu. Useful if sending images is infrequent.
  • Vision MaxRes:
    • Images will be shrunk to where the larger side is this. a 2048x2048 will become 1024x1024. a 2048x1024 will become 1024x512, etc.

  • Draft Model:

    • A smaller version of the same model used in Text Model, will "draft" a response for the text model and the text model will "validate" the response. This is especially useful for larger base models that were trained on the exact same data as the smaller variant.
    • Draft Amount:
      • How many tokens to draft before the primary text model will validate. Increasing MAY allow faster processing, but will potentially cause it to just be ignored and generated by the main text model instead if the draft fails.
    • Splits:
      • Separate split ratios from the main text model, allows you to use a single very fast card for drafting.
    • Layers:
      • Same as the Layers in the quick launch and hardware tab, this sets how much of the model to offload. If you cant fully offload the draft model, then chances are that it will not provide enough speed benefit to be worth loading.

  • Embeds Model:

    • A model to convert text to Embeddings.

  • Preload Story:

    • You can export a story/conversation from Kobold Lite and set it here as the default. Allows defining settings as well as content so that you can more easily set defaults for other users on your system (if you have any).

  • SaveData File:

    • Server side save file to allow loading from anywhere.

  • Chat Completions Adapter:

  • Allow Launch Without Models:

    • This is useful for using KoboldCPP for features OTHER than text. For instance, audio and image generation.