This page is machine-translated from French. Read the French original.
Choosing your models
The language model determines the quality, speed, and cost of responses, as well as whether data falls outside your scope. WivenLLM does not impose any specific language model on you.
Three families
1. Integrated local models
WivenLLM downloads and runs open models itself, without any external dependencies. An integrated catalog offers models of varying sizes, from lightweight models with a few billion parameters to significantly larger ones. You can also search for a GGUF-format model on Hugging Face from the same screen.
- No data is sent out, no API keys are exchanged, and there are no usage costs. Only the model download is done over the internet; in Strict sovereign mode, the host must be authorized.
- The speed depends entirely on your equipment.
- The model is loaded into memory on first use and automatically unloaded after a period of inactivity (15 minutes by default), to free up resources.
- The integrated engine is suitable for individual use. For multiple users, an inference server is preferable.
The hardware diagnostics (Settings → Hardware Diagnostics) analyzes your machine and rates its ability to run each class of model (Smooth, Fair, Slow, or Insufficient). A short test, which you can run yourself, refines these ratings in CPU mode, and each installed model can be measured to determine its actual speed. In the model list, each catalog entry indicates its compatibility with your hardware: Ideal compatibility, Good compatibility, Will work, but slowly, or Insufficient RAM/VRAM. This is the fastest way to determine what to download. On the desktop, the application can also select and download a suitable model on first launch.
2. Internal Inference Server
Does your organization already operate a model server? WivenLLM connects to it as a client. The principle remains the same—nothing leaves the network as long as the server is on the company's machine or network—but your server manages the model lifecycle, not WivenLLM.
Any server exposing an API compatible with the OpenAI standard is suitable, including the most common local solutions (Ollama, LM Studio, vLLM, llama.cpp). The interface then displays the «Local Model» badge.
3. External Suppliers
Major commercial providers are supported, as well as Infomaniak, a Swiss hosting company. They generally offer the best quality and speed, with no hardware requirements.
In return, your messages and associated documentation are transmitted to the provider, and usage is billed. This choice must be made consciously and clearly communicated to users. A badge in the sidebar reminds them of this.
The PII shield (Wiven Console) can replace personal data with pseudonyms before sending, but it is disabled by default and does not detect everything: it needs to be activated and configured, by listing the terms to be protected.
What equipment is needed for a local model?
These orders of magnitude relate to the quantified models of the integrated catalogue.
| Model size | Useful RAM | No GPU | With a suitable GPU |
|---|---|---|---|
| 3–4 billion parameters | 8 GB | Usable | Very fast (from 3–4 GB of VRAM) |
| 7–9 billion | 16 GB | Slow but doable | Fast (from 6–10 GB of VRAM) |
| 14 billion | 24 GB | Difficult | Comfortable (from 10 GB of VRAM) |
| 32 billion | 36 GB and more | Not recommended | Comfortable with a suitable GPU (starting at 20 GB of VRAM) |
| 70 billion | 64 GB and more | No | Requires a high-end GPU (40GB or more of VRAM) |
Two points that are often underestimated:
- Memory is the hard constraint. A model that doesn't fit into memory won't start, or will run via disk swaps at an unusable speed.
- The GPU changes the scale, not the result. It multiplies the generation speed; it doesn't make a small model smarter.
The agent model may be different
A workspace distinguishes between two models:
- the conversation model, which answers questions (tab «Chat Settings»);
- the agent model, which controls the sequences of tools (tab «Agent Configuration»).
The agents are demanding more: The model must decide which tool to call, with which arguments, and interpret the result. A model that converses correctly can fail as an agent.
Recommendation : If agents matter to you, assign them the most capable model you have available, even if the current conversation revolves around a lighter model.
The embeddings engine is a separate choice
It doesn't write anything: it translates the text into vectors for document retrieval. The integrated engine runs locally and is suitable for most uses. Its default model is designed for English; a multilingual model is available as an option, but there's no guarantee of better results: test it on your documents.
Structuring constraint: All vectors in an index must come from the same engine. Changing the engine, or its model, empties the index and removes documents from all spaces: they must be added again. Decide this at startup.
An external engine may be justified for demanding multilingual corpora or very high search quality needs — at the cost of a network output for each indexing and each query.
Arbitrate
| Your priority | Configuration |
|---|---|
| Strict confidentiality | Integrated local model + native embeddings + local vector base + strict sovereign mode |
| Maximum quality | External provider, explicitly mentioned to users |
| Zero cost in use | Any premises |
| Speed on modest equipment | External provider, or shared internal inference server |
| Both, depending on the case | Template router: local by default, escalates to rules |
Change your mind later
- Change language model: without consequence. The documents remain indexed, the conversations remain readable.
- Change embedding engine: clears the index of all spaces; the documents must be added there again.
- Change vector basis: same consequence.
In other words, only the first of these three choices is truly reversible without effort. That's why the other two deserve careful consideration during installation.
