AI
Managed Ollama on your own cluster
Open-weight language models on your own hardware, behind the API every Ollama client and most OpenAI-compatible SDKs speak.
Created inside a project, beside the apps that use it. Each project gets its own instance, storage and credentials.
A recent addition. It installs today, and the console marks it as new until it has run on a customer's cluster.
01What it does
Ollama, run for you
Ollama on the otwld community chart, pulling the models you name at start.
Run open-weight models where your data already is. Name the models to pull and the service downloads them at start, keeps them on its volume, and is not ready until every one is there. Bound apps get the host, port and URL, and most OpenAI-compatible SDKs point at it with one base-URL change.
A model needs roughly its file size in memory to run, so the size decides which models fit: a 7B model at 4-bit needs about 5Gi, so it wants the medium size. Switch on the GPU option and the service schedules onto a node with an NVIDIA GPU, which needs the NVIDIA device plugin on the cluster; without it the pod stays pending, and the setting says so.
02Options
What you choose
What the console asks when you create it, in the catalog's own numbers. Anything marked chosen once cannot change without creating a new instance.
- Sizes
- small 4Gi, medium 8Gi, large 16Gi of memory
- Storage
- 20Gi by default for model files, chosen once
- Models
- Any from the Ollama library, by name; llama3.2:1b by default
- GPU
- Optional; needs the NVIDIA device plugin on the cluster
- Pause
- Yes; the models stay on the volume
03Binding
What your app receives
Bind it to an app and these arrive as environment variables. The values are generated once into a secret and reach the app from there, rather than as something you copy.
- OLLAMA_HOST
- OLLAMA_PORT
- OLLAMA_URL
04Adding it
Add Ollama to a project
The same four steps as every service in the catalog.
- 01Open a project and go to Services.
- 02Pick it from the catalog and choose its options. Only the size is required; everything else has a default.
- 03Create it. Anyport installs it into the project's namespace, generates its credentials into a secret there, and tracks its health.
- 04Bind it to an app. The connection details arrive as environment variables, and rotating the credential never means editing an app.