Ollama Model Setup Guide

Overview

STING uses Ollama for local AI capabilities, including the Bee chat assistant. This guide helps you set up the required models.

For system requirements and initial installation, see the STING Platform Installation Guide.

Quick Start

1. Check if Ollama is running

ollama list

If you get an error, start Ollama:

ollama serve

2. Install recommended models

For general use (including Bee chat):

# Recommended - Latest Llama model with good performance
ollama pull llama3.3

# Alternative lightweight option
ollama pull phi3

For code-related tasks:

# Excellent for code analysis and generation
ollama pull deepseek-coder-v2

3. Verify installation

ollama list

You should see your installed models listed.

Model Recommendations

Troubleshooting

“No models available” error in Bee chat

1. Check if Ollama is running: curl http://localhost:11434/api/tags
2. Install a model: ollama pull llama3.3
3. Restart the external AI service: ./manage_sting.sh restart external-ai

Model downloading slowly

Models can be large. Consider:

• Using a faster internet connection
• Installing smaller models first (phi3:mini)
• Downloading during off-peak hours

Bee chat shows “online” but doesn’t respond

This usually means no models are installed. The service is running but has no AI model to use.

Configuration

The default model is configured in /conf/config.yml:

external_ai:
  providers:
    ollama:
      defaultModel: "llama3.3:latest"

After changing the configuration:

./manage_sting.sh restart external-ai

Memory Considerations

• Ollama models are loaded into memory when used
• Ensure you have sufficient RAM (8GB+ recommended)
• Models are automatically unloaded when idle

For Apple Silicon and NVIDIA GPU acceleration, see the Hardware Acceleration Guide.

Using Your Models

After installing models:

1. Test Bee chat in the STING UI
2. Check logs if issues persist: docker logs sting-ce-external-ai
3. Try different models to find the best fit for your use case