Your first local chat with Ollama

1. Install and start Ollama

Follow the installation steps for Mac, Windows or Linux. Open Terminal or PowerShell and run ollama --version. Keep the Ollama app or service running.

2. Download a small local model

Run:

ollama run qwen2.5:1.5b

The official model page checked on 17 October 2026 lists a 986 MB Q4_K_M download. That is the download size, not total RAM use. The first run downloads it; later runs can reuse it. This model name selects a local model, not a cloud model.

3. Ask a question

At the chat prompt, type: “Explain a rainbow in three short sentences.” Wait for the reply. Type /bye to leave the chat.

In the terminal, ollama list shows downloads, ollama ps shows loaded models, and ollama stop qwen2.5:1.5b unloads this model.

4. Optional: use a Hugging Bay GGUF

Download a complete GGUF supported by Ollama from Hugging Bay and check its SHA-256. In a plain-text file named Modelfile, put the following line, replacing the quoted path with your actual file path:

FROM "/absolute/path/to/model.gguf"

Open a terminal in the folder containing Modelfile and run:

ollama create my-local-model -f Modelfile
ollama run my-local-model

On Windows, use the real path with forward slashes, such as C:/Users/you/Downloads/model.gguf. The model architecture and chat template must be supported; a GGUF extension alone is not enough. See Ollama's import guide.