Run Llama locally on your computer

Start with a small local chat

Install Ollama using the Mac, Windows or Linux guide. In a terminal, run:

ollama run llama3.2:3b

The first run downloads the model from Ollama. Type a short question, then use /bye to leave chat. This command does not download a file from Hugging Bay. The linked Ollama model page documents the command; we have not measured its speed on your computer.

Use a GGUF you downloaded

Find the exact publisher, model version and file in Llama GGUF search. Check the file size, app support and upstream terms. A catalogue result can be source information only; check the file row for an actual download.

For a compatible local GGUF, save this line in a file named Modelfile, replacing the path with your file:

FROM /absolute/path/to/model.gguf

Then create a local Ollama name and open chat:

ollama create my-local-llama -f Modelfile
ollama run my-local-llama

Use the template and any extra files required by the model card and your runner. A GGUF extension alone does not establish support.

If it does not load

Start with Can I run this model?. Close other memory-heavy apps and use a modest context length. More context needs more memory. For a split GGUF, obtain all parts of the same release; one shard is not a complete model. Check a short answer against a known fact before using the model for your work.