No matches. Try a model format, tool name or error.
On this page
Open your download
Open a downloaded GGUF in LM Studio, Ollama or llama.cpp. Developer steps for safetensors and ONNX.
Next: open it
LFM2.5-350M-Q4_K_M.gguf
open it with LM Studio, Ollama or llama.cpp
LM Studio
Open LM Studio once. Run the import command below in your terminal and follow its prompts. Then open Chat, choose the imported model, wait for it to load and send a message. If lms is not found, see LM Studio's CLI setup instructions.
LM Studio setup and installlms import ~/Downloads/'LFM2.5-350M-Q4_K_M.gguf'
Other apps: Ollama, llama.cpp
Ollama
Run this in your terminal from the folder holding the file.
Get Ollamaprintf 'FROM ./LFM2.5-350M-Q4_K_M.gguf\n' > Modelfile && ollama create lfm2-5-350m -f Modelfile && ollama run lfm2-5-350mllama.cpp
Open your terminal and run. It opens a chat in the terminal.
Get llama.cppllama-cli -m ~/Downloads/'LFM2.5-350M-Q4_K_M.gguf'
Hugging Bay for Mac with Apple silicon
On a Mac with Apple silicon, try opening a supported GGUF in Hugging Bay. Whether it runs depends on the model and available memory.
Get Hugging Bay for MacIf double-clicking doesn't open the file, use one of the apps above.
Check it's the right file
This check proves the file arrived unchanged. Not that the model is safe.
Or check it in a terminal · SHA-256
shasum -a 256 ~/Downloads/'LFM2.5-350M-Q4_K_M.gguf'Compare with the recorded SHA-256: 7e6f72643caafc9a68256686638c4d7916f2cec76d1df478d4c3ddcd95a6aed4
Safetensors
This file is for developers. Load the model folder with transformers.
ONNX
This file is for developers. Load it with onnxruntime.