Module 3 · Files, tools and your own data Lesson 9 of 15
Running and tuning a model on your own machine — Llama's own feature
The open weights are the feature. You can download a model, run it with no network connection, point it at your own files, and — if you get that far — tune it on your own data so it answers in a way no hosted assistant can.
The concept
Because it changes what you can build. A hosted assistant cannot be shipped inside your product, cannot be told to work with no internet, and cannot be tuned on a company's private data. Open weights can, and the interview sentence "I ran and fine-tuned a model" is not something most freshers can say.
Start with a small quantised model and a local runner. Get it answering at all before you optimise anything. Then measure: how much memory, how fast, how good on five questions you know the answers to. Only then decide whether a bigger model is worth the slowdown.
Here is the shape of it, in the form you will actually use:
# One-time: install a local runner (Ollama is the simplest start)
# -> https://ollama.com/download
ollama pull llama3.2:3b # small, fast, runs on a laptop
ollama run llama3.2:3b # interactive chat in the terminal
# A single question, scriptable, with no network involved:
ollama run llama3.2:3b "Summarise this in three sentences: [text]"
# See exactly what is on disk and how big it is:
ollama listCommon mistakes
The mistake is judging a 3B model by the standard of a frontier model and concluding that open models are bad. They are a different tool: smaller, private, free to run, and yours to change.
Worked example
A summariser that runs offline on your laptop and never sends a word anywhere is a two-evening project, and it is the clearest demonstration of what open weights mean.
Your practice task
Run one model locally, build one prompt around it, and write down the numbers: memory used, seconds per answer, and how it compared with a hosted model on the same job.
Do this before moving on. Reading a solution teaches the shape of it; typing it teaches you.
Key takeaway
Because it changes what you can build.
Keep your place in this course
A free account saves which lessons you have finished and unlocks the certificate. The lesson itself stays free to read.
Create free account or sign inCourse complete
You finished every lesson.
Your certificate is ready, and the projects you built are the proof to attach to an application.
Almost there
15 lessons left of 15
Finish every lesson to unlock the certificate. Each one ends in a practice task — do them and the final project brief writes itself.