Local LLM: Your Own Model, Your Own Machine
Running a language model on your own machine costs you nothing per token and keeps your data in the room. It also means working inside the memory you actually have.
What running it locally actually buys you
Three things. Your data stays on the machine, there is no bill per token, and it keeps working when the internet does. The price is that the model has to be small enough for the memory you have, small models are noticeably weaker, and you have to do the setup yourself instead of pasting an API key into a web page.
Privacy is the one that matters in a school. Anything you type into a cloud tool may be stored and may be used to improve the service. A model running on a Raspberry Pi in your classroom sends nothing anywhere, so a student can experiment with real data without a permission form, and you can find out exactly what the software does because you can read it.
The RAM arithmetic, and where the Arduino fits
A 4-bit quantised model needs roughly 0.7 GB of RAM per billion parameters, plus room for the context cache. So a 3B model is about 2.0 GB resident and an 8B model about 5.0 GB. On a Raspberry Pi 5 with 8 GB you can hold a 3B model comfortably and an 8B model only just, once the operating system has taken its share. Speed follows memory bandwidth, not cleverness: a 3B model generates roughly 5 to 7 tokens per second, a 1B model about 14, and an 8B model about 1.4, which is painful.
The microcontroller is not the model, and pretending otherwise is the usual mistake. The model does one job: turn a sentence into a number. Send that number over serial at 9600 baud, roughly 10 ms for a short command, and the Arduino acts on it within a few milliseconds while the model is still thinking about the next question. It is the same division of labour as a car: the model decides, the microcontroller drives.
Step by step
-
Do the RAM sum before you download
Parameters times 0.7 GB, plus about 1 GB for the operating system and the context cache. If it does not fit in your RAM, it will not run at a useful speed, however good the runtime is.
-
Start on a desktop
A laptop with 16 GB and a graphics card runs the same model many times faster. Get the prompt and the output format right there before you move to a slower board.
-
Move to the Raspberry Pi 5
Use llama.cpp or Ollama, pull a 1B to 3B model at 4-bit, and measure the tokens per second yourself rather than trusting someone else's benchmark.
-
Make the output a number
Ask for a single integer or a short keyword. A model that answers 15 cm is easy to parse, while a model that writes a paragraph needs a parser you have to write yourself.
-
Send it to the Arduino over serial
At 9600 baud the link carries about 960 bytes per second. Print the number, read it with a non-blocking parser, and act on it without blocking the sensor loop.
-
Add a timeout and a default
If nothing arrives within 500 ms, fall back to the last known good value. A model that hangs must never be able to freeze the robot.
What you'll need
| Item | Qty | Why |
|---|---|---|
| Raspberry Pi 5 with 8 GB | 1 | Quad-core Cortex-A76 at 2.4 GHz and 8 GB of RAM, which is the working ceiling for a 1B to 3B model at 4-bit. Around $130. |
| microSD card 32 GB | 1 | Holds the operating system, the runtime and the model files. A single 4-bit model download is a couple of gigabytes. |
| Arduino Uno R3 | 1 | The real-time half: it reads the sensors and acts within milliseconds, whatever the model is doing. |
| HC-SR04 ultrasonic sensor | 1 | The observation the model reasons about and the board reports. |
| LED bar graph module | 1 | Shows what the model decided, so you can tell a bad decision from a bad parse. |
| USB cable and jumper wires | 1 set | The serial link between the Pi and the board, plus the sensor wiring. |
Practise it before you build it
Every step above has a matching circuit in the ThinkEdge Learn simulator. The simulator runs a real
Arduino sketch against a modified-nodal-analysis solver, so digitalWrite actually drives the
LED on screen and the readings change when you change the wiring — no hardware, no burnt components.
The Mega has 4 hardware serial ports (Serial, Serial1, Serial2, Serial3). That means you can talk to 4 different devices at the SAME time — like a GPS, a Bluetooth module, an XBee radio, and the USB. Use 3 of those ports to read 3 different sensors in parallel and stream their data to the Serial Monitor as a CSV row every second. This is the foundation of every real Arduino data logger.
A real Arduino project almost always involves a sensor + a way to see the data. The DHT11 reads temperature and humidity, and the Serial Monitor (the window in the Arduino IDE) shows the readings. The Nano is perfect for this because it plugs directly into a breadboard with no extra wiring for power — you can build a complete data logger in 5 minutes.
A 10-segment LED bar graph is the easiest way to visualize an analog input: spin a potentiometer and 1, 2, 3 ... 10 LEDs light up in sequence. This is a "VU meter" — the kind you see on stereo equipment. The Mega is ideal because it has 10 free digital pins without stealing any from the analog or serial pins. The same code can be extended to 20, 30, or 100 LEDs on a Mega.
Mount an HC-SR04 ultrasonic sensor on top of a servo. Sweep the servo from 0° to 180° and back, and at each step measure the distance. Print the results to the Serial Monitor as "angle,distance" pairs. Open the Serial Plotter in the Arduino IDE to see a live radar-like graph of the room. With Processing on your computer, you can build a full visual radar.
A real reaction-timer game: an LED lights up after a random delay, and you have to press a button as fast as possible. The Nano measures the time between the LED turning on and the button being pressed, then prints the result in milliseconds. This is the kind of small project the Nano was made for — fits on a single breadboard, perfect for school science fairs.
In factories, conveyor belts carry products past a sensor that counts them. A simple "break-beam" sensor (a light source + photodetector) registers each object as it passes. In this project, you'll simulate the conveyor with your hand: every time you press the button (mimicking a product passing the sensor), the counter increments and the LED blinks. Add a buzzer to beep once for every 10 items. This is the same logic as a people counter at a door, a coin counter at a bank, or a bottle counter on a bottling line.
Common questions
How fast is a local LLM on a Raspberry Pi?
On a Pi 5 with 8 GB, a 1B model at 4-bit generates about 14 tokens per second, a 3B model about 5 to 7, and an 8B model about 1.4, which is slow enough to be frustrating. The bottleneck is memory bandwidth, so a cleverer model does not automatically run faster. Treat 5 tokens per second as the planning number for a classroom project.
Do I need a GPU to run a local LLM?
Not to start. A Pi 5 runs a small model on its CPU, and the Raspberry Pi AI HAT with an NPU helps but changes the speed by a factor rather than unlocking a much bigger model. A desktop with 16 GB of memory and a graphics card is where comfortable token rates come from, and that card is the single biggest cost in the whole build.
Is a local model better than a cloud one?
For privacy and offline work, yes. For quality, no, and the gap is wide. A 3B model running on a Pi will not write a better essay than a large cloud model, and it will confidently get simple factual questions wrong. Use it where the answer is a number you can check, such as a distance, a temperature or a motor speed.
Related guides
Try the full simulator with your class
103 guided circuit projects, 21 lessons and an in-browser Arduino simulator. The first 15 projects are free, no account needed. NCERT Science mapping on 47 projects, with Arabic and English copy in the regional builds.