Lesson 46 · 3 min
Running AI offline on your own device
You can run some AI models directly on your laptop or phone with no internet, which keeps your text private but usually means a smaller, less capable model.
Most chatbots run on a company's computers. Your message travels over the internet, gets answered there, and comes back. A local model skips all that: the model file sits on your device and does the work there.
You need three things: a free app that runs local models, a model file small enough for your device, and enough memory. A recent laptop can handle small and medium models. Phones increasingly include small built-in models for tasks like summarising or suggesting replies.
The big plus is privacy. Your text never leaves the machine, which helps with sensitive notes and it also works on a plane or with a bad connection. Once it is set up, it costs nothing per question.
The trade-off is power. A model that fits on your device is far smaller than the ones in big data centres. It will be slower on weaker hardware, it can't look things up online unless you connect it, and it makes more mistakes on hard tasks.
Do it now
Plan a local AI setup
- 1. Check how much memory (RAM) your computer has in its system settings.
- 2. Ask a chatbot the prompt below with your real numbers.
- 3. If you try it, install only well-known apps from their official sites.
My computer has [X] GB of RAM and a [processor/chip name]. I want to run an AI model offline for private note-taking and summaries. Explain in plain words what size of model I can realistically run, what I should expect in speed and quality, and what kind of free app I need. Don't recommend anything you are unsure exists.
Quick check
Words from this lesson
- Local model
- An AI model that runs on your own computer or phone, so your data does not leave the device.
- Edge AI
- AI that runs directly on a phone, camera or other device instead of on distant servers, so data can stay local.
- Data privacy
- Keeping personal or confidential information under control, including deciding what you type into AI tools and where it is stored.