Constructing an AI

Jump to Examples

For most of my life, a "thinking machine" was something from films and books. Today, a program that can explain a Linux command, write a short story or describe a photo runs on the same kind of computer I use for everything else. In my note Communication with the Machines I looked at how we talk to computers; this note is about the machines that now talk back.

Just like "the cloud", artificial intelligence is not magic. A modern AI model is a very large file full of numbers, and a program that does a lot of simple calculations with them very quickly. Once you understand that, running one yourself becomes a normal home-server project.

A Bit of AI History

In 1950 Alan Turing asked a simple question: can machines think? He suggested a test, later called the Turing test: if a person cannot tell whether they are talking to a human or a machine, the machine can be called intelligent. The term "artificial intelligence" was first used in 1956 at a summer workshop at Dartmouth College, which is often seen as the start of AI as a field of research. In 1958 Frank Rosenblatt's perceptron showed a very early idea of a machine that learns, loosely inspired by brain cells. One of the best-known early programs was ELIZA (1966), a simple chat program that acted like a therapist. It did not understand anything – it only recognised keywords and returned prepared answers – but many people still felt they were talking to someone real.

For a long time, AI meant writing rules by hand: if this, then that. In the 1980s, so-called expert systems were used in medicine and industry. They worked well in narrow areas, but they were expensive to build and could not learn anything new on their own. When the results did not meet the high expectations, funding dropped twice. These periods are known as the AI winters. In the same years, the method that trains neural networks today, backpropagation, became widely known (1986), but computers were still too slow to use it on a large scale. A famous moment came in 1997, when IBM's Deep Blue beat world chess champion Garry Kasparov. Deep Blue was not "smart" in a human way – it simply calculated millions of positions per second.

The big change came when researchers stopped writing rules and let computers learn from examples instead. This is called machine learning. Three things came together: much more data (thanks to the internet), much faster hardware (especially graphics cards, which are very good at doing many simple calculations at once) and better algorithms for neural networks. In 2012 a neural network called AlexNet recognised objects in photos far better than anything before it, and the era of deep learning began. In 2016 AlphaGo beat one of the world's best players of Go, a game long considered too complex for computers.

In 2017 researchers at Google published the paper "Attention Is All You Need", which introduced the transformer. Almost every modern language model is based on it. Soon after came models such as GPT (2018) and BERT (2018), and at the end of 2022 ChatGPT made AI something that ordinary people use every day. At the same time, image generators such as Stable Diffusion (2022) appeared.

For home users, the most important step was the release of open-weight models – models that anyone can download and run, starting with Meta's Llama in 2023 and followed by many others. Together with open-source tools such as llama.cpp and Ollama, this made it possible to run a capable AI on a normal computer, without the cloud. Today, AI models can work with text, images, sound and video, write and explain code, and use other programs to complete tasks. Large models keep growing, but small models keep getting better too – and that is the part that matters for a home server. At the same time, laws such as the European AI Act (2024) started to set rules for how AI may be built and used.

A short timeline:

Some resources for history:

• History of artificial intelligence (Wikipedia)
• Timeline of artificial intelligence (Wikipedia)
• Alan Turing – Computing Machinery and Intelligence (1950)
• Attention Is All You Need (2017)
• Computer History Museum

How an AI Is Constructed

When people talk about AI today, they usually mean a large language model (LLM). Building one happens in several steps. The table below is a simplified overview; each step is explained in a few words, not in full detail.

From raw text to an assistant:

What makes local AI possible:

A few observations. First, an AI model is a file and a program, nothing more mysterious than that – which is why it can run on your own hardware. Second, training and using are very different: training a large model from scratch needs a data centre, but using, adapting and connecting an existing model fits on a desk. Third, an AI predicts likely text; it does not check facts. For anything important, check the answer.

Some resources for how AI works:

• How AI Works – my deeper look inside, with code examples
• 3Blue1Brown – Neural networks (visual video series)
• Jay Alammar – The Illustrated Transformer
• Hugging Face Learn – free courses
• Elements of AI – a free introduction for everyone
• My Local AI resources

Examples: Building Your Own AI Assistant

You cannot easily train a large model from scratch at home, but you can do the next best thing: download an open model, run it locally, give it your own instructions, connect it to your own documents and let other programs use it. All tools used here are free; most are open source. The commands are a starting point, not a tested recipe for your exact hardware – model names and installation steps change quickly, so always check the current documentation of each project before you run anything.

Which model size for which machine? (my own rough guide, not test results):

Example 1 – Run a model with Ollama

1. Install Ollama. On Windows and macOS, download the installer from ollama.com. On a Linux server, the documented way is an installation script – as always, download it and read it before you run it:

   curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
   less ollama-install.sh
   sh ollama-install.sh

2. Download and run a model:

   ollama run llama3.2

   The first time, the model is downloaded (a few gigabytes). After that you can chat with it directly in the terminal; type /bye to leave. Other models are listed in the Ollama library – it is worth trying a few, because they differ in speed, size and how well they handle languages such as Slovenian.
3. Useful commands:

   ollama list          # show downloaded models
   ollama ps            # show running models and whether they use the GPU
   ollama rm llama3.2   # delete a model

Example 2 – Give the AI your own personality with a Modelfile

This is where the AI becomes "yours". You do not change the model itself, but you give it permanent instructions and settings.

1. Create a text file called Modelfile:

   FROM llama3.2
   PARAMETER temperature 0.6
   SYSTEM """
   You are the assistant of my home server JPNAS.
   Answer briefly and clearly. When you explain Linux commands,
   always explain what each part of the command does.
   If you are not sure about something, say so.
   """

2. Build your own model from it and run it:

   ollama create jpnas-assistant -f Modelfile
   ollama run jpnas-assistant

   The temperature setting controls how creative the answers are: lower values give more predictable answers, higher values more varied ones.
3. Use it from other programs. Ollama runs a small web service on port 11434, so scripts and applications can talk to your AI through a simple API:

   curl http://localhost:11434/api/generate -d '{
     "model": "jpnas-assistant",
     "prompt": "How do I check free disk space on Linux?",
     "stream": false
   }'

   This is the same idea as in Communication with the Machines: a machine offers a service, and other programs talk to it through a defined protocol.

Example 3 – A private ChatGPT for the whole household with Open WebUI

The terminal is fine for testing, but a web interface is more comfortable – and the whole family can use it from phones and laptops. Open WebUI looks similar to ChatGPT and runs on your own server. Install Docker or Podman first, as described in Constructing a Cloud.

1. Start Open WebUI as a container (check the current documentation for the exact command):

   docker run -d -p 3000:8080 \
     --add-host=host.docker.internal:host-gateway \
     -v open-webui:/app/backend/data \
     --name open-webui --restart always \
     ghcr.io/open-webui/open-webui:main

2. Open it at http://SERVER-IP:3000, create the first account (it becomes the administrator) and select your model, for example jpnas-assistant.
3. Add your own documents. Open WebUI can work with uploaded files (RAG), so you can add manuals, notes or recipes and ask questions about them. If you prefer a desktop application instead of a server, Jan or AnythingLLM do a similar job on one computer.
4. Mind the firewall. As explained in Constructing a Cloud, ports published by Docker can bypass ufw rules. Check what is really reachable, or publish the port only on the local address (127.0.0.1:3000:8080) behind a reverse proxy.

Example 4 – Connect it to the smart home and add voice

1. Home Assistant has an Ollama integration, so your local model can answer questions in the Home Assistant assistant – the same smart home as in Constructing a Cloud and the Network Door Opener. Give it control over devices carefully, and only to the ones you really want.
2. Speech: Whisper (or the lighter whisper.cpp) turns speech into text, also in Slovenian, and Piper turns text into speech. Together with a local model, this is a voice assistant that does not send your voice to anyone.
3. Images: for creating pictures locally, look at ComfyUI or InvokeAI. They need a good GPU.

Keeping the AI Safe and Useful

• Keep it inside your network. Ollama and Open WebUI are meant for the home network. Do not open their ports to the internet; use a VPN such as WireGuard instead, as in Constructing a Cloud and Security Considerations.
• Download models from trusted sources. Use the official Ollama library or well-known publishers on Hugging Face, and read the licence of each model – in Ollama with ollama show MODEL --license (see How AI Works).
• Check important answers. A model can invent commands, facts or sources. Never run a command from an AI on a server without understanding what it does.
• Be careful with automation. The more you let an AI control (files, devices, e-mail), the more damage a wrong answer can do. Start with read-only access.
• Update regularly. Ollama, Open WebUI and container images get frequent updates and security fixes.
• Watch power and heat. A GPU under full load uses a lot of electricity and gets hot. When something does not start or stops working, the logs are the first place to look – see Troubleshooting Linux.

A Few More Thoughts

AI has come a long way – from hand-written rules to models that learn from enormous amounts of text. Building a large model from scratch still needs huge resources, and that will stay in the hands of a few organisations for some time. But using, adapting and connecting an open model is now something anyone can do at home, with free tools and ordinary hardware.

Running AI locally is a trade-off, just like the cloud. A local model is usually smaller and less capable than the best commercial services, but it is private, works without the internet, has no subscription and teaches you how these systems really work. For me, the sensible answer is again somewhere in the middle: use the big services when you need their power, and keep at home what is personal, experimental or simply fun to run. After you have built a small AI on your own machine, the big ones stop being mysterious – they are the same ideas, only much, much larger.