Debian 13 trixie

Ollama : Install2025/09/30

 

Install Ollama that is an application which allows you to run LLM locally.

It runs reasonably fast even on computers without a GPU.

[1] Install Ollama.
root@dlp:~#
wget https://ollama.ai/install.sh

root@dlp:~#
sh ./install.sh

>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
######################################################################################################################### 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink '/etc/systemd/system/default.target.wants/ollama.service' → '/etc/systemd/system/ollama.service'.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

# ollama service started

root@dlp:~#
systemctl status ollama

● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
     Active: active (running) since Tue 2025-09-30 14:20:43 JST; 38s ago
 Invocation: 7dceb73d796646f6b7d4ce41dd93f84a
   Main PID: 1158 (ollama)
      Tasks: 10 (limit: 19127)
     Memory: 13.6M (peak: 14.6M)
        CPU: 21ms
     CGroup: /system.slice/ollama.service
.....
.....

# ollama listens port 11434 on localhost

root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096       127.0.0.1:11434      0.0.0.0:*    users:(("ollama",pid=1158,fd=3))
[2] If you like to access it from other hosts, you can change the range it listens on.
root@dlp:~#
systemctl edit ollama.service
# add [OLLAMA_HOST] variable
# to listen all, specify [0.0.0.0]
# to listen internal network, specify internal IP address

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

root@dlp:~#
systemctl restart ollama.service
root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096               *:11434            *:*    users:(("ollama",pid=1206,fd=3))
[3] To use Ollama, download models first.
# example below is 8B model from llama3

root@dlp:~#
ollama pull llama3:8b

pulling 6a0746a1ec1a: 100% 4.7 GB
pulling 4fa551d4f938: 100%  12 KB
pulling 8ab4849b038c: 100%  254 B
pulling 577073ffcc6c: 100%  110 B
pulling 3f8eb4da87fa: 100%  485 B
verifying sha256 digest
writing manifest
success

# example below is phi4 model from Microsoft

root@dlp:~#
ollama pull phi4

pulling manifest
pulling fd7b6731c33c: 100%  9.1 GB
pulling 32695b892af8: 100%   275 B
pulling fa8235e5b48f: 100%  1.1 KB
pulling 45a1c652dddc: 100%    82 B
pulling f5d6f49c6477: 100%   486 B
verifying sha256 digest
writing manifest
success

# example below is Gemma model from Google

root@dlp:~#
ollama pull gemma2:2b

pulling manifest
pulling 7462734796d6: 100% 1.6 GB
pulling e0a42594d802: 100%  358 B
pulling 097a36493f71: 100% 8.4 KB
pulling 2490e7468436: 100%   65 B
pulling e18ad7af7efb: 100%  487 B
verifying sha256 digest
writing manifest
success

# example below is Granite Code model from IBM

root@dlp:~#
ollama pull granite3.3:2b

pulling manifest
pulling ac71e9e32c0b: 100% 1.5 GB
pulling 3da071a01bbe: 100% 6.6 KB
pulling 4a99a6dd617d: 100%  11 KB
pulling f9ed27df66e9: 100%  417 B
verifying sha256 digest
writing manifest
success
[4] How to use Ollama.
root@dlp:~#
ollama run llama3:8b

>>> Send a message (/? for help)
>>> Hello
Hello! It's nice to meet you. Is there something I can help you with, or would you like to chat?

>>> Tell me about yourself
I'd be happy to tell you about myself.

I am LLaMA, an AI model developed by Meta AI that can understand and respond to human input in a conversational manner. My
primary function is to generate text based on the input I receive, using patterns and structures learned from large
datasets of text.

I was trained on a massive corpus of text data, which includes but is not limited to:

1. Web pages: Articles, blog posts, and websites from across the internet.
2. Books and academic papers: A wide range of texts from various genres and disciplines.
3. Product reviews: Customer feedback and ratings for products and services.

This training enables me to generate responses that are natural-sounding and informative. I can provide information on a
vast array of topics, including but not limited to:

1. General knowledge: History, science, technology, literature, and more.
2. Entertainment: Movies, TV shows, music, books, and popular culture.
3. News and current events: Updates on local and global news, politics, and trends.

I'm designed to be helpful and engaging, so feel free to ask me anything that comes to mind!

to exit console, input [/bye]
>>> /bye

root@dlp:~#
ollama run phi4

>>> Hello
Hello! How can I assist you today? If you have any questions or need information, feel free to let me know.

>>> Tell me about yourself
I'm Phi, a language model developed by Microsoft. My purpose is to provide helpful and accurate information across a wide
range of topics. Here are some key aspects of what I can do:

1. **Answer Questions**: Whether it's general knowledge, science, history, or technology, I strive to give clear and
informative responses.

2. **Problem Solving**: I can assist with logic problems, basic math questions, programming concepts, and more.

3. **Explain Concepts**: If you're curious about how something works—like natural phenomena, technological processes, or
theoretical ideas—I can break it down for you.

4. **Generate Text**: From writing assistance to creative brainstorming, I can help with text generation in various styles
and formats.

5. **Privacy and Security**: I'm designed to respect user privacy by not retaining personal data from conversations.
However, I always encourage secure practices when handling sensitive information.

If there's something specific you'd like to know or discuss, feel free to ask!

>>> /bye

root@dlp:~#
ollama run gemma2:2b

>>> Hello
Hello!   How can I help you today? 

>>> Tell me about yourself
Hello! It's nice to meet you. 

I am Gemma, a large language model created by the Gemma team at Google AI.  Think of me as an AI assistant who loves to
chat and answer your questions in a comprehensive and informative way. I can generate text, translate languages, write
different kinds of creative content, and answer your questions in an engaging and informative way.

Here's what makes me unique:

* **Open-weights:** This means that my code is publicly available, allowing developers to study, modify, and build upon my
capabilities.
* **Focus on helpfulness and harmlessness:** My goal is to be a positive force in the world, providing information and
assistance while staying true to ethical principles.
* **Constantly learning:** As an AI, I'm always being trained on new data and improving my understanding of language and
the world around me!

What would you like to know more about? 

>>> /bye
[5] How to use Ollama via API.
root@dlp:~# curl http://localhost:11434/api/chat -d '{
  "model": "llama3:8b",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Where will the next Summer Olympics be held?" }
  ]
}'
{"model":"llama3:8b","created_at":"2025-09-30T05:41:27.025634251Z","message":{"role":"assistant","content":"The next Summer Olympics are scheduled to take place in Paris, France, from July 26 to August 11, 2024. The Olympic Games will feature a total of 32 sports and 5 new events: surfing, skateboarding, sport climbing, karate, and breakdancing.\n\nAfter that, the 2028 Summer Olympics are set to take place in Los Angeles, California, USA, from July 21 to August 6, 2028. This will be the third time that Los Angeles has hosted the Summer Olympics, after previously hosting in 1932 and 1984.\n\nIt's worth noting that the International Olympic Committee (IOC) has not yet announced the host city for the 2032 Summer Olympics, but the bidding process is expected to begin soon."},"done":true,"done_reason":"stop","total_duration":25507550312,"load_duration":3731693224,"prompt_eval_count":19,"prompt_eval_duration":582826305,"eval_count":160,"eval_duration":21191836382}

root@dlp:~# curl http://dlp.srv.world:11434/api/chat -d '{
  "model": "phi4",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Tell me the recommended sightseeing spots in Hiroshima, Japan." }
  ]
}'
{"model":"phi4","created_at":"2025-09-30T05:45:46.642076365Z","message":{"role":"assistant","content":"Hiroshima is a city rich in history and culture, offering numerous sightseeing spots that provide insight into its past and present. Here are some of the must-see locations:\n\n1. **Hiroshima Peace Memorial Park**: This park is dedicated to the victims of the atomic bomb. Key attractions include:\n   - The **Peace Memorial Museum**, which details the events surrounding the bombing.\n   - The iconic **Genbaku Dome (Atomic Bomb Dome)**, a UNESCO World Heritage site.\n\n2. **Morioka Residence**: Located near the Peace Memorial Park, this beautifully preserved traditional Japanese wooden merchant house offers insights into Hiroshima's pre-war life and architecture.\n\n3. **Itsukushima Shrine in Miyajima**: A short ferry ride from Hiroshima city, this famous Shinto shrine is renowned for its \"floating\" torii gate. The island of Miyajima itself is a scenic spot with hiking trails and deer roaming freely.\n\n4. **Hiroshima Castle (Carillon)**: While the original castle was destroyed during World War II, a new structure now serves as an observation tower offering panoramic views of the city.\n\n5. **Hiroshima City Asymmetric Building**: Known for its striking architecture, this building is also home to a museum focusing on urban development and architectural design in Hiroshima.\n\n6. **Shukkei-en Garden**: This beautiful landscape garden offers peaceful pathways, ponds, and scenic vistas, making it perfect for relaxation.\n\n7. **Hiroshima Central Park (Okamoto Kaijū Sculpture Museum)**: Besides being a lush green space, the park houses sculptures by artist Kiyoshi Matsuo, including famous pieces like \"Gulliver\" and \"Labyrinth.\"\n\n8. **Nagisa Pier**: This lively area is perfect for enjoying street food, shopping, and watching sunsets over Hiroshima Bay.\n\n9. **Shōwa-era Residential Districts (Machijaya)**: Explore areas like Koi in Naka-ku to experience the unique architecture from the Shōwa era.\n\n10. **Sakae Park**: Known for its cherry blossoms and stunning views, this park is a great spot for picnics and enjoying nature.\n\nEach of these locations offers a different perspective on Hiroshima's history, culture, and natural beauty."},"done":true,"done_reason":"stop","total_duration":121103889786,"load_duration":6509105786,"prompt_eval_count":23,"prompt_eval_duration":1687310798,"eval_count":468,"eval_duration":112903147368}
Matched Content