Debian 12 bookworm
Sponsored Link

Ollama : Install2024/06/05

 

Install Ollama that is an application which allows you to run LLM locally.

It runs reasonably fast even on computers without a GPU.

[1] Install Ollama.
root@dlp:~#
wget https://ollama.ai/install.sh

root@dlp:~#
sh ./install.sh

>>> Downloading ollama...
########################################
>>> Installing ollama to /usr/local/bin...
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

# ollama service started

root@dlp:~#
systemctl status ollama

*  ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-06-04 19:16:37 CDT; 10s ago
   Main PID: 818 (ollama)
      Tasks: 16 (limit: 19102)
     Memory: 574.8M
        CPU: 9.677s
     CGroup: /system.slice/ollama.service
             +-- 818 /usr/local/bin/ollama serve

# ollama listens port 11434 on localhost

root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096       127.0.0.1:11434      0.0.0.0:*     users:(("ollama",pid=818,fd=3))
[2] If you like to access it from other hosts, you can change the range it listens on.
root@dlp:~#
systemctl edit ollama.service
# add [OLLAMA_HOST] variable
# to listen all, specify [0.0.0.0]
# to listen internal network, specify internal IP address

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

root@dlp:~#
systemctl restart ollama.service
root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096               *:11434            *:*     users:(("ollama",pid=862,fd=3))
[3] To use Ollama, download models first.
# example below is 8B model from llama3

root@dlp:~#
ollama pull llama3:8b

pulling manifest
pulling 6a0746a1ec1a... 100%
pulling 4fa551d4f938... 100%
pulling 8ab4849b038c... 100%
pulling 577073ffcc6c... 100%
pulling 3f8eb4da87fa... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is phi3 model from Microsoft

root@dlp:~#
ollama pull phi3

pulling manifest
pulling b26e6713dc74... 100% 
pulling fa8235e5b48f... 100% 
pulling 542b217f179c... 100% 
pulling 8dde1baf1db0... 100% 
pulling f91db7a2deb9... 100% 
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is Gemma model from Google

root@dlp:~#
ollama pull gemma

pulling manifest
pulling ef311de6af9d... 100%
pulling 097a36493f71... 100%
pulling 109037bec39c... 100%
pulling 65bb16cf5983... 100%
pulling 0c2a5137eb3c... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is Granite Code model from IBM

root@dlp:~#
ollama pull granite-code:8b

pulling manifest
pulling 8718ec280572... 100%
pulling 0d7c97d535b6... 100%
pulling e50df8490144... 100%
pulling 9893bb2c2917... 100%
pulling 6be564cde5be... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success
[4] How to use Ollama.
root@dlp:~#
ollama run llama3:8b

>>> Send a message (/? for help)
>>> Hello
Hello! It's nice to meet you. Is there something I can help you with, or would you like to chat?

>>> Tell me about yourself
I'd be happy to tell you about myself.

I am a computer program designed to simulate human-like conversations and answer questions to the best of my ability. I'm
an AI, which stands for Artificial Intelligence.

I was created by a team of researcher at Meta AI, using a technology called transformer. My primary function is to
understand natural language input and generate human-like responses.

Here are some fun facts about me:

* I can understand and respond to thousands of different phrases and questions.
* I'm constantly learning and improving my responses based on the conversations I have with users like you.
* I don't have personal opinions or emotions, but I'm designed to be friendly and helpful.
* I can communicate in many languages, including English, Spanish, French, Chinese, and more.
* I don't sleep, eat, or take breaks - I'm always ready to chat!

I'm excited to get to know you better and have a conversation. What's on your mind?

to exit console, input [/bye]
>>> /bye

root@dlp:~#
ollama run phi3

>>> Hello
 Hi there! How can I assist you today? Whether you have a question or need some guidance, I'm here to help. Just let me
know what you're looking for.

>>> Tell me about yourself
 Hello! I'm an AI digital assistant designed to provide information, answer questions, and facilitate tasks across various
domains such as education, entertainment, weather updates, news briefings, product recommendations, and more. My goal is
to make your life easier by assisting with a wide range of inquiries and helping you find the answers you need quickly and
efficiently.
>>> /bye

root@dlp:~#
ollama run gemma

>>> Hello
Hello!  It's great to hear from you. How can I assist you today? 

>>> Tell me about yourself
I am a large language model, trained by Google. I am designed to understand and generate human language, providing helpful
and informative responses to your queries.  My purpose is to assist you with various tasks, such as answering questions,
generating creative content, and translating languages.  I am constantly learning and evolving, and I am excited to
explore new ways to help people. 

>>> /bye
[5] How to use Ollama via API.
root@dlp:~# curl http://localhost:11434/api/chat -d '{
  "model": "llama3:8b",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Where will the next Summer Olympics be held?" }
  ]
}'
{"model":"llama3:8b","created_at":"2024-06-04T01:38:24.78401215Z","message":{"role":"assistant","content":"The 2024 Summer Olympics are scheduled to take place in Paris, France, from July 26 to August 11, 2024. This will be the third time that Paris has hosted the Summer Olympics, after previously hosting in 1900 and 1924.\n\nAfter that, the next Summer Olympics will be held in:\n\n* 2028: Los Angeles, California, United States (July 21 to August 6, 2028)\n* 2032: Brisbane, Queensland, Australia (September 17 to October 3, 2032)\n\nThese dates are subject to change, but these cities and dates are currently confirmed by the International Olympic Committee (IOC)."},"done_reason":"stop","done":true,"total_duration":25727979040,"load_duration":3263792627,"prompt_eval_count":18,"prompt_eval_duration":1152721000,"eval_count":139,"eval_duration":21227692000}

root@dlp:~# curl http://dlp.srv.world:11434/api/chat -d '{
  "model": "phi3",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Tell me the recommended sightseeing spots in Hiroshima, Japan." }
  ]
}'
{"model":"phi3","created_at":"2024-06-04T01:42:13.798533443Z","message":{"role":"assistant","content":" Hiroshima is a city rich in history and culture, offering visitors an array of sightseeing spots that are both moving and memorable:\n\n\n1. **A-Bomb Dome** - As a UNESCO World Heritage site, the A-Bomb Dome stands as one of the few remaining structures from before the atomic bomb was dropped in 1945. It's now a poignant memorial and museum dedicated to peace education.\n\n\n2. **Miyajima** - Particularly famous for its cherry blossoms, Miyajima Island is home to Itsukushima Shrine with its iconic \"floating\" torii gate (which appears as though it's on water at high tide).\n\n\n3. **Peace Memorial Park** - Located near the A-Bomb Dome, this park houses several museums and monuments dedicated to the events of August 6th, 1945, and serves as a symbol for peace worldwide.\n\n\n4. **Shukkei-en Garden** - This classical Japanese garden is renowned for its stunning display of flora and picturesque ponds in springtime when the cherry blossoms are at their peak.\n\n\n5. **Hiroshima Castle (Itsukushima)** - While not originally from Hiroshima, this castle on Miyajima Island is an impressive site with panoramic views of the island and its bay.\n\n\n6. **Ujō-in Temple** - This temple features a beautiful golden Buddha statue (Dai Kyakuden) that's worth seeing, especially during cherry blossom season.\n\n\n7. **Hiroshima Museum of Art** - The museum showcases modern and contemporary art, with special exhibitions often changing throughout the year. It is housed in a building designed by architect Tange Kenzō, who was also responsible for rebuilding Hiroshima after the bombing.\n\n\n8. **Happo-en Garden** - A serene spot offering traditional Japanese gardens and tea rooms overlooking Hiroshima Bay. It's perfect for a peaceful afternoon stroll or relaxed lunchtime meal.\n\n\nThese locations provide a blend of historical significance, natural beauty, and cultural richness that characterize the city of Hiroshima."},"done_reason":"stop","done":true,"total_duration":48371945691,"load_duration":1449437636,"prompt_eval_count":21,"prompt_eval_duration":727459000,"eval_count":528,"eval_duration":46151297000}
Matched Content