OpenStack Zed : How to use Magnum2022/10/11 |
|
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+-----------------------------+-----------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached Nginx | | Neutron Server | | Nova Compute |
| Keystone httpd | | OVN-Northd | | Open vSwitch |
| Glance Nova API | | Nginx iSCSI Target | | OVN Metadata Agent |
| Cinder API | | Cinder Volume | | OVN-Controller |
| Barbican API | | Heat API/Engine | | |
| | | Magnum Services | | |
+-----------------------+ +-----------------------+ +-----------------------+
|
| [1] | On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance. |
|
root@dlp ~(keystone)# wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220424.3.0/x86_64/fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# xz -dv fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# openstack image create Fedora-CoreOS --file=fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public |
| [2] | How to use Magnum. For example, Create Kubernetes Cluster with [admin] user. |
|
root@dlp ~(keystone)# openstack flavor list +----+----------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+----------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | | 1 | m1.large | 8192 | 20 | 0 | 4 | True | +----+----------+------+------+-----------+-------+-----------+root@dlp ~(keystone)# openstack keypair list +--------+-------------------------------------------------+------+ | Name | Fingerprint | Type | +--------+-------------------------------------------------+------+ | my-key | 82:ec:cc:1a:8d:07:73:17:4c:82:3a:40:cd:dd:2c:a7 | ssh | +--------+-------------------------------------------------+------+root@dlp ~(keystone)# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 394ec727-7e9a-473e-8068-45aa841a9ac2 | public | ea6cb6cb-74ed-4007-bd25-b3fb84111e84 | | 7336a271-bec9-4d33-8dc5-e9f3a9892ed7 | private | a454cf3e-fef5-46f6-8468-4a26cbd14983 | +--------------------------------------+---------+--------------------------------------+root@dlp ~(keystone)# openstack subnet list +--------------------------------------+----------------+--------------------------------------+------------------+ | ID | Name | Network | Subnet | +--------------------------------------+----------------+--------------------------------------+------------------+ | a454cf3e-fef5-46f6-8468-4a26cbd14983 | private-subnet | 7336a271-bec9-4d33-8dc5-e9f3a9892ed7 | 192.168.100.0/24 | | ea6cb6cb-74ed-4007-bd25-b3fb84111e84 | public-subnet | 394ec727-7e9a-473e-8068-45aa841a9ac2 | 10.0.0.0/24 | +--------------------------------------+----------------+--------------------------------------+------------------+ # create Kubernetes Cluster template root@dlp ~(keystone)# openstack coe cluster template create k8s-cluster-template \ --image Fedora-CoreOS \ --external-network public \ --fixed-network private \ --fixed-subnet private-subnet \ --dns-nameserver 10.0.0.10 \ --network-driver calico \ --docker-storage-driver overlay2 \ --docker-volume-size 5 \ --master-flavor m1.large \ --flavor m1.large \ --coe kubernetes Request to create cluster template k8s-cluster-template accepted +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | private-subnet | | master_flavor_id | m1.large | | uuid | 5bc379ef-e462-46d1-bc58-ab125754a6d6 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | False | | http_proxy | - | | docker_volume_size | 5 | | server_type | vm | | external_network_id | public | | cluster_distro | fedora-coreos | | image_id | Fedora-CoreOS | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay2 | | apiserver_port | - | | name | k8s-cluster-template | | created_at | 2022-10-11T09:33:19+00:00 | | network_driver | calico | | fixed_network | private | | coe | kubernetes | | flavor_id | m1.large | | master_lb_enabled | False | | dns_nameserver | 10.0.0.10 | | hidden | False | | tags | - | +-----------------------+--------------------------------------+ # create Kubernetes Cluster with 2 nodes root@dlp ~(keystone)# openstack coe cluster create k8s-cluster \ --cluster-template k8s-cluster-template \ --master-count 1 \ --node-count 1 \ --keypair my-key Request to create cluster 9a548a60-0645-4226-abb9-e0bfc2e5a635 accepted # verify status # proceed to create cluster during [CREATE_IN_PROGRESS] state root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | 6de09900-2130-41fc-80f2-6b52b9ec6e5b | k8s-cluster | my-key | 1 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ # Heat orchestration System is used for creating root@dlp ~(keystone)# openstack stack list +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | 50f8e343-98f6-4453-a538-aa11061bc10c | k8s-cluster-kwixo7jj667g | 231a529f51394426b07c56f2c5ec580e | CREATE_IN_PROGRESS | 2022-10-11T09:33:44Z | None | +--------------------------------------+--------------------------+----------------------------------+------------ -----+----------------------+--------------+ # confirm checkpoints for creation root@dlp ~(keystone)# openstack stack list --nested | grep k8s-cluster | ccff612b-f6cb-4025-a101-0f2361b92da3 | k8s-cluster-kwixo7jj667g-api_address_floating_switch-phx23dpfl3ss | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:36:33Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | 9e47dd8f-afec-4834-9275-a84425bc8e81 | k8s-cluster-kwixo7jj667g-kube_minions-yrbxy7swqphg-0-rvyhkenhy654 | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:36:33Z | None | 3b10e3f8-271b-41fb-a18c-96d076fdd615 | | 3b10e3f8-271b-41fb-a18c-96d076fdd615 | k8s-cluster-kwixo7jj667g-kube_minions-yrbxy7swqphg | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:36:33Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | d26c6578-95a7-416a-ac09-a2f607ce0800 | k8s-cluster-kwixo7jj667g-api_address_lb_switch-wbillafc5jut | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:36:32Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | 1686ec93-a20c-4ddc-b051-be63f2a0234a | k8s-cluster-kwixo7jj667g-etcd_address_lb_switch-melqp53yixlk | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:36:32Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | 3e335984-9683-43a6-9991-b3af254c59f8 | k8s-cluster-kwixo7jj667g-kube_masters-bbhprjjb7jh5-0-635ycot4ewb7-api_address_switch-wys26ua5fwvh | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:34:07Z | None | 74c358a6-2250-4368-9c39-f6cb2d366ba5 | | 74c358a6-2250-4368-9c39-f6cb2d366ba5 | k8s-cluster-kwixo7jj667g-kube_masters-bbhprjjb7jh5-0-635ycot4ewb7 | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:52Z | None | 50304a43-8dfd-47c3-88fd-43294a86b767 | | 50304a43-8dfd-47c3-88fd-43294a86b767 | k8s-cluster-kwixo7jj667g-kube_masters-bbhprjjb7jh5 | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:51Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | f1ff2bf9-9737-4fcc-a2dc-7743426b1c5d | k8s-cluster-kwixo7jj667g-etcd_lb-43t5sohvumdp | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:50Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | 5e611c63-6d74-427b-b960-052d620bc76a | k8s-cluster-kwixo7jj667g-api_lb-aogajfne4pzy | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:50Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | d01d1ab1-f959-4791-a629-6366d53b9214 | k8s-cluster-kwixo7jj667g-network-qv5w4e7xrsr2-network_switch-7znct5ov42ef | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:48Z | None | 58af7e37-21be-432c-bd6c-002bac956d40 | | 58af7e37-21be-432c-bd6c-002bac956d40 | k8s-cluster-kwixo7jj667g-network-qv5w4e7xrsr2 | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:48Z | None | 50f8e343-98f6-4453-a538-aa11061bc10c | | 50f8e343-98f6-4453-a538-aa11061bc10c | k8s-cluster-kwixo7jj667g | 231a529f51394426b07c56f2c5ec580e | CREATE_COMPLETE | 2022-10-11T09:33:44Z | None | None | # if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY] root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 6de09900-2130-41fc-80f2-6b52b9ec6e5b | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ # instances are running root@dlp ~(keystone)# openstack server list +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | 3ca7c1ad-9f91-42de-aa18-36c059cf72ed | k8s-cluster-kwixo7jj667g-node-0 | ACTIVE | private=10.0.0.233, 192.168.100.47 | Fedora-CoreOS | m1.large | | e6a6ccc0-c12c-4266-82cf-cc68d2d76967 | k8s-cluster-kwixo7jj667g-master-0 | ACTIVE | private=10.0.0.218, 192.168.100.167 | Fedora-CoreOS | m1.large | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ |
| [3] | To access to use Kubernetes Cluster, Set like follows. |
|
# install [kubectl] root@dlp ~(keystone)# snap install kubectl --classic kubectl 1.25.2 from Canonical✓ installed openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 6de09900-2130-41fc-80f2-6b52b9ec6e5b | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
root@dlp ~(keystone)#
root@dlp ~(keystone)# openstack coe cluster config k8s-cluster export KUBECONFIG=/root/config root@dlp ~(keystone)# export KUBECONFIG=/root/config
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-cluster-kwixo7jj667g-master-0 Ready master 11m v1.23.3 k8s-cluster-kwixo7jj667g-node-0 Ready <none> 9m5s v1.23.3root@dlp ~(keystone)# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-54799d958b-dfd8z 1/1 Running 0 11m calico-node-2ddgr 1/1 Running 0 11m calico-node-vnpr5 1/1 Running 0 9m58s coredns-56448757b9-9fgpj 1/1 Running 0 11m coredns-56448757b9-brfk4 1/1 Running 0 11m dashboard-metrics-scraper-67f57ff746-r47l9 1/1 Running 0 11m k8s-keystone-auth-jhhjh 1/1 Running 0 11m kube-dns-autoscaler-6d5b5dc777-dpgfq 1/1 Running 0 11m kubernetes-dashboard-7b88d986b4-8vkkf 1/1 Running 0 11m magnum-metrics-server-6c4c77844b-965fn 1/1 Running 0 11m npd-7b7pt 1/1 Running 0 9m37s # verify cluster to create test pods root@dlp ~(keystone)# kubectl create deployment test-nginx --image=nginx --replicas=2 deployment.apps/test-nginx created kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-nginx-976fbbd77-f2nsn 1/1 Running 0 29s 10.100.154.68 k8s-cluster-kwixo7jj667g-node-0 <none> <none> test-nginx-976fbbd77-x4m45 1/1 Running 0 29s 10.100.154.67 k8s-cluster-kwixo7jj667g-node-0 <none> <none>
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl expose deployment test-nginx --type="NodePort" --port 80 service/test-nginx exposed kubectl get services test-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx NodePort 10.254.113.41 <none> 80:32743/TCP 5s
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 & Forwarding from 0.0.0.0:10443 -> 80 curl localhost:10443
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
|
| [4] | If you'd like to use Magnum with common users, it needs to change some settings. |
|
root@dlp ~(keystone)# openstack role list +----------------------------------+------------------+ | ID | Name | +----------------------------------+------------------+ | 119d976b7118436b86bb6a6eef561909 | heat_stack_owner | | 3eca53d4474f4d81bf2c85c8ea238f7c | reader | | 7eea1df857244828afbf10ac46356b23 | CloudUser | | 99fe824a4d5b40f1abb6efa9f6820dff | heat_stack_user | | dbd8fff588c74deaa4e2192ba1d92933 | member | | f125eaa73aed4ea2ba2b7cba02acd42c | admin | +----------------------------------+------------------+root@dlp ~(keystone)# openstack project list +----------------------------------+-----------+ | ID | Name | +----------------------------------+-----------+ | 231a529f51394426b07c56f2c5ec580e | admin | | 4dd1e1f6bac441ff9c77002c3ab4c58a | hiroshima | | ee51c09c765e4a4d958c2bcc4dbb5665 | service | +----------------------------------+-----------+root@dlp ~(keystone)# openstack user list +----------------------------------+---------------------+ | ID | Name | +----------------------------------+---------------------+ | 7c9dbb0cdc284113964d618b24cc722e | admin | | 0aef0c555a4f4cc885c80dd0eb4f7c72 | glance | | 4d912be26a4149d590dbfb7bc8b015fa | nova | | b3ad6ca00649400a8eae75d64a1c2181 | placement | | bcf45964c78c40fba4c125e31dee3721 | neutron | | 9345b7f4de21427abf31ad308cf952f8 | serverworld | | 3042ebda5502448097dd0bbbe3058812 | cinder | | 1bfd1646d0e04371a3c6199bc98d152f | heat | | b477de75cfbe4a5e863bc4d7ea1d2d5f | heat_domain_admin | | 55ad9ce9899d415ea9fba4d31a12597d | barbican | | aa26495baad24a9fac173217ab2cfc16 | magnum | | 8726616d293e45b992e976513b25fadd | magnum_domain_admin | +----------------------------------+---------------------+ # for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role root@dlp ~(keystone)# openstack role add --project hiroshima --user serverworld heat_stack_owner
# on the Node Neutron server is running, change settings like follows root@network:~# vi /etc/neutron/policy.json # create new # overwrite some settings
{
"create_port:fixed_ips:subnet_id": "",
"create_port:allowed_address_pairs": "",
"create_port:allowed_address_pairs:ip_address": "",
}
root@network:~# systemctl restart neutron-server
# that's OK, common users can create clusters ubuntu@dlp ~(keystone)$ openstack coe cluster list +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ | ab81cc22-8a2e-4dca-9e36-8f2a784b1b77 | k8s-cluster-1 | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+---------------+---------+------------+--------------+-----------------+---------------+ |
| Sponsored Link |