OpenStack Zed : How to use Magnum2022/11/11 |
|
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+-----------------------------+-----------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached Nginx | | Neutron Server | | Nova Compute |
| Keystone httpd | | OVN-Northd | | Open vSwitch |
| Glance Nova API | | Nginx iSCSI Target | | OVN Metadata Agent |
| Cinder API | | Cinder Volume | | OVN-Controller |
| Barbican API | | Heat API/Engine | | |
| | | Magnum Services | | |
+-----------------------+ +-----------------------+ +-----------------------+
|
| [1] | On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance. |
|
[root@dlp ~(keystone)]# wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220424.3.0/x86_64/fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2.xz [root@dlp ~(keystone)]# xz -dv fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2.xz [root@dlp ~(keystone)]# openstack image create Fedora-CoreOS --file=fedora-coreos-35.20220424.3.0-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public |
| [2] | How to use Magnum. For example, Create Kubernetes Cluster with [admin] user. |
|
[root@dlp ~(keystone)]# openstack flavor list +----+----------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+----------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | | 1 | m1.large | 8192 | 20 | 0 | 4 | True | +----+----------+------+------+-----------+-------+-----------+[root@dlp ~(keystone)]# openstack keypair list +--------+-------------------------------------------------+------+ | Name | Fingerprint | Type | +--------+-------------------------------------------------+------+ | my-key | 97:b9:5e:34:55:0b:59:d8:c9:e2:19:52:b3:78:28:db | ssh | +--------+-------------------------------------------------+------+[root@dlp ~(keystone)]# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 004ca400-3d79-4c1b-b8fb-bba1e273000e | public | 80213093-ac95-469a-b4d9-8c6adc93cfe1 | | c9c79a1d-136e-4036-8fe0-eaac79f05f5d | private | 85e66c6a-66eb-490f-a4fa-e21f91678c89 | +--------------------------------------+---------+--------------------------------------+[root@dlp ~(keystone)]# openstack subnet list +--------------------------------------+----------------+--------------------------------------+------------------+ | ID | Name | Network | Subnet | +--------------------------------------+----------------+--------------------------------------+------------------+ | 80213093-ac95-469a-b4d9-8c6adc93cfe1 | public-subnet | 004ca400-3d79-4c1b-b8fb-bba1e273000e | 10.0.0.0/24 | | 85e66c6a-66eb-490f-a4fa-e21f91678c89 | private-subnet | c9c79a1d-136e-4036-8fe0-eaac79f05f5d | 192.168.100.0/24 | +--------------------------------------+----------------+--------------------------------------+------------------+ # create Kubernetes Cluster template [root@dlp ~(keystone)]# openstack coe cluster template create k8s-cluster-template \ --image Fedora-CoreOS \ --external-network public \ --fixed-network private \ --fixed-subnet private-subnet \ --dns-nameserver 10.0.0.10 \ --network-driver calico \ --docker-storage-driver overlay2 \ --docker-volume-size 5 \ --master-flavor m1.large \ --flavor m1.large \ --coe kubernetes Request to create cluster template k8s-cluster-template accepted +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | private-subnet | | master_flavor_id | m1.large | | uuid | fe4f0d22-d03a-45c8-8f19-1d57c7345fb6 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | False | | http_proxy | - | | docker_volume_size | 5 | | server_type | vm | | external_network_id | public | | cluster_distro | fedora-coreos | | image_id | Fedora-CoreOS | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay2 | | apiserver_port | - | | name | k8s-cluster-template | | created_at | 2022-11-11T06:27:35+00:00 | | network_driver | calico | | fixed_network | private | | coe | kubernetes | | flavor_id | m1.large | | master_lb_enabled | False | | dns_nameserver | 10.0.0.10 | | hidden | False | | tags | - | +-----------------------+--------------------------------------+ # create Kubernetes Cluster with 2 nodes [root@dlp ~(keystone)]# openstack coe cluster create k8s-cluster \ --cluster-template k8s-cluster-template \ --master-count 1 \ --node-count 1 \ --keypair my-key Request to create cluster a93d1aaf-bc17-499d-aac6-43dcc4f43563 accepted # verify status # proceed to create cluster during [CREATE_IN_PROGRESS] state [root@dlp ~(keystone)]# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | a93d1aaf-bc17-499d-aac6-43dcc4f43563 | k8s-cluster | my-key | 1 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ # Heat orchestration System is used for creating [root@dlp ~(keystone)]# openstack stack list +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | k8s-cluster-gp2lk52qjexm | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_IN_PROGRESS | 2022-11-11T06:28:41Z | None | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ # confirm checkpoints for creation [root@dlp ~(keystone)]# openstack stack list --nested | grep k8s-cluster | 63bbd57c-9a9f-48c9-b025-7ca3b1885055 | k8s-cluster-gp2lk52qjexm-kube_minions-ilpjay5sq7jy-0-mp357vdhzihz | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:31:52Z | None | cbe2f480-22b9-46f0-9ad0-f96202794139 | | cbe2f480-22b9-46f0-9ad0-f96202794139 | k8s-cluster-gp2lk52qjexm-kube_minions-ilpjay5sq7jy | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:31:51Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 4cb8a1db-e9fd-406a-adb4-43974bfd8275 | k8s-cluster-gp2lk52qjexm-api_address_floating_switch-5ioiquyxrgjm | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:31:51Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 81953818-92ea-4745-aefa-8457e282e088 | k8s-cluster-gp2lk52qjexm-api_address_lb_switch-5mcli22dsrvw | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:31:49Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 7e9415ac-fc25-4087-b81a-f1bf6e4d77e1 | k8s-cluster-gp2lk52qjexm-etcd_address_lb_switch-vbojcqzoehl5 | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:31:49Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 23c372d7-ef69-4cfa-bb7f-bafbd6d24927 | k8s-cluster-gp2lk52qjexm-kube_masters-4itoqbkbeztc-0-3ivhjocnornd-api_address_switch-3igkdpvypbus | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:29:03Z | None | cced0d2c-d9ed-40dd-91e3-ce03b25237c4 | | cced0d2c-d9ed-40dd-91e3-ce03b25237c4 | k8s-cluster-gp2lk52qjexm-kube_masters-4itoqbkbeztc-0-3ivhjocnornd | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:47Z | None | 81e6151c-1924-461d-98be-93d667608461 | | 81e6151c-1924-461d-98be-93d667608461 | k8s-cluster-gp2lk52qjexm-kube_masters-4itoqbkbeztc | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:46Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 33b48200-f4fc-4da7-8310-cc5788198f91 | k8s-cluster-gp2lk52qjexm-api_lb-kx6mof3i43y6 | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:45Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | fcb5b6d2-9c30-42f3-82f8-caf4ed4ff19c | k8s-cluster-gp2lk52qjexm-etcd_lb-dt57ua253q6w | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:44Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 5728894e-a468-4c3a-b4aa-5384a3c848cb | k8s-cluster-gp2lk52qjexm-network-nhbm3pjgi4wk-network_switch-p7dzze45aywb | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:43Z | None | 3eb24927-c04f-49d2-ad95-b25cbc91023d | | 3eb24927-c04f-49d2-ad95-b25cbc91023d | k8s-cluster-gp2lk52qjexm-network-nhbm3pjgi4wk | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:42Z | None | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | | 70368ed7-a877-45dd-a183-d3cbc78ce7ac | k8s-cluster-gp2lk52qjexm | 939974d6fc8e400eb01f7dd749935ea3 | CREATE_COMPLETE | 2022-11-11T06:28:41Z | None | None | # if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY] [root@dlp ~(keystone)]# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | a93d1aaf-bc17-499d-aac6-43dcc4f43563 | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ # instances are running [root@dlp ~(keystone)]# openstack server list +--------------------------------------+-----------------------------------+--------+------------------------------------+---------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------------------------+--------+------------------------------------+---------------+----------+ | 49f37337-5776-4b08-962e-5374e104dfb3 | k8s-cluster-gp2lk52qjexm-node-0 | ACTIVE | private=10.0.0.238, 192.168.100.81 | Fedora-CoreOS | m1.large | | 887f2958-bd94-4554-8474-254eec36889e | k8s-cluster-gp2lk52qjexm-master-0 | ACTIVE | private=10.0.0.242, 192.168.100.41 | Fedora-CoreOS | m1.large | +--------------------------------------+-----------------------------------+--------+------------------------------------+---------------+----------+ |
| [3] | To access to use Kubernetes Cluster, Set like follows. |
|
# install [kubectl] from Snap [root@dlp ~(keystone)]# snap install kubectl --classic kubectl 1.25.3 from Canonical✓ installed openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | a93d1aaf-bc17-499d-aac6-43dcc4f43563 | k8s-cluster | my-key | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# openstack coe cluster config k8s-cluster export KUBECONFIG=/root/config [root@dlp ~(keystone)]# export KUBECONFIG=/root/config
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-cluster-gp2lk52qjexm-master-0 Ready master 7m45s v1.23.3 k8s-cluster-gp2lk52qjexm-node-0 Ready <none> 5m52s v1.23.3[root@dlp ~(keystone)]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-54799d958b-bh4g9 1/1 Running 0 8m21s calico-node-h2v4d 1/1 Running 0 6m42s calico-node-tgnrc 1/1 Running 0 8m21s coredns-56448757b9-hbbp7 1/1 Running 0 8m21s coredns-56448757b9-wzr5m 1/1 Running 0 8m21s dashboard-metrics-scraper-67f57ff746-675qx 1/1 Running 0 8m20s k8s-keystone-auth-tq8mr 1/1 Running 0 8m19s kube-dns-autoscaler-6d5b5dc777-xqtlm 1/1 Running 0 8m21s kubernetes-dashboard-7b88d986b4-5g5pg 1/1 Running 0 8m20s magnum-metrics-server-6c4c77844b-2fzhp 1/1 Running 0 8m13s npd-4gqmx 1/1 Running 0 6m22s # verify cluster to create test pods [root@dlp ~(keystone)]# kubectl create deployment test-nginx --image=nginx --replicas=2 deployment.apps/test-nginx created kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-nginx-976fbbd77-fp79q 1/1 Running 0 20s 10.100.81.67 k8s-cluster-gp2lk52qjexm-node-0 <none> <none> test-nginx-976fbbd77-xdrcr 1/1 Running 0 20s 10.100.81.68 k8s-cluster-gp2lk52qjexm-node-0 <none> <none>
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# kubectl expose deployment test-nginx --type="NodePort" --port 80 service/test-nginx exposed kubectl get services test-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx NodePort 10.254.39.123 <none> 80:30956/TCP 5s
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 & Forwarding from 0.0.0.0:10443 -> 80 curl localhost:10443
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
|
| [4] | If you'd like to use Magnum with common users, it needs to change some settings. |
|
[root@dlp ~(keystone)]# openstack role list +----------------------------------+------------------+ | ID | Name | +----------------------------------+------------------+ | 075291ce7c66499ea099a012a92f6695 | admin | | 1ace023940044b34b5a442eb38e97635 | heat_stack_owner | | 1c38efda26774c358fabbdfed25547ae | reader | | 245b582c47d744c1a6f706898534900a | CloudUser | | ae93621edfd4441096d4eb1b1e4be035 | heat_stack_user | | d0f53ed114fc4ed1b56679ba0b0ef33d | member | +----------------------------------+------------------+[root@dlp ~(keystone)]# openstack project list +----------------------------------+-----------+ | ID | Name | +----------------------------------+-----------+ | 28b6e37c787240e4a975d3614821cc71 | hiroshima | | 62f531f4d2934e75b8d7f11cd7d53be3 | service | | 939974d6fc8e400eb01f7dd749935ea3 | admin | +----------------------------------+-----------+[root@dlp ~(keystone)]# openstack user list +----------------------------------+---------------------+ | ID | Name | +----------------------------------+---------------------+ | f00e5072d7d6488f935ea680256af89c | admin | | 78009f5643d240da9996dff0aee5cc37 | glance | | 7ed52b257a47436389a60aef689d20f0 | nova | | 2080b84b7e304486b33365a447963bc1 | placement | | 6f4af63a84f944f3a7a0c018ef64a4a1 | neutron | | 9e8824a151c949ad9105535ead452501 | serverworld | | 42de8b3de0c74f3e9f1622c89fc1a178 | cinder | | 03b59cb43c8547d4bf0a055dd9edd7a8 | heat | | e8c47aafd9324805b90b52f963dc6cad | heat_domain_admin | | 17b493375a874f899cb40da602e1bec7 | barbican | | 81a4793c2dbd4839bcb5ecedf70d1916 | magnum | | 40b1f36c06434e249f1828d4cfbd290a | magnum_domain_admin | +----------------------------------+---------------------+ # for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role [root@dlp ~(keystone)]# openstack role add --project hiroshima --user serverworld heat_stack_owner
# on the Node Neutron server is running, change settings like follows [root@network ~]# vi /etc/neutron/policy.json # create new # overwrite some settings
{
"create_port:fixed_ips:subnet_id": "",
"create_port:allowed_address_pairs": "",
"create_port:allowed_address_pairs:ip_address": "",
}
[root@network ~]# systemctl restart neutron-server
# that's OK, common users can create clusters [cent@dlp ~(keystone)]$ openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | fe6d43ca-c2b6-412a-be87-c3eb6dbd515a | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ |
| Sponsored Link |
|
|