OpenStack Victoria : How to use Magnum2021/03/26 |
|
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+---------------------------+---------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ Control Node ] | | [ Network Node ] | | [ Compute Node ] |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached httpd | | Neutron Server | | Nova Compute |
| Keystone Glance | | OVN-Northd | | Open vSwitch |
| Nova API | | Cinder Volume | | OVN Metadata Agent |
| Cinder API | | iSCSI Target | | OVN-Controller |
| Barbican API | | Heat API/Engine | | |
| | | Magnum Services | | |
+-----------------------+ +-----------------------+ +-----------------------+
|
| [1] | On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance. |
|
[root@dlp ~(keystone)]# wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210301.3.1/x86_64/fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2.xz [root@dlp ~(keystone)]# xz -dv fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2.xz [root@dlp ~(keystone)]# openstack image create Fedora-CoreOS --file=fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public |
| [2] | How to use Magnum. For example, Create Kubernetes Cluster with [admin] user. |
|
[root@dlp ~(keystone)]# openstack flavor list +----+----------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+----------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | +----+----------+------+------+-----------+-------+-----------+[root@dlp ~(keystone)]# openstack keypair list +-------+-------------------------------------------------+ | Name | Fingerprint | +-------+-------------------------------------------------+ | mykey | 96:26:ef:f5:c0:81:40:67:d5:56:1e:17:5e:99:1d:83 | +-------+-------------------------------------------------+[root@dlp ~(keystone)]# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 9f53c3b1-ec6b-499d-854a-588d3b0b96d6 | private | 94afb0cc-1d58-4615-8fef-8852271bfc16 | | ea9725cf-a8bc-4eb5-a374-7d7d0e11453c | public | 63c29d7b-e870-477d-819b-e5dfe41af1d0 | +--------------------------------------+---------+--------------------------------------+[root@dlp ~(keystone)]# openstack subnet list +--------------------------------------+----------------+--------------------------------------+------------------+ | ID | Name | Network | Subnet | +--------------------------------------+----------------+--------------------------------------+------------------+ | 63c29d7b-e870-477d-819b-e5dfe41af1d0 | public-subnet | ea9725cf-a8bc-4eb5-a374-7d7d0e11453c | 10.0.0.0/24 | | 94afb0cc-1d58-4615-8fef-8852271bfc16 | private-subnet | 9f53c3b1-ec6b-499d-854a-588d3b0b96d6 | 192.168.100.0/24 | +--------------------------------------+----------------+--------------------------------------+------------------+ # create Kubernetes Cluster template [root@dlp ~(keystone)]# openstack coe cluster template create k8s-cluster-template \ --image Fedora-CoreOS \ --external-network public \ --fixed-network private \ --fixed-subnet private-subnet \ --dns-nameserver 10.0.0.10 \ --network-driver flannel \ --docker-storage-driver overlay2 \ --docker-volume-size 10 \ --master-flavor m1.small \ --flavor m1.small \ --coe kubernetes Request to create cluster template k8s-cluster-template accepted +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | private-subnet | | master_flavor_id | m1.small | | uuid | 4b2604e9-9e5c-4dbd-bbbc-592de6255905 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | False | | http_proxy | - | | docker_volume_size | 10 | | server_type | vm | | external_network_id | public | | cluster_distro | fedora-coreos | | image_id | Fedora-CoreOS | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay2 | | apiserver_port | - | | name | k8s-cluster-template | | created_at | 2021-03-26T03:37:45+00:00 | | network_driver | flannel | | fixed_network | private | | coe | kubernetes | | flavor_id | m1.small | | master_lb_enabled | False | | dns_nameserver | 10.0.0.10 | | hidden | False | +-----------------------+--------------------------------------+ # create Kubernetes Cluster with 2 nodes [root@dlp ~(keystone)]# openstack coe cluster create k8s-cluster \ --cluster-template k8s-cluster-template \ --master-count 1 \ --node-count 1 \ --keypair mykey Request to create cluster 2a013c5f-92a8-4799-b9d5-67dc24f96d94 accepted # verify status # proceed to create cluster during [CREATE_IN_PROGRESS] state [root@dlp ~(keystone)]# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey | 1 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ # Heat orchestration System is used for creating [root@dlp ~(keystone)]# openstack stack list +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | k8s-cluster-xw3duzjqjmv7 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_IN_PROGRESS | 2021-03-26T03:38:26Z | None | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ # confirm checkpoints for creation [root@dlp ~(keystone)]# openstack stack list --nested | grep k8s-cluster | 5fca1791-0219-44c0-83f0-a8d47107214c | k8s-cluster-xw3duzjqjmv7-kube_minions-tndsjafpi7ox-0-uzisit52mj5y | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:20Z | None | 9ff979f2-a66f-4bd0-9e6e-ef17b7cd21e6 | | 9ff979f2-a66f-4bd0-9e6e-ef17b7cd21e6 | k8s-cluster-xw3duzjqjmv7-kube_minions-tndsjafpi7ox | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:19Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | 5d270a23-2530-443a-a403-da90b7a46842 | k8s-cluster-xw3duzjqjmv7-api_address_floating_switch-2ehjcbduvjbg | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:19Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | dbd1473d-7700-4422-af33-d3bed4c7d06e | k8s-cluster-xw3duzjqjmv7-api_address_lb_switch-6ujh5orommsk | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:18Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | 334bd75a-c1b4-4c53-a196-adf986cfd52c | k8s-cluster-xw3duzjqjmv7-etcd_address_lb_switch-xp725ak5twze | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:18Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | d2603114-9ffa-4117-b1bd-04b2a9e4c39d | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa-0-mcw7quwoezqq-api_address_switch-p7s5fq3rlvfo | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:39:06Z | None | 7584870f-8d81-49fa-9b28-aac52db2619d | | 7584870f-8d81-49fa-9b28-aac52db2619d | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa-0-mcw7quwoezqq | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:37Z | None | b5c2f791-76a5-4f31-84e3-73b8895b52bc | | b5c2f791-76a5-4f31-84e3-73b8895b52bc | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:36Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | 8fa5750e-3320-404b-b40c-db6640091c52 | k8s-cluster-xw3duzjqjmv7-etcd_lb-u7iosmumufst | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:34Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | 3785f152-3da6-4625-b634-85cc84918c29 | k8s-cluster-xw3duzjqjmv7-api_lb-hkih2juzr4ew | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:34Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | f30a7eb3-7f9e-4fd7-a394-f167d2774b14 | k8s-cluster-xw3duzjqjmv7-network-tnenshonon32-network_switch-2p4qblfjnhut | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:32Z | None | 67852d07-1cde-4a66-980d-d79e51a7b6e0 | | 67852d07-1cde-4a66-980d-d79e51a7b6e0 | k8s-cluster-xw3duzjqjmv7-network-tnenshonon32 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:31Z | None | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | k8s-cluster-xw3duzjqjmv7 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:26Z | None | None | # if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY] [root@dlp ~(keystone)]# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ # instances are running [root@dlp ~(keystone)]# openstack server list +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | 7c3d81a9-df12-49ce-a5de-a0fc039271a4 | k8s-cluster-xw3duzjqjmv7-node-0 | ACTIVE | private=192.168.100.222, 10.0.0.225 | Fedora-CoreOS | m1.small | | 47d04e69-20f0-45fd-bb33-74b4764e2d4a | k8s-cluster-xw3duzjqjmv7-master-0 | ACTIVE | private=192.168.100.155, 10.0.0.237 | Fedora-CoreOS | m1.small | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ |
| [3] | To access to use Kubernetes Cluster, Set like follows. |
|
# install [kubectl] from Snap [root@dlp ~(keystone)]# snap install kubectl --classic kubectl 1.20.4 from Canonical✓ installed openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# openstack coe cluster config k8s-cluster export KUBECONFIG=/root/config [root@dlp ~(keystone)]# export KUBECONFIG=/root/config
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-cluster-xw3duzjqjmv7-master-0 Ready master 13m v1.18.2 k8s-cluster-xw3duzjqjmv7-node-0 Ready <none> 10m v1.18.2[root@dlp ~(keystone)]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-786ffb7797-47rmk 1/1 Running 0 13m coredns-786ffb7797-chmk6 1/1 Running 0 13m dashboard-metrics-scraper-6b4884c9d5-h4wjk 1/1 Running 0 13m k8s-keystone-auth-d5jkt 1/1 Running 0 13m kube-dns-autoscaler-75859754fd-4m6db 1/1 Running 0 13m kube-flannel-ds-cmwdg 1/1 Running 0 13m kube-flannel-ds-stffb 1/1 Running 0 11m kubernetes-dashboard-c98496485-gbdn6 1/1 Running 0 13m npd-th85x 1/1 Running 0 11m # verify cluster to create test pods [root@dlp ~(keystone)]# kubectl create deployment test-nginx --image=nginx --replicas=2 deployment.apps/test-nginx created kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-nginx-7b7d9954bd-9sm7t 1/1 Running 0 22s 10.100.1.5 k8s-cluster-xw3duzjqjmv7-node-0 <none> <none> test-nginx-7b7d9954bd-p9mht 1/1 Running 0 22s 10.100.1.4 k8s-cluster-xw3duzjqjmv7-node-0 <none> <none>
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# kubectl expose deployment test-nginx --type="NodePort" --port 80 service/test-nginx exposed kubectl get services test-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx NodePort 10.254.4.103 <none> 80:30388/TCP 7s
[root@dlp ~(keystone)]#
[root@dlp ~(keystone)]# kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 & Forwarding from 0.0.0.0:10443 -> 80 curl localhost:10443
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
|
| [4] | If you'd like to use Magnum with common users, it needs to change some settings. |
|
[root@dlp ~(keystone)]# openstack role list +----------------------------------+------------------+ | ID | Name | +----------------------------------+------------------+ | 4c700d9cf8e74d15939866751160b8e2 | CloudUser | | 50c665f9fef44e94859f3917fa5209d7 | reader | | 57f0601d9b9f4836b6f606c64e58b083 | heat_stack_owner | | bd7c20ef06864388806511e3c1f6f3a9 | heat_stack_user | | c8e2eb5f05d54f0292628e9304648c15 | member | | f7e97312fb04428bae7ff4506dde29dc | admin | +----------------------------------+------------------+[root@dlp ~(keystone)]# openstack project list +----------------------------------+------------------------------------------------------------------+ | ID | Name | +----------------------------------+------------------------------------------------------------------+ | 3a53d6ae5a9644ada9d4d9739824b1f3 | b9bf51c6436a4c5aa7a2df05688afb49-747014bf-b3e1-4f73-8fdf-b2a8cb5 | | affa3a6446154e37adfd233c437bacc1 | service | | b9bf51c6436a4c5aa7a2df05688afb49 | admin | | f1f04d774d2141fb9acabd28d0e00c33 | hiroshima | +----------------------------------+------------------------------------------------------------------+[root@dlp ~(keystone)]# openstack user list +----------------------------------+-----------------------------------------------------------------------+ | ID | Name | +----------------------------------+-----------------------------------------------------------------------+ | 431cac9bf8244de39edbbf9bf69ef84c | admin | | 95da4d31df43419ebb1e495e5eb180bd | glance | | ddc826fcee8f4695a6af91e71add144f | nova | | d32e7f6fce6f4f639d72c8a8eabc7dd4 | placement | | fdf65d0b49ee4bd6a2713219139d5af9 | neutron | | e08c35de3b6746bd9518c97f7068a71e | serverworld | | 37eac0fbdeb346a0b37e8e48d977f4ae | cinder | | a43acdb98b314569b8a837784dda00a2 | heat | | c19fd0eabc8947478bd3539f2357cde1 | heat_domain_admin | | f127d4ad0db14d64b703dd07aab3004d | barbican | | 6e7bbd6aad8a49808a04b91c50f15d9d | magnum | | 05260c80dd594d8f97ff2465ef47646c | magnum_domain_admin | | f2c7636edfbc4381a52a9b195486ae6f | 2a013c5f-92a8-4799-b9d5-67dc24f96d94_b9bf51c6436a4c5aa7a2df05688afb49 | +----------------------------------+-----------------------------------------------------------------------+ # for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role [root@dlp ~(keystone)]# openstack role add --project hiroshima --user serverworld heat_stack_owner
# on the Node Neutron Server is running, change settings for Neutron [root@network ~]# vi /etc/neutron/policy.json # create new # overwrite some settings
{
"create_port:fixed_ips:subnet_id": "",
"create_port:allowed_address_pairs": "",
"create_port:allowed_address_pairs:ip_address": "",
}
[root@network ~]# systemctl restart neutron-server
# that's OK, common users can create clusters [cent@dlp ~(keystone)]$ openstack coe cluster list +--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+ | 75264f97-5bd7-4247-ba4d-9ffd028b9b34 | k8s-cluster2 | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+ |
| Sponsored Link |
|
|