CentOS Stream 8
Sponsored Link

OpenStack Victoria : How to use Magnum2021/03/26

 
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+---------------------------+---------------------------+------------
            |                           |                           |
        eth0|10.0.0.30              eth0|10.0.0.50              eth0|10.0.0.51
+-----------+-----------+   +-----------+-----------+   +-----------+-----------+
|    [ Control Node ]   |   |    [ Network Node ]   |   |    [ Compute Node ]   |
|                       |   |                       |   |                       |
|  MariaDB    RabbitMQ  |   |      Open vSwitch     |   |        Libvirt        |
|  Memcached  httpd     |   |     Neutron Server    |   |     Nova Compute      |
|  Keystone   Glance    |   |       OVN-Northd      |   |      Open vSwitch     |
|  Nova API             |   |     Cinder Volume     |   |   OVN Metadata Agent  |
|  Cinder API           |   |      iSCSI Target     |   |     OVN-Controller    |
|  Barbican API         |   |     Heat API/Engine   |   |                       |
|                       |   |     Magnum Services   |   |                       |
+-----------------------+   +-----------------------+   +-----------------------+

[1] On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance.
[root@dlp ~(keystone)]#
wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210301.3.1/x86_64/fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2.xz

[root@dlp ~(keystone)]#
xz -dv fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2.xz

[root@dlp ~(keystone)]#
openstack image create Fedora-CoreOS --file=fedora-coreos-33.20210301.3.1-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public

[2] How to use Magnum. For example, Create Kubernetes Cluster with [admin] user.
[root@dlp ~(keystone)]#
openstack flavor list

+----+----------+------+------+-----------+-------+-----------+
| ID | Name     |  RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+----------+------+------+-----------+-------+-----------+
| 0  | m1.small | 2048 |   10 |         0 |     1 | True      |
+----+----------+------+------+-----------+-------+-----------+

[root@dlp ~(keystone)]#
openstack keypair list

+-------+-------------------------------------------------+
| Name  | Fingerprint                                     |
+-------+-------------------------------------------------+
| mykey | 96:26:ef:f5:c0:81:40:67:d5:56:1e:17:5e:99:1d:83 |
+-------+-------------------------------------------------+

[root@dlp ~(keystone)]#
openstack network list

+--------------------------------------+---------+--------------------------------------+
| ID                                   | Name    | Subnets                              |
+--------------------------------------+---------+--------------------------------------+
| 9f53c3b1-ec6b-499d-854a-588d3b0b96d6 | private | 94afb0cc-1d58-4615-8fef-8852271bfc16 |
| ea9725cf-a8bc-4eb5-a374-7d7d0e11453c | public  | 63c29d7b-e870-477d-819b-e5dfe41af1d0 |
+--------------------------------------+---------+--------------------------------------+

[root@dlp ~(keystone)]#
openstack subnet list

+--------------------------------------+----------------+--------------------------------------+------------------+
| ID                                   | Name           | Network                              | Subnet           |
+--------------------------------------+----------------+--------------------------------------+------------------+
| 63c29d7b-e870-477d-819b-e5dfe41af1d0 | public-subnet  | ea9725cf-a8bc-4eb5-a374-7d7d0e11453c | 10.0.0.0/24      |
| 94afb0cc-1d58-4615-8fef-8852271bfc16 | private-subnet | 9f53c3b1-ec6b-499d-854a-588d3b0b96d6 | 192.168.100.0/24 |
+--------------------------------------+----------------+--------------------------------------+------------------+

# create Kubernetes Cluster template
[root@dlp ~(keystone)]# openstack coe cluster template create k8s-cluster-template \
--image Fedora-CoreOS \
--external-network public \
--fixed-network private \
--fixed-subnet private-subnet \
--dns-nameserver 10.0.0.10 \
--network-driver flannel \
--docker-storage-driver overlay2 \
--docker-volume-size 10 \
--master-flavor m1.small \
--flavor m1.small \
--coe kubernetes 
Request to create cluster template k8s-cluster-template accepted
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| insecure_registry     | -                                    |
| labels                | {}                                   |
| updated_at            | -                                    |
| floating_ip_enabled   | True                                 |
| fixed_subnet          | private-subnet                       |
| master_flavor_id      | m1.small                             |
| uuid                  | 4b2604e9-9e5c-4dbd-bbbc-592de6255905 |
| no_proxy              | -                                    |
| https_proxy           | -                                    |
| tls_disabled          | False                                |
| keypair_id            | -                                    |
| public                | False                                |
| http_proxy            | -                                    |
| docker_volume_size    | 10                                   |
| server_type           | vm                                   |
| external_network_id   | public                               |
| cluster_distro        | fedora-coreos                        |
| image_id              | Fedora-CoreOS                        |
| volume_driver         | -                                    |
| registry_enabled      | False                                |
| docker_storage_driver | overlay2                             |
| apiserver_port        | -                                    |
| name                  | k8s-cluster-template                 |
| created_at            | 2021-03-26T03:37:45+00:00            |
| network_driver        | flannel                              |
| fixed_network         | private                              |
| coe                   | kubernetes                           |
| flavor_id             | m1.small                             |
| master_lb_enabled     | False                                |
| dns_nameserver        | 10.0.0.10                            |
| hidden                | False                                |
+-----------------------+--------------------------------------+

# create Kubernetes Cluster with 2 nodes
[root@dlp ~(keystone)]# openstack coe cluster create k8s-cluster \
--cluster-template k8s-cluster-template \
--master-count 1 \
--node-count 1 \
--keypair mykey 
Request to create cluster 2a013c5f-92a8-4799-b9d5-67dc24f96d94 accepted

# verify status

# proceed to create cluster during [CREATE_IN_PROGRESS] state

[root@dlp ~(keystone)]#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status             | health_status |
+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+
| 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey   |          1 |            1 | CREATE_IN_PROGRESS | None          |
+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+

# Heat orchestration System is used for creating

[root@dlp ~(keystone)]#
openstack stack list

+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+
| ID                                   | Stack Name               | Project                          | Stack Status       | Creation Time        | Updated Time |
+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+
| 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | k8s-cluster-xw3duzjqjmv7 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_IN_PROGRESS | 2021-03-26T03:38:26Z | None         |
+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+

# confirm checkpoints for creation

[root@dlp ~(keystone)]#
openstack stack list --nested | grep k8s-cluster

| 5fca1791-0219-44c0-83f0-a8d47107214c | k8s-cluster-xw3duzjqjmv7-kube_minions-tndsjafpi7ox-0-uzisit52mj5y                                 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:20Z | None         | 9ff979f2-a66f-4bd0-9e6e-ef17b7cd21e6 |
| 9ff979f2-a66f-4bd0-9e6e-ef17b7cd21e6 | k8s-cluster-xw3duzjqjmv7-kube_minions-tndsjafpi7ox                                                | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:19Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| 5d270a23-2530-443a-a403-da90b7a46842 | k8s-cluster-xw3duzjqjmv7-api_address_floating_switch-2ehjcbduvjbg                                 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:19Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| dbd1473d-7700-4422-af33-d3bed4c7d06e | k8s-cluster-xw3duzjqjmv7-api_address_lb_switch-6ujh5orommsk                                       | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:18Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| 334bd75a-c1b4-4c53-a196-adf986cfd52c | k8s-cluster-xw3duzjqjmv7-etcd_address_lb_switch-xp725ak5twze                                      | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:42:18Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| d2603114-9ffa-4117-b1bd-04b2a9e4c39d | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa-0-mcw7quwoezqq-api_address_switch-p7s5fq3rlvfo | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:39:06Z | None         | 7584870f-8d81-49fa-9b28-aac52db2619d |
| 7584870f-8d81-49fa-9b28-aac52db2619d | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa-0-mcw7quwoezqq                                 | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:37Z | None         | b5c2f791-76a5-4f31-84e3-73b8895b52bc |
| b5c2f791-76a5-4f31-84e3-73b8895b52bc | k8s-cluster-xw3duzjqjmv7-kube_masters-yydg5rglubqa                                                | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:36Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| 8fa5750e-3320-404b-b40c-db6640091c52 | k8s-cluster-xw3duzjqjmv7-etcd_lb-u7iosmumufst                                                     | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:34Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| 3785f152-3da6-4625-b634-85cc84918c29 | k8s-cluster-xw3duzjqjmv7-api_lb-hkih2juzr4ew                                                      | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:34Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| f30a7eb3-7f9e-4fd7-a394-f167d2774b14 | k8s-cluster-xw3duzjqjmv7-network-tnenshonon32-network_switch-2p4qblfjnhut                         | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:32Z | None         | 67852d07-1cde-4a66-980d-d79e51a7b6e0 |
| 67852d07-1cde-4a66-980d-d79e51a7b6e0 | k8s-cluster-xw3duzjqjmv7-network-tnenshonon32                                                     | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:31Z | None         | 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a |
| 747014bf-b3e1-4f73-8fdf-b2a8cb598f1a | k8s-cluster-xw3duzjqjmv7                                                                          | b9bf51c6436a4c5aa7a2df05688afb49 | CREATE_COMPLETE | 2021-03-26T03:38:26Z | None         | None                                 |

# if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY]

[root@dlp ~(keystone)]#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+

# instances are running

[root@dlp ~(keystone)]#
openstack server list

+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
| ID                                   | Name                              | Status | Networks                            | Image         | Flavor   |
+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
| 7c3d81a9-df12-49ce-a5de-a0fc039271a4 | k8s-cluster-xw3duzjqjmv7-node-0   | ACTIVE | private=192.168.100.222, 10.0.0.225 | Fedora-CoreOS | m1.small |
| 47d04e69-20f0-45fd-bb33-74b4764e2d4a | k8s-cluster-xw3duzjqjmv7-master-0 | ACTIVE | private=192.168.100.155, 10.0.0.237 | Fedora-CoreOS | m1.small |
+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
[3] To access to use Kubernetes Cluster, Set like follows.
# install [kubectl] from Snap

[root@dlp ~(keystone)]#
snap install kubectl --classic

kubectl 1.20.4 from Canonical✓ installed
[root@dlp ~(keystone)]#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| 2a013c5f-92a8-4799-b9d5-67dc24f96d94 | k8s-cluster | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+

[root@dlp ~(keystone)]#
openstack coe cluster config k8s-cluster

export KUBECONFIG=/root/config
[root@dlp ~(keystone)]#
export KUBECONFIG=/root/config
[root@dlp ~(keystone)]#
kubectl get nodes

NAME                                STATUS   ROLES    AGE   VERSION
k8s-cluster-xw3duzjqjmv7-master-0   Ready    master   13m   v1.18.2
k8s-cluster-xw3duzjqjmv7-node-0     Ready    <none>   10m   v1.18.2

[root@dlp ~(keystone)]#
kubectl get pods -n kube-system

NAME                                         READY   STATUS    RESTARTS   AGE
coredns-786ffb7797-47rmk                     1/1     Running   0          13m
coredns-786ffb7797-chmk6                     1/1     Running   0          13m
dashboard-metrics-scraper-6b4884c9d5-h4wjk   1/1     Running   0          13m
k8s-keystone-auth-d5jkt                      1/1     Running   0          13m
kube-dns-autoscaler-75859754fd-4m6db         1/1     Running   0          13m
kube-flannel-ds-cmwdg                        1/1     Running   0          13m
kube-flannel-ds-stffb                        1/1     Running   0          11m
kubernetes-dashboard-c98496485-gbdn6         1/1     Running   0          13m
npd-th85x                                    1/1     Running   0          11m

# verify cluster to create test pods

[root@dlp ~(keystone)]#
kubectl create deployment test-nginx --image=nginx --replicas=2

deployment.apps/test-nginx created
[root@dlp ~(keystone)]#
kubectl get pods -o wide

NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE                              NOMINATED NODE   READINESS GATES
test-nginx-7b7d9954bd-9sm7t   1/1     Running   0          22s   10.100.1.5   k8s-cluster-xw3duzjqjmv7-node-0   <none>           <none>
test-nginx-7b7d9954bd-p9mht   1/1     Running   0          22s   10.100.1.4   k8s-cluster-xw3duzjqjmv7-node-0   <none>           <none>

[root@dlp ~(keystone)]#
kubectl expose deployment test-nginx --type="NodePort" --port 80

service/test-nginx exposed
[root@dlp ~(keystone)]#
kubectl get services test-nginx

NAME         TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
test-nginx   NodePort   10.254.4.103   <none>        80:30388/TCP   7s

[root@dlp ~(keystone)]#
kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 &

Forwarding from 0.0.0.0:10443 -> 80
[root@dlp ~(keystone)]#
curl localhost:10443

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
[4] If you'd like to use Magnum with common users, it needs to change some settings.
[root@dlp ~(keystone)]#
openstack role list

+----------------------------------+------------------+
| ID                               | Name             |
+----------------------------------+------------------+
| 4c700d9cf8e74d15939866751160b8e2 | CloudUser        |
| 50c665f9fef44e94859f3917fa5209d7 | reader           |
| 57f0601d9b9f4836b6f606c64e58b083 | heat_stack_owner |
| bd7c20ef06864388806511e3c1f6f3a9 | heat_stack_user  |
| c8e2eb5f05d54f0292628e9304648c15 | member           |
| f7e97312fb04428bae7ff4506dde29dc | admin            |
+----------------------------------+------------------+

[root@dlp ~(keystone)]#
openstack project list

+----------------------------------+------------------------------------------------------------------+
| ID                               | Name                                                             |
+----------------------------------+------------------------------------------------------------------+
| 3a53d6ae5a9644ada9d4d9739824b1f3 | b9bf51c6436a4c5aa7a2df05688afb49-747014bf-b3e1-4f73-8fdf-b2a8cb5 |
| affa3a6446154e37adfd233c437bacc1 | service                                                          |
| b9bf51c6436a4c5aa7a2df05688afb49 | admin                                                            |
| f1f04d774d2141fb9acabd28d0e00c33 | hiroshima                                                        |
+----------------------------------+------------------------------------------------------------------+

[root@dlp ~(keystone)]#
openstack user list

+----------------------------------+-----------------------------------------------------------------------+
| ID                               | Name                                                                  |
+----------------------------------+-----------------------------------------------------------------------+
| 431cac9bf8244de39edbbf9bf69ef84c | admin                                                                 |
| 95da4d31df43419ebb1e495e5eb180bd | glance                                                                |
| ddc826fcee8f4695a6af91e71add144f | nova                                                                  |
| d32e7f6fce6f4f639d72c8a8eabc7dd4 | placement                                                             |
| fdf65d0b49ee4bd6a2713219139d5af9 | neutron                                                               |
| e08c35de3b6746bd9518c97f7068a71e | serverworld                                                           |
| 37eac0fbdeb346a0b37e8e48d977f4ae | cinder                                                                |
| a43acdb98b314569b8a837784dda00a2 | heat                                                                  |
| c19fd0eabc8947478bd3539f2357cde1 | heat_domain_admin                                                     |
| f127d4ad0db14d64b703dd07aab3004d | barbican                                                              |
| 6e7bbd6aad8a49808a04b91c50f15d9d | magnum                                                                |
| 05260c80dd594d8f97ff2465ef47646c | magnum_domain_admin                                                   |
| f2c7636edfbc4381a52a9b195486ae6f | 2a013c5f-92a8-4799-b9d5-67dc24f96d94_b9bf51c6436a4c5aa7a2df05688afb49 |
+----------------------------------+-----------------------------------------------------------------------+

# for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role

[root@dlp ~(keystone)]#
openstack role add --project hiroshima --user serverworld heat_stack_owner

# on the Node Neutron Server is running, change settings for Neutron

[root@network ~]#
vi /etc/neutron/policy.json
# create new

# overwrite some settings

{
  "create_port:fixed_ips:subnet_id": "",
  "create_port:allowed_address_pairs": "",
  "create_port:allowed_address_pairs:ip_address": "",
}

[root@network ~]#
systemctl restart neutron-server

# that's OK, common users can create clusters

[cent@dlp ~(keystone)]$
openstack coe cluster list

+--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name         | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+
| 75264f97-5bd7-4247-ba4d-9ffd028b9b34 | k8s-cluster2 | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+--------------+---------+------------+--------------+-----------------+---------------+
Matched Content