Ubuntu 20.04
Sponsored Link

OpenStack Victoria : How to use Magnum2020/10/24

 
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+---------------------------+---------------------------+------------
            |                           |                           |
        eth0|10.0.0.30              eth0|10.0.0.50              eth0|10.0.0.51
+-----------+-----------+   +-----------+-----------+   +-----------+-----------+
|    [ Control Node ]   |   |    [ Network Node ]   |   |    [ Compute Node ]   |
|                       |   |                       |   |                       |
|  MariaDB    RabbitMQ  |   |      Linux Bridge     |   |        Libvirt        |
|  Memcached  httpd     |   |   L2 Agent L3 Agent   |   |     Nova Compute      |
|  Keystone   Glance    |   |     Metadata Agent    |   |      Linux Bridge     |
|  Nova API  Cinder API |   |     Cinder Volume     |   |        L2 Agent       |
|  Neutron Server       |   |        Heat API       |   |                       |
|  Metadata Agent       |   |      Heat Engine      |   |                       |
|  Barbican API         |   |       Magnum API      |   |                       |
|                       |   |                       |   |                       |
+-----------------------+   +-----------------------+   +-----------------------+

[1] On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance.
root@dlp ~(keystone)#
wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/32.20201004.3.0/x86_64/fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz

root@dlp ~(keystone)#
xz -dv fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz

root@dlp ~(keystone)#
openstack image create Fedora-CoreOS --file=fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public

[2] How to use Magnum. For example, Create Kubernetes Cluster with [admin] user.
root@dlp ~(keystone)#
openstack flavor list

+----+----------+------+------+-----------+-------+-----------+
| ID | Name     |  RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+----------+------+------+-----------+-------+-----------+
| 0  | m1.small | 2048 |   10 |         0 |     1 | True      |
+----+----------+------+------+-----------+-------+-----------+

root@dlp ~(keystone)#
openstack keypair list

+-------+-------------------------------------------------+
| Name  | Fingerprint                                     |
+-------+-------------------------------------------------+
| mykey | bb:79:ba:d5:16:a6:ee:54:7c:b8:d1:88:e4:a2:63:d2 |
+-------+-------------------------------------------------+

root@dlp ~(keystone)#
openstack network list

+--------------------------------------+---------+--------------------------------------+
| ID                                   | Name    | Subnets                              |
+--------------------------------------+---------+--------------------------------------+
| 5d7b1b73-602d-40e2-a82f-352991de37e0 | public  | 7aa5fca7-efe4-4a30-b1f0-0653cc743a1c |
| 9f39959f-2f70-400c-aa80-5de54e92cf27 | private | e7e82cb2-29c0-4eea-a459-e212a2bc3650 |
+--------------------------------------+---------+--------------------------------------+

root@dlp ~(keystone)#
openstack subnet list

+--------------------------------------+----------------+--------------------------------------+------------------+
| ID                                   | Name           | Network                              | Subnet           |
+--------------------------------------+----------------+--------------------------------------+------------------+
| 7aa5fca7-efe4-4a30-b1f0-0653cc743a1c | public-subnet  | 5d7b1b73-602d-40e2-a82f-352991de37e0 | 10.0.0.0/24      |
| e7e82cb2-29c0-4eea-a459-e212a2bc3650 | private-subnet | 9f39959f-2f70-400c-aa80-5de54e92cf27 | 192.168.100.0/24 |
+--------------------------------------+----------------+--------------------------------------+------------------+

# create Kubernetes Cluster template
root@dlp ~(keystone)# openstack coe cluster template create k8s-cluster-template \
--image Fedora-CoreOS \
--external-network public \
--fixed-network private \
--fixed-subnet private-subnet \
--dns-nameserver 10.0.0.10 \
--network-driver flannel \
--docker-storage-driver overlay2 \
--docker-volume-size 10 \
--master-flavor m1.small \
--flavor m1.small \
--coe kubernetes 
Request to create cluster template k8s-cluster-template accepted
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| insecure_registry     | -                                    |
| labels                | {}                                   |
| updated_at            | -                                    |
| floating_ip_enabled   | True                                 |
| fixed_subnet          | private-subnet                       |
| master_flavor_id      | m1.small                             |
| uuid                  | 4dd6d1ac-d7d7-436e-ab2a-65455ce6aebb |
| no_proxy              | -                                    |
| https_proxy           | -                                    |
| tls_disabled          | False                                |
| keypair_id            | -                                    |
| public                | False                                |
| http_proxy            | -                                    |
| docker_volume_size    | 10                                   |
| server_type           | vm                                   |
| external_network_id   | public                               |
| cluster_distro        | fedora-coreos                        |
| image_id              | Fedora-CoreOS                        |
| volume_driver         | -                                    |
| registry_enabled      | False                                |
| docker_storage_driver | overlay2                             |
| apiserver_port        | -                                    |
| name                  | k8s-cluster-template                 |
| created_at            | 2020-10-24T06:51:38+00:00            |
| network_driver        | flannel                              |
| fixed_network         | private                              |
| coe                   | kubernetes                           |
| flavor_id             | m1.small                             |
| master_lb_enabled     | False                                |
| dns_nameserver        | 10.0.0.10                            |
| hidden                | False                                |
+-----------------------+--------------------------------------+

# create Kubernetes Cluster with 2 nodes
root@dlp ~(keystone)# openstack coe cluster create k8s-cluster \
--cluster-template k8s-cluster-template \
--master-count 1 \
--node-count 1 \
--keypair mykey 
Request to create cluster fc3358a4-1cac-470e-8db1-70b6c64c35c6 accepted

# verify status

# proceed to create cluster during [CREATE_IN_PROGRESS] state

root@dlp ~(keystone)#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status             | health_status |
+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+
| fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey   |          1 |            1 | CREATE_IN_PROGRESS | None          |
+--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+

# Heat orchestration System is used for creating

root@dlp ~(keystone)#
openstack stack list

+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+
| ID                                   | Stack Name               | Project                          | Stack Status       | Creation Time        | Updated Time |
+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+
| 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | k8s-cluster-7l2wbbc6lsrz | b573c9e160864f028fc2d681a929f5af | CREATE_IN_PROGRESS | 2020-10-24T06:52:22Z | None         |
+--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+

# confirm checkpoints for creation

root@dlp ~(keystone)#
openstack stack list --nested | grep k8s-cluster

| 61eac6af-b996-4bfe-94ec-67ed63b823b7 | k8s-cluster-7l2wbbc6lsrz-kube_minions-mxx4zqanyevk-0-qgupsa3zerek                                 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:40Z | None         | d34018c0-3b50-4baa-96b7-6011b088efad |
| d34018c0-3b50-4baa-96b7-6011b088efad | k8s-cluster-7l2wbbc6lsrz-kube_minions-mxx4zqanyevk                                                | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:39Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| 3cda31a8-d528-4744-a566-efe0172935f7 | k8s-cluster-7l2wbbc6lsrz-api_address_floating_switch-hwl3xisozljg                                 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:39Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| edbb3c8d-b4c4-42da-a95c-946bfd9c8673 | k8s-cluster-7l2wbbc6lsrz-api_address_lb_switch-rymy4gabknk6                                       | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:38Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| 54e76ca1-fa87-4943-aa58-d34dbee30b53 | k8s-cluster-7l2wbbc6lsrz-etcd_address_lb_switch-me65co3bwhj5                                      | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:38Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| eb88f9f4-5a4a-4e05-b9af-c0fd45c8e80a | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca-0-fbxdb6jsoyzh-api_address_switch-x2msyaoypidu | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:53:15Z | None         | 9090916b-24e8-48d7-a819-e0abf4e6ad0b |
| 9090916b-24e8-48d7-a819-e0abf4e6ad0b | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca-0-fbxdb6jsoyzh                                 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:33Z | None         | 4bcad585-6ab5-410d-b118-5e48687e8e53 |
| 4bcad585-6ab5-410d-b118-5e48687e8e53 | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca                                                | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:30Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| ff285c94-597b-4c70-ba87-0484ecf38259 | k8s-cluster-7l2wbbc6lsrz-etcd_lb-lx4bxmis2oo2                                                     | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:28Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| c2f89770-c0b9-406c-bb9d-ec74270d4a1f | k8s-cluster-7l2wbbc6lsrz-api_lb-gosqmuvkis25                                                      | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:28Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| 9b67c3d1-10f5-43c9-a80f-745160e9a8b1 | k8s-cluster-7l2wbbc6lsrz-network-ui4g22u6kbau-network_switch-fjzt3idykvrh                         | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:27Z | None         | 51859074-99de-4557-a01e-e966fa2a9edc |
| 51859074-99de-4557-a01e-e966fa2a9edc | k8s-cluster-7l2wbbc6lsrz-network-ui4g22u6kbau                                                     | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:26Z | None         | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e |
| 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | k8s-cluster-7l2wbbc6lsrz                                                                          | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:22Z | None         | None                                 |

# if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY]

root@dlp ~(keystone)#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+

# instances are running

root@dlp ~(keystone)#
openstack server list

+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
| ID                                   | Name                              | Status | Networks                            | Image         | Flavor   |
+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
| 828dc502-d974-4352-a230-ac759add5146 | k8s-cluster-7l2wbbc6lsrz-node-0   | ACTIVE | private=192.168.100.225, 10.0.0.220 | Fedora-CoreOS | m1.small |
| d06cbf3e-8f76-4587-8e40-73200f209418 | k8s-cluster-7l2wbbc6lsrz-master-0 | ACTIVE | private=192.168.100.102, 10.0.0.238 | Fedora-CoreOS | m1.small |
+--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+
[3] To access to use Kubernetes Cluster, Set like follows.
# install [kubectl]

root@dlp ~(keystone)#
snap install kubectl --classic

kubectl 1.19.3 from Canonical✓ installed
root@dlp ~(keystone)#
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+

root@dlp ~(keystone)#
openstack coe cluster config k8s-cluster

export KUBECONFIG=/root/config
root@dlp ~(keystone)#
export KUBECONFIG=/root/config
root@dlp ~(keystone)#
kubectl get nodes

NAME                                STATUS   ROLES    AGE   VERSION
k8s-cluster-7l2wbbc6lsrz-master-0   Ready    master   18m   v1.18.2
k8s-cluster-7l2wbbc6lsrz-node-0     Ready    <none>   14m   v1.18.2

root@dlp ~(keystone)#
kubectl get pods -n kube-system

NAME                                         READY   STATUS    RESTARTS   AGE
coredns-786ffb7797-4tnjm                     1/1     Running   0          18m
coredns-786ffb7797-f56sc                     1/1     Running   0          18m
dashboard-metrics-scraper-6b4884c9d5-ltrh9   1/1     Running   0          18m
k8s-keystone-auth-b6th6                      1/1     Running   0          18m
kube-dns-autoscaler-75859754fd-shwzc         1/1     Running   0          18m
kube-flannel-ds-bx6gh                        1/1     Running   0          14m
kube-flannel-ds-jnc8h                        1/1     Running   0          18m
kubernetes-dashboard-c98496485-d6wl5         1/1     Running   0          18m
magnum-metrics-server-79556d6999-tnvb2       1/1     Running   0          18m
npd-rh8wr                                    1/1     Running   0          14m

# verify cluster to create test pods

root@dlp ~(keystone)#
kubectl create deployment test-nginx --image=nginx --replicas=2

deployment.apps/test-nginx created
root@dlp ~(keystone)#
kubectl get pods -o wide

NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE                              NOMINATED NODE   READINESS GATES
test-nginx-7b7d9954bd-99pq9   1/1     Running   0          43s   10.100.1.5   k8s-cluster-7l2wbbc6lsrz-node-0   <none>           <none>
test-nginx-7b7d9954bd-xq9wp   1/1     Running   0          43s   10.100.1.6   k8s-cluster-7l2wbbc6lsrz-node-0   <none>           <none>

root@dlp ~(keystone)#
kubectl expose deployment test-nginx --type="NodePort" --port 80

service/test-nginx exposed
root@dlp ~(keystone)#
kubectl get services test-nginx

NAME         TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
test-nginx   NodePort   10.254.145.185   <none>        80:31395/TCP   5s

root@dlp ~(keystone)#
kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 &

Forwarding from 0.0.0.0:10443 -> 80
root@dlp ~(keystone)#
curl localhost:10443

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
[4] If you'd like to use Magnum with common users, it needs to change some settings.
root@dlp ~(keystone)#
openstack role list

+----------------------------------+------------------+
| ID                               | Name             |
+----------------------------------+------------------+
| 1838ec90094c480bae0979228a321c88 | heat_stack_user  |
| 5528fea7004044cfbd06ba1c2684af43 | CloudUser        |
| 624a27603cdf44a78fa802bf59a2ff8c | member           |
| 7e0d807bb67e4c239349385a85113bef | heat_stack_owner |
| 8cd056a250054dddb2a15853e0a7f441 | admin            |
| f24b4181288a42669c42e24405c92374 | reader           |
+----------------------------------+------------------+

root@dlp ~(keystone)#
openstack project list

+----------------------------------+-----------+
| ID                               | Name      |
+----------------------------------+-----------+
| 37197271a1954ddb90207a95d5f46488 | service   |
| 6c44eafd4f614985bf74b94f2aee82fb | hiroshima |
| b573c9e160864f028fc2d681a929f5af | admin     |
+----------------------------------+-----------+

root@dlp ~(keystone)#
openstack user list

+----------------------------------+-----------------------------------------------------------------------+
| ID                               | Name                                                                  |
+----------------------------------+-----------------------------------------------------------------------+
| ddcdc9a445bd45e7bdb71244343e7f78 | admin                                                                 |
| 03d8beaafa3045d58c3417bfec3bcefa | glance                                                                |
| d605621cc0f44bdcb93864d3347b2300 | nova                                                                  |
| 319fe43139464ecbb178e217253929f1 | placement                                                             |
| 2eadb99a37544406bc01b71eb7fb1b1c | neutron                                                               |
| a13cfae0e5eb466fae71a636a6ffb6b4 | serverworld                                                           |
| fd14a5e3cd654faba1b1e7923d298711 | cinder                                                                |
| 740a0d7b450c4949a6ea7af78c8f8565 | heat                                                                  |
| 147869d311494a9cb06c3a67f219541e | heat_domain_admin                                                     |
| 2473f94fed154b5290df73ed4e62763b | barbican                                                              |
| 42282050760e440392b589d313577658 | gnocchi                                                               |
| a954b69f9b8345d9a797abbc0a949108 | ceilometer                                                            |
| 08052d6e306c4ec7986996e31df01729 | magnum                                                                |
| c9bf3b2582274d47ba6629e157e9cc0b | magnum_domain_admin                                                   |
| b5c5dee78291406780b4d8fcb1fdfe5b | fc3358a4-1cac-470e-8db1-70b6c64c35c6_b573c9e160864f028fc2d681a929f5af |
+----------------------------------+-----------------------------------------------------------------------+

# for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role

root@dlp ~(keystone)#
openstack role add --project hiroshima --user serverworld heat_stack_owner
root@dlp ~(keystone)#
vi /etc/neutron/policy.json
# create new

# overwrite some settings

{
  "create_port:fixed_ips:subnet_id": "",
  "create_port:allowed_address_pairs": "",
  "create_port:allowed_address_pairs:ip_address": "",
}

root@dlp ~(keystone)#
systemctl restart neutron-server

# that's OK, common users can create clusters

ubuntu@dlp ~(keystone)$
openstack coe cluster list

+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| uuid                                 | name        | keypair | node_count | master_count | status          | health_status |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
| abcdc4d1-3bd5-47b5-a8d6-e7a46a2db88e | k8s-cluster | mykey   |          1 |            1 | CREATE_COMPLETE | HEALTHY       |
+--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
Matched Content