OpenStack Victoria : How to use Magnum2020/10/24 |
|
Install OpenStack Container Infrastructure Management Service (Magnum).
This example is based on the environment like follows.
------------+---------------------------+---------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ Control Node ] | | [ Network Node ] | | [ Compute Node ] |
| | | | | |
| MariaDB RabbitMQ | | Linux Bridge | | Libvirt |
| Memcached httpd | | L2 Agent L3 Agent | | Nova Compute |
| Keystone Glance | | Metadata Agent | | Linux Bridge |
| Nova API Cinder API | | Cinder Volume | | L2 Agent |
| Neutron Server | | Heat API | | |
| Metadata Agent | | Heat Engine | | |
| Barbican API | | Magnum API | | |
| | | | | |
+-----------------------+ +-----------------------+ +-----------------------+
|
| [1] | On Control Node, Download a VM image for containers (Fedora CoreOS) and add it to Glance. |
|
root@dlp ~(keystone)# wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/32.20201004.3.0/x86_64/fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# xz -dv fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz root@dlp ~(keystone)# openstack image create Fedora-CoreOS --file=fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public |
| [2] | How to use Magnum. For example, Create Kubernetes Cluster with [admin] user. |
|
root@dlp ~(keystone)# openstack flavor list +----+----------+------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+----------+------+------+-----------+-------+-----------+ | 0 | m1.small | 2048 | 10 | 0 | 1 | True | +----+----------+------+------+-----------+-------+-----------+root@dlp ~(keystone)# openstack keypair list +-------+-------------------------------------------------+ | Name | Fingerprint | +-------+-------------------------------------------------+ | mykey | bb:79:ba:d5:16:a6:ee:54:7c:b8:d1:88:e4:a2:63:d2 | +-------+-------------------------------------------------+root@dlp ~(keystone)# openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 5d7b1b73-602d-40e2-a82f-352991de37e0 | public | 7aa5fca7-efe4-4a30-b1f0-0653cc743a1c | | 9f39959f-2f70-400c-aa80-5de54e92cf27 | private | e7e82cb2-29c0-4eea-a459-e212a2bc3650 | +--------------------------------------+---------+--------------------------------------+root@dlp ~(keystone)# openstack subnet list +--------------------------------------+----------------+--------------------------------------+------------------+ | ID | Name | Network | Subnet | +--------------------------------------+----------------+--------------------------------------+------------------+ | 7aa5fca7-efe4-4a30-b1f0-0653cc743a1c | public-subnet | 5d7b1b73-602d-40e2-a82f-352991de37e0 | 10.0.0.0/24 | | e7e82cb2-29c0-4eea-a459-e212a2bc3650 | private-subnet | 9f39959f-2f70-400c-aa80-5de54e92cf27 | 192.168.100.0/24 | +--------------------------------------+----------------+--------------------------------------+------------------+ # create Kubernetes Cluster template root@dlp ~(keystone)# openstack coe cluster template create k8s-cluster-template \ --image Fedora-CoreOS \ --external-network public \ --fixed-network private \ --fixed-subnet private-subnet \ --dns-nameserver 10.0.0.10 \ --network-driver flannel \ --docker-storage-driver overlay2 \ --docker-volume-size 10 \ --master-flavor m1.small \ --flavor m1.small \ --coe kubernetes Request to create cluster template k8s-cluster-template accepted +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | private-subnet | | master_flavor_id | m1.small | | uuid | 4dd6d1ac-d7d7-436e-ab2a-65455ce6aebb | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | False | | http_proxy | - | | docker_volume_size | 10 | | server_type | vm | | external_network_id | public | | cluster_distro | fedora-coreos | | image_id | Fedora-CoreOS | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay2 | | apiserver_port | - | | name | k8s-cluster-template | | created_at | 2020-10-24T06:51:38+00:00 | | network_driver | flannel | | fixed_network | private | | coe | kubernetes | | flavor_id | m1.small | | master_lb_enabled | False | | dns_nameserver | 10.0.0.10 | | hidden | False | +-----------------------+--------------------------------------+ # create Kubernetes Cluster with 2 nodes root@dlp ~(keystone)# openstack coe cluster create k8s-cluster \ --cluster-template k8s-cluster-template \ --master-count 1 \ --node-count 1 \ --keypair mykey Request to create cluster fc3358a4-1cac-470e-8db1-70b6c64c35c6 accepted # verify status # proceed to create cluster during [CREATE_IN_PROGRESS] state root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ | fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey | 1 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+-------------+---------+------------+--------------+--------------------+---------------+ # Heat orchestration System is used for creating root@dlp ~(keystone)# openstack stack list +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | k8s-cluster-7l2wbbc6lsrz | b573c9e160864f028fc2d681a929f5af | CREATE_IN_PROGRESS | 2020-10-24T06:52:22Z | None | +--------------------------------------+--------------------------+----------------------------------+--------------------+----------------------+--------------+ # confirm checkpoints for creation root@dlp ~(keystone)# openstack stack list --nested | grep k8s-cluster | 61eac6af-b996-4bfe-94ec-67ed63b823b7 | k8s-cluster-7l2wbbc6lsrz-kube_minions-mxx4zqanyevk-0-qgupsa3zerek | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:40Z | None | d34018c0-3b50-4baa-96b7-6011b088efad | | d34018c0-3b50-4baa-96b7-6011b088efad | k8s-cluster-7l2wbbc6lsrz-kube_minions-mxx4zqanyevk | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:39Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | 3cda31a8-d528-4744-a566-efe0172935f7 | k8s-cluster-7l2wbbc6lsrz-api_address_floating_switch-hwl3xisozljg | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:39Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | edbb3c8d-b4c4-42da-a95c-946bfd9c8673 | k8s-cluster-7l2wbbc6lsrz-api_address_lb_switch-rymy4gabknk6 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:38Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | 54e76ca1-fa87-4943-aa58-d34dbee30b53 | k8s-cluster-7l2wbbc6lsrz-etcd_address_lb_switch-me65co3bwhj5 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:57:38Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | eb88f9f4-5a4a-4e05-b9af-c0fd45c8e80a | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca-0-fbxdb6jsoyzh-api_address_switch-x2msyaoypidu | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:53:15Z | None | 9090916b-24e8-48d7-a819-e0abf4e6ad0b | | 9090916b-24e8-48d7-a819-e0abf4e6ad0b | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca-0-fbxdb6jsoyzh | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:33Z | None | 4bcad585-6ab5-410d-b118-5e48687e8e53 | | 4bcad585-6ab5-410d-b118-5e48687e8e53 | k8s-cluster-7l2wbbc6lsrz-kube_masters-l2aywhn7ssca | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:30Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | ff285c94-597b-4c70-ba87-0484ecf38259 | k8s-cluster-7l2wbbc6lsrz-etcd_lb-lx4bxmis2oo2 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:28Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | c2f89770-c0b9-406c-bb9d-ec74270d4a1f | k8s-cluster-7l2wbbc6lsrz-api_lb-gosqmuvkis25 | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:28Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | 9b67c3d1-10f5-43c9-a80f-745160e9a8b1 | k8s-cluster-7l2wbbc6lsrz-network-ui4g22u6kbau-network_switch-fjzt3idykvrh | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:27Z | None | 51859074-99de-4557-a01e-e966fa2a9edc | | 51859074-99de-4557-a01e-e966fa2a9edc | k8s-cluster-7l2wbbc6lsrz-network-ui4g22u6kbau | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:26Z | None | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | | 0c0a04c9-3a68-486c-92cc-1d8e0b4f921e | k8s-cluster-7l2wbbc6lsrz | b573c9e160864f028fc2d681a929f5af | CREATE_COMPLETE | 2020-10-24T06:52:22Z | None | None | # if sucessfully finished, state is [CREATE_COMPLETE] + [HEALTHY] root@dlp ~(keystone)# openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ # instances are running root@dlp ~(keystone)# openstack server list +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ | 828dc502-d974-4352-a230-ac759add5146 | k8s-cluster-7l2wbbc6lsrz-node-0 | ACTIVE | private=192.168.100.225, 10.0.0.220 | Fedora-CoreOS | m1.small | | d06cbf3e-8f76-4587-8e40-73200f209418 | k8s-cluster-7l2wbbc6lsrz-master-0 | ACTIVE | private=192.168.100.102, 10.0.0.238 | Fedora-CoreOS | m1.small | +--------------------------------------+-----------------------------------+--------+-------------------------------------+---------------+----------+ |
| [3] | To access to use Kubernetes Cluster, Set like follows. |
|
# install [kubectl] root@dlp ~(keystone)# snap install kubectl --classic kubectl 1.19.3 from Canonical✓ installed openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | fc3358a4-1cac-470e-8db1-70b6c64c35c6 | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+
root@dlp ~(keystone)#
root@dlp ~(keystone)# openstack coe cluster config k8s-cluster export KUBECONFIG=/root/config root@dlp ~(keystone)# export KUBECONFIG=/root/config
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-cluster-7l2wbbc6lsrz-master-0 Ready master 18m v1.18.2 k8s-cluster-7l2wbbc6lsrz-node-0 Ready <none> 14m v1.18.2root@dlp ~(keystone)# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-786ffb7797-4tnjm 1/1 Running 0 18m coredns-786ffb7797-f56sc 1/1 Running 0 18m dashboard-metrics-scraper-6b4884c9d5-ltrh9 1/1 Running 0 18m k8s-keystone-auth-b6th6 1/1 Running 0 18m kube-dns-autoscaler-75859754fd-shwzc 1/1 Running 0 18m kube-flannel-ds-bx6gh 1/1 Running 0 14m kube-flannel-ds-jnc8h 1/1 Running 0 18m kubernetes-dashboard-c98496485-d6wl5 1/1 Running 0 18m magnum-metrics-server-79556d6999-tnvb2 1/1 Running 0 18m npd-rh8wr 1/1 Running 0 14m # verify cluster to create test pods root@dlp ~(keystone)# kubectl create deployment test-nginx --image=nginx --replicas=2 deployment.apps/test-nginx created kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES test-nginx-7b7d9954bd-99pq9 1/1 Running 0 43s 10.100.1.5 k8s-cluster-7l2wbbc6lsrz-node-0 <none> <none> test-nginx-7b7d9954bd-xq9wp 1/1 Running 0 43s 10.100.1.6 k8s-cluster-7l2wbbc6lsrz-node-0 <none> <none>
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl expose deployment test-nginx --type="NodePort" --port 80 service/test-nginx exposed kubectl get services test-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx NodePort 10.254.145.185 <none> 80:31395/TCP 5s
root@dlp ~(keystone)#
root@dlp ~(keystone)# kubectl port-forward service/test-nginx --address 0.0.0.0 10443:80 & Forwarding from 0.0.0.0:10443 -> 80 curl localhost:10443
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
.....
.....
|
| [4] | If you'd like to use Magnum with common users, it needs to change some settings. |
|
root@dlp ~(keystone)# openstack role list +----------------------------------+------------------+ | ID | Name | +----------------------------------+------------------+ | 1838ec90094c480bae0979228a321c88 | heat_stack_user | | 5528fea7004044cfbd06ba1c2684af43 | CloudUser | | 624a27603cdf44a78fa802bf59a2ff8c | member | | 7e0d807bb67e4c239349385a85113bef | heat_stack_owner | | 8cd056a250054dddb2a15853e0a7f441 | admin | | f24b4181288a42669c42e24405c92374 | reader | +----------------------------------+------------------+root@dlp ~(keystone)# openstack project list +----------------------------------+-----------+ | ID | Name | +----------------------------------+-----------+ | 37197271a1954ddb90207a95d5f46488 | service | | 6c44eafd4f614985bf74b94f2aee82fb | hiroshima | | b573c9e160864f028fc2d681a929f5af | admin | +----------------------------------+-----------+root@dlp ~(keystone)# openstack user list +----------------------------------+-----------------------------------------------------------------------+ | ID | Name | +----------------------------------+-----------------------------------------------------------------------+ | ddcdc9a445bd45e7bdb71244343e7f78 | admin | | 03d8beaafa3045d58c3417bfec3bcefa | glance | | d605621cc0f44bdcb93864d3347b2300 | nova | | 319fe43139464ecbb178e217253929f1 | placement | | 2eadb99a37544406bc01b71eb7fb1b1c | neutron | | a13cfae0e5eb466fae71a636a6ffb6b4 | serverworld | | fd14a5e3cd654faba1b1e7923d298711 | cinder | | 740a0d7b450c4949a6ea7af78c8f8565 | heat | | 147869d311494a9cb06c3a67f219541e | heat_domain_admin | | 2473f94fed154b5290df73ed4e62763b | barbican | | 42282050760e440392b589d313577658 | gnocchi | | a954b69f9b8345d9a797abbc0a949108 | ceilometer | | 08052d6e306c4ec7986996e31df01729 | magnum | | c9bf3b2582274d47ba6629e157e9cc0b | magnum_domain_admin | | b5c5dee78291406780b4d8fcb1fdfe5b | fc3358a4-1cac-470e-8db1-70b6c64c35c6_b573c9e160864f028fc2d681a929f5af | +----------------------------------+-----------------------------------------------------------------------+ # for example, add [serverworld] user in [hiroshima] project to [heat_stack_owner] role root@dlp ~(keystone)# openstack role add --project hiroshima --user serverworld heat_stack_owner
root@dlp ~(keystone)#
vi /etc/neutron/policy.json # create new # overwrite some settings
{
"create_port:fixed_ips:subnet_id": "",
"create_port:allowed_address_pairs": "",
"create_port:allowed_address_pairs:ip_address": "",
}
root@dlp ~(keystone)# systemctl restart neutron-server
# that's OK, common users can create clusters ubuntu@dlp ~(keystone)$ openstack coe cluster list +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ | abcdc4d1-3bd5-47b5-a8d6-e7a46a2db88e | k8s-cluster | mykey | 1 | 1 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------------+---------+------------+--------------+-----------------+---------------+ |
| Sponsored Link |
|
|