Using the OCI VCN-Native Networking CNI plugin to provide networking services to containers inside Oracle Container Engine for Kubernetes - OKE
By default, the Oracle Kubernetes Engine (OKE) uses the OCI VCN-Native CNI plugin to provide network (security) features to containerized applications. In this tutorial, I will show you how you can verify what CNI plugin is used and how we can use this default CNI plugin (VCN-Native CNI) to configure a Load Balancer Service and attach that to an application that is running inside a container.
The Steps
- STEP 01: Deploy a Kubernetes Cluster (using OKE)
- STEP 02: Verify the installed CNI Plugin
- STEP 03: Deploy a sample application
- STEP 04: Configure Kubernetes Services of Type LoadBalancer
- STEP 05: Removing the sample application and Kubernetes Services of Type LoadBalancer
STEP 01 - Deploy a Kubernetes Cluster using OKE
In [this] article you can read about the different OKE deployment models you can choose from.
The flavors are as follows:
- Example 1: Cluster with Flannel CNI Plugin, Public Kubernetes API Endpoint, Private Worker Nodes, and Public Load Balancers
- Example 2: Cluster with Flannel CNI Plugin, Private Kubernetes API Endpoint, Private Worker Nodes, and Public Load Balancers
- Example 3: Cluster with OCI CNI Plugin, Public Kubernetes API Endpoint, Private Worker Nodes, and Public Load Balancers
- Example 4: Cluster with OCI CNI Plugin, Private Kubernetes API Endpoint, Private Worker Nodes, and Public Load Balancers
I have chosen to go for the Example 3 deployment model and you can read how this was deployed in this article. (ADD ARTICLE LATER)
STEP 02 - Verify the installed CNI Plugin
When the Oracle Kubernetes Engine (OKE) is fully deployed and you have access to this you can issue the following command
- 1. Issue the following command:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get all -n kube-system NAME READY STATUS RESTARTS AGE pod/coredns-64ffdf5cf7-lvrhq 1/1 Running 0 2d pod/coredns-64ffdf5cf7-rmxt8 1/1 Running 0 2d pod/coredns-64ffdf5cf7-vq76p 1/1 Running 0 2d pod/csi-oci-node-ghff6 1/1 Running 0 2d pod/csi-oci-node-jrjpr 1/1 Running 0 2d pod/csi-oci-node-r68qz 1/1 Running 1 (2d ago) 2d pod/kube-dns-autoscaler-5bb955d5c-r2j2q 1/1 Running 0 2d pod/kube-proxy-5cznp 1/1 Running 0 2d pod/kube-proxy-fddrd 1/1 Running 0 2d pod/kube-proxy-sb769 1/1 Running 0 2d pod/proxymux-client-7s7f9 1/1 Running 0 2d pod/proxymux-client-lngrm 1/1 Running 0 2d pod/proxymux-client-qxlf2 1/1 Running 0 2d pod/vcn-native-ip-cni-hkfjz 1/1 Running 0 2d pod/vcn-native-ip-cni-pdv4c 1/1 Running 0 2d pod/vcn-native-ip-cni-qfvk8 1/1 Running 0 2d NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kube-dns ClusterIP 10.96.5.5 <none> 53/UDP,53/TCP,9153/TCP 2d NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/csi-oci-node 3 3 3 3 3 <none> 2d daemonset.apps/kube-proxy 3 3 3 3 3 beta.kubernetes.io/os=linux 2d daemonset.apps/node-termination-handler 0 0 0 0 0 oci.oraclecloud.com/oke-is-preemptible=true 2d daemonset.apps/nvidia-gpu-device-plugin 0 0 0 0 0 <none> 2d daemonset.apps/proxymux-client 3 3 3 3 3 node.info.ds_proxymux_client=true 2d daemonset.apps/vcn-native-ip-cni 3 3 3 3 3 <none> 2d NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/coredns 3/3 3 3 2d deployment.apps/kube-dns-autoscaler 1/1 1 1 2d NAME DESIRED CURRENT READY AGE replicaset.apps/coredns-64ffdf5cf7 3 3 3 2d replicaset.apps/kube-dns-autoscaler-5bb955d5c 1 1 1 2d iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 2. Notice the name “vcn-native” in the output in the pod section.
- 3. Notice the name “vcn-native” in the output in the deamonset section.
This will tell you that the OCI VCN-Native CNI plugin is currently used for this deployed OKE deployment.
STEP 03 - Deploy a sample application
Now I am going to deploy a sample application. We will use this sample application together with the OCI VCN-Native CNI plugin and enable the Load Balancer Service type in the next step.
- 1. Issue the following command to deploy a sample (NGINX) application inside OKE:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl apply -f https://k8s.io/examples/application/deployment.yaml deployment.apps/nginx-deployment created
- 2. Issue the following command to verify the details of the deployed sample (NGINX) application:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl describe deployment nginx-deployment Name: nginx-deployment Namespace: default CreationTimestamp: Fri, 08 Mar 2024 07:57:02 +0000 Labels: <none> Annotations: deployment.kubernetes.io/revision: 1 Selector: app=nginx Replicas: 2 desired
- 3. Notice that the application is deployed using two pods.
- 1. Issue this command to take a closer look at the deployed pods:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-86dcfdf4c6-fdxgz 1/1 Running 0 3m46s nginx-deployment-86dcfdf4c6-fqrkh 1/1 Running 0 3m46s iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 2. Notice that there are two instances/pods/replicas of the NGINX application and the status is set to RUNNING.
A visual representation of the deployment can be found in the diagram below. Focus on the two deployed pods inside the worker nodes.
STEP 04 - Configure Kubernetes Services of Type LoadBalancer
Now that we have our sample application running inside the OKE it is time to expose the application to the network (the internet) by attaching a network service of the type LoadBalancer to the application.
- 1. Issue the following command to review the existing running services in the Kubernetes Cluster:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP,12250/TCP 2d23h iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 2. Notice that the only service that is running belongs to the Kubernetes Control Plane itself.
- 1. Issue the following command to deploy a new (VCN-Native CNI plugin) network service of the type LoadBalancer and expose this new service to the application we just deployed:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl expose deployment nginx-deployment --type=LoadBalancer --name=my-service service/my-service exposed iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 2. Notice the message that the service is successfully exposed.
- 1. Issue the following command again to review the existing running services in the Kubernetes Cluster:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP,12250/TCP 2d23h my-service LoadBalancer 10.96.209.132 xxx.xxx.xxx.167 80:32291/TCP 53s iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 2. Notice that the service we have just configured is now on the list
- 3. Notice the EXTERNAL-IP (Public IP address) that has been assigned to the Load Balancer (ending with .167)
- 4. Click on the minimize button to minimize the Cloud Shell Window.
Open a new browser.
When we copy this Public IP address and paste it into our browser address field we can now access the NGINX webserver that is deployed on a container inside the Oracle Kubernetes Engine (OKE).
We can also take a closer look at what is happening under the hood by using the OCI console.
Of we browse to Networking > Loadbalancers
Click on Load Balancer.
- 1. Notice there is a new load balancer deployed with the public IP address ending with .167.
- 2. Click on the name of the load balancer.
- 1. Notice that the load balancer is ACTIVE.
- 2. Scroll down.
- 1. Review the configuration details of the deployed load balancer.
- 2. Click on the Restore button to restore the Cloud Shell Window.
A visual representation of the load balancer deployment can be found in the diagram below. Focus on the load balancer.
STEP 05 - Removing the sample application and Kubernetes Services of Type LoadBalancer
Now that we have deployed a sample application and created a new Kubernetes network service of the type Load Balancer it is time to clean up the application and the service.
- 1. Issue this command to delete the Load Balancer service:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl delete service my-service -n default service "my-service" deleted
- 2. Notice the message that the service (load balancer) is successfully deleted.
- 3. Issue the following command to verify if the service (load balancer) is deleted:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP,12250/TCP 2d23h iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 4. Notice that the service (load balancer) is deleted.
- 1. Issue the following command to retrieve the existing pods:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-86dcfdf4c6-fdxgz 1/1 Running 0 36m nginx-deployment-86dcfdf4c6-fqrkh 1/1 Running 0 36m
- 2. Notice that the NGINX application is still running.
- 3. Issue the following command to delete the deployment of the NGINX application:
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl delete deployment nginx-deployment --namespace default deployment.apps "nginx-deployment" deleted
- 4. Issue the following command to retrieve the existing pods again (to verify the deployment is deleted):
iwan_hooge@cloudshell:~ (eu-amsterdam-1)$ kubectl get pods No resources found in default namespace. iwan_hooge@cloudshell:~ (eu-amsterdam-1)$
- 5. Notice that the NGINX application is deleted.
- 6. Click on the minimize button to minimize the Cloud Shell Window.
- 1. Notice that the OCI Console will display a message that the Load Balancer is deleted.
- 2. Click on the Close button
Notice that there is no load balancer deployed anymore (because we just deleted the service).
A visual representation of the load balancer deletion can be found in the diagram below. Focus on the part where the load balancer is no longer deployed.
Conclusion
In this article, we have seen how we can verify the default CNI plugin that is used by the Oracle Kubernetes Engine (OKE). We have also deployed a new NGINX containerized application to test some basic networking features of the default VCN-Native CNI plugin. We leveraged the VCN-Native CNI plugin to create a new network service of the type load balancer and exposed that service to our deployed NGINX application. In the end, we cleared up the application and the load-balancing service.