Kubernetes is a system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery, it takes advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads between them. It has features like
- Service discovery and load balancing
- Storage orchestration
- Automated rollouts and rollbacks
- Self-healing
- Horizontal scaling
Some of the key Kubernetes terminology
| Master (control plane) | the container orchestration layer that exposes the API and interfaces to define, deploy, and manage the lifecycle of containers |
| Node (worker machine) | is a worker machine in Kubernetes |
| Cluster | a set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node |
| Pod | the smallest and simplest Kubernetes object. A Pod represents a set of running containers on your cluster |
| Deployment | an API object that manages a replicated application, typically by running Pods with no local state |
| minikube | a tool for running Kubernetes locally, for enterprise solutions you can use the following
|
| kubectl | a command line tool for communicating with a Kubernetes API server |
| Kube-proxy | kube-proxy is a network proxy that runs on each node in your cluster, it proxies network traffic to Service Objects (network objects) to communicate to the POD/Container. There is only one kube-proxy per node and thus all network traffic flows through it. |
The below diagram is a high-level view of how Kubernetes works, basically the master controls the nodes and pod on what desired configuration you need, you access the master using the minikube or kubectl commands, many of the cloud provides have options to setup Kubernete clusters very easy so all you need to do is create the configuration files which I cover below.
In Kubernetes you create configuration files that will be used to create kubernetes objects, objects service different purposes for example running a container, monitoring a container, setting up networking, etc.
Below are some (but no all) brief descriptions of a kubernetes configuration file
- apiVersion - the version of the Kubernetes API you're using to create this object, different versions offer different objects types for example versions are v1, app/v1
- kind - the kind of object you want to create (Cluster, Pods (DEV), Services, Deployments (Production), Volumes, StatefulSet, ReplicaController, etc)
- metadata - data that helps uniquely identify the object, including a name string, UID, and optional namespace
- spec - what state you desire for the object
To expand on the kind object we have the following
- Pod - the smallest and simplest Kubernetes object. A Pod represents a set of running containers (grouping containers) on your cluster.
- Service - an abstract way to expose an application running on a set of Pods as a network service
- ClusterIP - used to expose a set of pods to other objects in the cluster, choosing this makes the Service only reachable from within the cluster
- NodePort - used to expose a container to the outside world (only use in DEV)
- LoadBalancer - legacy way of getting network traffic into a kubernetes cluster
- Ingress - an API object that manages external access to the services in a cluster, typically HTTP/HTTPS. Ingress may provide load balancing, SSL termination and name-based virtual hosting.
- Volume - a directory containing data, accessible to the containers in a Pod, there are Volumes, Persistent Volumes (PV) and Persistent Volume Claim (PVC).
Below is a diagram on what a pod could look like, its a grouping of your application and any shared resources that those containers needs, resources could be shared storage (PVC's) IP addresses, information about how to run each container (what ports to use, etc), its a tight coupling of an application.
When using a development environment you can use the NodePort, however in production environments you should use the Ingress and ClusterIP service objects. The ClusterIP is used to allow other ClusterIP's to communicate with each other inside the nodes (no external access). You can only access ClusterIP's via the Ingress service, these are generally the ports and IP addresses that you want the application to expose. There are a number of different Ingress options for example Ingress-nginx or kubernetes-ingress, etc, also the Ingress will be differet if setting up in Azure, AWS or Google cloud. You create a Ingress config which is loaded into a Ingress controller which watches for changes and updates the Ingress service which is used to route network traffic. Keep an eye on this area as its moving all the time and different solutions may be available.
There are three terminologies regarding data storage that you need to understand in Kubernetes
| Volume (Kubernetes) | is a specific type of object that allows a container to store persistent data at the Pod level and if the pod dies so does the volume. Kubernetes supports many types of volumes an a Pod can use any number of them simultaneously. This type of volume is not idea to store data inside a database, its idea for caching volumes, etc. Examples of volume types are: azureDisk, azureFile, awsElasticBlockStore, nfs, and many others |
| Persistent Volume | is a volume that is not tied to a pod, it exists outside the pod. PV's can be of two types
|
| Persistent Volume Claim | is a request for storage by a user, these persistent volumes are created inside the pod configuration file (static), however kubernetes can create a persistent volume (dynamic) on the fly. It is similar to a Pod, Claims can request specific size and access modes which can be any of the following:
|
The deployment configuration file (uses the deployment object type which can run a set of identical pods) instructs Kubernetes what containers and how many should be running, Kubernetes keeps a count of this and adjusts if less are running across the nodes to make sure that the correct number is running (desired state).
There are two types of deployment
- Imperative Delpoyments - do these steps to arrive at this container setup (manual via CLI)
- Declarative Delpoyments - our container setup should look like this go away and do it for me (automatic via files)
Below are some examples of Kubernetes configuration files
| Pods and Deployments | |
| Pod config example (DEV only) | apiVersion: v1 # version and what objects will be available
kind: Pod
metadata: # uniquely ID this object
name: client-pod
labels:
component: web # used to link other objects to this Pod, does have to be called component
spec:
containers:
- name: client # the container name
image: datadisk/multi-client # the image to retrieve from docker hub
ports:
- containerPort: 3000 # expose port to outside world |
| Deployment config example (Production) | apiVersion: apps/v1 # version and what objects will be available
kind: Deployment # a deployment object
metadata: # uniquely ID this object
name: client-deployment
spec:
replicas: 1 # number of identical pods
selector: # used to identify the pod (handle on the pod)
matchLabels:
component: web
template: # information used by all pods
metadata:
labels:
component: web # all pods will have a component label called web
spec:
volumes:
- name: web-storage # name will be used below
persistentVolumeClaim:
claimName: web-pvc # the external storage (PVC)
containers: # each pod will use the same image be called client and expose port 3000
- name: client
image: datadisk/multi-client
ports:
- containerPort: 3000
env: # how to include environment values, including a secret
- name: APP_USER
value: user1
- name: APP_PASSWORD
valueFrom: # secret object
secretKeyRef:
name: mysecrets # name we give to the secret_name location
key: APP_PASSWORD # key of the key/value pair we stored in the above secret_name location
volumeMounts:
- name: web-storage # use same name as above
mountPath: /web_app |
| Services (NodePort, ClusterIP and Ingress) | |
| Service config example (NodePort) | apiVersion: v1
kind: Service # setting up networking
metadata:
name: client-node-port
spec:
type: NodePort # NodePort service, expose container to outside world
ports: # array of ports
- port: 3050 # used to allow other POD access, matches the Pod containerPort above
targetPort: 3000 # port inside the Pod/container
nodePort: 31515 # used by browser or other applications externally
selector: #
component: web # link to above Pod, uses Pod component label |
| Service config example (ClusterIP) | apiVersion: v1
kind: Service
metadata:
name: client-cluster-ip-service
spec:
type: ClusterIP # ClusterIP service object
selector:
component: web
ports:
- port: 3000 # used to allow other POD access, matches the Pod containerPort above
targetPort: 3000 # port inside the Pod/container |
| Service config example (Ingress-nginx) | apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx # create nginx controller based on nginx project
nginx.ingress.kubernetes.io/rewrite-target: /$1 # how nginx behaves, rewrite rule
# UPDATE THIS LINE ABOVE
spec:
rules:
- http: # use HTTP protocol
paths:
- path: /?(.*) # route one - based on the code route
# UPDATE THIS LINE ABOVE
backend:
serviceName: client-cluster-ip-service # route to this service
servicePort: 3000 # route to this port
- path: /api/?(.*) # route two - based on the code route
# UPDATE THIS LINE ABOVE
backend:
serviceName: server-cluster-ip-service # route to this serice
servicePort: 5000 # route to this port
Note: need to run "minikube addons enable ingress" |
| Persistent Volume and Persistent Volume Claims | |
| Persistent volume claim config example | apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: web-pvc
spec:
accessModes:
- ReadWriteOnce # ReadOnlyMany or ReadWriteMany
resources:
requests:
storage: 2Gi # 2GB in size |
| Introduction | some code |
Below are some of the commonly used kubernetes commands
| Object Types | pods # runs one or more closely related containers (used in dev only) services # sets up networking in a kubernetes cluster deployment # maintains a set of indentical pods, ensuring that they have the correct config and that the right number exists (use in production) |
| Basic commands | minikube start # start a kubernetes cluster minikube stop # stop the kubernetes cluster minikube status # get the status of the cluster minikube delete # delete the kubernetes cluster minikube ip # get IP of kubernetes node, DON'T USE LOCALHOST minkube dashboard # a GUI dash board of your kubernetes cluster kubectl version kubectl cluster-info kubectl get [nodes|pods|services|deployments] # get information from kubernetes cluster kubectl get <object type> [-o wide] # get additional columns of information (IP, node information, etc) kubectl describe <object type> <object name> # get detailed information on a object type (i.e list containers in a pod) kubectl apply -f <config file> # create a pod, service, etc kubectl delete -f <config file> # delete (remove) a pod, services, etc kubectl logs <pod name> # look at pod container logs kubectl exec -it <pod name> sh # connect to a container inside a pod |
| Secrets | kubectl create secret [generic|docker-registery|tls] <secret_name> --from-literal key=value kubectl create secret generic mysecrets --from-literal APP_PASSWORD=1234asdf # example of above command kubectl get [secrets] # get information from kubernetes cluster |
| Images | kubectl set image <object type>/<object name> <container name>=<new image> # update a image in pod kubectl set image deployments/client-deployment client=datadiskpfv/multi-client:v5 # example of above command |
| Persistent Volumes commands | kubectl get [storageclass|pv|pvc] # get information about persistent volumes |
| Connect Docker client to Kubernets | eval $(minikube docker-env) # point docker cli to kubernetes docker server (linux)
minikube docker-env # point docker cli to kubernetes docker server, follow instructions (window)
# c:\docker> @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i |