80:32643/TCP 17m
```
Now the state is pending :)
- Run `netstat -nltp`, and notice the `kube-proxy`
```diff
++ tcp 0 0 0.0.0.0:32643 0.0.0.0:* LISTEN 13095/kube-proxy
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 7024/kubelet
++ tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 13095/kube-proxy
```
See the magic.
```bash
root@vagrant:/home/vagrant/kubedata# curl 0.0.0.0:32643
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
```
- The `LoadBalancer` exposed the service endpoints out of Kubernetes cluster IP interface and in our vagrant host we can access it now directly :)
- The next challenge to to expose this `kube-proxy` interface to host machine. And hack is done, then we can access the service running in Pod(replica set deployment) from our host interface directly.
- This is how the network now looks like. The port `32643` is not exposed through kube-proxy over host/control-plane node.
```bash
Kubernetes Cluster
+---------------------------------------------+
| POD |
| +---------+ |
| +------> NGINX | |
| | +---------+ |
| LB | |
+--------------+ | +---------------+ POD |
0.0.0.0:32643| Kube Proxy |80 | | | +---------+ |
<------------------>----------->+ SERVICE +------> NGINX | |
| | | 80| | +---------+ |
+--------------+ | +---------------+ |
HOST | | POD |
| | +---------+ |
| +------> NGINX | |
| +---------+ |
+---------------------------------------------+
```
## Stateless workloads
- Deployments and Replicasets that we had deployed so far are stateless workloads.
- There is no state related information stored at Pods/Service, so request from kube-proxy via serivce resource can be routed to any of the Pod in the cluster.
- This constitutes stateless workload.
- Next section is to create a Stateful workload.
## Stateful workloads
- Preserve the state of data present on Pods.
- Two situations can be possible:
- Multi pod stateful workload
- If multiple pods are connecting to stateful workload, there should be worker based synchronization
- Else, stateful data may go out of sync.
- Single pod stateful workload
- Create persitant volumes
- Create persitant volume claims to access persitant volumes in a synchronized way, just to prevent ensure data atomicity.
### Persistent Volumes
- PV are like volumes in Docker, just that their lifecycle is independent of Pods.
- This is an API object. Captures details about storage implementation.
- Provised by Kubernetes administrator.
- Way to abstract storage resource.
- Create a persistent volume for MySQL server. [File](files/pv.yml)
```yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/data"
```
This spec specifies the volume is at `/data` on cluster's node.
Apply it : `kubectl apply -f pv.yml`
```bash
root@vagrant:/home/vagrant/kubedata# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv 5Gi RWO Retain Available manual 62s
```
### Persistent Volume Claims
- Storage requeest by a user.
- PVCs consume PV resources.
- Way to access abstract storage.
- PVC can request specific size and access mode: `ReadWriteOnce`, `ReadOnlyMany`, `ReadWriteMany`
|Access Mode| Meaning|
|---|---|
|ReadWriteOnce | volume can be mounted as read-write by a single node|
|ReadOnlyMany | volume can be mounted read-only by many nodes|
|ReadWriteMany | volume can be mounted as read-write by many nodes|
- Create a PVC spec. [File](files/pvc.yml)
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
```
Apply it.
```bash
root@vagrant:/home/vagrant/kubedata# kubectl apply -f pv-claim.yml
persistentvolumeclaim/pv-claim created
root@vagrant:/home/vagrant/kubedata# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pv-claim Bound pv 5Gi RWO manual 8s
root@vagrant:/home/vagrant/kubedata# kubectl describe pvc pv-claim
Name: pv-claim
Namespace: default
StorageClass: manual
Status: Bound
Volume: pv
Labels:
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 5Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By:
Events:
root@vagrant:/home/vagrant/kubedata#
```
- Pods use PersistentVolumeClaims to request physical storage
- After creating the PersistentVolumeClaim, the Kubernetes control plane looks for a PersistentVolume that satisfies the claim's requirements. If the control plane finds a suitable PersistentVolume with the same StorageClass, it binds the claim to the volume.
- Lets create a POD which will use PV as Volume using PVC. [File](files/nginx-pod-with-pv.yml)
```yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod-with-pvc
spec:
volumes:
- name: nginx-pv-storage
persistentVolumeClaim:
claimName: pv-claim
containers:
- name: nginx-with-pv
image: nginx
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: nginx-pv-storage
```
```bash
root@vagrant:/home/vagrant/kubedata# kubectl get pods nginx-pod-with-pvc
NAME READY STATUS RESTARTS AGE
nginx-pod-with-pvc 1/1 Running 0 16s
root@vagrant:/home/vagrant/kubedata# kubectl exec -it nginx-pod-with-pvc -c nginx-with-pv -- /bin/bash
root@nginx-pod-with-pvc:/# curl localhost
Hi PV
```
- The file we just created in storage is made accessible to Nginx POD.
### Summary
```
+--------------------------------------+
| +------------+ |
| | POD | +--------------->
| +-----+------+ | | |
| | | | |
| | +-----+------+ | v
| | | PV | | /data
| | +------+-----+ |
| +-----v------+ ^ |
| | PVC +---------+ |
| +------------+ |
| |
+--------------------------------------+
```
- PV to PVC bind is automatic, based on storage class.
- Pod/Deployment/K8s-Resource link to PVC has to has to be done manually in spec file.
## Sample Application Example
1. This End to End setup will include:
1. MySQL setup through PV and PVC.
2. Building Custom Dockerfile for sprinboot application.
3. Creating Deployment for SpringBoot application.
1. Setup the environment for application to connect to DB.
2. Setting up PVC setup in deployment.
3. Creating Serivce for springboot application access outside pod.
1. Service setup through LB
Once we create spec.yml in bits, we will create a big spec to show our Infrastructure as a Code and deploy that :smile:.
### MySQL Resource
#### Step 1: Create PV for MYSQL DB
```yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: mysql-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/data/mysql"
```
#### Step 2: Create PVC for PV
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mysql-pvc
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
```
#### Step 3: Create MySQL deployment Spec
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dbserver
labels:
app: dbserver
spec:
selector:
matchLabels:
app: dbserver
template:
metadata:
labels:
app: dbserver
spec:
containers:
- image: mysql
name: mysql
imagePullPolicy: Never
env:
- name: MYSQL_ROOT_PASSWORD
value: mysecretpassword
ports:
- containerPort: 3306
name: dbserver
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pvc
```
- Once the DB server is up, please go adhead and login to MySQL and create `peopledb` for sprinboot application to access.
- `mysql -- mysql -u root -pmysecretpassword` & `create database peopledb`
#### Step 4: Expose MySQL server via Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: dbservice
spec:
selector:
app: dbserver
ports:
- protocol: TCP
port: 3306
targetPort: 3306
```
- This will expose this service over/inside cluster for other services to access.
### Springboot Application
#### Step 1: Build and Deploy AppServer
- Build the Docker image with name `appserver` from this [File](files/Dockerfile).
```bash
docker build -t appserver .
```
- Create Deployment spec for appserver.
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: appserver
spec:
replicas: 1
selector:
matchLabels:
app: appserver
template:
metadata:
labels:
app: appserver
spec:
containers:
- name: appserver
image: appserver
imagePullPolicy: Never
env:
- name: DB_HOST
value: dbservice
```
#### Step 2: Expose AppServer service via Service type LB to host.
```yaml
apiVersion: v1
kind: Service
metadata:
name: contacts
spec:
type: LoadBalancer
selector:
app: appserver
ports:
- protocol: TCP
port: 80
targetPort: 8080
```
### Infrastructure as a Code
#### MySQL Full Spec
- You can find the full spec file here : [File](files/mysql-spec.yml)
```yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: mysql-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/data/mysql"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mysql-pvc
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dbserver
labels:
app: dbserver
spec:
selector:
matchLabels:
app: dbserver
template:
metadata:
labels:
app: dbserver
spec:
containers:
- image: mysql
name: mysql
imagePullPolicy: Never
env:
- name: MYSQL_ROOT_PASSWORD
value: mysecretpassword
ports:
- containerPort: 3306
name: dbserver
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pvc
---
apiVersion: v1
kind: Service
metadata:
name: dbservice
spec:
selector:
app: dbserver
ports:
- protocol: TCP
port: 3306
targetPort: 3306
```
- `kuebctl apply -f mysql-spec.yml` :smile:
#### AppServer Full Spec
- You can find the full spec file here: [File](files/appserver-spec.yml)
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: appserver
spec:
replicas: 1
selector:
matchLabels:
app: appserver
template:
metadata:
labels:
app: appserver
spec:
containers:
- name: appserver
image: appserver
imagePullPolicy: Never
env:
- name: DB_HOST
value: dbservice
---
apiVersion: v1
kind: Service
metadata:
name: contacts
spec:
type: LoadBalancer
selector:
app: appserver
ports:
- protocol: TCP
port: 80
targetPort: 8080
```
Quickly apply it with `kubectl apply -f appserver-spec.yml`
## Understanding Advance Kubernetes Resources
### Namespace
Namespace are software level cluster virtualization over same physical k8s cluster.
```bash
root@vagrant:/home/vagrant# kubectl get ns
NAME STATUS AGE
default Active 19d
kube-node-lease Active 19d
kube-public Active 19d
kube-system Active 19d
```
Kubernetes starts with 4 namespaces:
1. **default**: The default namespace for objects with no other namespace.
2. **kube-system**: The namespace for objects created by the Kubernetes system.
3. **kube-public**: This namespace is created automatically and is readable by all users (including those not **authenticated**). This namespace is mostly reserved for cluster usage, in case that some resources should be visible and readable publicly throughout the whole cluster. The public aspect of this namespace is only a convention, not a requirement.
4. **kube-node-lease**: This namespace for the lease objects associated with each node which improves the performance of the node heartbeats as the cluster scales.
Get Pods from specific namespace
``kubectl get pods --namespace=default`` OR `kubectl get pods -n default`
```bash
root@vagrant:/home/vagrant# kubectl get pods --namespace=kube-system
NAME READY STATUS RESTARTS AGE
coredns-f9fd979d6-g9wxg 1/1 Running 5 19d
coredns-f9fd979d6-zrdvs 1/1 Running 5 19d
etcd-vagrant 1/1 Running 5 19d
kube-apiserver-vagrant 1/1 Running 5 19d
kube-controller-manager-vagrant 1/1 Running 7 19d
kube-flannel-ds-64l2p 1/1 Running 6 19d
kube-proxy-4j4kw 1/1 Running 5 19d
kube-scheduler-vagrant 1/1 Running 7 19d
```
#### Creating Namespace & Adding resource
- Create namespace : `kubectl create namespace qa`
- Once the namespace is created, just add the metadata field : `namespace: qa`, [File](files/pod-qa.yml)
```diff
apiVersion: v1
kind: Pod
metadata:
name: nginx
++ namespace: qa
spec:
containers:
- name: nginx
image: nginx
```
- Most Kubernetes resources (e.g. pods, services, replication controllers, and others) are in some namespaces. However namespace resources are not themselves in a namespace. And low-level resources, such as nodes and persistentVolumes, are not in any namespace.
- To see the list of resource not in namespace : `kubectl api-resources --namespaced=false`
### Context
- Is a tuple of **cluster**, **user**, **namespace**. This is useful when you connect to multiple clusters from one control plane.
- Get the current context: `kubectl config get-contexts`
```bash
root@vagrant:/home/vagrant/kubedata# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
```
- You can create kubernetes context using config file or using commands.
- Create a qa-config: `kubectl config set-context dev-env --cluster=kubernetes --user=new-admin --namespace=dev-env`
```bash
root@vagrant:/home/vagrant/kubedata# kubectl config set-context dev-env --cluster=kubernetes --user=new-admin --namespace=dev-env
Context "dev-env" created.
```
```bash
root@vagrant:/home/vagrant/kubedata# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
dev-env kubernetes new-admin dev-env
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
```
- Now use the created context using : `kubectl config use-context dev-env`
- All your k8s resource will now be in DEV name space under kubernetes cluster :smile:
- But to create resource you will need user `new-admin` authentication. This is the user created during context creation.
- Create username & password for user `new-admin` to use the resource in context and create a role binding: **Run this before switching context**
`kubectl config set-credentials new-admin --username=adm --password=changeme`
```bash
cat << EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: new-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: marry@example.com
EOF
```
## CheatSheet
- I plan to write a simple cheat sheet covering the commands in this repo. But for now Try : [k8s-official-cheat-sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/)
## Next Steps
- [In detail K8s Reference](https://kubernetes.io/docs/reference/)
- [API Guide](https://kubernetes.io/docs/reference/)
- [CLI Guide](https://kubernetes.io/docs/reference/)
- [K8s Design Docs](https://kubernetes.io/docs/reference/)
- Raising a PR makes me happy, take that as a next step.
- Issues are more than welcome.
- If you like it, share it.
