Kubernetes Deployment
Contents
Kubernetes Deployment#
Install Kubernetes to Alienware.
https://www.padok.fr/en/blog/minikube-kubeadm-kind-k3s
Using kind.
Install with choco from elevated PowerShell
choco install kind
Create a cluster
kind create cluster
Default kind settings
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXhOekE1TWpFMU1sb1hEVE14TVRFeE5UQTVNakUxTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTnQ5CjFBUjZsRlp6cE9qNHJMTnp6MUoxL3l2M0NGU0RDWEhCR2V6Szg1YnVqTEgzTEQ5UEVaeVhBRzFVWkRZZEIybDUKZkJXUUF6cDZkdUFIbm5kYUFoUlBvSkdYdjB4eTFGTisyTWlIeW5kWkIwQVk1OUh4c0dMQkZVR3BoWi9JMVowZApKUkt3ZGYyMjMyUktYSFN0S3NHeWttbGtGVHlBRDNpaXFONnpJLzJaTDRBcG9McnFpVTJXTUxGRmVsWE55RDR2CjRZMi9LSmlCZXAzRVpKUDhEeEF4NVZrWHErbGl0em1FODBIcVhDOTd0a2JnS29MdXZZTmhtdnpvV2ZWTnRqZFkKc09BVzlybmR0cGdHWE1mUjU5RDV1WllEbENwNHFmeW84N2x5RlIyanJXYXcwaDduanYxaGluZklxN1U0NWJ6aQplWVVjN1dHTjJzcExSRGI3S1AwQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZKWGxTR3VYNEhxb2NtL3ppSGNnSUw3NmQ5QXVNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBOUdDdVl6ZzVXTUQ3bTdtNncxYUNYanFpcGkyZTBrKytHU3JDNFFubDBYdVl2UXg0eQpmUE9ieFhhZ3hxcnZCQi90b25uOEt4SlR6SUNKNE9BNGxLQ0UwcGV5U3ljbnpDRUZSVVFrSFl4VFhjN1JIVURDCnFwZ0x0aEo3RVJnZWVDMk51WUpMSlJEZ3c2czVPNjZ4VUFORUU5TW5UWTljMGtuSkRaWnUxMDhYR1U5N3M3dFgKeWx4T1ZqdzZITUhYMy91cEw1cGRKWFNDa3ZHQmRUT2xUaldZcHV5QVVtYUIwRFJQMG81N2dHQzZFME1GMyszMApSL1pQbzVQQVpwTDFiMlp0SFFpR3JiNkhaRjE4UXNEeG8vTjFoQTk1eSt0dUdzYTk0NlJHSEs1TUtpTUpNSXVsCkV5dW9pdFJraGVhQkRkOUlUK3JsaXB3dDkzeWlFdHQ4ayszegotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://127.0.0.1:60884
name: kind-kind
contexts:
- context:
cluster: kind-kind
user: kind-kind
name: kind-kind
current-context: kind-kind
kind: Config
preferences: {}
users:
- name: kind-kind
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJVXBPbkVsOHlwZ2d3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE1UY3dPVEl4TlRKYUZ3MHlNakV4TVRjd09USXhOVE5hTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXBUS0FqUy84alNrT2lBZ2IKeTNMWm56TkpEQVE2K1VUZTJ0Uk5WaHl0ZFJwM0I1ajVxOGg2OWtpVjdPQ1Q2Tjk2N1dkeERqZy9nVUY3OTZmYwpLUG1JUFhwOGxUUllwYkFzRG1IVndtWTRQbkloTzhZckRweXJjY2NuOGI2TFBzSndQUmFVTk42WHJWU3NtR3BaCkFBa1RaREpNS0ZUdVVjWnc5bytUcW5vdEZueWxUaGorUjZOM25xSGNnWlFBN3dMK2JEUFFTNGF2WUViK3QrZy8KbFRwT1hIMDlxTlZRWlpONlQ3eS83Nlc0WnBSaWlha28zZ1BXMG9HUWFDTkY5bkw5TkdHbVNBTUxPTzIybHdiQwphdWlEbnBGcUV1clZVUEpnWlViVTVjT21DZUZUTVowVTBubUF2NzRUZ2J1ODlCc0ZNMjhBNEF0bDM1dUxnUTVLCjMvbW5XUUlEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTVjVVaHJsK0I2cUhKdjg0aDNJQ0MrK25mUQpMakFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBWjdpbzlHcExGTE1LVEFwR3VXNmc1RDJ4L0dtYTMxeW1MWllCClVGeUtwNFFobXRZcldMemVnd3NDUDE4OGJqMWhDa0h3WDNIYUhTR3VySGtiazRrb0xxbG9YOGlRYnM2QlQ3dHoKWkxZTVB5dWFXdUhJTWJ4eE43dVlJNE43dkxISTJFNDFWRGo5a0krWGl4dzZYNUNobU50K2xQNUxkZWNLcDdCegpsb3ZNVGJzVURUQzFJQWZNUmRqNEZockQvSk9qU2N3KzVoMWJwRzE2b1FaTGZrVXhDaUZLSmsycEJqcGxyQU0wCnN0dDRJTzNzUE5MRUREVlpWZWZCTGRJVXZlVitkT2xuM2VoSmFhVkJoSzhtOG54bXg3d3gzSDJJYUFNRml3bHUKQnhaaXB2SlB5Y042dVI1R1dVL21zUWQxSUVMdGY2c2tRU25NU1l3dHZaUDdWWDliSGc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBcFRLQWpTLzhqU2tPaUFnYnkzTFpuek5KREFRNitVVGUydFJOVmh5dGRScDNCNWo1CnE4aDY5a2lWN09DVDZOOTY3V2R4RGpnL2dVRjc5NmZjS1BtSVBYcDhsVFJZcGJBc0RtSFZ3bVk0UG5JaE84WXIKRHB5cmNjY244YjZMUHNKd1BSYVVOTjZYclZTc21HcFpBQWtUWkRKTUtGVHVVY1p3OW8rVHFub3RGbnlsVGhqKwpSNk4zbnFIY2daUUE3d0wrYkRQUVM0YXZZRWIrdCtnL2xUcE9YSDA5cU5WUVpaTjZUN3kvNzZXNFpwUmlpYWtvCjNnUFcwb0dRYUNORjluTDlOR0dtU0FNTE9PMjJsd2JDYXVpRG5wRnFFdXJWVVBKZ1pVYlU1Y09tQ2VGVE1aMFUKMG5tQXY3NFRnYnU4OUJzRk0yOEE0QXRsMzV1TGdRNUszL21uV1FJREFRQUJBb0lCQUFzNnhxQlNYY0dldzkwZApaLzVKZlYra0ZkZ2FSWVFDV1gvWXBMVXcxOTI1QWRSallyMTVzRm9qandWK01xL0cyVTFnRmNCWXRJQnQyaVo0Cm82K3BYc2lIRmdieWV3QXdYSjVScWowL3c1MERGeDFBR0ZjeThUaVh5REJYdXBGMm1EaWUyMDVJdjFqS0E3Mk0KcFNZcWQ4WDg0VTV3ajQ3QmhaWk5Ka1dWYnBFenBlYjJKcTBoM0gyd3dRMEtFUUVURVdnUDZ6U2h0SjJIS1NoZQoxRVFFdmJMRnVJQlJ2OEF5bGtTOHg0TkJGbllFTkNsU2RmcXNZNkV5QzNsNTM1TTdkSVJEKzRuUFVWVlo1TklOCkVHQXY4bEhDc1lpSzVpQ2JkRE5QMDY3NllmUmxBNENRY0tiVVRxc1dJejIwajI1ZnRZS0RwakJKeGNyRDBjZG0KMWQ0T1JMRUNnWUVBM0NYRjFjcUhBMFNpa0kyUFJ0QW83STU0SEVhZXlKN1lmWGdHajdtNVZxdi9WdHg5VUVwWgovd1BJdDJONUtHelozcjdBZTcxY3FiUVBtYXhqTTM5dzR6N1NQQ2FuT29qWWpyY0lLRTMweStVZWIrSTdJcVFvCm5NdDdmK3ZTTDZFUW1YSE5UK3g4Y0NxWWJxN3RnZDVEV08rb3hiem42bDU4Nk1oRS9MbzhMTzBDZ1lFQXdCbkkKazdNSjI2T213cUNUSG9CT1NuWFNDb2NRMjlMNGpPSlU3QmFFdUF3OHFpd0JrajA1T1pqazUvYThBZm9tYTh6VApyMDJ1NnBtYnM0ZFMxTDQwS3FQQ3pGanZKTGZ4TEhtb25LODhVb3hJTlNKYVlqSnFOcGxHaUxSb2xwb1FPb3lkCmF5NnVwK3M5RWZTcHU4Nnh6NlpCOTVCU1Q0T09NdFBmczFqUlFwMENnWUI5YmhnV0ExcUJoaTJKWnQ1blhscXkKelVtcDhxS3RjN090YVNGejZrR3JIRHhUWlRVUTlndm92R016K2IwaVIwS3JJc3ZRdE1lQytWQ2FJaG96VVQwawpmTHFVQXdIczIrTm5ET25jN08yNDdrWTFRUlNvWEJlM25rZ3pUazFuVGNHRGE4K3E1ZmJRZ2JSdis4TGU1NkEvCm4renRIUUNXREZmbTJaUlUybGRHMFFLQmdFdm9wRnJBNzg5TDdyU1AwVExIVHFpYndDRUsyRDJWTTBRd2w1QW4KSWRDOGZTaGRkVTE5UmVTZHB5NkdVQ3FUMTZUWGJucWlQdEZUVzdSb2IvSERLMGdDS0JLcHZlNExoWkNtamx0RAozNWxhOTVtUGRwbkJCRG1JL2RjOU91cnYvQ08ydHVuSXE1bU5HamZETDJNWVBTa3BzSDJNR1FwMG1MYzdBL1dSCnpqYjlBb0dCQU5qQmxlM05BeS9OK1dRcXBOUzZxNHlGTlYzYXh5OXc5QTdQdDdVcDF4NUpKRmZrbHV3U3loUzYKV3A5N0FwMEFZeVVsYzBxb0xsMlFIYTFCOWhYS0RvTzh6cXJQUGE5RDUyUE9VVHZFRFljSXVIYlF1a1Fpb0dtOQpBVUZLdmxRQ281QmxscGpsR2kydGVVclNXbWNtYktDd0twbU45emxzNXpzajNlWXRtamZVCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
Change server to 0.0.0.0:60884
Configure advanced settings
# cluster-config.yml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30000
hostPort: 30000
protocol: TCP
Configure according to docs https://kind.sigs.k8s.io/docs/user/using-wsl2/
kind create cluster --config=cluster-config.yml
kubectl create deployment nginx --image=nginx --port=80
kubectl create service nodeport nginx --tcp=80:80 --node-port=30000
curl localhost:30000
Repeat this step - reached docker limit? https://about.gitlab.com/blog/2020/11/18/docker-hub-rate-limit-monitoring/
100 per 6 hours, wait for a bit then do the code again
https://kind.sigs.k8s.io/docs/user/private-registries/#add-credentials-to-the-nodes
Need to add the credentials to the nodes.
- role: control-plane
extraMounts:
- containerPath: /var/lib/kubelet/config.json
hostPath: /home/uet200/kubernetes/config.json
Check with the code below to make sure it works.
kubectl describe pod
Install kubectl on the Alienware
Run the code.
Run the API
kubectl proxy --address='0.0.0.0' --accept-hosts='.*'
Open port 8001
Run this code instead
https://kubernetes.io/blog/2020/05/21/wsl-docker-kubernetes-on-the-windows-desktop/
Get the image manually to let docker cache
Might need to restart it to make it work.
Do port forward to access 8001 on alienware
Delete just in case
https://stackoverflow.com/questions/47128586/how-to-delete-all-resources-from-kubernetes-one-time
netsh interface portproxy add v4tov4 listenaddress=10.208.2.135 listenport=8080 connectaddress=127.0.0.1 connectport=38911
#Token
eyJhbGciOiJSUzI1NiIsImtpZCI6Ik5SSnZSblVRal84YXdOdklJSGFiZUJ3OUJHTFFtTnM3MG9oaEVvc1ByNDQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLW05Z2toIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxNjEwZjhkZi04NDgxLTRhZTItYmE1Zi02NDdhZWZjZjk0MjgiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.fTwHJC-snKoeUHyL0ouUseAQQTuZ_n1WFrHyADPXYWzPTDE_BImSn-pv1PCy1R3sBaP7lzs_hXxciQebM3G6uc3_5a-apQ0_7_r-pskm1lFeFoyOkqwPeljkhZNsRZrN7p-AFosw4TsSs4SH9S3dvggkhs41sgth5A8_UwxKTWKHp3duwhW7McwqhVG4nx-ERzb6CFcUY1pbj8YoIbJtyTI_vkT_SaKk87Ni_uMHlkUPZxCJYD66t8BER0YpD8tUZimFywUd-vniR5emM45HD4A170oQ476LkL7lHxAs7WcBeKoEx5wZqRLbp1irOtfMlSuEshEK4VEMRnl7-CWmkgkub
Install kubectl in MacBook
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl"
Validate
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl.sha256"
echo "$(<kubectl.sha256) kubectl" | shasum -a 256 --check
Make it executable and move to path
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
sudo chown root: /usr/local/bin/kubectl
Run it
kubectl version --client
Build the image in Macbook.
tutor images build openedx \
--build-arg EDX_PLATFORM_REPOSITORY="https://[email protected]/uetuluk/openedx-tutor.git" \
--build-arg EDX_PLATFORM_VERSION=master \
--docker-arg="--no-cache"
Quick start tutor.
tutor local quickstart
Build the dev image.
tutor images build openedx-dev
Start image - left here
tutor dev start
Connect to kind from Macbook#
Create the cluster with the alienware ip
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- containerPath: /var/lib/kubelet/config.json
hostPath: /home/uet200/kubernetes/config.json
extraPortMappings:
- containerPort: 30000
hostPort: 30000
protocol: TCP
- role: worker
extraMounts:
- containerPath: /var/lib/kubelet/config.json
hostPath: /home/uet200/kubernetes/config.json
- role: worker
extraMounts:
- containerPath: /var/lib/kubelet/config.json
hostPath: /home/uet200/kubernetes/config.json
Get the creds
docker --config . login
Need to enable API/dashboard access to external users
Trying minikube instead#
Install according to official guides.
https://zepworks.com/posts/access-minikube-remotely-kvm/
Set up cluster
minikube start --memory 4096 --cpus 8 --apiserver-name="k8.alienware.ritsdev.top" --ports="9000:8443"
Forward port - compare the connect address and connect port to the config port
Enable dashboard
minikube dashboard --url
Transfer the files
mkdir ~/.minikube_remote
cd ~/.minikube_remote
scp pc1:.minikube/ca.crt .
scp pc1:.minikube/profiles/minikube/client.crt .
scp pc1:.minikube/profiles/minikube/client.key .
Go to MacBook#
MacBook config
apiVersion: v1
clusters:
- cluster:
certificate-authority: C:\Users\uet200\.minikube\ca.crt
extensions:
- extension:
last-update: Fri, 19 Nov 2021 09:58:44 CST
provider: minikube.sigs.k8s.io
version: v1.24.0
name: cluster_info
server: https://k8.alienware.ritsdev.top:8443
name: minikube
contexts:
- context:
cluster: minikube
extensions:
- extension:
last-update: Fri, 19 Nov 2021 09:58:44 CST
provider: minikube.sigs.k8s.io
version: v1.24.0
name: context_info
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: C:\Users\uet200\.minikube\profiles\minikube\client.crt
client-key: C:\Users\uet200\.minikube\profiles\minikube\client.key
Use the config file inside Macbook
kubectl config --kubeconfig=./config view
Create an alias to make it easier
https://wpbeaches.com/make-an-alias-in-bash-or-zsh-shell-in-macos-with-terminal/
alias kubectl='kubectl --kubeconfig=/Users/uetuluk/Documents/tutor-dev/kubernetes/config'
Error with certificates → wrong certificate → use command shift dot to enable hidden files
Fix taint
kubectl taint nodes minikube node-role.kubernetes.io/master-
kubectl taint nodes minikube-m02 node-role.kubernetes.io/master-
Find the error
kubectl describe node
Eror
container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Solution is to first delete everything.
Do multi nodes later.
Check this command
minikube start --memory 4096 --cpus 8 --apiserver-name="k8.alienware.ritsdev.top" --ports="8443"
There should be a better way of doing it.
Try the other options presented in:
https://minikube.sigs.k8s.io/docs/commands/start/
Change the exposed port
minikube start --memory 4096 --cpus 8 --apiserver-port=9000 --apiserver-name="k8.alienware.ritsdev.top" --ports="9000:8443"
Find a way to expose the port, but keep it stable.
Check commands
minikube start --apiserver-port=9000 --apiserver-name="k8.alienware.ritsdev.top" --ports="9000:8443"
The flags correspond to kubernetes config
apiserver-port → nodeport
apiserver-name → apiservernames
apiserver-ip → apiserverips
—ports → docker expose
Check commands
This combination worked!#
minikube start --apiserver-port=8443 --apiserver-name="k8.alienware.ritsdev.top" --ports="8443:8443"
Networking
netsh interface portproxy add v4tov4 listenaddress=10.208.2.135 listenport=8443 connectaddress=127.0.0.1 connectport=8443
Change with the IP of the exposed port for 8443
netsh interface portproxy add v4tov4 listenaddress=10.208.2.135 listenport=56902 connectaddress=127.0.0.1 connectport=56902
The IP itself is not needed.
Check what causes it to expose the ip
→ make it expose the port to 0.0.0.0 instead
Trying to use WSL instead
Add the driver flag to point to docker
minikube start --apiserver-port=8443 --apiserver-name="k8.alienware.ritsdev.top" --ports="8443:8443" --driver=docker
Networking
netsh interface portproxy add v4tov4 listenaddress=10.208.2.135 listenport=8443 connectaddress=127.0.0.1 connectport=8443
Debug with the flag -v=4