참고사이트 :
https://www.howtoforge.com/tutorial/centos-kubernetes-docker-cluster/
설치버전 :
CentOS버전 :
$ cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
리눅스 커널버전 :
$ uname -sr
Linux 3.10.0-693.el7.x86_64
호스트 :
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.2 master
192.168.0.3 slave01
모든 노드에서 진행해준다.
Disable selinux
$ setenforce 0
$ sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
Enable br_netfilter Kernel Module
$ modprobe br_netfilter
$ echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
$ modprobe br_netfilter
위에 modprobe을 하면 bridge관련 모듈이 추가된다.
$ ll /proc/sys/net/
total 0
dr-xr-xr-x. 1 root root 0 May 24 22:23 core
dr-xr-xr-x. 1 root root 0 May 24 02:30 ipv4
dr-xr-xr-x. 1 root root 0 May 24 02:30 ipv6
dr-xr-xr-x. 1 root root 0 May 24 02:30 netfilter
-rw-r--r--. 1 root root 0 May 24 22:23 nf_conntrack_max
dr-xr-xr-x. 1 root root 0 May 24 02:30 unix
$ ll /proc/sys/net/
total 0
dr-xr-xr-x. 1 root root 0 May 24 01:19 bridge
dr-xr-xr-x. 1 root root 0 May 24 01:06 core
dr-xr-xr-x. 1 root root 0 May 23 21:09 ipv4
dr-xr-xr-x. 1 root root 0 May 23 21:09 ipv6
dr-xr-xr-x. 1 root root 0 May 23 21:09 netfilter
-rw-r--r--. 1 root root 0 May 24 01:06 nf_conntrack_max
dr-xr-xr-x. 1 root root 0 May 23 21:09 unix
Disable SWAP
$ swapoff -a
# swap 파티션이나 swap 파일의 구동을 중단시키는 명령어
swap이란 하드디스크를 메모리처럼 사용하는 기법
-> 물리적인 메모리가 모자라면 하드디스크를 메모리처럼 데이터를 기록하여 메모리를 확보
프로그램들을 많이 실행해서 메모리가 부족해지면, 메모리 상에 적재된 프로그램 중 당장 필요하지 않은 프로그램 데이터를 하드디스크에 옮겨서 메모리 공간을 확보
출처: http://nextcube.tistory.com/137 [중성자 별의 충돌 에너지]
$ vi /etc/fstab
/dev/mapper/centos-root / xfs defaults 0 0
UUID=e68efdc4-b1c1-4f94-ab67-72d611499e13 /boot xfs defaults 0 0
UUID=E849-B774 /boot/efi vfat umask=0077,shortname=winnt 0 0
#/dev/mapper/centos-swap swap swap defaults 0 0
Install Docker
$ yum install -y yum-utils device-mapper-persistent-data lvm2
$ yum install -y docker-ce
Install Kubernetes
$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
# /etc/yum.repos.d/kubernetes.repo
$ systemctl start docker && systemctl enable docker
$ systemctl start kubelet && systemctl enable kubelet
Change the cgroup-driver
$ docker info | grep -i cgroup
Cgroup Driver: cgroupfs
$ sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
$ systemctl daemon-reload
$ systemctl restart kubelet
k8S Cluster Initialization
Master서버에서 ( apiserver-advertise-address 에 master서버 아이피 입력 )
$ kubeadm init --apiserver-advertise-address=192.168.0.2 --pod-network-cidr=10.244.0.0/16
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join 192.168.0.2:6443 --token bi09ej.5z5q8osipp4r9w5u --discovery-token-ca-cert-hash sha256:0aa7b7489d097ae88ad17c7dad7a591d2da711fca6ed533b4063979c917747a5
# 따로 메모 해둔다. 슬레이브서버에서 조인할때 필요
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-jframe-master 1/1 Running 0 22s
kube-system kube-apiserver-jframe-master 1/1 Running 0 28s
kube-system kube-controller-manager-jframe-master 1/1 Running 0 42s
kube-system kube-dns-86f4d74b45-nxqfr 3/3 Running 0 1m
kube-system kube-flannel-ds-7dmtt 1/1 Running 0 1m
kube-system kube-proxy-w6hm8 1/1 Running 0 1m
kube-system kube-scheduler-jframe-master 1/1 Running 0 19s
슬레이브노드에서 진행 :
$ kubeadm join 192.168.0.2:6443 --token bi09ej.5z5q8osipp4r9w5u --discovery-token-ca-cert-hash sha256:0aa7b7489d097ae88ad17c7dad7a591d2da711fca6ed533b4063979c917747a5
This node has joined the cluster:
* Certificate signing request was sent to master and a response
was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.
마스터노드에서 확인 : ( slave01이 제대로 안 올라오면 위에 과정을 제대로 입력했는지 확인해본다 )
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 21m v1.10.3
slave01 Ready <none> 57s v1.10.3
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-jframe-master 1/1 Running 0 46m
kube-system kube-apiserver-jframe-master 1/1 Running 0 46m
kube-system kube-controller-manager-jframe-master 1/1 Running 0 46m
kube-system kube-dns-86f4d74b45-nxqfr 3/3 Running 0 47m
kube-system kube-flannel-ds-7dmtt 1/1 Running 0 47m
kube-system kube-flannel-ds-f9vbs 1/1 Running 0 56s
kube-system kube-proxy-9xgll 1/1 Running 0 56s
kube-system kube-proxy-w6hm8 1/1 Running 0 47m
kube-system kube-scheduler-jframe-master 1/1 Running 0 46m
마지막으로 팟을 만들어서 확인해본다.
Testing Create First Pod
$ kubectl create deployment nginx --image=nginx
$ kubectl describe deployment nginx
$ kubectl create service nodeport nginx --tcp=80:80
$ kubectl get svc
( 이 명령어를 통해 PORT에 포워딩한 포트를 기억한다 )
$ curl master:31280
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
$ curl slave01:31280
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
