admin 发表于 2018-9-20 11:08:15

k8s学习二:k8s编译安装集群搭建——单master多node简易部署

服务器环境

centos7.5
mac装的pd虚拟机
作用        IP        部署服务        配置
master        10.211.55.10        etcd、kube-apiserver、kube-controller-manager、kube-scheduler        2C、2G
node1        10.211.55.11        docker 、kubelet、kube-proxy        2C、2G
node2        10.211.55.12        docker 、kubelet、kube-proxy        2C、2G
- 计划采用二进制包进行部署:

所需二进制包下载地址:
1.https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
2.https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
3.https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
注意所有服务器都需要关闭防火墙
Master部署

二进制安装基本都是以下几个步骤:
1、复制对应的二进制文件到/usr/bin目录下
2、创建systemd service启动服务文件
3、创建service中对应的配置参数文件
4、将该应用加入到开机自启
5、启动服务并查看服务状态
etcd部署

下载二进制安装包并安装:
wget https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
cd etcd-v3.2.22-linux-amd64/
cp etcd /usr/bin/
cp etcdctl /usr/bin/
mkdir /var/lib/etcd
mkdir /etc/etcd

编辑systemd管理文件
vim /usr/lib/systemd/system/etcd.service


Description=Etcd Server
After=network.target


Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd


WantedBy=multi-user.target


启动服务,并设置开机启动
systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

查看服务状态的三种命令
systemctl status etcd.service

curl -L http://127.0.0.1:2379/version

etcdctl cluster-health

这个安装的还挺顺利,很快就ok了。继续。。。。
kube-apiserver

下载并安装
wget https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
tar -xzvf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp kube-apiserver /usr/bin/

# 一起拷贝吧,后面就直接配置了
cp kube-controller-manager /usr/bin/
cp kube-scheduler /usr/bin/


编辑systemd的启动文件
vim /usr/lib/systemd/system/kube-apiserver.service


Description=Kubernetes API Server
Documentation=https://kubernetes.io/docs/concepts/overview
After=network.target
After=etcd.service


EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536


WantedBy=multi-user.target



配置参数文件
mkdir /etc/kubernetes/
vim /etc/kubernetes/apiserver

KUBE_API_ARGS="--storage-backend=etcd3 \
               --etcd-servers=http://127.0.0.1:2379 \
               --bind-address=0.0.0.0 \
               --secure-port=6443\
               --service-cluster-ip-range=192.168.2.0/16\
               --service-node-port-range=1-65535 \
               --client-ca-file=/etc/kubernetes/ssl/ca.crt \
               --tls-private-key-file=/etc/kubernetes/ssl/server.key\
               --tls-cert-file=/etc/kubernetes/ssl/server.crt\
               --enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,DefaultStorageClass,ResourceQuota \
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"


service-cluster-ip-range是servcies的虚拟IP的IP范围,这里可以自己定义,不能当前的宿主机网段重叠。
bind-addres 指定的apiserver监听地址,对应的监听端口是6443,使用的https的方式。(0.0.0.0 表示绑定所有地址)
client-ca-file 这是认证的相关文件,这预先定义,后面会创建证书文件,并放置到对应的路径。
创建日志目录和证书目录
mkdir -p /etc/kubernetes/ssl
mkdir -p /var/log/kubernete

kube-controller-manager

kube-controller-manager 依赖 kube-apiserver服务
编辑systemd启动文件
vim /usr/lib/systemd/system/kube-controller-manager.service


Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service


EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536


WantedBy=multi-user.target


配置启动参数
vim /etc/kubernetes/controller-manager

KUBE_CONTROLLER_MANAGER_ARGS="--master=https://10.211.55.10:6443   \
               --service-account-private-key-file=/etc/kubernetes/ssl/server.key\
               --root-ca-file=/etc/kubernetes/ssl/ca.crt \
               --kubeconfig=/etc/kubernetes/kubeconfig \
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"


kube-scheduler

kube-scheduler也依赖kubu-apiserver
- 编辑systemd启动文件
vim /usr/lib/systemd/system/kube-scheduler.service


Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service


EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536


WantedBy=multi-user.target
配置参数文件
vim /etc/kubernetes/scheduler

KUBE_SCHEDULER_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig \
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"

创建CA证书

注意生成证书前先同步一下服务器时间:ntpdate s2m.time.edu.cn
创建kube-apiserver的CA证书和私钥文件
cd/etc/kubernetes/ssl/
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=10.211.55.10" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048

创建master_ssl.cnf文件
vim master_ssl.cnf


req_extensions = v3_req
distinguished_name = req_distinguished_name

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = k8s_master
IP.1 = 192.168.2.1   # ClusterIP 地址
IP.2 = 10.211.55.10    # master IP地址



生成apiserver证书
openssl req -new -key server.key -subj "/CN=10.211.55.10" -config master_ssl.cnf -out server.csr

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt

设置kube-controller-manager相关证书
openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=10.211.55.10" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000

创建kubeconfig文件,kube-controller-manager和kube-scheduler公用的配置文件
vim /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: controllermanager
user:
    client-certificate: /etc/kubernetes/ssl/cs_client.crt
    client-key: /etc/kubernetes/ssl/cs_client.key
clusters:
- name: local
cluster:
    certificate-authority: /etc/kubernetes/ssl/ca.crt
contexts:
- context:
    cluster: local
    user: controllermanager
name: my-context
current-context: my-context

启动服务

启动kube-apiserver
systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver

启动kube-controller-manager
systemctl enable kube-controller-manager
systemctl start kube-controller-manager

启动kube-scheduler
systemctl enable kube-scheduler
systemctl start kube-scheduler

Node

安装docker

使用aliyun的yum源
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache

yum安装docker工具
yum install docker-ce
systemctl start docker
systemctl enable docker

docker -v

安装kubelet服务

安装包下载,整理
wget https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
tar -xzvf kubernetes-node-linux-amd64.tar.gz
cd kubernetes/node/bin
cp * /usr/bin

添加systemctl启动配置
vim /usr/lib/systemd/system/kubelet.service
mkdir -p /var/lib/kubelet
mkdir -p /etc/kubernetes/
mkdir -p /var/log/kubernetes


Description=Kubelet Service
After=docker.service
Requires=docker.service

WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
LimitNOFILE=65536


WantedBy=multi-user.target

kuberlet运行参数配置
安装kube-proxy服务

添加systemctl启动配置
vim /usr/lib/systemd/system/kube-proxy.service


Description=K8s kube-proxy Service
After=network.target
After=docker.service
After=network.target
After=network.service


EnvironmentFile=/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536


WantedBy=multi-user.target

生成CA证书

将master节点上的kube-apiserver证书ca.crt和ca.key拷贝到Node上
使用ca.crt和ca.key生成node证书
openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=10.211.55.11" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

mkdir /etc/kubernetes/ssl
mv kubelet_client.* /etc/kubernetes/ssl/
mv ca.crt /etc/kubernetes/ssl/

配置kubeconfig
vim /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
user:
      client-certificate: /etc/kubernetes/ssl/kubelet_client.crt
      client-key: /etc/kubernetes/ssl/kubelet_client.key
clusters:
- name: local
cluster:
      certificate-authority: /etc/kubernetes/ssl/ca.crt
      server: https://10.211.55.10:6443
contexts:
- context:
      cluster: local
      user: kubelet
name: my-context
current-context: my-context

kubelet启动参数配置
vim /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=10.211.55.11 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --fail-swap-on=false"
这里要注意–fail-swap-on=false或者禁用swap,我这里选择配置–fail-swap-on=false
设置kube-proxy启动参数
vim /etc/kubernetes/kube-proxy

KUBE_PROXY_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
启动服务

systemctl daemon-reload
systemctl start kubelet.service
systemctl status kubelet.service

systemctl start kube-proxy
systemctl status kube-proxy
node 2就按照上面的步骤进行安装即可

admin 发表于 2018-9-20 11:11:21

搭建私有库

私有库用于系统内部存储成品镜像,能够快速进行下载及被k8s调度。

1.下载并启动私有库

:docker run --name registry -v /etc/localtime:/etc/localtime -v /opt/registry:/var/lib/registry -p 5000:5000 -itd docker.io/registry

#--name 表示启动的容器后名称,此处为registry
#-v 表示挂载路径格式为宿主机路径:容器内路径
#-p 表示映射端口格式为宿主机端口:容器内端口
#-itd   docker的内部参数,此处声明后台运行容器并分配一个伪终端并绑定到容器的标准输入上,后跟镜像名称此处为docker.io/registry

2.创建一个secret服务,用于k8s调度私有库容器时的“令牌”。简单来说,secret服务就是一个存储密码的服务

:kubectl create secret docker-registry registrykey --docker-server=registry.evehicle.cn --docker-username=docker --docker-password=docker --docker-email=lienhua@zhongchuangsanyou.com

:kubectl get secret
NAME          TYPE                      DATA      AGE
registrykey   kubernetes.io/dockercfg   1         6s

此时登录时会提示认证错误

:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn
Flag --email has been deprecated, will be removed in 1.13.
Error response from daemon: login attempt to https://registry.evehicle.cn/v2/ failed with status: 401 Unauthorized

这是因为Docker官方是推荐采用Secure Registry的工作模式的,即transport采用tls。这样我们就需要为Registry配置tls所需的key和crt文件了

3.配置nginx反向代理
: cat registry.evehicle.cn.conf

# For versions of nginx > 1.3.9 that include chunked transfer encoding support
# Replace with appropriate values where necessary

upstream docker-registry {
server 192.168.121.9:5000;
#server 10.44.170.95:5000;
}

# uncomment if you want a 301 redirect for users attempting to connect
# on port 80
# NOTE: docker client will still fail. This is just for convenience
# server {
#   listen *:80;
#   server_name my.docker.registry.com;
#   return 301 https://$server_name$request_uri;
# }

server {
    listen 443;
    server_name registry.evehicle.cn;

    ssl on;
    ssl_certificate ssl/registry.evehicle.cn.crt;
    ssl_certificate_key ssl/registry.evehicle.cn.key;

    client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

    # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
    chunked_transfer_encoding on;

    location / {
      auth_basic"Restricted";
      auth_basic_user_filepasswd;
      add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

      proxy_pass                        http://docker-registry;
      proxy_set_headerHost            $http_host;   # required for docker client's sake
      proxy_set_headerX-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_headerX-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_headerX-Forwarded-Proto $scheme;
      proxy_read_timeout                  900;
      }

    location /_ping {
      auth_basic off;
      include               docker-registry.conf;
    }

    location /v1/_ping {
      auth_basic off;
      include               docker-registry.conf;
    }

    location /v2/_ping {
      auth_basic off;
      include               docker-registry.conf;
    }
}

将key及crt证书文件放到../ssl目录下。使用htpasswd生成密码放于./上一级目录

htpasswd -bcm passwd docker docker
#-c:创建一个加密文件
#-m:md5加密,默认可不填写
#-b:表示用户名密码在命令行中一并输入,不用分别填写

4.再次登录

:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn

Login Succeeded
表示成功,此时再pull\push既在私有库中进行

构建服务

docker的本意是将代码包含在容器内制作成镜像形成“产品”。但出于公司的(频繁修改代码及服务器资源受限)的特殊性,我们将代码以“外挂”的形式运行在宿主机上。下面以部署官网(apache)服务为例:
1.从docker的公有库里下载centos7的原生镜像

:docker pull centos

Using default tag: latest
Trying to pull repository docker.io/library/centos ...
latest: Pulling from docker.io/library/centos
d9aaf4d82f24: Downloading [>            ]   540 kB/73.39 MB
d9aaf4d82f24: Pulling fs layer
Digest: sha256:eba772bac22c86d7d6e72421b4700c3f894ab6e35475a34014ff8de74c10872e
Status: Downloaded newer image for centos:latest

2.编写Dockerfile制造apache基础镜像

######httpd####
FROM centos
MAINTAINER lienhua lienhua@zhongchuangsanyou.com
RUN yum -y install epel-release
RUN yum -y install httpdphp php-mysql php-memcache* php-mbstring
ADD httpd.conf /etc/httpd/conf/httpd.conf

EXPOSE 80

CMD ["/usr/sbin/apachectl", "-D", "FOREGROUND"]

其中httpd.conf文件需要在当前目录下真实存在,此处其内容为

ServerRoot "/etc/httpd"
Listen 80
Listen 8080
Include conf.modules.d/*.conf
Include zcsy/*.conf
User apache
Group apache
ServerAdmin root@localhost
<Directory />
    AllowOverride none
    Require all denied
</Directory>
DocumentRoot "/var/www/html"
<Directory "/var/www">
    AllowOverride None
    Require all granted
</Directory>
<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>
<IfModule dir_module>
    DirectoryIndex index.html
</IfModule>
<Files ".ht*">
    Require all denied
</Files>
ErrorLog "logs/error_log"
LogLevel warn
<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common
    <IfModule logio_module>
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>
    CustomLog "logs/access_log" combined
</IfModule>
<IfModule alias_module>
    ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
</IfModule>
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Require all granted
</Directory>
<IfModule mime_module>
    TypesConfig /etc/mime.types
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
    AddType application/x-httpd-php .php
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
</IfModule>
AddDefaultCharset UTF-8
<IfModule mime_magic_module>
    MIMEMagicFile conf/magic
</IfModule>
EnableSendfile off
EnableMMAP off
IncludeOptional conf.d/*.conf

执行:docker build -t registry.evehicle.cn/httpd . 命令制作名为”registry.evehicle.cn/httpd”的镜像(注意此处的点必须要有,并且其意义代表当前目录下的Dockerfile文件)

3.将制作好的镜像上传到私有库

docker push registry.evehicle.cn/httpd

4.编写启动apache服务的yaml文件

:cat 13-rc-httpd.yaml

apiVersion: v1
kind: ReplicationController
metadata:
name: 13-rc-httpd
labels:
    name: 13-rc-httpd
spec:
replicas: 2
selector:
    name: 13-rc-httpd
template:
    metadata:
      labels:
      name: 13-rc-httpd
    spec:
      containers:
      - name: 13-rc-httpd
      image: registry.evehicle.cn/httpd
      env:
      - name: LANG
          value: en_US.UTF-8
      ports:
      - containerPort: 80
          hostPort: 80
      volumeMounts:
      - name: time
          mountPath: /etc/localtime
      - name: zcsy
          mountPath: /etc/httpd/zcsy
      - name: deploy
          mountPath: /docker/httpd/deploy
      - name: log
          mountPath: /var/log/httpd
      volumes:
      - name: time
          hostPath:
            path: /etc/localtime
      - name: zcsy
          hostPath:
            path: /docker/httpd/zcsy
      - name: deploy
          hostPath:
            path: /docker/httpd/deploy
      - name: log
          hostPath:
            path: /docker/httpd/log
      nodeSelector:
      slave: "13"
      imagePullSecrets:
      - name: registrykey

5.给其中一个node加上标签为“13”

kubectl label nodes centos-minion-1 slave=13

6.此时拥有标签“13”的nodes应具备的条件

/docker/httpd/zcsy下需要有官网的配置文件

<VirtualHost *:80>
   ServerName www.evehicle.cn
DocumentRoot /var/deploy/wordpress/
      RewriteEngine on
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !^.*\.(ico|pdf|flv|jpe?g|js|gif|png|html|shtml|zip|xml|gz|rar|swf|txt|apk|bmp|css|m4a|ogg|mp3|ipa|plist)$
      RewriteCond %{REQUEST_URI} !^/server-status$
      RewriteRule . /index.php

</VirtualHost>
<Directory /var/deploy/wordpress/>
    Options FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

以及/docker/httpd/deploy下需要有官网的代码

7.运行yaml文件启动容器

: kuberctl create -f 13-rc-httpd.yaml

8.查看服务

: kuberctl get rc

NAME               DESIRED   CURRENT   AGE
13-rc-httpd          2         2         168d

9.程序中涉及的mysql\redis\memcache等服务也需使用容器运行起来

: docker pull redis
: docker tag registry.evehicle.cn/redis redis
: docker push registry.evehicle.cn/redis
: kubectl create -f rc-redis.yaml
: cat rc-redis.yaml

apiVersion: v1
kind: ReplicationController
metadata:
name: redis
labels:
    name: redis
spec:
replicas: 2
selector:
    name: redis
template:
    metadata:
      labels:
      name: redis
    spec:
      containers:
      - name: redis
      image: registry.evehicle.cn/redis
      ports:
      - containerPort: 6379
          hostPort: 6379
      volumeMounts:
      - name: data
          mountPath: /data
      - name: time
          mountPath: /etc/localtime
      volumes:
      - name: data
          hostPath:
            path: /docker/redis/6379
      - name: time
          hostPath:
            path: /etc/localtime
      nodeSelector:
      slave: "13"
      imagePullSecrets:
      - name: registrykey

启动memcache
: docker pull memcache
: docker tag registry.evehicle.cn/memcached memcache
: docker push registry.evehicle.cn/memcached
: kubectl create -f rc-memcached.yaml
: cat rc-memcached.yaml

apiVersion: v1
kind: ReplicationController
metadata:
name: memcached
labels:
    name: memcached
spec:
replicas: 3
selector:
    name: memcached
template:
    metadata:
      labels:
      name: memcached
    spec:
      containers:
      - name: memcached
      image: registry.evehicle.cn/memcached
      ports:
      - containerPort: 11211
          hostPort: 11211
      #nodeSelector:
      #slave: "13"
      imagePullSecrets:
      - name: registrykey

制造mysql镜像
: cat Dockerfile

FROM alpine


COPY startup.sh /startup.sh
RUN addgroup mysql && \
    adduser -H -D -s /bin/false -G mysql mysql && \
    apk add --update mysql mysql-client && rm -f /var/cache/apk/* && \
    mkdir /data && \
    chown -R mysql:mysql /data /etc/mysql && \
    chmod 755 /startup.sh \
    ;


WORKDIR /data
VOLUME /data
VOLUME /etc/mysql


EXPOSE 3306
CMD ["/startup.sh"]

启动mysql(建议mysql在宿主机启动)
: docker build -t registry.evehicle.cn/mysql
: docker push registry.evehicle.cn/mysql
: kubectl create -f rc-mysql.yaml
: cat rc-mysql.yaml

apiVersion: v1
kind: ReplicationController
metadata:
name: 13-rc-mysql
labels:
    name: 13-rc-mysql
spec:
replicas: 2
selector:
    name: 13-rc-mysql
template:
    metadata:
      labels:
      name: 13-rc-mysql
    spec:
      containers:
      - name: 13-rc-mysql
      image: registry.evehicle.cn/mysql
      env:
      - name: MYSQL_DATABASE
          value: admin
      - name: MYSQL_USER
          value: tony
      - name: MYSQL_PASSWORD
          value: 456
      - name: MYSQL_ROOT_PASSWORD
          value: 123
      ports:
      - containerPort: 3306
          hostPort: 3306
      volumeMounts:
      - name: time
          mountPath: /etc/localtime
      - name: data
          mountPath: /data
      - name: etc
          mountPath: /etc/mysql
      - name: run
          mountPath: /run/mysqld
      volumes:
      - name: time
          hostPath:
            path: /etc/localtime
      - name: data
          hostPath:
            path: /docker/mysql/data
      - name: etc
          hostPath:
            path: /docker/mysql/etc
      - name: run
          hostPath:
            path: /docker/mysql/run
      nodeSelector:
      slave: "13"
      imagePullSecrets:
      - name: registrykey

为方便代码编写及统一管理,应提前做好内部DNS解析。将所负责的应用规整到对应的机器上。

admin 发表于 2018-9-20 16:11:19

kubectl config set-cluster default-cluster --server=http://192.168.121.9:8080
kubectl config set-context default-context --cluster=default-cluster --user=default-admin
kubectl config use-context default-context

admin 发表于 2018-9-20 21:31:29

搭建私有库

私有库用于系统内部存储成品镜像,能够快速进行下载及被k8s调度。

1.下载并启动私有库

:docker run --name registry -v /etc/localtime:/etc/localtime -v /opt/registry:/var/lib/registry -p 5000:5000 -itd docker.io/registry

#--name 表示启动的容器后名称,此处为registry
#-v 表示挂载路径格式为宿主机路径:容器内路径
#-p 表示映射端口格式为宿主机端口:容器内端口
#-itd   docker的内部参数,此处声明后台运行容器并分配一个伪终端并绑定到容器的标准输入上,后跟镜像名称此处为docker.io/registry

2.创建一个secret服务,用于k8s调度私有库容器时的“令牌”。简单来说,secret服务就是一个存储密码的服务

:kubectl create secret docker-registry registrykey --docker-server=registry.evehicle.cn --docker-username=docker --docker-password=docker --docker-email=lienhua@zhongchuangsanyou.com

:kubectl get secret
NAME          TYPE                      DATA      AGE
registrykey   kubernetes.io/dockercfg   1         6s

此时登录时会提示认证错误

:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn
Flag --email has been deprecated, will be removed in 1.13.
Error response from daemon: login attempt to https://registry.evehicle.cn/v2/ failed with status: 401 Unauthorized

这是因为Docker官方是推荐采用Secure Registry的工作模式的,即transport采用tls。这样我们就需要为Registry配置tls所需的key和crt文件了

3.配置nginx反向代理
: cat registry.evehicle.cn.conf

# For versions of nginx > 1.3.9 that include chunked transfer encoding support
# Replace with appropriate values where necessary

upstream docker-registry {
server 192.168.121.9:5000;
#server 10.44.170.95:5000;
}

# uncomment if you want a 301 redirect for users attempting to connect
# on port 80
# NOTE: docker client will still fail. This is just for convenience
# server {
#   listen *:80;
#   server_name my.docker.registry.com;
#   return 301 https://$server_name$request_uri;
# }

server {
    listen 443;
    server_name registry.evehicle.cn;

    ssl on;
    ssl_certificate ssl/registry.evehicle.cn.crt;
    ssl_certificate_key ssl/registry.evehicle.cn.key;

    client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

    # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
    chunked_transfer_encoding on;

    location / {
      auth_basic"Restricted";
      auth_basic_user_filepasswd;
      add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

      proxy_pass                        http://docker-registry;
      proxy_set_headerHost            $http_host;   # required for docker client's sake
      proxy_set_headerX-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_headerX-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_headerX-Forwarded-Proto $scheme;
      proxy_read_timeout                  900;
      }

    location /_ping {
      auth_basic off;
      include               docker-registry.conf;
    }

    location /v1/_ping {
      auth_basic off;
      include               docker-registry.conf;
    }

    location /v2/_ping {
      auth_basic off;
      include               docker-registry.conf;
    }
}

将key及crt证书文件放到../ssl目录下。使用htpasswd生成密码放于./上一级目录

htpasswd -bcm passwd docker docker
#-c:创建一个加密文件
#-m:md5加密,默认可不填写
#-b:表示用户名密码在命令行中一并输入,不用分别填写

4.再次登录

:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn

Login Succeeded
表示成功,此时再pull\push既在私有库中进行
页: [1]
查看完整版本: k8s学习二:k8s编译安装集群搭建——单master多node简易部署