k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

1 持续集成/部署/交付概述

持续集成(Continuous Integration,CI): 代码合并、构建、部署、测试都在一起,不断地执行这个过程,并对结果反馈。 持续部署(Continuous Deployment,CD): 部署到测试环境、预生产环境、生产环境。 持续交付(Continuous Delivery,CD): 将最终产品发布到生产环境,给用户使用。
  • 发布流程设计 k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
  • 工作流程

工作流程:手动/自动构建-> Jenkins 调度K8S API->动态生成Jenkins Slave pod ->Slave pod 拉取Git 代码/编译/打包镜像->推送到镜像仓库Harbor ->Slave 工作完成,Pod 自动销毁->部署到测试或生产Kubernetes平台。

  • 部署到k8s平台流程
(1). 制作镜像
(2). 容器放到Pod
(3). 控制器管理Pod
(4). 暴露应用
(5). 对外发布应用
(6). 日志管理/监控

2 部署jenkins

2.1 环境准备 

准备K8S、Ingress和PV自动供给(NFS),
部署harbor:https://devopstack.cn/k8s/1301.html
构建jenkins-slave镜像:https://github.com/jenkinsci/docker-jnlp-slave
将构建好的jenkins-slave镜像推送至harbor仓库
在kubernetes中部署jenkins:https://github.com/jenkinsci/kubernetes-plugin/tree/fc40c869edfd9e3904a9a56b0f80c5a25e988fa1/src/main/kubernetes
部署git,创建用户,jenkins保存git凭据
Kubernetes插件:Jenkins在Kubernetes集群中运行动态代理。插件介绍:https://github.com/jenkinsci/kubernetes-plugin

2.2 部署jenkins

2.2.1 部署有状态的jenkins Pod

2.2.2 创建jenkins ingress

cat ingress-jenkins.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: "true"
    # 如果上传插件超出默认会报"413 Request Entity Too Large", 增加 client_max_body_size
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    # nginx-ingress controller版本小于 0.9.0.beta-18 的配置
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
  rules:
  - host: jenkins.devopstack.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 80

kubectl apply -f ingress-jenkins.yml 
ingress.extensions/jenkins created

2.2.3 jenkins认证授权

# cat service-account.yml
---
# 创建名为jenkins的ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins

---
# 创建名为jenkins的Role,授予允许管理API组的资源Pod
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
# 将名为jenkins的Role绑定到名为jenkins的ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

2.3 配置jenkins

获取jenkins密码,登录jenkins
[root@k8s-master01 jenkins]#  kubectl exec -it jenkins-0 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
jenkins@jenkins-0:/$  cat /var/jenkins_home/secrets/initialAdminPassword
9b3bba656b3f4afe8ec4eb788497c215
登录jenkins.devopstack.cn查看
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
安装插件:pipeline、git、kubernetes Kubernetes Continuous Deploy

配置jenkins加速


/data/k8s-volume/default-jenkins-data-pvc-6e3a3b9f-8906-4adc-ac85-e9684bad3e68/updates [root@k8s-node01 updates]# sed -i 's#http://updates.jenkins-ci.org/download#https://mirrors.tuna.tsinghua.edu.cn/jenkins#g;s#http://www.google.com#https://www.baidu.com#g' default.json
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

2.4 Jenkins在K8S中动态创建代理

K8S中Jenkins Master/Slave架构
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
jenkins插件连接K8S配置 k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

2.5 构建Jenkins Slave镜像

代码拉取:git,安装git命令 单元测试:忽略,这不是我们擅长的,如果公司有可以写进来 代码编译:maven,安装maven包 构建镜像:Dockerfile文件、docker命令(通过挂载宿主机docker) 推送镜像:docker命令(通过挂载宿主机docker) 镜像启动后支持slave: 下载官方slave.jar包 启动 slave.ja包:jenkins-slave启动脚步(通过参考文档URL) maven配置文件:settings.xml (这里配置阿里云的仓库源)

2.5.1 Dockerfile配置文件

cat dockerfile-jenkins-slave 
FROM centos:7
MAINTAINER lizhenliang

RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \ 
    yum clean all && \
    rm -rf /var/cache/yum/* && \
    mkdir -p /usr/share/jenkins

COPY slave.jar /usr/share/jenkins/slave.jar  
COPY jenkins-slave /usr/bin/jenkins-slave
COPY settings.xml /etc/maven/settings.xml
RUN chmod +x /usr/bin/jenkins-slave

ENTRYPOINT ["jenkins-slave"]

2.5.2 jenkins-slave启动脚步

cat jenkins-slave 
#!/usr/bin/env sh

if [ # -eq 1 ]; then

    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec "@"

else

    # if -tunnel is not provided try env vars
    case "@" in
        *"-tunnel "*) ;;
        *)
        if [ ! -z "JENKINS_TUNNEL" ]; then
            TUNNEL="-tunnel JENKINS_TUNNEL"
        fi ;;
    esac

    # if -workDir is not provided try env vars
    if [ ! -z "JENKINS_AGENT_WORKDIR" ]; then
        case "@" in
            *"-workDir"*) echo "Warning: Work directory is defined twice in command-line arguments and the environment variable" ;;
            *)
            WORKDIR="-workDirJENKINS_AGENT_WORKDIR" ;;
        esac
    fi

    if [ -n "JENKINS_URL" ]; then
        URL="-urlJENKINS_URL"
    fi

    if [ -n "JENKINS_NAME" ]; then
        JENKINS_AGENT_NAME="JENKINS_NAME"
    fi  

    if [ -z "JNLP_PROTOCOL_OPTS" ]; then
        echo "Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior"
        JNLP_PROTOCOL_OPTS="-Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true"
    fi

    # If both required options are defined, do not pass the parameters
    OPT_JENKINS_SECRET=""
    if [ -n "JENKINS_SECRET" ]; then
        case "@" in
            *"{JENKINS_SECRET}"*) echo "Warning: SECRET is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_SECRET="{JENKINS_SECRET}" ;;
        esac
    fi

    OPT_JENKINS_AGENT_NAME=""
    if [ -n "JENKINS_AGENT_NAME" ]; then
        case "@" in
            *"{JENKINS_AGENT_NAME}"*) echo "Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_AGENT_NAME="{JENKINS_AGENT_NAME}" ;;
        esac
    fi

    #TODO: Handle the case when the command-line and Environment variable contain different values.
    #It is fine it blows up for now since it should lead to an error anyway.

    exec javaJAVA_OPTS JNLP_PROTOCOL_OPTS -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headlessTUNNEL URLWORKDIR OPT_JENKINS_SECRETOPT_JENKINS_AGENT_NAME "$@"
fi

2.5.3 maven源配置文件settings.xml

# cat settings.xml
<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <pluginGroups>
  </pluginGroups>

  <proxies>
  </proxies>

  <servers>
  </servers>

  <mirrors>
    <mirror>
      <id>central</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>
  </mirrors>

  <profiles>
  </profiles>

</settings>

2.5.4 构建镜像, 并推送至私有镜像仓库

[root@k8s-master01 jenkins-salve]# ls
Dockerfile  jenkins-slave  settings.xml  slave.jar
构建docker镜像
docker build -t 192.168.10.20:8081/library/jenkins-slave-jdk:1.8 .
登录docker仓库
docker login 192.168.10.20:8081
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

推送镜像
 docker push 192.168.10.20:8081/library/jenkins-slave-jdk:1.8
The push refers to repository [192.168.10.20:8081/library/jenkins-slave-jdk]
cc6b65234afb: Pushed 
1d14b4ca7bd2: Pushed 
9b1f1ed0660e: Pushed 
86146a421624: Pushed 
a48b0a9ab7f5: Pushed 
174f56854903: Pushed 
1.8: digest: sha256:fecb8589bbe1bd3189a948edba8761f9d674aec3bd30f9eed3a62a2a8d61eab0 size: 1574
如上已推送成功

3 pipeline的使用

3.1 Jenkins Pipeline构建流水线发布

Jenkins Pipeline是一套插件,支持在Jenkins中实现持续集成、交付管道;
Pipeline通过特定语法从简单到复杂的传输管道进行建模;
① 声明式:遵循与Groovy相同语法。pipeline { }
② 脚本式:支持Groovy大部分功能,也是非常表达和灵活的工具。node { }
Jenkins Pipeline的定义被写入一个文本文件,称为Jenkinsfile。

Jenkins Pipeline 核心概念:
Node: 节点,一个 Node 就是一个 Jenkins 节点,Master 或者 Agent,是执行 Step 的具体运行环境,比如我们之前动态运行的 Jenkins Slave 就是一个 Node 节点

Stage: 阶段,一个 Pipeline 可以划分为若干个 Stage,每个 Stage 代表一组操作,如:Build、Test、Deploy等,Stage 是一个逻辑分组的概念,可以跨多个 Node

Step: 步骤,Step 是最基本的操作单元,可以是打印一句话,也可以是构建一个 Docker 镜像,由各类 Jenkins 插件提供,比如命令:sh ‘make’,就相当于我们平时 shell 终端中执行 make 命令一样。

如下图是我们编写的pipeline

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
// Uses Declarative syntax to run commands inside a container.
pipeline {
    agent {
        kubernetes {
            label "jenkins-slave"
            yaml '''
apiVersion: v1
kind: Pod
metadata:
  name: jenkins-slave
spec:
  containers:
  - name: jnlp
    image: 192.168.10.20:8081/library/jenkins-slave-jdk:1.8

'''

        }
    }
    stages {
        stage('Main') {
            steps {
                sh 'hostname'
            }
        }
    }
}

构建任务,k8s会启动一个jenkins-slave跑我们的任务,跑完任务,jenkins-slave的pod会自动删除的.达到即开机用的目的.
如下图是我们跑取pipeline的最终结果

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

如下图是创建pod和删除pod的过程

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

3.2 生成git流水线语法

生成git拉取代码的pipeline k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

3.3 pipeline语法

脚本式写法:

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

声明式写法实验

pipeline {
    agent {
        kubernetes {
            label "jenkins-slave"
            yaml '''
apiVersion: v1
kind: Pod
metadata:
  name: jenkins-slave
spec:
  containers:
  - name: jnlp
    image: 192.168.10.20:8081/library/jenkins-slave-jdk:1.8

'''

        }
    }
    stages {
        stage('拉取代码') {
            steps {
                git credentialsId: 'ea5947ff-50ec-4438-8323-0336c9398cb4', url: 'http://192.168.10.20:8088/root/java-demo.git'
            }
        }
        stage('代码编译') {
            steps {
                sh 'mvn clean package -Dmaven.test.skip=true'
            }
        }
        stage('构建镜像') {
            steps {
             echo '构建镜像'
            }
        }

        stage('发布代码到k8s') {
            steps {
               echo '发布代码到k8s'
            }
        }
    }
}

jenkins构建效果

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4 通过jenkins部署代码到k8s

4.1 编写Pipeline脚本完成CI阶段

安装插件Extended Choice Parameter,git Parameter开启参数化构建
pipeline script注意:
podTemplate 中label和node括号里的字符串要一致
podTemplate 中cloud为之前配置jenkins Cloud 名称
containerTemplate中name最好保持为"jnlp"
pipeline script变量docker_registry_auth、git_auth、k8s_auth通过保存在jenkins凭据中相应的凭据ID。

4.1.1 创建pipeline流水线中各个凭证

创建docker拉取认证
kubectl create secret docker-registry  dockerpullauth  --docker-username=admin --docker-password=Harbor12345 --docker-server=192.168.10.20:8081
在jenkins中创建harbor凭据
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

查看凭据id

b61a6724-b13f-42c7-bfb7-3b81c0028aeb    
拉取Git代码秘钥配置
ba956237-7a53-4bff-acab-e0a187bed3a9
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
创建k8s auth凭据

[root@k8s-master01 ~]# cat .kube/config

c30859ee-4984-49fe-bb6e-f30a583eebde
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4.1.2 parameters参数化生成

方式一:

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

方式二

parameters {    
        gitParameter branch: '', branchFilter: '.*', defaultValue: 'master', description: '选择发布的分支', name: 'Branch', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH'
        choice (choices: ['1', '3', '5', '7'], description: '副本数', name: 'ReplicaCount')
        choice (choices: ['dev','test','prod'], description: '命名空间', name: 'Namespace')
    }

4.1.3 拉取代码片段pipeline生成

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群
checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[credentialsId: 'ba956237-7a53-4bff-acab-e0a187bed3a9', url: 'http://192.168.10.20:8088/root/java-demo.git']]])

4.1.4 完整的基于jenkins pipeine脚本CI阶段构建

如下是完整的基于jenkins pipeine脚本CI阶段代码

// 公共
def registry = "192.168.10.20:8081"
// 项目
def project = "javademo"
def app_name = "javademo"
def image_name = "{registry}/{project}/{app_name}:{BUILD_NUMBER}"
def git_address = "http://192.168.10.20:8088/root/java-demo.git"
// 认证
def secret_name = "dockerpullauth"
def docker_registry_auth = "b61a6724-b13f-42c7-bfb7-3b81c0028aeb"  //Harbor login auth
def git_auth = "ba956237-7a53-4bff-acab-e0a187bed3a9" //git login auth
def k8s_auth = "c30859ee-4984-49fe-bb6e-f30a583eebde"  // k8s auth,CI阶段不用到, CD 部署时才用到

pipeline {
  agent {
    kubernetes {
        label "jenkins-slave"
        yaml """
kind: Pod
metadata:
  name: jenkins-slave
spec:
  containers:
  - name: jnlp
    image: {registry}/library/jenkins-slave-jdk:1.8
    imagePullPolicy: Always
    volumeMounts:
      - name: docker-cmd
        mountPath: /usr/bin/docker
      - name: docker-sock
        mountPath: /var/run/docker.sock
      - name: maven-cache
        mountPath: /root/.m2
  volumes:
    - name: docker-cmd
      hostPath:
        path: /usr/bin/docker
    - name: docker-sock
      hostPath:
        path: /var/run/docker.sock
    - name: maven-cache
      hostPath:
        path: /tmp/m2
"""
        }

      }
    parameters {           gitParameter branch: '', branchFilter: '.*', defaultValue: 'master', description: '选择发布的分支', name: 'Branch', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH'
        choice (choices: ['1', '3', '5', '7'], description: '副本数', name: 'ReplicaCount')
        choice (choices: ['dev','test','prod'], description: '命名空间', name: 'Namespace')
    }

    stages {
        stage('拉取代码'){
            steps {
                checkout([class: 'GitSCM', 
                branches: [[name: "{params.Branch}"]],                doGenerateSubmoduleConfigurations: false,                extensions: [], submoduleCfg: [],                userRemoteConfigs: [[credentialsId: "{git_auth}", url: "{git_address}"]]
                ])
            }
        }

        stage('代码编译'){
           steps {
             sh """
                mvn clean package -Dmaven.test.skip=true
                """           }
        }

        stage('构建镜像'){
           steps {
                withCredentials([usernamePassword(credentialsId: "{docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
                sh """
                  echo '
                    FROM {registry}/javademo/javademo
                    LABEL maitainer suixiaofeng
                    RUN rm -rf /usr/local/tomcat/webapps/*
                    ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
                  '>Dockerfile
                  docker build -t{image_name} .
                  docker login -u {username} -p '{password}' {registry}
                  docker push{image_name}
                """
                }
           } 
        }
 }
} 

触发jenkins构建完成ci阶段操作

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4.2 编写Pipeline脚本完成CD阶段

4.2.1 持续部署pipeline片段生成

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4.2.2 持续部署pipeline代码

      stage('部署到K8S平台'){
         steps {
          sh """
          sed -i 's#IMAGE_NAME#{image_name}#' deploy.yaml
          sed -i 's#SECRET_NAME#{secret_name}#' deploy.yaml
          sed -i 's#REPLICAS#{ReplicaCount}#' deploy.yaml
          sed -i 's#NS#{Namespace}#' deploy.yaml
          """
          kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "${k8s_auth}"

         }
      }

编写deploy.yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-demo
  namespace: NS
spec:
  replicas: REPLICAS
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets: 
      - name: "SECRET_NAME"
      containers:
      - image: IMAGE_NAME
        name: java-demo
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 1Gi
          limits:
            cpu: 1
            memory: 2Gi
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: java-demo
  name: java-demo
  namespace: NS
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
    nodePort: 30018
  selector:
    app: java-demo
  type: NodePort

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: java-demo 
  namespace: NS
spec:
  rules:
    - host: java.devopstack.cn
      http:
        paths:
        - path: /
          backend:
            serviceName: java-demo 
            servicePort: 80

创建dev命名空间
[root@k8s-master01 jenkins]# kubectl create ns dev 
namespace/dev created
创建对应命名空间的harbor认证
[root@k8s-master01 jenkins]# kubectl create secret docker-registry  dockerpullauth  --docker-username=admin --docker-password=Harbor12345 --docker-server=192.168.10.20:8081 -n dev
secret/dockerpullauth created

4.2.3 构建完成cd阶段

完整的构建pipeline代码如下

// 公共
def registry = "192.168.10.20:8081"
// 项目
def project = "javademo"
def app_name = "javademo"
def image_name = "{registry}/{project}/{app_name}:{BUILD_NUMBER}"
def git_address = "http://192.168.10.20:8088/root/java-demo.git"
// 认证
def secret_name = "dockerpullauth"
def docker_registry_auth = "b61a6724-b13f-42c7-bfb7-3b81c0028aeb"  //Harbor login auth
def git_auth = "ba956237-7a53-4bff-acab-e0a187bed3a9" //git login auth
def k8s_auth = "c30859ee-4984-49fe-bb6e-f30a583eebde"  // k8s auth,CI阶段不用到, CD 部署时才用到

pipeline {
  agent {
    kubernetes {
        label "jenkins-slave"
        yaml """
kind: Pod
metadata:
  name: jenkins-slave
spec:
  containers:
  - name: jnlp
    image: {registry}/library/jenkins-slave-jdk:1.8
    imagePullPolicy: Always
    volumeMounts:
      - name: docker-cmd
        mountPath: /usr/bin/docker
      - name: docker-sock
        mountPath: /var/run/docker.sock
      - name: maven-cache
        mountPath: /root/.m2
  volumes:
    - name: docker-cmd
      hostPath:
        path: /usr/bin/docker
    - name: docker-sock
      hostPath:
        path: /var/run/docker.sock
    - name: maven-cache
      hostPath:
        path: /tmp/m2
"""
        }

      }
    parameters {           gitParameter branch: '', branchFilter: '.*', defaultValue: 'master', description: '选择发布的分支', name: 'Branch', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH'
        choice (choices: ['1', '3', '5', '7'], description: '副本数', name: 'ReplicaCount')
        choice (choices: ['dev','test','prod'], description: '命名空间', name: 'Namespace')
    }

    stages {
        stage('拉取代码'){
            steps {
                checkout([class: 'GitSCM', 
                branches: [[name: "{params.Branch}"]],                doGenerateSubmoduleConfigurations: false,                extensions: [], submoduleCfg: [],                userRemoteConfigs: [[credentialsId: "{git_auth}", url: "{git_address}"]]
                ])
            }
        }

        stage('代码编译'){
           steps {
             sh """
                mvn clean package -Dmaven.test.skip=true
                """           }
        }

        stage('构建镜像'){
           steps {
                withCredentials([usernamePassword(credentialsId: "{docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
                sh """
                  echo '
                    FROM {registry}/javademo/javademo
                    LABEL maitainer suixiaofeng
                    RUN rm -rf /usr/local/tomcat/webapps/*
                    ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
                  '>Dockerfile
                  docker build -t{image_name} .
                  docker login -u {username} -p '{password}' {registry}
                  docker push{image_name}
                """
                }
           } 
        }

      stage('部署到K8S平台'){
         steps {
          sh """
          sed -i 's#IMAGE_NAME#{image_name}#' deploy.yaml
          sed -i 's#SECRET_NAME#{secret_name}#' deploy.yaml
          sed -i 's#REPLICAS#{ReplicaCount}#' deploy.yaml
          sed -i 's#NS#{Namespace}#' deploy.yaml
          """
          kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "${k8s_auth}"

         }
      }
  }
}

选择构建参数,副本数,放在哪个环境下,然后构建

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

查看是否构建成功,如图,已经部署成功.

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

通过域名去访问

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4.2.4 添加回滚操作pipeline

如下pipeline,我们加入了回退的选项


// 公共 def registry = "192.168.10.20:8081" // 项目 def project = "javademo" def app_name = "javademo" def image_name = "{registry}/{project}/{app_name}:{BUILD_NUMBER}" def git_address = "http://192.168.10.20:8088/root/java-demo.git" def rollback_image_name = "{registry}/{project}/{app_name}:{version}" // 认证 def secret_name = "dockerpullauth" def docker_registry_auth = "b61a6724-b13f-42c7-bfb7-3b81c0028aeb" //Harbor login auth def git_auth = "ba956237-7a53-4bff-acab-e0a187bed3a9" //git login auth def k8s_auth = "c30859ee-4984-49fe-bb6e-f30a583eebde" // k8s auth,CI阶段不用到, CD 部署时才用到 pipeline { agent { kubernetes { label "jenkins-slave" yaml """ kind: Pod metadata: name: jenkins-slave spec: containers: - name: jnlp image: {registry}/library/jenkins-slave-jdk:1.8 imagePullPolicy: Always volumeMounts: - name: docker-cmd mountPath: /usr/bin/docker - name: docker-sock mountPath: /var/run/docker.sock - name: maven-cache mountPath: /root/.m2 volumes: - name: docker-cmd hostPath: path: /usr/bin/docker - name: docker-sock hostPath: path: /var/run/docker.sock - name: maven-cache hostPath: path: /tmp/m2 """ } } parameters { gitParameter branch: '', branchFilter: '.*', defaultValue: 'master', description: '选择发布的分支', name: 'Branch', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH' choice (choices: ['1', '3', '5', '7'], description: '副本数', name: 'ReplicaCount') choice (choices: ['dev','test','prod'], description: '命名空间', name: 'Namespace') choice choices: ['deploy', 'rollback'], description: '''deploy发布新代码rollback回滚''', name: 'deploy_env' string defaultValue: '0', description: '选择回滚版本号', name: 'version', trim: false } stages { stage('拉取代码'){ steps { checkout([class: 'GitSCM', branches: [[name: "{params.Branch}"]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: "{git_auth}", url: "{git_address}"]] ]) } } stage('代码编译'){ when { environment name: 'deploy_env', value: 'deploy' } steps { sh """ mvn clean package -Dmaven.test.skip=true """ } } stage('构建镜像'){ when { environment name: 'deploy_env', value: 'deploy' } steps { withCredentials([usernamePassword(credentialsId: "{docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) { sh """ echo ' FROM {registry}/javademo/javademo LABEL maitainer suixiaofeng RUN rm -rf /usr/local/tomcat/webapps/* ADD target/*.war /usr/local/tomcat/webapps/ROOT.war '>Dockerfile docker build -t{image_name} . docker login -u {username} -p '{password}' {registry} docker push{image_name} """ } } } stage('部署到K8S平台'){ when { environment name: 'deploy_env', value: 'deploy' } steps { sh """ sed -i 's#IMAGE_NAME#{image_name}#' deploy.yaml sed -i 's#SECRET_NAME#{secret_name}#' deploy.yaml sed -i 's#REPLICAS#{ReplicaCount}#' deploy.yaml sed -i 's#NS#{Namespace}#' deploy.yaml """ kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "{k8s_auth}" } } stage('回滚指定的镜像'){ when { environment name: 'deploy_env', value: 'rollback' } steps { sh """ sed -i 's#IMAGE_NAME#{rollback_image_name}#' deploy.yaml sed -i 's#SECRET_NAME#{secret_name}#' deploy.yaml sed -i 's#REPLICAS#{ReplicaCount}#' deploy.yaml sed -i 's#NS#{Namespace}#' deploy.yaml """ kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "{k8s_auth}" } } } }
k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

验证回退是否有效
目前界面背景图是如下照片中,这边迭代一个版本把背景照换掉.

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

先git提交一个版本,然后jenkins再次构建

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

然后执行回滚上一个版本的操作

k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群k8s+jenkins+gitlab+harbor实现自动化部署应用至k8s集群

4.3 添加构建状态告警pipeline

添加个邮件通知
安装插件:Extended E-mail Notification

  post {
        success {
            emailext (
                subject: "SUCCESSFUL: Job '{env.JOB_NAME} [{env.BUILD_NUMBER}]'",
                body: """<p>SUCCESSFUL: Job '{env.JOB_NAME} [{env.BUILD_NUMBER}]':</p>
                    <p>Check console output at "<a rel="nofollow" href="{env.BUILD_URL}">{env.JOB_NAME} [{env.BUILD_NUMBER}]</a>"</p>""",
                to: "carp@qq.com",
                from: "carp@qq.com"
            )
        }
        failure {
            emailext (
                subject: "FAILED: Job '{env.JOB_NAME} [{env.BUILD_NUMBER}]'",
                body: """<p>FAILED: Job '{env.JOB_NAME} [{env.BUILD_NUMBER}]':</p>
                    <p>Check console output at "<a rel="nofollow" href="{env.BUILD_URL}">{env.JOB_NAME} [{env.BUILD_NUMBER}]</a>"</p>""",
                to: "carp@qq.com",
                from: "carp@qq.com"
            )
        }
    }

5 jenkins的核心

pipeline是灵魂,后面将系统学习pipeline语法使用.
  • 我的微信
  • 这是我的微信扫一扫
  • weinxin
  • 我的微信公众号
  • 我的微信公众号扫一扫
  • weinxin
avatar

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: