k8s基础入门之外部服务发现之 ingress

内容 隐藏
1 1 ingress介绍
2 2 pod和ingress的关系
3 3 ingress流程图
4 4 部署ingress-nginx controller
5 5 ingress的使用
5.1 5.1 基于URL路径进行流量分发
5.2 5.2 基于主机名称的虚拟主机
5.3 5.3 Ingress https代理
5.4 5.4 annotations对ingress个性化配置
6 6 ingress controller高可用方案
7 7 Traefik

1 ingress介绍

Ingress就是一组基于DNS名称（host）或URL路径把请求转发至指定的Service资源的规则，用于将集群外部的请求流量转发至集群内部完成服务发布。然而，Ingress资源自身并不能进行“流量穿透”，它仅是一组路由规则的集合，这些规则要想真正发挥作用还需要其他功能的辅助，如监听某套接字，然后根据这些规则的匹配机制路由请求流量。这种能够为Ingress资源监听套接字并转发流量的组件称为Ingress控制器（Ingress Controller）。 Ingress控制器并不直接运行为kube-controller-manager的一部分，它是Kubernetes集群的一个重要附件，类似于CoreDNS，需要在集群上单独部署。 Ingress控制器可以由任何具有反向代理（HTTP/HTTPS）功能的服务程序实现，如Nginx、Envoy、HAProxy、Vulcand和Traefik等。Ingress控制器自身也是运行于集群中的Pod资源对象，它与被代理的运行为Pod资源的应用运行于同一网络中 使用Ingress资源进行流量分发时，Ingress控制器可基于某Ingress资源定义的规则将客户端的请求流量直接转发至与Service对应的后端Pod资源之上，这种转发机制会绕过Service资源，从而省去了由kube-proxy实现的端口代理开销。

2 pod和ingress的关系

  • 通过label-selector相关联
  • 通过Ingress Controller实现Pod的负载均衡
  • 支持TCP/UDP 4层和HTTP 7层

3 ingress流程图

k8s基础入门之外部服务发现之 ingress

4 部署ingress-nginx controller

本次实验使用ingress-nginx部署 其他主流控制器 traefik: http反向代理,负载均衡工具 istio: 服务治理,控制入口流量.
k8s基础入门之外部服务发现之 ingress
下载对应的yaml
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/mandatory.yaml
内容如下
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1
          args:
            - /nginx-ingress-controller
            - --configmap=(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=(POD_NAMESPACE)/udp-services
            - --publish-service=(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

修改
  spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1
          args:

创建ingress controller
[root@k8s-master01 ~]# kubectl apply -f mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
[root@k8s-master01 ~]# 

查看pod创建情况
[root@k8s-master01 ~]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-6d8d7c675b-r7bxl   1/1     Running   0          89s

5 ingress的使用 

单Service资源型Ingress
使用Ingress来暴露服务，此时只需要为Ingress指定“default backend”即可
[root@k8s-master01 ~]# cat my-ingress_default.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    name: my-ingress
spec:
    backend:
        serviceName: web
        servicePort: 80
[root@k8s-master01 ~]# kubectl apply -f my-ingress_default.yaml
ingress.extensions/my-ingress created

访问查看

k8s基础入门之外部服务发现之 ingress

5.1 基于URL路径进行流量分发 


[root@k8s-master01 ~]# cat myapp03.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: svc3
  labels:
    app: svc3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: svc3
  template:
    metadata:
      labels:
        app: svc3
    spec:
      containers:
      - name: svc3
        image: cnych/example-web-service
        env:
        - name: APP_SVC
          value: svc3
        ports:
        - containerPort: 8080
          protocol: TCP

kind: Service
apiVersion: v1
metadata:
  labels:
    app: svc3
  name: svc3
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: svc3

创建测试的svc和pods
[root@k8s-master01 ~]# kubectl apply -f myapp01.yaml 
service/svc1 created
[root@k8s-master01 ~]# kubectl apply -f myapp02.yaml 
service/svc2 created
[root@k8s-master01 ~]# kubectl apply -f myapp03.yaml 
service/svc3 created

查看svc
[root@k8s-master01 ~]# kubectl get svc |grep svc
svc1         ClusterIP   10.110.207.5     <none>        8080/TCP       2m54s
svc2         ClusterIP   10.110.235.231   <none>        8080/TCP       87s
svc3         ClusterIP   10.98.100.177    <none>        8080/TCP       83s

查看pods
[root@k8s-master01 ~]# kubectl get pods  |grep svc
svc1-5ff98455f6-8g78g    1/1     Running     0          19m
svc2-678c5fb65-rnt8t     1/1     Running     0          56s
svc3-55569d594-59zjf     1/1     Running     0          23s

创建一个 ingress 对象来访问上面的3个服务
[root@k8s-master01 ~]# cat ingress_app.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-web-url
  annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: ingress.devopstack.com
    http:
      paths:
      - path: /s1
        backend:
          serviceName: svc1
          servicePort: 8080
      - path: /s2
        backend:
          serviceName: svc2
          servicePort: 8080
      - path: /s3
        backend:
          serviceName: svc3
          servicePort: 8080
创建对应的ingress
[root@k8s-master01 ~]# kubectl apply -f ingress_app.yaml 
ingress.extensions/ingress-web-url created

验证
[root@k8s-master01 ~]# kubectl get ingress
NAME              CLASS    HOSTS                    ADDRESS   PORTS   AGE
ingress-web-url   <none>   ingress.devopstack.com             80      6m45s

查看具体规则内容
[root@k8s-master01 ~]# kubectl describe ingress ingress-web-url
Name:             ingress-web-url
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                    Path  Backends
  ----                    ----  --------
  ingress.devopstack.com  
                          /s1   svc1:8080 (10.244.2.181:8080)
                          /s2   svc2:8080 (10.244.2.218:8080)
                          /s3   svc3:8080 (10.244.2.221:8080)
Annotations:              nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  7m47s  nginx-ingress-controller  Ingress default/ingress-web-url
  Normal  CREATE  7m47s  nginx-ingress-controller  Ingress default/ingress-web-url

访问验证下
upsmart@upsmart-System-Product-Name:~$ cat /etc/hosts
192.168.10.72 ingress.devopstack.com

k8s基础入门之外部服务发现之 ingressk8s基础入门之外部服务发现之 ingressk8s基础入门之外部服务发现之 ingress

5.2 基于主机名称的虚拟主机

主机名通配符

主机名可以是精确匹配（例如“foo.bar.com”）或者使用通配符来匹配 （例如“.foo.com”）。 精确匹配要求 HTTP host 头部字段与 host 字段值完全匹配。 通配符匹配则要求 HTTP host 头部字段与通配符规则中的后缀部分相同。
主机 host 头部 匹配与否？
.foo.com bar.foo.com 基于相同的后缀匹配
.foo.com baz.bar.foo.com 不匹配，通配符仅覆盖了一个 DNS 标签
.foo.com foo.com 不匹配，通配符仅覆盖了一个 DNS 标签

实例:
[root@k8s-master01 ~]# cat ingress_app_host.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-web-host
spec:
  rules:
  - host: svc1.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
            serviceName: svc1
            servicePort: 8080

  - host: svc2.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
            serviceName: svc2
            servicePort: 8080

  - host: svc3.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          serviceName: svc3
          servicePort: 8080

执行yaml生成对应的ingress策略
[root@k8s-master01 ~]# kubectl apply -f ingress_app_host.yaml 
ingress.extensions/ingress-web-host created

查看ingress
[root@k8s-master01 ~]# kubectl get ingress
NAME               CLASS    HOSTS                                                      ADDRESS   PORTS   AGE
ingress-web-host   <none>   svc1.devopstack.cn,svc2.devopstack.cn,svc3.devopstack.cn             80      7s

查看 ingress策略详情
[root@k8s-master01 ~]# kubectl describe ingress ingress-web-host
Name:             ingress-web-host
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  svc1.devopstack.cn  
                      /   svc1:8080 (10.244.2.181:8080)
  svc2.devopstack.cn  
                      /   svc2:8080 (10.244.2.218:8080)
  svc3.devopstack.cn  
                      /   svc3:8080 (10.244.2.221:8080)
Annotations:          Events:
  Type                Reason  Age    From                      Message
  ----                ------  ----   ----                      -------
  Normal              CREATE  2m21s  nginx-ingress-controller  Ingress default/ingress-web-host
  Normal              CREATE  2m21s  nginx-ingress-controller  Ingress default/ingress-web-host

查看nginx-crontroller具体策略
kubectl  exec -it nginx-ingress-controller-b5p69 bash -n ingress-nginx

绑定 cat /etc/hosts
192.168.10.73 svc1.devopstack.cn
192.168.10.73 svc2.devopstack.cn
192.168.10.73 svc3.devopstack.cn

浏览器访问测试
k8s基础入门之外部服务发现之 ingressk8s基础入门之外部服务发现之 ingressk8s基础入门之外部服务发现之 ingress

5.3 Ingress https代理 

TLS 认证
在现在大部分场景下面我们都会使用 https 来访问我们的服务，这节课我们将使用一个自签名的证书，当然你有在一些正规机构购买的 CA 证书是最好的，这样任何人访问你的服务的时候都是受浏览器信任的证书。使用下面的 openssl 命令生成 CA 证书：
openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt
现在我们有了证书，创建 secret 资源:
$ kubectl create secret  generic ingressnginxcert  --from-file=tls.crt --from-file=tls.key -n kube-system

查看:
[root@k8s-master01 ~]# kubectl describe secret ingressnginxcert
Name:         ingressnginxcert
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
tls.crt:  1220 bytes
tls.key:  1704 bytes

[root@k8s-master01 ~]# kubectl get ing
NAME               CLASS    HOSTS                                                      ADDRESS   PORTS     AGE
ingressssl         <none>   blogs.devopstack.com                                                 80, 443   12h

绑定hosts
cat /etc/hosts
192.168.10.72 blogs.devopstack.com

验证结果:
k8s基础入门之外部服务发现之 ingress

查看证书

k8s基础入门之外部服务发现之 ingress

5.4 annotations对ingress个性化配置

名称 描述 值
nginx.ingress.kubernetes.io/rewrite-target 必须重定向流量的目标URL value
nginx.ingress.kubernetes.io/ssl-redirect 指示位置部分是否仅可访问SSL(当ingress包含证书时，默认为True) 布尔
nginx.ingress.kubernetes.io/force-redirect 即使Ingress未启用TLS，也强制重定向到HTTPS 布尔
nginx.ingress.kubernetes.io/app-root 定义Controller必须重定向的应用程序根，如果它在"/"上下文中 value
nginx.ingress.kubernetes.io/use-regex 指示 Ingress 上定义的路径是否使用正则表达式 布尔
参考: https://kubernetes.github.io/ingress-nginx/examples/rewrite/

annotations用法:

https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md

nginx配置
https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/index.md
实例:

 my-ingress_default.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    name: my-ingress
    annotations:
        kubernetes.io/ingress.class: "nginx"  #选择ingress控制器类型
        nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-body-size: "100m"
        nginx.ingress.kubernetes.io/ssl-redirect: "true" #http->https
        nginx.ingress.kubernetes.io/server-snippet: |     ##可以通过这个参数加nginx具体配置
            set  agentflag 0;
            if (http_user_agent ~* "(Mobile)"){
               set agentflag 1;
            }
             if (agentflag  =1 ){
              return 301 https://default.devopstack.cn;
            }
 spec:
    backend:
        serviceName: web
        servicePort: 80

6 ingress controller高可用方案

1.固定ingress controller 到两个node上(daemonset+nodeselector)->user-> 域名-> vip(keepalive)ha->pod 2. 固定ingress controller 到两个node上(daemonset+nodeselector)->user-> 域名-> lB(nginx,lvs,haproxy)-> ingress controller -> pod

7 Traefik

  • 我的微信
  • 这是我的微信扫一扫
  • weinxin
  • 我的微信公众号
  • 我的微信公众号扫一扫
  • weinxin
avatar

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: