k8s基础入门之外部服务发现之 ingress

1 ingress介绍

Ingress就是一组基于DNS名称（host）或URL路径把请求转发至指定的Service资源的规则，用于将集群外部的请求流量转发至集群内部完成服务发布。然而，Ingress资源自身并不能进行“流量穿透”，它仅是一组路由规则的集合，这些规则要想真正发挥作用还需要其他功能的辅助，如监听某套接字，然后根据这些规则的匹配机制路由请求流量。这种能够为Ingress资源监听套接字并转发流量的组件称为Ingress控制器（Ingress Controller）。 Ingress控制器并不直接运行为kube-controller-manager的一部分，它是Kubernetes集群的一个重要附件，类似于CoreDNS，需要在集群上单独部署。 Ingress控制器可以由任何具有反向代理（HTTP/HTTPS）功能的服务程序实现，如Nginx、Envoy、HAProxy、Vulcand和Traefik等。Ingress控制器自身也是运行于集群中的Pod资源对象，它与被代理的运行为Pod资源的应用运行于同一网络中 使用Ingress资源进行流量分发时，Ingress控制器可基于某Ingress资源定义的规则将客户端的请求流量直接转发至与Service对应的后端Pod资源之上，这种转发机制会绕过Service资源，从而省去了由kube-proxy实现的端口代理开销。

2 pod和ingress的关系

  • 通过label-selector相关联
  • 通过Ingress Controller实现Pod的负载均衡
  • 支持TCP/UDP 4层和HTTP 7层

3 ingress流程图

k8s基础入门之外部服务发现之 ingress

4 部署ingress-nginx controller

本次实验使用ingress-nginx部署 其他主流控制器 traefik: http反向代理,负载均衡工具 istio: 服务治理,控制入口流量.
k8s基础入门之外部服务发现之 ingress 
下载对应的yaml
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/mandatory.yaml
内容如下
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

修改
  spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1
          args:

创建ingress controller
[root@k8s-master01 ~]# kubectl apply -f mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
[root@k8s-master01 ~]# 

查看pod创建情况
[root@k8s-master01 ~]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-6d8d7c675b-r7bxl   1/1     Running   0          89s

5 ingress的使用

单Service资源型Ingress
使用Ingress来暴露服务，此时只需要为Ingress指定“default backend”即可
[root@k8s-master01 ~]# cat my-ingress_default.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    name: my-ingress
spec:
    backend:
        serviceName: web
        servicePort: 80
[root@k8s-master01 ~]# kubectl apply -f my-ingress_default.yaml
ingress.extensions/my-ingress created

访问查看

k8s基础入门之外部服务发现之 ingress

5.1 基于URL路径进行流量分发


[root@k8s-master01 ~]# cat myapp03.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: svc3
  labels:
    app: svc3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: svc3
  template:
    metadata:
      labels:
        app: svc3
    spec:
      containers:
      - name: svc3
        image: cnych/example-web-service
        env:
        - name: APP_SVC
          value: svc3
        ports:
        - containerPort: 8080
          protocol: TCP

kind: Service
apiVersion: v1
metadata:
  labels:
    app: svc3
  name: svc3
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: svc3

创建测试的svc和pods
[root@k8s-master01 ~]# kubectl apply -f myapp01.yaml 
service/svc1 created
[root@k8s-master01 ~]# kubectl apply -f myapp02.yaml 
service/svc2 created
[root@k8s-master01 ~]# kubectl apply -f myapp03.yaml 
service/svc3 created

查看svc
[root@k8s-master01 ~]# kubectl get svc |grep svc
svc1         ClusterIP   10.110.207.5     <none>        8080/TCP       2m54s
svc2         ClusterIP   10.110.235.231   <none>        8080/TCP       87s
svc3         ClusterIP   10.98.100.177    <none>        8080/TCP       83s

查看pods
[root@k8s-master01 ~]# kubectl get pods  |grep svc
svc1-5ff98455f6-8g78g    1/1     Running     0          19m
svc2-678c5fb65-rnt8t     1/1     Running     0          56s
svc3-55569d594-59zjf     1/1     Running     0          23s

创建一个 ingress 对象来访问上面的3个服务
[root@k8s-master01 ~]# cat ingress_app.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-web-url
  annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: ingress.devopstack.com
    http:
      paths:
      - path: /s1
        backend:
          serviceName: svc1
          servicePort: 8080
      - path: /s2
        backend:
          serviceName: svc2
          servicePort: 8080
      - path: /s3
        backend:
          serviceName: svc3
          servicePort: 8080
创建对应的ingress
[root@k8s-master01 ~]# kubectl apply -f ingress_app.yaml 
ingress.extensions/ingress-web-url created

验证
[root@k8s-master01 ~]# kubectl get ingress
NAME              CLASS    HOSTS                    ADDRESS   PORTS   AGE
ingress-web-url   <none>   ingress.devopstack.com             80      6m45s

查看具体规则内容
[root@k8s-master01 ~]# kubectl describe ingress ingress-web-url
Name:             ingress-web-url
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                    Path  Backends
  ----                    ----  --------
  ingress.devopstack.com  
                          /s1   svc1:8080 (10.244.2.181:8080)
                          /s2   svc2:8080 (10.244.2.218:8080)
                          /s3   svc3:8080 (10.244.2.221:8080)
Annotations:              nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  7m47s  nginx-ingress-controller  Ingress default/ingress-web-url
  Normal  CREATE  7m47s  nginx-ingress-controller  Ingress default/ingress-web-url

访问验证下
upsmart@upsmart-System-Product-Name:~$ cat /etc/hosts
192.168.10.72 ingress.devopstack.com

k8s基础入门之外部服务发现之 ingress k8s基础入门之外部服务发现之 ingress k8s基础入门之外部服务发现之 ingress

5.2 基于主机名称的虚拟主机

主机名通配符

主机名可以是精确匹配（例如“foo.bar.com”）或者使用通配符来匹配 （例如“_.foo.com”）。 精确匹配要求 HTTP host 头部字段与 host 字段值完全匹配。 通配符匹配则要求 HTTP host 头部字段与通配符规则中的后缀部分相同。
主机 host 头部 匹配与否？
_.foo.com bar.foo.com 基于相同的后缀匹配
_.foo.com baz.bar.foo.com 不匹配，通配符仅覆盖了一个 DNS 标签
_.foo.com foo.com 不匹配，通配符仅覆盖了一个 DNS 标签

实例:
[root@k8s-master01 ~]# cat ingress_app_host.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-web-host
spec:
  rules:
  - host: svc1.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
            serviceName: svc1
            servicePort: 8080

  - host: svc2.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
            serviceName: svc2
            servicePort: 8080

  - host: svc3.devopstack.cn
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          serviceName: svc3
          servicePort: 8080

执行yaml生成对应的ingress策略
[root@k8s-master01 ~]# kubectl apply -f ingress_app_host.yaml 
ingress.extensions/ingress-web-host created

查看ingress
[root@k8s-master01 ~]# kubectl get ingress
NAME               CLASS    HOSTS                                                      ADDRESS   PORTS   AGE
ingress-web-host   <none>   svc1.devopstack.cn,svc2.devopstack.cn,svc3.devopstack.cn             80      7s

查看 ingress策略详情
[root@k8s-master01 ~]# kubectl describe ingress ingress-web-host
Name:             ingress-web-host
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  svc1.devopstack.cn  
                      /   svc1:8080 (10.244.2.181:8080)
  svc2.devopstack.cn  
                      /   svc2:8080 (10.244.2.218:8080)
  svc3.devopstack.cn  
                      /   svc3:8080 (10.244.2.221:8080)
Annotations:          Events:
  Type                Reason  Age    From                      Message
  ----                ------  ----   ----                      -------
  Normal              CREATE  2m21s  nginx-ingress-controller  Ingress default/ingress-web-host
  Normal              CREATE  2m21s  nginx-ingress-controller  Ingress default/ingress-web-host

查看nginx-crontroller具体策略
kubectl  exec -it nginx-ingress-controller-b5p69 bash -n ingress-nginx

绑定 cat /etc/hosts
192.168.10.73 svc1.devopstack.cn
192.168.10.73 svc2.devopstack.cn
192.168.10.73 svc3.devopstack.cn

浏览器访问测试
k8s基础入门之外部服务发现之 ingress k8s基础入门之外部服务发现之 ingress k8s基础入门之外部服务发现之 ingress

5.3 Ingress https代理

TLS 认证
在现在大部分场景下面我们都会使用 https 来访问我们的服务，这节课我们将使用一个自签名的证书，当然你有在一些正规机构购买的 CA 证书是最好的，这样任何人访问你的服务的时候都是受浏览器信任的证书。使用下面的 openssl 命令生成 CA 证书：
openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt
现在我们有了证书，创建 secret 资源:
$ kubectl create secret  generic ingressnginxcert  --from-file=tls.crt --from-file=tls.key -n kube-system

查看:
[root@k8s-master01 ~]# kubectl describe secret ingressnginxcert
Name:         ingressnginxcert
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
tls.crt:  1220 bytes
tls.key:  1704 bytes

[root@k8s-master01 ~]# kubectl get ing
NAME               CLASS    HOSTS                                                      ADDRESS   PORTS     AGE
ingressssl         <none>   blogs.devopstack.com                                                 80, 443   12h

绑定hosts
cat /etc/hosts
192.168.10.72 blogs.devopstack.com

验证结果:
k8s基础入门之外部服务发现之 ingress

查看证书

k8s基础入门之外部服务发现之 ingress

5.4 annotations对ingress个性化配置

名称 描述 值
nginx.ingress.kubernetes.io/rewrite-target 必须重定向流量的目标URL value
nginx.ingress.kubernetes.io/ssl-redirect 指示位置部分是否仅可访问SSL(当ingress包含证书时，默认为True) 布尔
nginx.ingress.kubernetes.io/force-redirect 即使Ingress未启用TLS，也强制重定向到HTTPS 布尔
nginx.ingress.kubernetes.io/app-root 定义Controller必须重定向的应用程序根，如果它在"/"上下文中 value
nginx.ingress.kubernetes.io/use-regex 指示 Ingress 上定义的路径是否使用正则表达式 布尔
参考: https://kubernetes.github.io/ingress-nginx/examples/rewrite/

annotations用法:

https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md

nginx配置
https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/index.md
实例:

 my-ingress_default.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    name: my-ingress
    annotations:
        kubernetes.io/ingress.class: "nginx"  #选择ingress控制器类型
        nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
        nginx.ingress.kubernetes.io/proxy-body-size: "100m"
        nginx.ingress.kubernetes.io/ssl-redirect: "true" #http->https
        nginx.ingress.kubernetes.io/server-snippet: |     ##可以通过这个参数加nginx具体配置
            set  $agentflag 0;
            if ($http_user_agent ~* "(Mobile)"){
               set $agentflag 1;
            }
             if ($agentflag  =1 ){
              return 301 https://default.devopstack.cn;
            }
 spec:
    backend:
        serviceName: web
        servicePort: 80

6 ingress controller高可用方案

1.固定ingress controller 到两个node上(daemonset+nodeselector)->user-> 域名-> vip(keepalive)ha->pod 2. 固定ingress controller 到两个node上(daemonset+nodeselector)->user-> 域名-> lB(nginx,lvs,haproxy)-> ingress controller -> pod

7 Traefik

  • 我的微信
  • 这是我的微信扫一扫
  • weinxin
  • 我的微信公众号
  • 我的微信公众号扫一扫
  • weinxin
avatar

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: