of

of

关注
发信
关注(28)粉丝(399)

同一K8S集群上的两个入口控制器

本文介绍了同一K8S集群上的两个入口控制器的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我在DigitalOcean管理的K8S群集上安装了以下两个不同的入口控制器:

I have installed the following two different ingress controllers on my DigitalOcean managed K8S cluster:

  • Nginx

  • Nginx

Istio

，并且它们已分配给两个不同的IP地址.我的问题是，在同一个K8S集群上拥有两个不同的入口控制器是否错误?

and they have been assigned to two different IP addresses. My question is, if it is wrong to have two different ingress controllers on the same K8S cluster?

之所以这样做，是因为nginx用于harbour，argocd等工具，而istio用于微服务.

The reason, why I have done it, because nginx is for tools like harbor, argocd, etc. and istio for microservices.

我还发现，当两者并排安装时，有时在部署过程中，K8S突然掉线了.

I have also figured out, when both are installed alongside each other, sometimes during the deployment, the K8S suddenly goes down.

例如，我已经部署:

apiVersion: v1
kind: Service
metadata:
  name: hello-kubernetes-first
  namespace: dev
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: hello-kubernetes-first
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-kubernetes-first
  namespace: dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-kubernetes-first
  template:
    metadata:
      labels:
        app: hello-kubernetes-first
    spec:
      containers:
        - name: hello-kubernetes
          image: paulbouwer/hello-kubernetes:1.7
          ports:
            - containerPort: 8080
          env:
            - name: MESSAGE
              value: Hello from the first deployment!
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: istio
  name: helloworld-ingress
  namespace: dev
spec:
  rules:
    - host: hello.service.databaker.io
      http:
        paths:
          - path: /*
            backend:
              serviceName: hello-kubernetes-first
              servicePort: 80
---

然后我得到了:

Error from server (InternalError): error when creating "istio-app.yml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://ingress-nginx-controller-admission.nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s: dial tcp 10.245.107.175:443: i/o timeout

推荐答案

您提出了几点-在回答问题之前，让我们退后一步.

You have raised several points - before answering your question, let's take a step back.

重要的是要注意Istio不建议使用K8s Ingress:

It is important to note how Istio does not recommend using K8s Ingress:

参考: https://istio.io/Latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/

如前所述，Istio网关(Istio IngressGateway和EgressGateway)充当边缘，您可以在"> https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/.

As noted, Istio Gateway (Istio IngressGateway and EgressGateway) acts as the edge, which you can find more in https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/.

如果您需要为业务需求分配一个公共端点，而为监视分配另一个公共端点(如您提到的Argo CD，Harbor)，则可以仅使用Istio来实现.大约有两种方法.

If you need to assign one public endpoint for business requirement, and another for monitoring (such as Argo CD, Harbor as you mentioned), you can achieve that by using Istio only. There are roughly 2 approaches to this.

  1. 创建单独的Istio IngressGateways-一个用于主要流量，另一个用于监视
  2. 创建一个Istio IngressGateway，然后使用网关定义处理多种访问模式
  1. Create separate Istio IngressGateways - one for main traffic, and another for monitoring
  2. Create one Istio IngressGateway, and use Gateway definition to handle multiple access patterns

这两种方法都是有效的，并且根据要求，您可能需要选择一种或另一种方法.

Both are valid, and depending on requirements, you may need to choose one way or the other.

关于方法2，正是Istio的流量管理系统大放异彩的地方.这是Istio强大功能的一个很好的例子，但是如果您不熟悉Istio，它的设置会有些复杂.因此，这里有一个例子.

As to the Approach #2., it is where Istio's traffic management system shines. It is a great example of Istio's power, but the setup is slightly complex if you are new to it. So here goes an example.

方法2的示例

按照默认安装创建Istio IngressGateway时，它会会像下面那样创建istio-ingressgateway(我过于简化了YAML定义):

When you create Istio IngressGateway by following the default installation, it would create istio-ingressgateway like below (I overly simplified YAML definition):

apiVersion: v1
kind: Service
metadata:
  labels:
    app: istio-ingressgateway
    istio: ingressgateway
  name: istio-ingressgateway
  namespace: istio-system
  # ... other attributes ...
spec:
  type: LoadBalancer
  # ... other attributes ...

此LB服务将成为您的端点. (我对DigitalOcean K8s env并不熟悉，但我想他们会处理LB的创建.)

This LB Service would then be your endpoint. (I'm not familiar with DigitalOcean K8s env, but I suppose they would handle LB creation.)

然后，您可以创建如下的网关定义:

Then, you can create Gateway definition like below:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: your-gateway
  namespace: istio-system
spec:
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  servers:
    - port:
        number: 3000
        name: https-your-system
        protocol: HTTPS
      hosts:
        - "your-business-domain.com"
        - "*.monitoring-domain.com"
      # ... other attributes ...

然后您可以创建2个或多个 VirtualService 定义.

You can then create 2 or more VirtualService definitions.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: business-virtsvc
spec:
  gateways:
    - istio-ingressgateway.istio-system.svc.cluster.local
  hosts:
    - "your-business-domain.com"
  http:
    - match:
        - port: 3000
      route:
        - destination:
            host: some-business-pod
            port:
              number: 3000
    # ... other attributes ...

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: monitoring-virtsvc
spec:
  gateways:
    - istio-ingressgateway.istio-system.svc.cluster.local
  hosts:
    - "harbor.monitoring-domain.com"
  http:
    - match:
        - port: 3000
      route:
        - destination:
            host: harbor-pod
            port:
              number: 3000
    # ... other attributes ...

注意:以上假设是很多事情，例如端口映射，流量处理等.请查看官方文档以了解详细信息.

NOTE: The above is assuming a lot of things, such as port mapping, traffic handling, etc.. Please check out the official doc for details.

所以，回到长途绕道之后的问题:

So, back to the question after long detour:

我相信这是可以的，尽管这可能会导致错误，就像您看到的那样，因为两个Ingress控制器争夺K8s Ingress资源.

I believe it is OK, though this can cause an error like you are seeing, as two ingress controller fight for the K8s Ingress resource.

如上所述，如果您使用的是Istio，最好坚持使用Istio IngressGateway而不是K8s Ingress.如果出于某些特定原因需要K8s Ingress，则可以将其他Ingress控制器用于K8s Ingress，例如Nginx.

As mentioned above, if you are using Istio, it's better to stick with Istio IngressGateway instead of K8s Ingress. If you need K8s Ingress for some specific reason, you could use other Ingress controller for K8s Ingress, like Nginx.

关于您看到的错误，它来自Nginx部署的Webhook，ingress-nginx-controller-admission.nginx.svc不可用.这意味着您已经创建了带有kubernetes.io/ingress.class: istio批注的K8s Ingress helloworld-ingress，但是Nginx webhook正在干扰K8s Ingress处理.然后Webhook无法处理资源，因为未找到负责Webhook流量的Pod/Svc.

As to the error you saw, it's coming from Nginx deployed webhook, that ingress-nginx-controller-admission.nginx.svc is not available. This means you have created a K8s Ingress helloworld-ingress with kubernetes.io/ingress.class: istio annotation, but Nginx webhook is interfering with K8s Ingress handling. The webhook is then failing to handle the resource, as the Pod / Svc responsible for webhook traffic is not found.

错误本身仅表明K8中存在某些问题-可能没有足够的Node分配给集群，因此Pod分配没有发生.还需要注意的是，Istio确实需要占用一些CPU和内存，这可能会给群集带来更多压力.

The error itself just says something is unhealthy in K8s - potentially not enough Node allocated to the cluster, and thus Pod allocation not happening. It's also good to note that Istio does require some CPU and memory footprint, which may be putting more pressure to the cluster.

这篇关于同一K8S集群上的两个入口控制器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

09-01 21:11
Powered by LMLPHP ©2024 of 0.037247