问题描述
我已经开始使用Cass-Operator,并且安装过程像个魅力! https://github.com/datastax/cass-operator .
I have started using Cass-Operator and the setup worked like a charm! https://github.com/datastax/cass-operator.
但是我有一个问题.我的集群已启动并在GCP上运行.但是,如何从笔记本电脑(基本上是从外部)访问它?抱歉,我是Kubernetes的新手,所以我不知道如何从外部访问集群?
I have an issue though. My cluster is up and running on GCP. But how do I access it from my laptop (basically from outside)? Sorry, I'm new to Kubernetes so I do not know how to access the cluster from outside?
我可以看到GCP仪表板上的节点都在上面.我可以从笔记本电脑ping节点的外部IP,但是当我运行cqlsh external_ip 9042
时,连接将失败.
I can see the nodes are up on the GCP dashboard. I can ping the external IP of the nodes from my laptop but when I run cqlsh external_ip 9042
then the connection fails.
如何将K8s/Cassandra群集连接到外部工作,以便我的Web应用程序可以访问它?
How do I go about connecting the K8s/Cassandra cluster to outside work so that my web application can access it?
我想:
- 具有一个URL,以便我的Web应用程序使用该URL而不是IP地址来连接到cassandra/K8s集群.因此,我需要一个DNS. K8S是否默认提供它?会是网址吗?在某些节点中为我管理dns映射的K8会重启吗?
- 我的Web应用程序应该能够在9042上访问Cassandra.似乎已经为http/https完成了负载平衡.该Cassandra应用程序不是http/https请求.所以我不需要端口80或443
我已经阅读了一些有关Service,Loadbalancer和Ingress的教程.但是我无法开始.
I have read few tutorials which talk about Service, Loadbalancer and Ingress. But I am unable to make a start.
我创建了这样的服务
kind: Service
apiVersion: v1
metadata:
name: cass-operator-service
spec:
type: LoadBalancer
ports:
- port: 9042
selector:
name: cass-operator
然后创建服务-kubectl apply -f ./cass-operator-service.yaml
我检查了服务是否使用kubectl get svc
创建并获得了输出
I checked if the service was created using kubectl get svc
and got output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cass-operator-service LoadBalancer 10.51.249.224 34.91.214.233 9042:30136/TCP 4m17s
kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 10h.
但是当我运行cqlsh 34.91.214.233 9042
时,连接失败
But when I run cqlsh 34.91.214.233 9042
then the connection fails
似乎将端口9042的请求转发到30136.但是应该将它们转发到9042,因为那是Pod中的Cassandra映像侦听传入请求的地方
It seems that the requests to port 9042 would be forwarded to 30136. But They should be forwarded to 9042 as that is where the Cassandra image in the pods is listening for incoming requests
更新
尝试了targetPort但还是没有运气
Tried targetPort but still no luck
manuchadha25@cloudshell:~ (copper-frame-262317)$ cat cass-operator-service.yaml
kind: Service
apiVersion: v1
metadata:
name: cass-operator-service
spec:
type: LoadBalancer
ports:
- port: 9042
targetPort: 9042
selector:
name: cass-operator
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl apply -f ./cass-operator-service.yaml
service/cass-operator-service created
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cass-operator-service LoadBalancer 10.51.255.184 <pending> 9042:30024/TCP 12s
kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cass-operator-service LoadBalancer 10.51.255.184 <pending> 9042:30024/TCP 37s
kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cass-operator-service LoadBalancer 10.51.255.184 34.91.214.233 9042:30024/TCP 67s
kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ ping 34.91.214.233
PING 34.91.214.233 (34.91.214.233) 56(84) bytes of data.
64 bytes from 34.91.214.233: icmp_seq=1 ttl=109 time=7.89 ms
查询所有名称空间将显示以下内容
Querying all names spaces reveal the following
但是使用命名空间cass-operator查询pod会返回空结果
But querying pods with namespace cass-operator returns empty result
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get pods -l name=cass-operator
No resources found in default namespace.
推荐答案
- 由于您是Kubernetes的新手,所以您可能不熟悉 StatefulSets :
- Since you are new to Kubernetes, you probably are not familiar with StatefulSets:
- 我建议您阅读以下文章,以了解有关其机制的更多信息:
- Kubernetes.io-Statefulsets
- Megalix-Statefulsets 101
- ITnext-在Kubernetes中公开状态集
- I recommend you to read these articles to learn more about it's mechanisms:
- Kubernetes.io - Statefulsets
- Megalix - Statefulsets 101
- ITNext - Exposing Statefulsets in Kubernetes
- 我发现datastax/cass-operator仍在开发其文档,我发现此文档尚未合并到主文档中,但是它很好地解释了如何连接到Cassandra,我强烈建议您阅读.
- 有几个未解决的问题,用于记录从群集外部进行连接的方法.
- I found out that datastax/cass-operator is still developing their documentation, I found this document that is not merged to master yet, but it explains very well about how to connect to Cassandra, I strongly recommend reading.
- There are several open issues for documenting methods for connection from outside the cluster.
我在 https://github.com/datastax/cass-operator从映像中部署cass-operator + Cassandra数据中心示例,我相信您也遵循:
I followed the guide in https://github.com/datastax/cass-operator to deploy the cass-operator + Cassandra Datacenter Example as from your images I believe you followed as well:
$ kubectl create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/docs/user/cass-operator-manifests-v1.15.yaml namespace/cass-operator created serviceaccount/cass-operator created secret/cass-operator-webhook-config created customresourcedefinition.apiextensions.k8s.io/cassandradatacenters.cassandra.datastax.com created clusterrole.rbac.authorization.k8s.io/cass-operator-cluster-role created clusterrolebinding.rbac.authorization.k8s.io/cass-operator created role.rbac.authorization.k8s.io/cass-operator created rolebinding.rbac.authorization.k8s.io/cass-operator created service/cassandradatacenter-webhook-service created deployment.apps/cass-operator created validatingwebhookconfiguration.admissionregistration.k8s.io/cassandradatacenter-webhook-registration created $ kubectl create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/operator/k8s-flavors/gke/storage.yaml storageclass.storage.k8s.io/server-storage created $ kubectl -n cass-operator create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/operator/example-cassdc-yaml/cassandra-3.11.6/example-cassdc-minimal.yaml cassandradatacenter.cassandra.datastax.com/dc1 created $ kubectl get all -n cass-operator NAME READY STATUS RESTARTS AGE pod/cass-operator-78c6469c6-6qhsb 1/1 Running 0 139m pod/cluster1-dc1-default-sts-0 2/2 Running 0 138m pod/cluster1-dc1-default-sts-1 2/2 Running 0 138m pod/cluster1-dc1-default-sts-2 2/2 Running 0 138m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/cass-operator-metrics ClusterIP 10.21.5.65 <none> 8383/TCP,8686/TCP 138m service/cassandradatacenter-webhook-service ClusterIP 10.21.0.89 <none> 443/TCP 139m service/cluster1-dc1-all-pods-service ClusterIP None <none> <none> 138m service/cluster1-dc1-service ClusterIP None <none> 9042/TCP,8080/TCP 138m service/cluster1-seed-service ClusterIP None <none> <none> 138m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/cass-operator 1/1 1 1 139m NAME DESIRED CURRENT READY AGE replicaset.apps/cass-operator-78c6469c6 1 1 1 139m NAME READY AGE statefulset.apps/cluster1-dc1-default-sts 3/3 138m $ CASS_USER=$(kubectl -n cass-operator get secret cluster1-superuser -o json | jq -r '.data.username' | base64 --decode) $ CASS_PASS=$(kubectl -n cass-operator get secret cluster1-superuser -o json | jq -r '.data.password' | base64 --decode) $ echo $CASS_USER cluster1-superuser $ echo $CASS_PASS _5ROwp851l0E_2CGuN_n753E-zvEmo5oy31i6C0DBcyIwH5vFjB8_g
- 从上面的
kubectl get all
命令中,我们可以看到有一个名为statefulset.apps/cluster1-dc1-default-sts
的状态集,该状态集可以控制卡桑德拉豆荚. - 为了创建一个LoadBalancer服务,使该
statefulset
管理的所有Pod都可用,我们需要使用分配给它们的相同标签: - From the
kubectl get all
command above we can see there is an statefulset calledstatefulset.apps/cluster1-dc1-default-sts
which controls the cassandra pods. - In order to create a LoadBalancer service that makes available all the pods managed by this
statefulset
we need to use the same labels assigned to them:
$ kubectl describe statefulset cluster1-dc1-default-sts -n cass-operator Name: cluster1-dc1-default-sts Namespace: cass-operator CreationTimestamp: Tue, 30 Jun 2020 12:24:34 +0200 Selector: cassandra.datastax.com/cluster=cluster1,cassandra.datastax.com/datacenter=dc1,cassandra.datastax.com/rack=default Labels: app.kubernetes.io/managed-by=cass-operator cassandra.datastax.com/cluster=cluster1 cassandra.datastax.com/datacenter=dc1 cassandra.datastax.com/rack=default
- 现在让我们创建LoadBalancer服务yaml,并将该标签用作该服务的
selectors
: - Now let's create the LoadBalancer service yaml and use the labels as
selectors
for the service:
apiVersion: v1 kind: Service metadata: name: cassandra-loadbalancer namespace: cass-operator labels: cassandra.datastax.com/cluster: cluster1 cassandra.datastax.com/datacenter: dc1 cassandra.datastax.com/rack: default spec: type: LoadBalancer ports: - port: 9042 protocol: TCP selector: cassandra.datastax.com/cluster: cluster1 cassandra.datastax.com/datacenter: dc1 cassandra.datastax.com/rack: default
-
创建类型为
LoadBalancer
的服务时,Google Cloud控制器将唤醒并配置网络负载平衡器在您的项目中.负载平衡器具有一个稳定的IP地址,可以从您的项目外部访问该地址.When you create a Service of type
LoadBalancer
, a Google Cloud controller wakes up and configures a network load balancer in your project. The load balancer has a stable IP address that is accessible from outside of your project.网络负载平衡器支持任何和所有端口.您可以使用网络负载平衡来负载平衡TCP和UDP通信.由于负载平衡器是传递负载平衡器,因此后端将终止负载平衡的TCP连接或UDP数据包本身.
The network load balancer supports any and all ports. You can use Network Load Balancing to load balance TCP and UDP traffic. Because the load balancer is a pass-through load balancer, your backends terminate the load-balanced TCP connection or UDP packets themselves.
现在,我们应用yaml并注意列出的Pod的端点IP:
Now let's apply the yaml and note the Endpoint IPs of the pods being listed:
$ kubectl apply -f cassandra-loadbalancer.yaml service/cassandra-loadbalancer created $ kubectl get service cassandra-loadbalancer -n cass-operator NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cassandra-loadbalancer LoadBalancer 10.21.4.253 146.148.89.7 9042:30786/TCP 5m13s $ kubectl describe svc cassandra-loadbalancer -n cass-operator Name: cassandra-loadbalancer Namespace: cass-operator Labels: cassandra.datastax.com/cluster=cluster1 cassandra.datastax.com/datacenter=dc1 cassandra.datastax.com/rack=default Annotations: Selector: cassandra.datastax.com/cluster=cluster1,cassandra.datastax.com/datacenter=dc1,cassandra.datastax.com/rack=default Type: LoadBalancer IP: 10.21.4.253 LoadBalancer Ingress: 146.148.89.7 Port: <unset> 9042/TCP TargetPort: 9042/TCP NodePort: <unset> 30786/TCP Endpoints: 10.24.0.7:9042,10.24.2.7:9042,10.24.3.9:9042 Session Affinity: None External Traffic Policy: Cluster Events: <none>
- 要进行测试,我将使用带有Cassandra容器的云外壳,使用上面提供的
LoadBalancer
IP来模拟您的笔记本: - To test it, I'll use my cloud shell with a cassandra container to emulate your notebook using the
LoadBalancer
IP provided above:
$ docker run -it cassandra /bin/sh # cqlsh -u cluster1-superuser -p _5ROwp851l0E_2CGuN_n753E-zvEmo5oy31i6C0DBcyIwH5vFjB8_g 146.148.89.7 9042 Connected to cluster1 at 146.148.89.7:9042. [cqlsh 5.0.1 | Cassandra 3.11.6 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. cluster1-superuser@cqlsh> select * from system.peers; peer | data_center | host_id | preferred_ip | rack | release_version | rpc_address | schema_version | tokens -----------+-------------+--------------------------------------+--------------+---------+-----------------+-------------+--------------------------------------+-------------------------- 10.24.3.9 | dc1 | bcec6c12-49a1-41d5-be58-5150e99f5dfb | null | default | 3.11.6 | 10.24.3.9 | e84b6a60-24cf-30ca-9b58-452d92911703 | {'2248175870989649036'} 10.24.0.7 | dc1 | 68409f08-9d6e-4e40-91ff-f43581c8b6f3 | null | default | 3.11.6 | 10.24.0.7 | e84b6a60-24cf-30ca-9b58-452d92911703 | {'-1105923522927946373'} (2 rows)
- 关于cassandra-operator的文档也有关于的部分. Ingress ,我也建议阅读.
- Kubernetes没有默认的DNS名称.
- 您将必须注册一个域,将DNS指向负载均衡器的IP,这样它将解析网络LoadBalancer的IP.
- 网络负载平衡器已绑定到静态公共IP,Kubernetes节点中的任何更改都不会导致服务不可用.
- That documentation on cassandra-operator also has a section about Ingress, I recommend reading as well.
- Kubernetes does not come with a default DNS name.
- You will have to register a domain, point the DNS to the IP of the load balancer this way it will resolve the IP of the Network LoadBalancer.
- The Network LoadBalancer is bound to a Static Public IP, any changes in Kubernetes nodes will not cause service unavailability.
如果您有任何问题,请在评论中告诉我.
If you have any question, let me know in the comments.
这篇关于如何从集群外部访问我的Cassandra/Kubernetes集群?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!
- 要进行测试,我将使用带有Cassandra容器的云外壳,使用上面提供的
管理一组 Pods 的部署和扩展,并提供有关这些Pod的顺序和唯一性的保证.
Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods.
像部署一样,StatefulSet管理基于相同容器规范的Pod. 与部署不同,StatefulSet为其每个Pod维护一个粘性标识.这些Pod是根据相同的规范创建的,但不能互换:每个Pod都有一个永久性标识符,可以在任何重新计划中维护.
Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.