Skip to content

Commit c83d3cd

Browse files
leandroberettajmazzitelli
leandroberetta
and
jmazzitelli
authored
Multicluster tutorial (#681)
* tutorial * mc tutorial * fixes * addressing feedback * Update content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md Co-authored-by: John Mazzitelli <mazz@redhat.com> * Apply suggestions from code review Co-authored-by: John Mazzitelli <mazz@redhat.com> --------- Co-authored-by: John Mazzitelli <mazz@redhat.com>
1 parent 4aeb588 commit c83d3cd

16 files changed

+477
-2
lines changed

‎content/en/docs/Tutorials/multicluster/01-Introduction.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
title: "Introduction"
3+
description: "Observe the Travels application deployed in multiple clusters with the new capabilities of Kiali."
4+
weight: 1
5+
---
6+
7+
So far, we know how good Kiali can be to understand applications, their relationships with itself and also with external applications.
8+
9+
In the past, Kiali was installed just to observe one cluster with all the applications that conforms to it. Today, we are expanding its capabilities to also observe more than one cluster. The extra clusters are remotes, meaning that there is not a control plane on them, they only have user applications.
10+
11+
This topology is called [primary-remote](http://istio.io/latest/docs/setup/install/multicluster/primary-remote/) and it is very useful to spread applications into different clusters having just one primary cluster, which is where Istio and Kiali are installed.
12+
13+
This scenario is a good choice when as an application administrator or architect, you want to give a different set of clusters to different sets of developers and you also want that all these applications belong to the same mesh. This scenario is also very helpful to give applications high availability capabilities while keeping the observability together (we are referring to just applications in terms of high availability, for Istio, we might want to install a multi-primary deployment model, which is on the [roadmap](http://github.com/kiali/kiali/issues/5618) for the multicluster journey for Kiali).
14+
15+
At first, we will install one cluster with Istio, then we will add a new cluster, the remote, and we will join it to the mesh and we will see how Kiali allows us to observe and manage both of them and their applications.

‎content/en/docs/Tutorials/multicluster/02-Prerequisites.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
title: "Prerequisites"
3+
description: "How to prepare for running the tutorial."
4+
weight: 2
5+
---
6+
7+
This tutorial is a walkthrough guide to install everything. For this reason, we will need:
8+
9+
* minikube
10+
* istioctl
11+
* helm
12+
13+
This tutorial was tested on:
14+
15+
* Minikube v1.30.1
16+
* Istio v1.18.1
17+
* Kiali v1.70
18+
19+
Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations.
20+
21+
We will set up some environment variables for the following commands:
22+
23+
```
24+
CLUSTER_EAST="east"
25+
CLUSTER_WEST="west"
26+
ISTIO_DIR="absolute-path-to-istio-folder"
27+
```
28+
29+
As Istio will be installed on more than one cluster and needs to communicate between clusters, we need to create certificates for the Istio installation. We will follow the [Istio documentation related to certificates](http://istio.io/latest/docs/tasks/security/cert-management/plugin-ca-cert/) to achieve this:
30+
31+
```
32+
mkdir -p certs
33+
pushd certs
34+
35+
make -f $ISTIO_DIR/tools/certs/Makefile.selfsigned.mk root-ca
36+
37+
make -f $ISTIO_DIR/tools/certs/Makefile.selfsigned.mk $CLUSTER_EAST-cacerts
38+
make -f $ISTIO_DIR/tools/certs/Makefile.selfsigned.mk $CLUSTER_WEST-cacerts
39+
40+
popd
41+
```
42+
43+
The result is two certificates for then use when installing Istio in the future.

‎content/en/docs/Tutorials/multicluster/03-Deploy-east-cluster.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: "Deploy East cluster"
3+
description: "Deploy the East cluster which will be the primary cluster"
4+
weight: 3
5+
---
6+
7+
Run the following commands to deploy the first cluster:
8+
9+
```
10+
minikube start -p $CLUSTER_EAST --network istio --memory 8g --cpus 4
11+
```
12+
13+
For both clusters, we need to configure MetalLB, which is a load balancer. This is because we need to assign an external IP to the required ingress gateways to enable cross cluster communication between Istio and the applications installed.
14+
15+
```
16+
minikube addons enable metallb -p $CLUSTER_EAST
17+
```
18+
19+
We set up some environment variables with IP ranges that MetalLB will then assign to the services:
20+
21+
```
22+
MINIKUBE_IP=$(minikube ip -p $CLUSTER_EAST)
23+
MINIKUBE_IP_NETWORK=$(echo $MINIKUBE_IP | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+)\.[0-9]+/\1/')
24+
MINIKUBE_LB_RANGE="${MINIKUBE_IP_NETWORK}.20-${MINIKUBE_IP_NETWORK}.29"
25+
26+
cat <<EOF | kubectl --context $CLUSTER_EAST apply -f -
27+
apiVersion: v1
28+
kind: ConfigMap
29+
metadata:
30+
namespace: metallb-system
31+
name: config
32+
data:
33+
config: |
34+
address-pools:
35+
- name: default
36+
protocol: layer2
37+
addresses: [${MINIKUBE_LB_RANGE}]
38+
EOF
39+
```
40+
41+
We should have the first cluster deployed and ready to use.

‎content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
title: "Install Istio on East cluster"
3+
description: "Install Istio on the primary cluster"
4+
weight: 4
5+
---
6+
7+
The east cluster is the primary one, consequently is where the istiod process will be installed alongside other applications like Kiali.
8+
9+
Run the following commands to install Istio:
10+
11+
```
12+
kubectl create namespace istio-system --context $CLUSTER_EAST
13+
14+
kubectl create secret generic cacerts -n istio-system --context $CLUSTER_EAST \
15+
--from-file=certs/$CLUSTER_EAST/ca-cert.pem \
16+
--from-file=certs/$CLUSTER_EAST/ca-key.pem \
17+
--from-file=certs/$CLUSTER_EAST/root-cert.pem \
18+
--from-file=certs/$CLUSTER_EAST/cert-chain.pem
19+
20+
kubectl --context=$CLUSTER_EAST label namespace istio-system topology.istio.io/network=network1
21+
22+
cat <<EOF > $CLUSTER_EAST.yaml
23+
apiVersion: install.istio.io/v1alpha1
24+
kind: IstioOperator
25+
spec:
26+
values:
27+
global:
28+
meshID: mesh1
29+
multiCluster:
30+
clusterName: $CLUSTER_EAST
31+
network: network1
32+
EOF
33+
34+
istioctl install -y --set values.pilot.env.EXTERNAL_ISTIOD=true --context=$CLUSTER_EAST -f $CLUSTER_EAST.yaml
35+
```
36+
37+
After the installation, we need to create what we called an “east-west” gateway. It’s an ingress gateway just for the cross cluster configuration as we are opting to use the installation for different networks (this will be the case in the majority of the production scenarios).
38+
39+
```
40+
$ISTIO_DIR/samples/multicluster/gen-eastwest-gateway.sh \
41+
--mesh mesh1 --cluster $CLUSTER_EAST --network network1 | \
42+
istioctl --context=$CLUSTER_EAST install -y -f -
43+
```
44+
45+
Then, we need to expose the istiod service as well as the applications for the cross cluster communication:
46+
47+
```
48+
kubectl apply --context=$CLUSTER_EAST -n istio-system -f \
49+
$ISTIO_DIR/samples/multicluster/expose-istiod.yaml
50+
51+
kubectl --context=$CLUSTER_EAST apply -n istio-system -f \
52+
$ISTIO_DIR/samples/multicluster/expose-services.yaml
53+
54+
export DISCOVERY_ADDRESS=$(kubectl \
55+
--context=$CLUSTER_EAST \
56+
-n istio-system get svc istio-eastwestgateway \
57+
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')
58+
```
59+
60+
Finally, we need to install Prometheus, which is important and required for Kiali to operate:
61+
62+
```
63+
kubectl --context $CLUSTER_EAST -n istio-system apply -f $ISTIO_DIR/samples/addons/prometheus.yaml
64+
```

‎content/en/docs/Tutorials/multicluster/05-Install-Kiali.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
title: "Install Kiali"
3+
description: "Install Kiali on the primary cluster"
4+
weight: 5
5+
---
6+
7+
Run the following command to install Kiali using the Kiali operator:
8+
9+
```
10+
kubectl config use-context $CLUSTER_EAST
11+
12+
helm upgrade --install --namespace istio-system --set auth.strategy=anonymous --set deployment.logger.log_level=debug --set deployment.ingress.enabled=true --repo http://kiali.org/helm-charts kiali-server kiali-server
13+
```
14+
15+
Verify that Kiali is running with the following command:
16+
17+
```
18+
istioctl dashboard kiali
19+
```
20+
21+
There are other alternatives to expose Kiali or other Addons in Istio. Check [Remotely Accessing Telemetry Addons for more information](http://istio.io/latest/docs/tasks/observability/gateways/).

‎content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
title: "Install Travels on East cluster"
3+
description: "Install the Travels application just on East cluster"
4+
weight: 6
5+
---
6+
7+
Run the following commands to install Travels application on east cluster:
8+
9+
```
10+
kubectl create namespace travel-agency --context $CLUSTER_EAST
11+
kubectl create namespace travel-portal --context $CLUSTER_EAST
12+
kubectl create namespace travel-control --context $CLUSTER_EAST
13+
14+
kubectl label namespace travel-agency istio-injection=enabled --context $CLUSTER_EAST
15+
kubectl label namespace travel-portal istio-injection=enabled --context $CLUSTER_EAST
16+
kubectl label namespace travel-control istio-injection=enabled --context $CLUSTER_EAST
17+
18+
kubectl apply -f <(curl -L http://raw.githubusercontent.com/kiali/demos/master/travels/travel_agency.yaml) -n travel-agency --context $CLUSTER_EAST
19+
kubectl apply -f <(curl -L http://raw.githubusercontent.com/kiali/demos/master/travels/travel_portal.yaml) -n travel-portal --context $CLUSTER_EAST
20+
kubectl apply -f <(curl -L http://raw.githubusercontent.com/kiali/demos/master/travels/travel_control.yaml) -n travel-control --context $CLUSTER_EAST
21+
```
22+
23+
After the installation, we can see that the Travels application is running on the east cluster:
24+
25+
![Overview](/images/mc-tutorial/01.png "Overview")
26+
27+
It is important to note that Kiali only observes one istio-system namespace as we did not configure it for multicluster yet.
28+
29+
Go to the Graph page and select the three namespaces related to the Travels demo in the namespace dropdown menu. This shows you the in-cluster traffic:
30+
31+
![Graph](/images/mc-tutorial/02.png "Graph")
32+
33+
So far, we installed everything on one cluster, similarly to the Travels tutorial for a single cluster.
34+
35+
Now we will expand this topology to include a remote cluster. As we commented this situation can be very common in a production scenario, either because we might want to split some applications into different clusters, generally because they are maintained by different developers or for high availability or just making applications available in other zones to reduce latencies.

‎content/en/docs/Tutorials/multicluster/07-Deploy-west-cluster.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
title: "Deploy West cluster"
3+
description: "Deploy the West cluster which will be the remote cluster"
4+
weight: 7
5+
---
6+
7+
Run the following commands to deploy the second cluster:
8+
9+
```
10+
minikube start -p $CLUSTER_WEST --network istio --memory 8g --cpus 4
11+
```
12+
13+
Similar to the east cluster, we configure MetalLB:
14+
15+
```
16+
minikube addons enable metallb -p $CLUSTER_WEST
17+
18+
MINIKUBE_IP=$(minikube ip -p $CLUSTER_WEST)
19+
MINIKUBE_IP_NETWORK=$(echo $MINIKUBE_IP | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+)\.[0-9]+/\1/')
20+
MINIKUBE_LB_RANGE="${MINIKUBE_IP_NETWORK}.30-${MINIKUBE_IP_NETWORK}.39"
21+
22+
cat <<EOF | kubectl --context $CLUSTER_WEST apply -f -
23+
apiVersion: v1
24+
kind: ConfigMap
25+
metadata:
26+
namespace: metallb-system
27+
name: config
28+
data:
29+
config: |
30+
address-pools:
31+
- name: default
32+
protocol: layer2
33+
addresses: [${MINIKUBE_LB_RANGE}]
34+
EOF
35+
```

‎content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
title: "Install Istio on West cluster"
3+
description: "Install Istio on the remote cluster"
4+
weight: 8
5+
---
6+
7+
This installation will be different as this cluster will be a remote. In a remote cluster, it won't be an Istio control plane. Istio will install some resources that allows the primary control plane to configure the workloads in the remote cluster like injecting the sidecars and configuring the low level routing.
8+
9+
```
10+
kubectl create namespace istio-system --context $CLUSTER_WEST
11+
12+
kubectl create secret generic cacerts -n istio-system --context $CLUSTER_WEST \
13+
--from-file=certs/$CLUSTER_WEST/ca-cert.pem \
14+
--from-file=certs/$CLUSTER_WEST/ca-key.pem \
15+
--from-file=certs/$CLUSTER_WEST/root-cert.pem \
16+
--from-file=certs/$CLUSTER_WEST/cert-chain.pem
17+
18+
kubectl --context=$CLUSTER_WEST annotate namespace istio-system topology.istio.io/controlPlaneClusters=$CLUSTER_EAST
19+
kubectl --context=$CLUSTER_WEST label namespace istio-system topology.istio.io/network=network2
20+
21+
cat <<EOF > $CLUSTER_WEST.yaml
22+
apiVersion: install.istio.io/v1alpha1
23+
kind: IstioOperator
24+
spec:
25+
profile: remote
26+
values:
27+
istiodRemote:
28+
injectionPath: /inject/cluster/$CLUSTER_WEST/net/network2
29+
global:
30+
remotePilotAddress: ${DISCOVERY_ADDRESS}
31+
EOF
32+
33+
istioctl install -y --context=$CLUSTER_WEST -f $CLUSTER_WEST.yaml
34+
```
35+
36+
We will also install a Prometheus instance on the remote. We will federate both Prometheus, with the east's one being the place where all metrics will be gathered together:
37+
38+
```
39+
kubectl apply -f $ISTIO_DIR/samples/addons/prometheus.yaml --context $CLUSTER_WEST
40+
```
41+
42+
An important step is to create a secret on east cluster allowing it to fetch information of the remote cluster:
43+
44+
```
45+
istioctl x create-remote-secret \
46+
--context=$CLUSTER_WEST \
47+
--name=$CLUSTER_WEST | \
48+
kubectl apply -f - --context=$CLUSTER_EAST
49+
```
50+
51+
Finally, we create the east-west gateway
52+
53+
```
54+
$ISTIO_DIR/samples/multicluster/gen-eastwest-gateway.sh \
55+
--mesh mesh1 --cluster $CLUSTER_WEST --network network2 | \
56+
istioctl --context=$CLUSTER_WEST install -y -f -
57+
58+
```
59+
60+
## Prometheus federation
61+
62+
An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus.
63+
64+
We will configure east's Prometheus to fetch west's metrics:
65+
66+
```
67+
kubectl patch svc prometheus -n istio-system --context $CLUSTER_WEST -p "{\"spec\": {\"type\": \"LoadBalancer\"}}"
68+
69+
WEST_PROMETHEUS_ADDRESS=$(kubectl --context=$CLUSTER_WEST -n istio-system get svc prometheus -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
70+
71+
curl -L -o prometheus.yaml http://raw.githubusercontent.com/kiali/kiali/master/hack/istio/multicluster/prometheus.yaml
72+
73+
sed -i "s/WEST_PROMETHEUS_ADDRESS/$WEST_PROMETHEUS_ADDRESS/g" prometheus.yaml
74+
75+
kubectl --context=$CLUSTER_EAST apply -f prometheus.yaml -n istio-system
76+
```

‎content/en/docs/Tutorials/multicluster/09-Configure-Kiali.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
title: "Configure Kiali for multicluster"
3+
description: "In this section we will add some configuration for Kiali to start observing the remote cluster."
4+
weight: 9
5+
---
6+
7+
We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster:
8+
9+
```
10+
curl -L -o kiali-prepare-remote-cluster.sh http://raw.githubusercontent.com/kiali/kiali/master/hack/istio/multicluster/kiali-prepare-remote-cluster.sh
11+
12+
chmod +x kiali-prepare-remote-cluster.sh
13+
14+
./kiali-prepare-remote-cluster.sh --kiali-cluster-context $CLUSTER_EAST --remote-cluster-context $CLUSTER_WEST
15+
```
16+
17+
Finally, upgrade the installation for Kiali to pick up the secret:
18+
19+
```
20+
kubectl config use-context $CLUSTER_EAST
21+
22+
helm upgrade --install --namespace istio-system --set auth.strategy=anonymous --set deployment.logger.log_level=debug --set deployment.ingress.enabled=true --repo http://kiali.org/helm-charts kiali-server kiali-server
23+
```
24+
25+
As result, we can quickly see that a new namespace appear in the Overview, the istio-system namespace from west cluster:
26+
27+
![Kiali MC](/images/mc-tutorial/03.png "Kiali MC")

0 commit comments

Comments
 (0)