11.2 ingress controller - linkerd does not suit with nginx ingress controller

chaosreload

I found that nginx ingress controller does not work with linkerd, when both of them are new version:

Version of the images:

1. nginx ingress controller: v1.0.4 (with nginx/1.19.9 running)
2. linkerd-proxy: stable-2.11.0
3. kubernetes: v1.21.4

The output of the logs in controller pod:

W1019 08:09:30.643739       6 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1019 08:09:30.643931       6 main.go:221] "Creating API client" host="https://10.96.0.1:443"
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.0.4
  Build:         9b78b6c197b48116243922170875af4aa752ee59
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.9

-------------------------------------------------------------------------------

It hanged at the 'creating API client' part, and crashed after the threshold of liveness-probe failed with 502 response:

  Warning  Unhealthy  3s (x4 over 33s)  kubelet            Liveness probe failed: HTTP probe failed with statuscode: 502
  Warning  Unhealthy  3s (x4 over 33s)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 502

After trial and experiment, I figured out a work-around, which adding the '443' to the '--skip-inbound-ports' and '--skip-outbound-ports' flags then execute the linkerd inject:

kubectl get daemonset myingress-controller-ingress-nginx-controller -o yaml | linkerd inject --ingress --skip-inbound-ports 443 --skip-outbound-ports 443  - | kubectl apply -f -

and it works.

This may because linkerd-proxy blocks all of the non-HTTP outbound traffic: https://github.com/linkerd/linkerd2/blob/main/CHANGES.md#stable-2110

Traffics from nginx ingress controller to API server is 'HTTPS' (https://10.96.0.1:443) so i may be blocked.

Should I add the '443' port to '--opaque-ports' flag instead?

jitapichab

HI, I change the version to stable-2.9.3 and it works too.

gfnord

Thanks!

tranthepq

Thanks a lot!