Infrastructure as Code and Deployment Pipeline for Large number of Application Instances

tinphamtan

Use Case

Application is deployed to hundreds of instances for different customers, each has different configuration. Code based is stable, not frequently change. Following tools are used:

• Multi tenant approach can not be considered.
• Application is packed as container image.
• Kubernetes environment, strictly protected, no access to cluster admin privileges.
• Application is deployed with Helm Charts and Azure DevOps pipeline
• Each prod environments has slightly different configuration, like secrets, ConfigMap, K8s resources.

Question and Discussion

Normal CI/CD process is for application development, with frequent code change and one final destination which is prod env.
But is there a standard practice, tools set for deploying and managing one application to hundreds of different prod environments?
Managing one Application instance at a time is currently used, but it is not very GitOps in my opinion, and also not efficient for hundreds of instances.

I found ArgoCD is quite closed for the use case, with IaC pull approach, but can not be used because it needs cluster-admin privilege. (https://argo-cd.readthedocs.io/en/stable)

Overview

tinphamtan

@luisviveropena more details about my question.

gouravshah

@tinphamtan I just watched the recording of your conversation with @luisviveropena. Your situation seems to be unique in the sense you are trying to work with different clients, each with their own cluster. The direction you are exploring is right, in the way you have arrived at ArgoCD.

My opinion is you could possibly solve this using FluxCD + a repository design approach + a layer of RBAC that I have authored in the GitOps course. It would be best if you drop by during my next office hour on Monday, Feb 21st and we could have a discussion about it.

luisviveropena

Hi @gouravshah , many thanks for the assistance with the @tinphamtan 's question! In fact it's a very special use case.

Regards,
Luis.