# So You Want to Build An... ![rw-book-cover](https://pbs.twimg.com/profile_images/1326958623587700736/_sXRf1ch.jpg) URL:: https://twitter.com/sidpalas/status/1520068212448137217 Author:: @sidpalas on Twitter ## Highlights > So you want to build an application platform with Kubernetes? > There are MANY decisions you will need to make along your journey. > A thread 🧵: > (1/n) > ![](https://pbs.twimg.com/media/FRhd-ThUcAAP8vm.png) ([View Tweet](https://twitter.com/sidpalas/status/1520068212448137217)) > Should you even be using Kubernetes? > If you are operating many, containerized, microservices k8s can provide a great base for building an app platform. > If you have a single monolithic app, you will almost certainly be better served by a different approach! > (2/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068219100225537)) > Should you self-host Kubernetes? > If you are making this decision based on a Twitter thread... probably not 😅 > There are use cases for large organizations that have mature infrastructure teams, but if getting started with k8s, using a managed service is much easier. > (3/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068221939765249)) > Which managed cluster should you use? > There are many great options here. @learnk8s maintains an informative guide comparing many of the services https://t.co/cXwvAQl2et > (4/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068224745803778)) > How do you want to handle infrastructure as code? > Sure, you CAN create your clusters with click ops via the GUI, but you SHOULD be encoding your configuration using infrastructure as code. > This takes more work up-front, but will save you time in the long run! > (5/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068227602157568)) > What level of environment isolation do you need (e.g. between teams or dev/staging/prod)? > You might be able to use namespaces for separation, you might want to provision separate clusters, or maybe even locating those separate clusters in isolated cloud accounts! > (6/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068230521393153)) > Which container registry should you use? > If you are using a managed k8s service and it comes with a container registry preconfigured (e.g. GKE + GCR) probably just go with that. > (7/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068233360863232)) > How should you manage k8s manifests? > Managing raw yaml files can quickly become cumbersome, especially as you deploy to multiple environments. > Tools like Helm and Kustomize can help with this, or you may even want to go as far as defining CRDs that provide guardrails! > (8/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068236133294085)) > Where will you store your source code? > GitHub and GitLab seem to be the top dogs here these days. The choice will likely depend on whether you have a preference for one of their CI/CD tooling. > (9/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068239023394817)) > How to structure your repo (mono vs. multi)? > Many people have strong opinions about this, and both options have benefits/tradeoffs. > If you go with a monorepo, you may need to invest in building tooling to help manage it as it grows... > (10/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068241934090245)) > How to handle Continuous Integration? > There are seemingly endless CI tooling options these days, which mostly boil down to "running a sequence of processes based on event triggers" > IMO, the developer experience for iterating on pipeline key when making a choice! > (11/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068244681285632)) > What about Continuous Delivery? > The big choice for here is whether you want to go with a "push" model (your CI pipeline calling some version of kubectl apply) or a "pull" model (agent in your cluster automatically syncing with a repo config... "GitOps") > (12/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068247378219009)) > How to get traffic into the cluster? > For simple use cases, a LoadBalancer type service might be sufficient, but you will likely graduate to using an ingress controller at some point > Lots of options here, again @learnk8s has a solid resource: https://t.co/oDgYRDLNFy > (13/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068250221961218)) > What about networking? > Most managed clusters will have an opinionated networking implementation that will be enough to get started. > If you need something more exotic, you can take a look at the various CNI plugins. > (14/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068253086720000)) > Do you need a service mesh? > Service meshes provide improved intra-network network features including mTLS between services, service discovery, and improved observability! > If you need/want these things you will need to choose and install/configure one. > (15/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068255888465922)) > What if my applications need storage? > If you are using a managed k8s, your cluster will probably have a way to automatically provision storage in your cloud provider based on the volumes you specify. > If you need something more exotic, take a look at Rook! > (16/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068258707124224)) > How can I observe the health of my systems? > Aggregating your logs, metrics, and tracing is key to operating distributed systems. > The cloud providers' tooling may be enough to get started, otherwise you can self-host relevant tools or pay a 3rd party SaaS (can be pricey) > (17/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068261462781954)) > How should we handle credentials? > Your applications will need credentials for various things. Managing them security is important. > Even if you store them as k8s secrets, you still likely want a separate source of truth to manage them. > (18/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068264298041345)) > What is the right developer experience? > Depending on the size and k8s familiarity of your team the approach will vary. > docker-compose might suffice, but you also might want people developing directly against remote k8s clusters with something like Tilt or Skaffold. > (19/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068267099918338)) > How should we back up our clusters? > There are free open source options (velero) and paid SaaS options (kasten), or your IaC configuration may be sufficient for disaster recovery. > Whichever route you choose, it is useful to have a plan in case something goes awry! > (20/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068269889048576)) > What about security? > Security really should be earlier in the decision path (😱), and there is certainly a lot to consider! > RBAC! > Container image scanning! > Configuration validation! > Admission controllers! > Network policies! > Runtime threat detection! > Oh my! > (21/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068272787312640)) > Phew! That was quite a journey 😅 > As you can see, building an application platform with Kubernetes can be quite an undertaking. That being said, if done properly can be a force multiplier for your team. 💪 > Good luck! > (22/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068275668877313)) > Found this thread helpful? > Like/retweet the first tweet and follow me @sidpalas for more Kubernetes content! 🙏 > https://t.co/VCRRObwroJ ([View Tweet](https://twitter.com/sidpalas/status/1520068943074889731)) --- Title: So You Want to Build An... Author: @sidpalas on Twitter Tags: readwise, tweets date: 2024-01-30 --- # So You Want to Build An... ![rw-book-cover](https://pbs.twimg.com/profile_images/1326958623587700736/_sXRf1ch.jpg) URL:: https://twitter.com/sidpalas/status/1520068212448137217 Author:: @sidpalas on Twitter ## AI-Generated Summary None ## Highlights > So you want to build an application platform with Kubernetes? > There are MANY decisions you will need to make along your journey. > A thread 🧵: > (1/n) > ![](https://pbs.twimg.com/media/FRhd-ThUcAAP8vm.png) ([View Tweet](https://twitter.com/sidpalas/status/1520068212448137217)) > Should you even be using Kubernetes? > If you are operating many, containerized, microservices k8s can provide a great base for building an app platform. > If you have a single monolithic app, you will almost certainly be better served by a different approach! > (2/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068219100225537)) > Should you self-host Kubernetes? > If you are making this decision based on a Twitter thread... probably not 😅 > There are use cases for large organizations that have mature infrastructure teams, but if getting started with k8s, using a managed service is much easier. > (3/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068221939765249)) > Which managed cluster should you use? > There are many great options here. @learnk8s maintains an informative guide comparing many of the services https://t.co/cXwvAQl2et > (4/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068224745803778)) > How do you want to handle infrastructure as code? > Sure, you CAN create your clusters with click ops via the GUI, but you SHOULD be encoding your configuration using infrastructure as code. > This takes more work up-front, but will save you time in the long run! > (5/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068227602157568)) > What level of environment isolation do you need (e.g. between teams or dev/staging/prod)? > You might be able to use namespaces for separation, you might want to provision separate clusters, or maybe even locating those separate clusters in isolated cloud accounts! > (6/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068230521393153)) > Which container registry should you use? > If you are using a managed k8s service and it comes with a container registry preconfigured (e.g. GKE + GCR) probably just go with that. > (7/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068233360863232)) > How should you manage k8s manifests? > Managing raw yaml files can quickly become cumbersome, especially as you deploy to multiple environments. > Tools like Helm and Kustomize can help with this, or you may even want to go as far as defining CRDs that provide guardrails! > (8/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068236133294085)) > Where will you store your source code? > GitHub and GitLab seem to be the top dogs here these days. The choice will likely depend on whether you have a preference for one of their CI/CD tooling. > (9/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068239023394817)) > How to structure your repo (mono vs. multi)? > Many people have strong opinions about this, and both options have benefits/tradeoffs. > If you go with a monorepo, you may need to invest in building tooling to help manage it as it grows... > (10/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068241934090245)) > How to handle Continuous Integration? > There are seemingly endless CI tooling options these days, which mostly boil down to "running a sequence of processes based on event triggers" > IMO, the developer experience for iterating on pipeline key when making a choice! > (11/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068244681285632)) > What about Continuous Delivery? > The big choice for here is whether you want to go with a "push" model (your CI pipeline calling some version of kubectl apply) or a "pull" model (agent in your cluster automatically syncing with a repo config... "GitOps") > (12/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068247378219009)) > How to get traffic into the cluster? > For simple use cases, a LoadBalancer type service might be sufficient, but you will likely graduate to using an ingress controller at some point > Lots of options here, again @learnk8s has a solid resource: https://t.co/oDgYRDLNFy > (13/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068250221961218)) > What about networking? > Most managed clusters will have an opinionated networking implementation that will be enough to get started. > If you need something more exotic, you can take a look at the various CNI plugins. > (14/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068253086720000)) > Do you need a service mesh? > Service meshes provide improved intra-network network features including mTLS between services, service discovery, and improved observability! > If you need/want these things you will need to choose and install/configure one. > (15/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068255888465922)) > What if my applications need storage? > If you are using a managed k8s, your cluster will probably have a way to automatically provision storage in your cloud provider based on the volumes you specify. > If you need something more exotic, take a look at Rook! > (16/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068258707124224)) > How can I observe the health of my systems? > Aggregating your logs, metrics, and tracing is key to operating distributed systems. > The cloud providers' tooling may be enough to get started, otherwise you can self-host relevant tools or pay a 3rd party SaaS (can be pricey) > (17/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068261462781954)) > How should we handle credentials? > Your applications will need credentials for various things. Managing them security is important. > Even if you store them as k8s secrets, you still likely want a separate source of truth to manage them. > (18/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068264298041345)) > What is the right developer experience? > Depending on the size and k8s familiarity of your team the approach will vary. > docker-compose might suffice, but you also might want people developing directly against remote k8s clusters with something like Tilt or Skaffold. > (19/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068267099918338)) > How should we back up our clusters? > There are free open source options (velero) and paid SaaS options (kasten), or your IaC configuration may be sufficient for disaster recovery. > Whichever route you choose, it is useful to have a plan in case something goes awry! > (20/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068269889048576)) > What about security? > Security really should be earlier in the decision path (😱), and there is certainly a lot to consider! > RBAC! > Container image scanning! > Configuration validation! > Admission controllers! > Network policies! > Runtime threat detection! > Oh my! > (21/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068272787312640)) > Phew! That was quite a journey 😅 > As you can see, building an application platform with Kubernetes can be quite an undertaking. That being said, if done properly can be a force multiplier for your team. 💪 > Good luck! > (22/n) ([View Tweet](https://twitter.com/sidpalas/status/1520068275668877313)) > Found this thread helpful? > Like/retweet the first tweet and follow me @sidpalas for more Kubernetes content! 🙏 > https://t.co/VCRRObwroJ ([View Tweet](https://twitter.com/sidpalas/status/1520068943074889731))