# [[TestOps]] TestOps is a trend in [[Software Testing]] where testing is moving closer to [[Site Reliability Engineering]] (traditionally called "ops" or "operations") in terms of skills, techniques, and tools. This movement is the testing equivalent of [[DevOps]] and is an implementation of the trend [[Ops all the things]]. (However, some [^ahmed] consider TestOps as a practice within DevOps.) There are three main ideas that people refer to when they speak of TestOps: shift-right testing; the setup, automation, and maintenance of test infrastructure; and the merging of testing and operations itself. ## [[Shift-right testing]] In the context of shift-right testing, TestOps can be seen as a shortcut for the idea that testing should extend into production and beyond. This includes practices such as [[Testing in production]] or in the immediate pre- or post-production environments. Testing practices could involve: - [[Synthetic monitoring]] - doing small [[Load Testing|load tests]] - production monitoring - canary testing - crowd testing - CX-based testing (checking user behaviour against test requirements) [^ahmed] In this regard, someone practicing TestOps might advocate the [[Continuous Improvement|continuity]] of testing and [[Observability]] as a way to get the same data that traditional testing seeks. ## Test infrastructure management Another step towards TestOps involves testers employing practices from [[Site Reliability Engineering|SRE]] to fulfill primarily testing-related tasks. For example, testers learning how to use [[Kubernetes]] to spin up pre-production or production environments frees them from the dependency on Ops support and environment schedules. Gone are the days when "the test tool" just involved a single monolithic application, deployed on the same infrastructure as the application. Now, testing tools are fragmenting into more modular components ([[Microservices]]) and can be thought of as a "testing stack". This testing stack must then also be managed separately from the application stack so as not to contaminate the results of the test or the metrics from the application. If there is test-specific infrastructure that must be provisioned, the TestOps engineer may also be responsible for automating and streamlining the provisioning. TestOps also encourages testers to understand the automation behind deploying application code itself, because the process is not immune to defects. [^katalon] Of particular note are the following aspects: - Test suite management and collaboration - Test execution - Test analysis - DevOps integration (testers could own parts of the DevOps process related to testing) ## The true merging of Test and Ops TestOps can also refer to the merging of two disciplines, testing and SRE, into something else entirely. This includes the expansion of the definition of quality to include quality in deployment. Testers can apply testing attitudes and techniques to ensure that the architecture of applications, as well as their deployment configuration, are not just functional but performant and optimal. Unlike the previous two approaches to TestOps, in this context, the line between Testing and Operations as activities, skills, and as teams begins to blur. [[Containerization]] and the rise of [[Kubernetes]] and other [[Distributed computing]] technologies have enabled this progression into Ops. ### [[SLAs]] and [[SLOs]] [[Requirements]] from software testing have merged with the similar, but more operations-flavored SLAs and SLOs. ### [[Chaos Engineering]] [[Chaos engineering is a testing discipline]], but its roots in Operations means testers and SREs alike could be responsible for carrying out chaos experiments. ### [[Performance testing is overlapping with SRE]] ## Benefits of the overlap of performance and SRE - It breaks down silos between testers and operations engineers [^arnott] - Faster feedback cycles - Better use of automation to reduce toil for testers [^arnott] - Reduction in blame culture, because testers are allowed on the "other side of the fence" and thus share in the reponsibility for deployments: ![[How SRE Will Transform the Role of QA Engineers - DZone DevOps#^181044775]] ## Why is TestOps a thing? What's driving the proliferation of TestOps? - Shorter timelines in the [[Agile]] world are rarely sufficient for traditional testing - [[Continuous Improvement|Continuous Integration]] places a heavy emphasis on "testing as we go" - The complexity of some systems makes them hard to simulate in test environments - Prevention of siloes within teams - Emergence of customer experience as a key quality metric to be measured in production [^arnott]: Arnott, E. (2021). _QA engineers, this is how SRE will transform your role_. Blameless. Retrieved from https://www.blameless.com/sre/how-sre-will-transform-qa . [[How SRE Will Transform the Role of QA Engineers - DZone DevOps|My highlights]]. [^katalon]: Katalon. _TestOps: The future of software testing_. Retrieved from https://katalon.com/resources-center/blog/what-is-testops. [[TestOps The Future of Software Testing|My highlights]]. [^ahmed]: Ahmed, S. (2020). _Shift-right testing: The emergence of TestOps_. Retrieved from https://devops.com/shift-right-testing-the-emergence-of-testops/. [[Shift-Right Testing The Emergence of TestOps - DevOps.com|My highlights]].