<iframe width="560" height="315" src="https://www.youtube.com/embed/SFag8ggkXKo" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> # In search of the best pokémon: Browser automation and load testing in one script with k6 ## Speaker bio ```ad-info title: Short Bio Nicole is a performance engineer with over a decade of experience in breaking software and learning to build it back up again. She has lived in the Philippines, the US, Australia, and now the Netherlands, helping teams all over the world scale up their load tests on the cloud. She currently works at k6 as a Developer Advocate for performance testing, which means she learns new stuff she has no business learning for a living and then tells people about how she did it. ``` ## Abstract ```ad-abstract title: Abstract Being a tester is like being a pokémon trainer: we want to choose the right pokémon for the job. The problem with doing different types of testing is that we also end up with multiple scripts in different languages and testing tools. We end up choosing a squad of pokémon: one for unit testing, one for automation testing, one for load testing, and so on. Making sure they all work together is a task in itself! But what if we could just use one? What if we only needed one Mega-evolved pokémon that could fulfill multiple roles? Is it really possible to write a browser automation script and then reuse it for load testing? In this talk, I will talk about my search for the elusive one pokémon that does multiple jobs, the advantages of hybrid load testing, and how to pick pokémon that make our life as trainers and testers easier. ``` <iframe width="560" height="315" src="https://www.youtube.com/embed/NlE8p5Mxp8M" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> ## Requirements ```ad-info title: From Joe - The perfect video length is _25_minutes. - The first few minutes are _just you talking to the camera_ about what you are about to show us. - Screen share showing the use of a tool, technique or tip that will help us in our day-to-day testing efforts (you don’t need to have your camera turned on during this part if you don’t want to). - The Guild prefers hands-on, practical screen sharing of tips, tricks, and demos that can help fellow automation/test engineers create better automation & testing over boring, high-level, philosophical PowerPoint slides. _Of course, this is just a good practice. If you are doing a more high-level presentation than at least make the slides interesting_  - Also, since the videos are pre-recorded, speakers are encouraged to dive into the actual code to “show us,” not just “tell us” how something can be achieved. This is what will set us apart from a “live,” in person conference — more in-depth with real, hands-on, screen-sharing-type demos. Once again this varies depending on your topic. - The last few minutes should just be you on camera wrapping up your session. - Since this is an online conference it’s critical that you have a quality microphone and video camera. Also since we will be LIVE for QA please make sure that you join using a wired internet connection whenever possible. Use the Guild Conferences slide template for your conference type ==>[http://rcl.ink/yH](http://rcl.ink/yHt) ``` ## Technical setup Test app: - Script for protocol-based testing - Script for browser-based testing - Hybrid script - How to view results: Grafana via Prometheus Remote Write ## Hook There are a lot of reasons I love Pokémon. - First, it first came out in 1996, and I happened to be 10 years old-- the _perfect age_ for fawning all over the cuteness that was these pocket monster. - Second, the hero of the franchise, Ash Ketchum, is the underdog. He's not the best pokémon trainer YET-- but he's trying. - And third, Ash doesn't just rely on ONE pokémon. He goes out in search of the best pokémon to add to his team, and then he fights with the one he thinks will do the best in a certain situation. But what does this have to do with testing? ## Intro Hi, I'm Nicole van der Hoeven. For the last decade, I've secretly pictured myself as the Ash Ketchum of performance testing. I started out as a performance testing consultant, where I cut my teeth on tools like Neoload, LoadRunner, JMeter, and Silk Performer. 4 years ago, I joined the Tricentis Flood team, where I helped companies use JMeter, Gatling, Flood Element, and Selenium. And just over a year ago, I joined k6 (now acquired by Grafana Labs), where I am a developer advocate for performance testing for the tool that is also called k6. What can I say? I like new tools. "Catch 'em all" is a bit of my life philosophy. In the next 20 minutes, I'm going to talk about k6, because that's the tool I'm most familiar with at this point - but beyond that, I'm talking about hybrid load testing in general, and why it's so important that I've been harping on this subject for a few years now. Everything I'm going to show today is 100% free and open-source. ## Structure - What is a hybrid load testing tool? (brief) - What's protocol-based? - What's browser-based? - Approaches to hybrid load testing - Protocol + browser (Pikachu + Charmander) - Protocol + performance profiling (Pikachu + Bulbasaur) - Browser only (Charmander only) - (new from PerfGuild 2020) Use one tool to do it all (Eevee) - Disadvantages of other approaches - Advantages of the Eevee Approach - What is k6? - Installation - Execution - Scripting of protocol-based - Results in Grafana - What is xk6? - Some cool extensions - How to install xk6? - What is xk6-browser? - How to install xk6-browser - Scripting in xk6-browser - Putting them together: protocol + browser - Disadvantages - xk6-browser is not yet fully-fledged; Playwright would be a better bet for functionality - xk6-browser is in beta, so you may run into a few bugs - Alternatives: [[Karate]], [[LoadRunner]] (with [[TruClient]]), [[NeoLoad]] (via [[Selenium WebDriver]], [[Tosca]]) ## Outro The Eevee Approach to performance testing is a multitool approach: it is an acknowledgement that specialized tools, as good as they _can_ be, have a hidden cost: and that's integration. The combination of tools we choose can sometimes be more important, or more difficult to live with, than the actual testing. So, what's the best pokémon? What's the best load testing tool? It's not Eevee, and it's not k6. The truth is that it's too simplistic to always fall back on any one tool. But when you're starting out, or you're not very clear on the kind of testing you want to do, I think a multitool with multiple purposes and uses is an excellent pokémon to add to your team. ## Related videos and resources Pokémon: I Choose You! episode, from The Official Pokémon YouTube Channel: https://www.youtube.com/watch?v=Zyt2GKb6qWw ## Post-production - [ ] Promote on personal social media. - [ ] Twitter - [ ] LinkedIn - [ ] Create a blog post on [nicolevdh.com](http://nicolevdh.com) - [ ] Include in a weekly newsletter. ## Q&A Prep > Why are browser-based testing tools more expensive to run at scale? Isn't it simpler to run an automation testing script with multiple users and drive load that way? - Browser-based testing tools are resource intensive. For example, you can expect to be able to safely run about 5 Selenium users per AWS m5.xlarge instance. You can compare that to JMeter, with which you can run 1000 users on the same machine. - Yes, it's simpler, but it will probably cost you! > What do you think about using JMeter and the JMeter WebDriver sampler plugin to do hybrid load testing? - JMeter + Selenium has a higher footprint (CPU and memory) than either JMeter or Selenium on its own. > Is there a way to get graphs for k6 results? - You can use: https://github.com/benc-uk/k6-reporter - you import it in your script and it's like the Gatling HTML dashboard - You can export output in many forms (junit, JSON, CSV, Prometheus Remote Write) and visualize in another tool like Grafana or Tableau > Can you do distributed load testing with k6 without using k6 Cloud? - Yes! Check out the k6 operator: https://github.com/grafana/k6-operator > Why is xk6-browser an extension and not included with the basic k6 package? - The k6 philosphy is to reduce the default binary as much as possible. Test your test tools! They can have a bigger effect on your test results than you think. Hey! Thanks all for watching. Some links in case you want to learn more! xk6-browser: https://github.com/grafana/xk6-browser Scripts I used in this presentation: https://github.com/nicolevanderhoeven/k6-scripts/tree/main/automationguild2022/xk6_0.5.0_mac_amd64 k6: https://k6.io k6 YouTube Channel: https://youtube.com/k6test Me on Twitter: https://twitter.com/n_vanderhoeven Me on LinkedIn: https://www.linkedin.com/in/nvanderhoeven My site: https://nicolevanderhoeven.com My YouTube Channel: https://youtube.com/nicolevanderhoeven Using GitHub Actions with k6: https://www.youtube.com/watch?v=GzR2Z25-Yqo Basics of load testing with k6 and Grafana in 20 minutes: https://www.youtube.com/watch?v=gvounvDSDGg Chaos testing with k6, Prometheus, and Grafana: https://www.youtube.com/watch?v=2QHs_HEX7r0 What you need to know about Grafana Labs acquisition of k6: https://www.youtube.com/watch?v=t1agJsLwrGE k6 extensions - k6-operator - xk6-chaos - xk6-browser - xk6-benc-uk - xk6-exec k6 vs JMeter blog post: https://k6.io/blog/k6-vs-jmeter/ k6 vs JMeter video: https://k6.io/blog/k6-vs-jmeter/ k6 metrics: https://k6.io/docs/using-k6/metrics/