%% date:: [[2021-02-09]], [[2020-12-07]], [[2020-12-04]], [[2022-07-14]], [[2021-02-20]], [[2021-04-02]], [[2021-04-12]], [[2021-04-13]], [[2021-04-19]], [[2021-05-19]], [[2022-07-19]] Parents:: [[Load Testing]] [[Load Testing Tool]] [[Performance Testing]] %% # [[JMeter]] JMeter is an extremely popular open-source load testing tool, and for good reason. It’s got a solid history of being able to deliver results and is an industry standard. To start with JMeter load testing, you’ll need to make sure you’ve downloaded Java as well as JMeter itself. Unlike the Test Builder, where you did your scripting in a web interface, you’ll be doing your scripting within the JMeter program. One big reason to use JMeter is its UI. While not the best looking, it is relatively easy to use. It’s a step above the Test Builder in complexity, because there’s still a learning curve here, but it’s still a step below other tools where scripts are written entirely in code.  You can use JMeter to create robust load testing scripts without coding experience, but your JMeter load testing script can also include code in the form of Beanshell or Groovy post- and pre-processors. This flexibility means you can do a lot more with JMeter than you can with Test Builder alone. Because JMeter is so popular, it’s very easy to get support online. Regardless of what you’re trying to do in JMeter, typing “JMeter load testing” in the search engine is likely to return a lot of results for you to learn from. It is also very well-supported by the community: the JMeter project on GitHub boasts 15,722 commits in the 20 years it’s been around, and the number of custom plugins for JMeter load testing seems to increase every day. ## Basics - [[JMeter Terminology]] - [[JMeter Installation and Setup]] - [[Recording traffic in JMeter]] - [[Getting Started with JMeter]] - [[Excluding static resources in JMeter]] - [[JMeter Clearing results in listeners]] - [[Parameterization in JMeter]] ## Scripting - [[JMeter/Assertions]] - [[JMeter Controllers]] - [[JMeter Config Elements]] - [[JMeter Listeners]] - [[JMeter Samplers]] - [[JMeter Extractors]] - [[JMeter Processors]] - [[JMeter Test Fragments]] - [[JMeter CSV Data Set Config]] ## Protocols - [[Testing LDAP with JMeter]] ## [[Troubleshooting in JMeter]] ## Executing a test - [[JMeter CLI]] - [[Load Profile]]s - [[Setting threads dynamically with Jexl in JMeter]] - Stepped load profile -  - [[Load distribution in JMeter]] ## Analysis - [[Tableau and JMeter]] - [[Custom metrics in JMeter]] ## JMeter's place in the market - [[JMeter Competitive Disadvantages]] - [[JMeter Competitive Advantages]] ## History Released in 1998 [[Philippe Mouawad]] is one of the biggest contributors to JMeter > JMeter has more than 10.000 commits representing almost 500K lines of code. 33 people contributed to the project so far. [[Philippe Mouawad]]. is one of the biggest contributor to the project, mostly in Java. He is the founder of [[UbikLoadPack]], a French company offering commercial JMeter plugins. ([View Highlight](https://instapaper.com/read/1368934176/14835178)) - [[Jérôme Loisel]] ## Criticisms of JMeter ### JMeter doesn't measure front-end performance (well) While JMeter can be used in conjunction with [[Selenium WebDriver]] to run browser-based tests, this setup is incredibly inefficient. In this video, I talk about why JMeter alone isn't enough (in Spanish): <iframe width="560" height="315" src="https://www.youtube.com/embed/1KB8e2Ld08M" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> - [[/k6 vs JMeter]] ## Key features of a load testing tool ## Scripting ### Creating a script - [x] [[Recorder]] - [x] [[Exclude third-party or configured domains]] - [ ] [[Sample script packaged in]] - [x] [[GUI Test Builder]] - [x] [[Code-based scripting]] - Some, using [[Beanshell]], [[Groovy]], or other [[Java]]-based languages. - [x] [[Protocol-based load testing]] - [x] [[Browser-based testing]] ### Making the script robust - Protocols Supported - [x] [[HTTP/1.1] - [x] [[HTTP/2] - [x] [[WebSockets]] - [x] [[gRPC] - Yes, but with a custom beanshell or java sampler. - [x] [[Citrix ICA] - [x] [[LDAP] - [x] [[Modify headers]] - [x] [[Response assertions]] - [[JMeter/Assertions]] - [[Error handling]] - [x] [[Output to log upon error]] - [ ] [[Screenshots]] - [x] [[Continue on error or restart iteration]] - [[Debugging]] - [x] [[See request and response headers and bodies on single iteration]] - [x] [[Change log levels]] - [x] [[Transaction groups]] - "Thread Group" - [x] [[Custom functions or test fragments]] - [[JMeter Test Fragments]] ### Making the script realistic - [[Correlation of dynamic values]] - [[JMeter Extractors]] - [x] [[JMeter Regular Expression Extractor]] - [[JMeter Regular Expression Extractor]] - [x] [[CSS Selector]] - [[JMeter CSS Extractor]] - [x] [[JSON Extractor]] - [x] [[Dynamic think time and pacing]] - [x] [[Conditional logic]] - [x] [[Parallel execution of requests]] - [x] [[Download embedded resources]] - [x] [[Follow redirects]] - [x] [[Network bandwidth throttling]] - [[Network bandwidth throttling in JMeter]] - [x] [[Cache management]] - [x] [[Startup and teardown thread groups]] ### Test data - [x] [[Static test data]] - [x] [[Unique data generation during runtime]] - [x] [[Virtual Table Server]] - [[HTTP Simple Table Server JMeter Plugin]] ### Collaborating on a script - [x] [[Version Control System]] - [x] [[Test Scaffolding]] ## Execution ### [x] [[Test Scenarios]] ### [x] [[Configurable Load profile]] -  - [x] [[Number of virtual users]] - [x] [[Duration]] - [x] [[Origin or Region]] - [x] [[Ramp-up]] - [x] [[Ramp down]] - [x] [[Number of iterations]] - [x] [[Fully customizable load profile]] ### SLAs - [ ] [[Set multiple thresholds]] - Not as such, but a few things can be used to accomplish the same thing. - Custom code via [[Beanshell]] + [[JMeter Simple Data Writer]] - Timers: [[JMeter Constant Throughput Timer]] or [[Throughput Shaping Timer JMeter Plugin]] - Duration [[JMeter/Assertions]] on the request level - Global [[JMeter/Assertions]] placed at the "Test Plan" level will still just apply it to every sampler - [[Performance Plugin JMeter Plugin]] if integrating with [[Jenkins]] ### [x] [[Execution Parameters]] ### Memory consumption benchmark 1000 users on AWS m5.xlarge ## Analysis ### Dashboard - [x] [[Pre-generated reports]] - [[JMeter Listeners]] for different specific formats of results - [[JMeter HTML Report]] - A little fiddly, because you first have to use a listener to output results to a `.jtl` or `.csv`, then fill out these fields before the HTML report is generated. The output directory must have been created and must be empty. These fields must be filled out each time you want to generate the report. -  -  -  - Sample: ![[jmeter-sample-html-report.zip]] - More information: https://jmeter.apache.org/usermanual/generating-dashboard.html - [x] [[Custom dashboard creation]] - To some extent. You can't create __every__ type of graph, but you can configure all of the graphs JMeter already makes in `user.properties`. - [[Custom metrics in JMeter]]: http://www.perftuned.com/jmeter-generating-custom-charts/ - [x] [[Reusable dashboards]] - Yes - just save your `user.properties` settings. - Easy error troubleshooting - [ ] [[Screenshots]] - [x] [[Response bodies]] -  - [x] [[Assertion success or failure]] - [x] [[Log for transaction failure]] ### Metrics - [x] [[Response time]] - [x] [[Request throughput]] - [x] [[Number of virtual users]] - [x] [[Error rate]] - [x] [[Network throughput]] - [x] [[Load generator CPU]] - [[PerfMon Metrics Collector JMeter Plugin]] - [x] [[Load generator memory]] - [[PerfMon Metrics Collector JMeter Plugin]] ### Graphs - [x] [[Different Graph Types]] - [x] [[View by transaction]] - [ ] [[View by load generator or region]] - [x] [[Custom views]] ### Results - [x] [[Raw data]] - [x] [[At least 1s granularity]] - [x] [[Export as plain text]] ## Continuous Load Testing - [x] [[Jenkins]] - [x] [[Azure Pipelines]] - [x] [[Travis CI]] - [x] [[CircleCI]]