%% Last Updated: - [[2021-03-06]] - [[2021-02-09]] - [[2020-12-17]] - [[2020-12-11]] - [[2020-12-04]] %% # CA BlazeMeter URL: https://blazemeter.com ## What is BlazeMeter? BlazeMeter is a SaaS platform built around ramping up load testing scripts from JMeter and Selenium. It has recently expanded its offerings to include a mocking service. [[BlazeMeter Mission Statement]]: Deliver as One ## What are BlazeMeter's goals? [[BlazeMeter 2021 Roadmap]] - BlazeMeter have already expanded from load testing into performance testing and functional testing, and now they appear to be heading towards becoming a DevOps tool. - BlazeMeter will likely move towards integrations with [[CA Technologies]]'s other tools - [[CA Lisa]] - [[CA Technologies APM]] ## History In 2019, BlazeMeter was acquired by [[CA Technologies]], a [[Broadcom]] company. ## Competitive Advantages - Every test can send an email notification - You can run tests on AWS, Azure, GCP - Executive summary report, PDF-able ![[Blazemeter-PDF-Report.pdf]] - [[BlazeData]] [^01] test data store ## Advantages of [[k6 (Company)]] vs Blazemeter (as a platform): - price - support - Free tier is better - Blazemeter's free tier uses shared infrastructure, which you should never use for a load test. - Starting instances with k6 is often significantly faster (<1 minute) because we use pre-started instances: takes waiting time out of the equation - Blazemeter has no web script editor, so you have to modify your script and re-upload to change something - Performance insights! ## Key features of a load testing platform ### Scripting #### Creating a script - [x] [[Recorder]] - [x] [[Exclude third-party or configured domains]] - [x] [[GUI Test Builder]] -  - [ ] [[Web-based script editor]] - [x] [[Upload of scripts from other tools]] - [x] [[Protocol-based load testing]] - [[JMeter]] - [x] [[Browser-based testing]] - [[Selenium WebDriver]] #### Making the script robust - [x] [[Supports all HTTP methods]] - [x] [[Sending parameters with request]] - [x] [[Modify headers]] - [x] [[Response assertions]] - Error handling - [ ] [[Output to log upon error]] - Not in the UI, but this can be set within the tool. - [ ] [[Continue on error or restart iteration]] - Debugging - [ ] [[See request and response headers and bodies on single iteration]] - [ ] [[Transaction groups]] - Not in the UI, but this can be set within the tool. #### Making the script realistic - [ ] [[Correlation of dynamic values]] - [ ] [[Pass variables in request]] - [ ] [[Dynamic think time and pacing]] ### Execution #### [x] [[Test Scenarios]] - Called "scenarios". They can be combined in a "multi-test". -  - Source: https://guide.blazemeter.com/hc/en-us/articles/360000591677-Creating-a-Multi-Test #### [ ] [[Configurable Load profile]] -  - [x] [[Number of virtual users]] - [x] [[Duration]] - [x] [[Origin or Region]] - [ ] [[Fully customizable load profile]] - [x] [[Ramp-up]] - [ ] [[Ramp down]] - [x] [[Number of iterations]] #### [ ] [[Data partitioning across load generators]] #### SLAs - [x] [[Set multiple thresholds]] - Called "test failure criteria" -  #### [x ] [[No shared infrastructure]] - They do allow shared infrastructure in "sandbox mode", but real executions are not shared #### [x ] [[Bandwidth throttling]] - Yes, "network emulation" feature in UI -  - Source: https://guide.blazemeter.com/hc/en-us/articles/115005924849-Network-Emulation-NTC--Network-Emulation-(NTC) #### Infrastructure - [x] [[Control over how many users per load generator to run]] - [x] [[Self-hosted]] - [ ] [[On-premise]] - [x] [[Own cloud accounts]] - [x] [[AWS]] - [x] [[Microsoft Azure]] - [x] [[Google Cloud Platform]] - [ ] [[DigitalOcean]] - [x] [[On demand]] #### [ ] [[Multiple IP addresses]] #### [ ] [[Execution Parameters]] ### Analysis #### Dashboard - [ ] [[Custom dashboard creation]] - [ ] [[Reusable dashboards]] - Easy error troubleshooting - [ ] [[Screenshots]] - [ ] [[Response bodies]] - [x] [[Assertion success or failure]] - [x] [[Log for transaction failure]] -  #### Metrics - [x] [[Response time]] - [x] [[Request throughput]] - [x] [[Number of virtual users]] - [x] [[Error rate]] - [ ] [[Network throughput]] - [ ] [[Load generator CPU]] - [ ] [[Load generator memory]] #### Graphs - [ ] [[Different Graph Types]] - [x] [[View by transaction]] - [ ] [[View by load generator or region]] - [x] [[Custom views]] -  #### Comparison - [x] [[Compare two tests]] -  - Source: https://guide.blazemeter.com/hc/en-us/articles/207421555-Compare-Reports - [x] [[Overlay results by relative time]] #### Results - [ ] [[Raw data]] - [x] [[At least 1s granularity]] - [ ] [[Export as plain text]] - [ ] [[Integration with APM or Data Visualization tools]] -  - Source: https://guide.blazemeter.com/hc/en-us/articles/115005911509-APM-Integration - [x] [[New Relic]] - [x] [[Dynatrace]] - [x] [[AppDynamics]] - [ ] [[Tableau]] - [ ] [[InfluxDB]] - [x] [[Datadog]] - [x] [[CloudWatch]] - [x] [[CA Technologies APM]] - [x] [[Splunk Cloud]] - [x] [[Keen]] - [x] [[BigPanda]] ### Continuous Load Testing #### Scheduling - [x] [[Run tests on a schedule]] -  - Source: https://guide.blazemeter.com/hc/en-us/articles/360000835118-Scheduling-a-test - [x] [[Email results]] -  - [x] [[Alerts]] - [x] [[Slack]] - [x] [[Microsoft Teams]] - [x] [[PagerDuty]] - [x] [[VictorOps]] - [x] [[OpsGenie]] - [x] [[ServiceNow]] - [x] [[Flowdock]] - [x] [[Zapier]] - [x] [[Grove]] #### [ ] [[API hooks]] #### [ ] [[Graph of continuous performance over time]] [^01]: [[Introducing BlazeData]]