%% Last Updated: - [[2021-02-28]] - [[2021-02-11]] Related: - [[Performance Testing]] [[Load Testing]] %% Failures to find and fix performance bottlenecks can be very public. Just last year, we found that [many sites were not properly prepared](https://www.theage.com.au/business/companies/online-betting-sites-crashed-in-lead-up-to-melbourne-cup-20181106-p50ebb.html) for the influx of new users that [[Melbourne Cup]] can bring. If you're a site selling bets online for the Melbourne Cup, like the unfortunate ones in that article, chances are high that your inability to keep up with the load would not have gone unnoticed. ## Fails in the wild ### Amazon, 2018 Prime Day outage due to load could have cost them up to $72 million. [^01] ### Costco, 2019 Goldman, M. C. (2019). Costco's Thanksgiving Day website crash cost it nearly $11M. Accessed on [[2020-09-16]] from https://www.thestreet.com/technology/costco-thanksgiving-day-website-crash-cost-it-nearly-11million-15185344. > Bulk-discount retail giant Costco got a rude and costly awakening Thanksgiving Day when its website went down for more than 16 hours, costing the retailer nearly $11 million in lost pre-Black Friday potential sales, according to retail sales aggregator website [LovetheSales.com](https://www.lovethesales.com/help/about-us). ### Google, 2020 - December: "https://www.cbsnews.com/news/google-services-restored-global-outage-today-2020-12-14/" - "GMail outages a day later: https://www.forbes.com/sites/joewalsh/2020/12/15/gmail-users-struggle-with-glitches-a-day-after-google-suffered-major-global-outage/" - "two hours of downtime or YouTube in November" ### AWS, 2020 - [[Summary of the Amazon Kinesis Event in the Northern Virginia (US-EAST-1) Region]] (November) ### Slack, 2021 - three hours: https://edition.cnn.com/2021/01/04/business/nightcap-slack-outage-haven-folds/index.html ("Welcome back to work, everything is broken") ### Binance, 2021 - February - They blamed an [[AWS]] outage for causing them to display incorrect account balances - [[Tweets From Tired_Momma#^147880841|@tiredmomma8]], [[Tweets From Binance#^147881013|@binance]] - [Postmortem](https://www.binance.com/en/support/announcement/63070a39c12e46f58e7f1ff2f83aa9d7) [^01]: [[It's Time for Shift-Left Performance Testing]]