# How It Works: The Novel HTTP/2 ‘Rapid Reset’ DDoS Attack
URL:: https://cloud.google.com/blog/products/identity-security/how-it-works-the-novel-http2-rapid-reset-ddos-attack
Author:: Juho Snellman
## Highlights
> A primary design goal of HTTP/2 was efficiency, and unfortunately the features that make HTTP/2 more efficient for legitimate clients can also be used to make DDoS attacks more efficient. ([View Highlight](https://read.readwise.io/read/01hcvk0axc5mk65eack9v2jbsw))
### Stream multiplexing
> HTTP/2 uses "streams", bidirectional abstractions used to transmit various messages, or "frames", between the endpoints. “Stream multiplexing” is the core HTTP/2 feature which allows higher utilization of each TCP connection. Streams are multiplexed in a way that can be tracked by both sides of the connection while only using one Layer 4 connection. Stream multiplexing enables clients to have multiple in-flight requests without managing multiple individual connections. ([View Highlight](https://read.readwise.io/read/01hcvk1fqkd3y7zemtp9mgbpn7))
> One of the main constraints when mounting a Layer 7 DoS attack is the number of concurrent transport connections. Each connection carries a cost, including operating system memory for socket records and buffers, CPU time for the TLS handshake, as well as each connection needing a unique four-tuple, the IP address and port pair for each side of the connection, constraining the number of concurrent connections between two IP addresses.
> In HTTP/1.1, each request is processed serially. The server will read a request, process it, write a response, and only then read and process the next request. In practice, this means that the rate of requests that can be sent over a single connection is one request per round trip, where a round trip includes the network latency, proxy processing time and backend request processing time. While HTTP/1.1 pipelining is available in some clients and servers to increase a connection's throughput, it is not prevalent amongst legitimate clients.
> With HTTP/2, the client can open multiple concurrent streams on a single TCP connection, each stream corresponding to one HTTP request. The maximum number of concurrent open streams is, in theory, controllable by the server, but in practice clients may open 100 streams per request and the servers process these requests in parallel. It’s important to note that server limits can not be unilaterally adjusted.
> For example, the client can open 100 streams and send a request on each of them in a single round trip; the proxy will read and process each stream serially, but the requests to the backend servers can again be parallelized. The client can then open new streams as it receives responses to the previous ones. This gives an effective throughput for a single connection of 100 requests per round trip, with similar round trip timing constants to HTTP/1.1 requests. This will typically lead to almost 100 times higher utilization of each connection. ([View Highlight](https://read.readwise.io/read/01hcvk2ta4rfp0g8a1zkn99ac7))
### The HTTP/2 Rapid Reset attack
> The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RST_STREAM frame. The protocol does not require the client and server to coordinate the cancellation in any way, the client may do it unilaterally ([View Highlight](https://read.readwise.io/read/01hcvmtpct8z5khgq4m5qzp0zx))
> This attack is called Rapid Reset because it relies on the ability for an endpoint to send a RST_STREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request. The request is canceled, but leaves the HTTP/2 connection open. ([View Highlight](https://read.readwise.io/read/01hcvmv5vjts0dm2tb6hmtxftm))
> The HTTP/2 Rapid Reset attack built on this capability is simple: The client opens a large number of streams at once as in the standard HTTP/2 attack, but rather than waiting for a response to each request stream from the server or proxy, the client cancels each request immediately.
> The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight. By explicitly canceling the requests, the attacker never exceeds the limit on the number of concurrent open streams. The number of in-flight requests is no longer dependent on the round-trip time (RTT), but only on the available network bandwidth. ([View Highlight](https://read.readwise.io/read/01hcvmwxmn9h2kyapqvdf500an))
> In a typical HTTP/2 server implementation, the server will still have to do significant amounts of work for canceled requests, such as allocating new stream data structures, parsing the query and doing header decompression, and mapping the URL to a resource. For reverse proxy implementations, the request may be proxied to the backend server before the RST_STREAM frame is processed. The client on the other hand paid almost no costs for sending the requests. This creates an exploitable cost asymmetry between the server and the client.
> Another advantage the attacker gains is that the explicit cancellation of requests immediately after creation means that a reverse proxy server won't send a response to any of the requests. Canceling the requests before a response is written reduces downlink (server/proxy to attacker) bandwidth. ([View Highlight](https://read.readwise.io/read/01hcvmy419hz8skqvhv7t4d9ay))
### HTTP/2 Rapid Reset attack variants
> The first variant does not immediately cancel the streams, but instead opens a batch of streams at once, waits for some time, and then cancels those streams and then immediately opens another large batch of new streams. ([View Highlight](https://read.readwise.io/read/01hcvmz9ahg76t89mzp7zb0afs))
> The second variant does away with canceling streams entirely, and instead optimistically tries to open more concurrent streams than the server advertised. ([View Highlight](https://read.readwise.io/read/01hcvmzs4pega1vp10mprtc8g3))
### A multifaceted approach to mitigations
> We don't expect that simply blocking individual requests is a viable mitigation against this class of attacks — instead the entire TCP connection needs to be closed when abuse is detected. HTTP/2 provides built-in support for closing connections, using the GOAWAY frame type. T ([View Highlight](https://read.readwise.io/read/01hcvn235r5dg94cbn9bsgezmn))
> However, this graceful GOAWAY process is usually not implemented in a way which is robust against malicious clients. This form of mitigation leaves the connection vulnerable to Rapid Reset attacks for too long, and should not be used for building mitigations as it does not stop the inbound requests. Instead, the GOAWAY should be set up to limit stream creation immediately. ([View Highlight](https://read.readwise.io/read/01hcvn2ejrpn1hfq3say06vsbc))
> Mitigations for this attack vector can take multiple forms, but mostly center around tracking connection statistics and using various signals and business logic to determine how useful each connection is. For example, if a connection has more than 100 requests with more than 50% of the given requests canceled, it could be a candidate for a mitigation response. The magnitude and type of response depends on the risk to each platform, but responses can range from forceful GOAWAY frames as discussed before to closing the TCP connection immediately. ([View Highlight](https://read.readwise.io/read/01hcvn3kjertsf3tea1gw7b8ct))
> To mitigate against the non-cancelling variant of this attack, we recommend that HTTP/2 servers should close connections that exceed the concurrent stream limit. This can be either immediately or after some small number of repeat offenses. ([View Highlight](https://read.readwise.io/read/01hcvn448v0qd40shryf7ysmsc))
## New highlights added October 17, 2023 at 4:16 PM
> We do not believe these attack methods translate directly to HTTP/3 (QUIC) due to protocol differences, and Google does not currently see HTTP/3 used as a DDoS attack vector at scale. Despite that, our recommendation is for HTTP/3 server implementations to proactively implement mechanisms to limit the amount of work done by a single transport connection, similar to the HTTP/2 mitigations discussed above. ([View Highlight](https://read.readwise.io/read/01hcyxev18pnbpj7ev99n6w2a2))
---
Title: How It Works: The Novel HTTP/2 ‘Rapid Reset’ DDoS Attack
Author: Juho Snellman
Tags: readwise, articles
date: 2024-01-30
---
# How It Works: The Novel HTTP/2 ‘Rapid Reset’ DDoS Attack
URL:: https://cloud.google.com/blog/products/identity-security/how-it-works-the-novel-http2-rapid-reset-ddos-attack
Author:: Juho Snellman
## AI-Generated Summary
Learn how the new DDoS attack technique Rapid Reset works, and how to mitigate it
## Highlights
> A primary design goal of HTTP/2 was efficiency, and unfortunately the features that make HTTP/2 more efficient for legitimate clients can also be used to make DDoS attacks more efficient. ([View Highlight](https://read.readwise.io/read/01hcvk0axc5mk65eack9v2jbsw))
### Stream multiplexing
> HTTP/2 uses "streams", bidirectional abstractions used to transmit various messages, or "frames", between the endpoints. “Stream multiplexing” is the core HTTP/2 feature which allows higher utilization of each TCP connection. Streams are multiplexed in a way that can be tracked by both sides of the connection while only using one Layer 4 connection. Stream multiplexing enables clients to have multiple in-flight requests without managing multiple individual connections. ([View Highlight](https://read.readwise.io/read/01hcvk1fqkd3y7zemtp9mgbpn7))
> One of the main constraints when mounting a Layer 7 DoS attack is the number of concurrent transport connections. Each connection carries a cost, including operating system memory for socket records and buffers, CPU time for the TLS handshake, as well as each connection needing a unique four-tuple, the IP address and port pair for each side of the connection, constraining the number of concurrent connections between two IP addresses.
> In HTTP/1.1, each request is processed serially. The server will read a request, process it, write a response, and only then read and process the next request. In practice, this means that the rate of requests that can be sent over a single connection is one request per round trip, where a round trip includes the network latency, proxy processing time and backend request processing time. While HTTP/1.1 pipelining is available in some clients and servers to increase a connection's throughput, it is not prevalent amongst legitimate clients.
> With HTTP/2, the client can open multiple concurrent streams on a single TCP connection, each stream corresponding to one HTTP request. The maximum number of concurrent open streams is, in theory, controllable by the server, but in practice clients may open 100 streams per request and the servers process these requests in parallel. It’s important to note that server limits can not be unilaterally adjusted.
> For example, the client can open 100 streams and send a request on each of them in a single round trip; the proxy will read and process each stream serially, but the requests to the backend servers can again be parallelized. The client can then open new streams as it receives responses to the previous ones. This gives an effective throughput for a single connection of 100 requests per round trip, with similar round trip timing constants to HTTP/1.1 requests. This will typically lead to almost 100 times higher utilization of each connection. ([View Highlight](https://read.readwise.io/read/01hcvk2ta4rfp0g8a1zkn99ac7))
### The HTTP/2 Rapid Reset attack
> The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RST_STREAM frame. The protocol does not require the client and server to coordinate the cancellation in any way, the client may do it unilaterally ([View Highlight](https://read.readwise.io/read/01hcvmtpct8z5khgq4m5qzp0zx))
> This attack is called Rapid Reset because it relies on the ability for an endpoint to send a RST_STREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request. The request is canceled, but leaves the HTTP/2 connection open. ([View Highlight](https://read.readwise.io/read/01hcvmv5vjts0dm2tb6hmtxftm))
> The HTTP/2 Rapid Reset attack built on this capability is simple: The client opens a large number of streams at once as in the standard HTTP/2 attack, but rather than waiting for a response to each request stream from the server or proxy, the client cancels each request immediately.
> The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight. By explicitly canceling the requests, the attacker never exceeds the limit on the number of concurrent open streams. The number of in-flight requests is no longer dependent on the round-trip time (RTT), but only on the available network bandwidth. ([View Highlight](https://read.readwise.io/read/01hcvmwxmn9h2kyapqvdf500an))
> In a typical HTTP/2 server implementation, the server will still have to do significant amounts of work for canceled requests, such as allocating new stream data structures, parsing the query and doing header decompression, and mapping the URL to a resource. For reverse proxy implementations, the request may be proxied to the backend server before the RST_STREAM frame is processed. The client on the other hand paid almost no costs for sending the requests. This creates an exploitable cost asymmetry between the server and the client.
> Another advantage the attacker gains is that the explicit cancellation of requests immediately after creation means that a reverse proxy server won't send a response to any of the requests. Canceling the requests before a response is written reduces downlink (server/proxy to attacker) bandwidth. ([View Highlight](https://read.readwise.io/read/01hcvmy419hz8skqvhv7t4d9ay))
### HTTP/2 Rapid Reset attack variants
> The first variant does not immediately cancel the streams, but instead opens a batch of streams at once, waits for some time, and then cancels those streams and then immediately opens another large batch of new streams. ([View Highlight](https://read.readwise.io/read/01hcvmz9ahg76t89mzp7zb0afs))
> The second variant does away with canceling streams entirely, and instead optimistically tries to open more concurrent streams than the server advertised. ([View Highlight](https://read.readwise.io/read/01hcvmzs4pega1vp10mprtc8g3))
### A multifaceted approach to mitigations
> We don't expect that simply blocking individual requests is a viable mitigation against this class of attacks — instead the entire TCP connection needs to be closed when abuse is detected. HTTP/2 provides built-in support for closing connections, using the GOAWAY frame type. T ([View Highlight](https://read.readwise.io/read/01hcvn235r5dg94cbn9bsgezmn))
> However, this graceful GOAWAY process is usually not implemented in a way which is robust against malicious clients. This form of mitigation leaves the connection vulnerable to Rapid Reset attacks for too long, and should not be used for building mitigations as it does not stop the inbound requests. Instead, the GOAWAY should be set up to limit stream creation immediately. ([View Highlight](https://read.readwise.io/read/01hcvn2ejrpn1hfq3say06vsbc))
> Mitigations for this attack vector can take multiple forms, but mostly center around tracking connection statistics and using various signals and business logic to determine how useful each connection is. For example, if a connection has more than 100 requests with more than 50% of the given requests canceled, it could be a candidate for a mitigation response. The magnitude and type of response depends on the risk to each platform, but responses can range from forceful GOAWAY frames as discussed before to closing the TCP connection immediately. ([View Highlight](https://read.readwise.io/read/01hcvn3kjertsf3tea1gw7b8ct))
> To mitigate against the non-cancelling variant of this attack, we recommend that HTTP/2 servers should close connections that exceed the concurrent stream limit. This can be either immediately or after some small number of repeat offenses. ([View Highlight](https://read.readwise.io/read/01hcvn448v0qd40shryf7ysmsc))
> We do not believe these attack methods translate directly to HTTP/3 (QUIC) due to protocol differences, and Google does not currently see HTTP/3 used as a DDoS attack vector at scale. Despite that, our recommendation is for HTTP/3 server implementations to proactively implement mechanisms to limit the amount of work done by a single transport connection, similar to the HTTP/2 mitigations discussed above. ([View Highlight](https://read.readwise.io/read/01hcyxev18pnbpj7ev99n6w2a2))