QUIC Retry Offload

1.1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].¶

In this document, these words will appear with that interpretation only when in ALL CAPS. Lower case uses of these words are not to be interpreted as carrying significance described in RFC 2119.¶

For brevity, "Connection ID" will often be abbreviated as "CID".¶

A "Retry Offload" is a hardware or software device that is conceptually separate from a QUIC server that terminates QUIC connections. This document assumes that the Retry Offload and the server have an administrative relationship that allows them to accept common configuation.¶

A "configuration agent" is some entity that determines the common configuration to be distributed to the servers and the Retry Offload.¶

This document uses "QUIC" to refer to the protocol in QUIC version 1 [RFC9000]. Retry offloads can be applied to other versions of QUIC that use Retry packets and have identical information requirements for Retry validation. However, note that source and destination connection IDs are the only relevant data fields that are invariant across QUIC versions [RFC8999].¶

1.2. Notation

All wire formats will be depicted using the notation defined in Section 1.3 of [RFC9000].¶

The example below illustrates the basic framework:¶

Example Structure {
  One-bit Field (1),
  7-bit Field with Fixed Value (7) = 61,
  Field with Variable-Length Integer (i),
  Arbitrary-Length Field (..),
  Variable-Length Field (8..24),
  Field With Minimum Length (16..),
  Field With Maximum Length (..128),
  [Optional Field (64)],
  Repeated Field (8) ...,
}

2.1. Consistent Treatment of Initials

Retry Offloads SHOULD treat Initial packets from the same connection with a uniform policy. Initial packets of the first and second client flight can be difficult to distinguish without expensive decryption of the contents, which is unsuitable under the conditions of a DDoS attack. If the first packet of a connection is admitted without Retry, but the second triggers a Retry, that Retry packet will be ignored and the loss of an Initial coalesced with other packets can impair performance. In some situations, the client does not yet have handshake keys, and dropping further client Initial packets creates a deadlock where the connection cannot progress.¶

The simplest means to ensure this is to require, when active, a Retry Token for all incoming Initial packets, and send a Retry packet otherwise. If the Retry Offload is to be more selective, one technique keeps state on which address/port 4-tuples have been admitted. Another would be to apply a secure hash to the source IP address, port, and connection ID to deterministically compute whether the Initial requires a Retry Token or not. These source values remain consistent over the handshake.¶

However, even with these techniques there is a potential problem when a Retry Offload switches from inactive to active mode. The Retry Offload could admit the first packet while in inactive mode, and then drop subsequent Initials in active mode.¶

If the Retry Offload is always on-path, it MAY keep state on incoming connections while in inactive mode to avoid this problem. If it cannot or will not keep such state, it SHOULD implement "transition mode" for an interval chosen to include the likely Initial packet exchange of most clients (200ms is a sensible default).¶

In transition mode, Retry Offloads process Initial packets with Retry tokens as in active mode. When the Retry Offload receives an Initial packet with no token, it issues a Retry AND forwards the packet to the server. If the client has already received a packet from the server, it will ignore the Retry and the connection will progress normally. If not, the client will reconnect based on the Retry, the server's response to the first initial will be discarded, and the connection will progress normally based on the client's second Initial. Appendix B explores the various possible packet sequences in transition mode.¶

Note that transition mode provides no actual DDoS relief to the server, so its duration should be as short as possible. The Retry Offload can choose not to implement transition mode and cause some client connections to fail.¶

Servers operating behind a Retry Offload SHOULD implement a mechanism that operates whenever a client Initial arrives with a valid Retry token. If there is another connection with identical client Connection ID, IP, and Port, but with an unvalidated address, that connection is immediately and silently terminated. This mechanism eliminates incorrect connection state that is an artifact of transition mode, as explained in Appendix B.¶

2.2. Considerations for Non-Initial Packets

Initial Packets are especially effective at consuming server resources because they cause the server to create connection state. Even when mitigating this load with Retry Packets, the act of validating an Initial Token and sending a Retry Packet is more expensive than the response to a non-Initial packet with an unknown Connection ID: simply dropping it and/or sending a Stateless Reset.¶

Nevertheless, a Retry Offload in Active Mode might desire to shield servers from non-Initial packets that do not correspond to a previously admitted Initial Packet. This has a number of considerations.¶

• If a Retry Offload maintains no per-flow state, it cannot distinguish between valid and invalid non-Initial packets and MUST forward all non-Initial Packets to the server.¶

• For QUIC versions the Retry Offload does not support and are present on the Allow-List (or absent from the Deny-List), the Retry Offload cannot distinguish Initial Packets from other long headers and therefore MUST admit all long headers.¶

• If a Retry Offload keeps per-flow state, it can identify 4-tuples that have been previously approved, admit non-Initial packets from those flows, and drop all others. However, dropping short headers will effectively break Address Migration and NAT Rebinding when in Active Mode, as post-migration packets will arrive with a previously unknown 4-tuple. This policy will also break connection attempts using any new QUIC versions that begin connections with a short header.¶

• If a Retry Offload is integrated with a QUIC-LB routable load balancer [I-D.ietf-quic-load-balancers], it can verify that the Destination Connection ID is routable, and only admit non-Initial packets with routable DCIDs. As the Connection ID encoding is invariant across QUIC versions, the Retry Offload can do this for all short headers.¶

Nothing in this section prevents Retry Offloads from making basic syntax correctness checks on packets with QUIC versions that it understands (e.g., enforcing the Initial Packet datagram size minimum in version 1).¶

3.1. Configuration Agent Actions

See Section 2.¶

3.2. Offload Requirements

A no-shared-state Retry Offload MUST be present on all paths from potential clients to the server. These paths MUST fail to pass QUIC traffic should the offload fail for any reason. That is, if the offload is not operational, the server MUST NOT be exposed to client traffic. Otherwise, servers that have already disabled their Retry capability would be vulnerable to attack.¶

The path between offload and server MUST be free of any potential attackers. Note that this and other requirements above severely restrict the operational conditions in which a no-shared-state Retry Offload can safely operate.¶

Retry tokens generated by the offload MUST have the format below.¶

No-Shared-State Retry Offload Token {
  Token Type (1) = 0,
  ODCIL (7) = 8..20,
  Original Destination Connection ID (64..160),
  Opaque Data (..),
}

The first bit of retry tokens generated by the offload MUST be zero. The token has the following additional fields:¶

ODCIL: The length of the original destination connection ID from the triggering Initial packet. This is in cleartext to be readable for the server, but authenticated later in the token. The Retry Offload SHOULD reject any token in which the value is less than 8.¶

Original Destination Connection ID: This also in cleartext and authenticated later.¶

Opaque Data: This data contains the information necessary to authenticate the Retry token in accordance with the QUIC specification. A straightforward implementation would encode the Retry Source Connection ID, client IP address, and a timestamp in the Opaque Data. A more space-efficient implementation would use the Retry Source Connection ID and Client IP as associated data in an encryption operation, and encode only the timestamp and the authentication tag in the Opaque Data. If the Initial packet alters the Connection ID or source IP address, authentication of the token will fail.¶

Upon receipt of an Initial packet with a token that begins with '0', the Retry Offload MUST validate the token in accordance with the QUIC specification.¶

In active mode, the offload MUST issue Retry packets for all client Initial packets that contain no token, or a token that has the first bit set to '1'. It MUST NOT forward the packet to the server. The offload MUST validate all tokens with the first bit set to '0'. If successful, the offload MUST forward the packet with the token intact. If unsuccessful, it MUST drop the packet. The Retry Offload MAY send an Initial Packet containing a CONNECTION_CLOSE frame with the INVALID_TOKEN error code when dropping the packet.¶

Note that this scheme has a performance drawback. When the Retry Offload is in active mode, clients with a token from a NEW_TOKEN frame will suffer a 1-RTT penalty even though its token provides proof of address.¶

In inactive mode, the offload MUST forward all packets that have no token or a token with the first bit set to '1'. It MUST validate all tokens with the first bit set to '0'. If successful, the offload MUST forward the packet with the token intact. If unsuccessful, it MUST drop the packet.¶

3.3. Server Requirements

A server behind a non-shared-state Retry Offload MUST NOT send Retry packets for a QUIC version the Retry Offload understands. It MAY send Retry for QUIC versions the Retry Offload does not understand.¶

Tokens sent in NEW_TOKEN frames MUST have the first bit set to '1'.¶

If a server receives an Initial Packet with the first bit in the token set to '1', it could be from a server-generated NEW_TOKEN frame and should be processed in accordance with the QUIC specification. If a server receives an Initial Packet with the first bit to '0', it is a Retry token and the server MUST NOT attempt to validate it. Instead, it MUST assume the address is validated, MUST include the packet's Destination Connection ID in a Retry Source Connection ID transport parameter, and MUST extract the Original Destination Connection ID from the token cleartext for use in the transport parameter of the same name.¶

4.1. Token Protection with AEAD

On the wire, the token is presented as:¶

Shared-State Retry Offload Token {
  Token Type (1),
  Key Sequence (7),
  Unique Token Number (96),
  Encrypted Shared-State Retry Offload Token Body (64..),
  AEAD Integrity Check Value (128),
}

The tokens are protected using AES128-GCM as follows:¶

• The Key Sequence is the 7 bit identifier to retrieve the token key and IV.¶

• The AEAD IV, is 96 bits generated by the configuration agent.¶

• The AEAD nonce, N, is formed by XORing the AEAD IV with the 96 bit unique token number.¶

• The associated data is a formatted as a pseudo header by combining the cleartext part of the token with the IP address of the client. The format of the pseudoheader depends on whether the Token Type bit is '1' (a NEW_TOKEN token) or '0' (a Retry token).¶

Shared-State Retry Offload Token Pseudoheader {
  IP Address (128),
  Token Type (1),
  Key Sequence (7),
  Unique Token Number (96),
  [RSCIL (8)],
  [Retry Source Connection ID (0..20)],
}

RSCIL: The Retry Source Connection ID Length in octets. This field is only present when the Token Type is '0'.¶

Retry Source Connection ID: To create a Retry Token, populate this field with the Source Connection ID the Retry packet will use. To validate a Retry token, populate it with the Destination Connection ID of the Initial packet that carries the token. This field is only present when the Token Type is '0'.¶

• The input plaintext for the AEAD is the token body. The output ciphertext of the AEAD is transmitted in place of the token body.¶

• The AEAD Integrity Check Value(ICV), defined in Section 6 of [RFC4106], is computed as part of the AEAD encryption process, and is verified during decryption.¶

4.2. Configuration Agent Actions

The configuration agent generates and distributes a "token key", a "token IV", a key sequence, and the information described in Section 2.¶

4.3. Offload Requirements

In inactive mode, the Retry Offload forwards all packets without further inspection or processing. The rest of this section only applies to a offload in active mode.¶

Retry Offloads MUST NOT issue Retry packets except where explicitly allowed below, to avoid sending a Retry packet in response to a Retry token.¶

The offload MUST generate Retry tokens with the format described above when it receives a client Initial packet with no token.¶

If there is a token of either type, the offload MUST attempt to decrypt it.¶

To decrypt a packet, the offload checks the Token Type and constructs a pseudoheader with the appropriate format for that type, using the bearing packet's Destination Connection ID to populate the Retry Source Connection ID field, if any.¶

A token is invalid if:¶

• it uses an unknown key sequence,¶

• the AEAD ICV does not match the expected value (By construction, it will only match if the client IP Address, and any Retry Source Connection ID, also matches),¶

• the ODCIL, if present, is invalid for a client-generated CID (less than 8 or more than 20 in QUIC version 1),¶

• the Timestamp of a token points to time in the past (however, in order to allow for clock skew, it SHOULD NOT consider tokens to be expired if the Timestamp encodes less than two seconds in the past), or¶

• the port number, if present, does not match the source port in the encapsulating UDP header.¶

Packets with valid tokens MUST be forwarded to the server.¶

The offload MUST drop packets with invalid tokens. If the token is of type '1' (NEW_TOKEN), it MUST respond with a Retry packet. If of type '0', it MUST NOT respond with a Retry packet.¶

4.4. Server Requirements

The server MAY issue Retry or NEW_TOKEN tokens in accordance with [RFC9000]. When doing so, it MUST follow the format above.¶

The server MUST validate all tokens that arrive in Initial packets, as they may have bypassed the Retry Offload. It determines validity using the procedure in Section 4.3.¶

If a valid Retry token, the server populates the original_destination_connection_id transport parameter using the corresponding token field. It populates the retry_source_connection_id transport parameter with the Destination Connection ID of the packet bearing the token.¶

In all other respects, the server processes both valid and invalid tokens in accordance with [RFC9000].¶

For QUIC versions the offload does not support, the server MAY use any token format.¶

5.1. Shared-State Retry Keys

The Shared-State Retry Offload defined in Section 4 describes the format of retry tokens or new tokens protected and encrypted using AES128-GCM. Each token includes a 96 bit randomly generated unique token number, and an 8 bit identifier used to get the AES-GCM encryption context. The AES-GCM encryption context contains a 128 bit key and an AEAD IV. There are three important security considerations for these tokens:¶

• An attacker that obtains a copy of the encryption key will be able to decrypt and forge tokens.¶

• Attackers may be able to retrieve the key if they capture a sufficently large number of retry tokens encrypted with a given key.¶

• Confidentiality of the token data will fail if separate tokens reuse the same 96 bit unique token number and the same key.¶

To protect against disclosure of keys to attackers, offload and servers MUST ensure that the keys are stored securely. To limit the consequences of potential exposures, the lifetime of any given key should be limited.¶

Section 6.6 of [RFC9001] states that "Endpoints MUST count the number of encrypted packets for each set of keys. If the total number of encrypted packets with the same key exceeds the confidentiality limit for the selected AEAD, the endpoint MUST stop using those keys." It goes on with the specific limit: "For AEAD_AES_128_GCM and AEAD_AES_256_GCM, the confidentiality limit is 2^23 encrypted packets; see Appendix B.1." It is prudent to adopt the same limit here, and configure the offload in such a way that no more than 2^23 tokens are generated with the same key.¶

In order to protect against collisions, the 96 bit unique token numbers should be generated using a cryptographically secure pseudorandom number generator (CSPRNG), as specified in Appendix C.1 of the TLS 1.3 specification [RFC8446]. With proper random numbers, if fewer than 2^40 tokens are generated with a single key, the risk of collisions is lower than 0.001%.¶

A.1. Tree Diagram

This summary of the YANG models uses the notation in [RFC8340].¶

module: retry-offload-config
  +--rw retry-offload-config
     +--rw supported-versions*            uint32
     +--rw unsupported-version-default?   enumeration
     +--rw version-exceptions*            uint32
     +--rw token-keys* [key-sequence-number]
        +--rw key-sequence-number    uint8
        +--rw token-key              quic-lb-key
        +--rw token-iv               yang:hex-string

A.2. Shared State Retry Token Test Vectors

In this case, the shared-state retry token is issued by Retry Offload, so the opaque data of shared-state retry token body would be null (Section 4).¶

Configuration:
key_seq 0x00
encrypt_key 0x30313233343536373839303132333435
AEAD_IV 0x313233343536373839303132

Shared-State Retry Offload Token Body:
ODCIL 0x12
RSCIL 0x10
port 0x1a0a
original_destination_connection_id 0x0c3817b544ca1c94313bba41757547eec937
retry_source_connection_id 0x0301e770d24b3b13070dd5c2a9264307
timestamp 0x0000000060c7bf4d

Shared-State Retry Offload Token:
unique_token_number 0x59ef316b70575e793e1a8782
key_sequence 0x00
encrypted_shared_state_retry_offload_token_body
0x7d38b274aa4427c7a1557c3fa666945931defc65da387a83855196a7cb73caac1e28e5346fd76868de94f8b62294
AEAD_ICV 0xf91174fdd711543a32d5e959867f9c22

AEAD related parameters:
client_ip_addr 127.0.0.1
client_port 6666
AEAD_nonce 0x68dd025f45616941072ab6b0
AEAD_associated_data 0x7f00000100000000000000000000000059ef316b70575e793e1a878200

B.1. Handshakes in Progress

If the client hello was admitted in inactive mode, then the client has already received a packet from the server. Although subsequent client Initial packets will trigger a Retry, the client will ignore these packets. Those Initials will also be processed by the server to continue the handshake.¶

B.2. New Connections

After sending a Client Hello in Initial Packet A, a client will rapidly receive a Retry Packet from the Offload and attempt to reconnect accordingly with Initial Packet B.¶

The client will discard any server response to Initial A. If a Retry, it is a second Retry on the connection. If an Initial, its is encrypted with keys derived from Initial A, which have already been discarded, and will be a decryption failure.¶

Initial B's destination connection ID will be new, so the server will process it as a new connection and proceed normally.¶

Unfortunately, the server connection state initiated by Initial A will remain. For this reason, this document suggests that servers silently terminate the older connection. Requiring the address to be validated avoids cases where an attacker simply replays a client Initial with a new Destination Connection ID to terminate a valid connection.¶

Note that there are corner cases involving further packet loss that result in connection timeout. For instance, if the Retry Offload's response to Initial A is lost, then the connection will proceed based on Initial A. If the Retry Offload then switches from transition mode to active mode before the client's second flight arrives, the Retry Offload will drop the Initial packet in that flight, and the connection might deadlock.¶

D.1. since draft-duke-quic-retry-offload-00

• Converted to adopted IETF draft¶

• Cleaner transition from inactive to active mode¶

D.2. since draft-ietf-quic-load-balancers-12

• Separated from the QUIC-LB draft¶

• Renamed "Retry Service" to "Retry Offload"¶

D.3. since draft-ietf-quic-load-balancers-11

• Fixed mistakes in test vectors¶

D.4. since draft-ietf-quic-load-balancers-10

• Refactored algorithm descriptions; made the 4-pass algorithm easier to implement¶

• Revised test vectors¶

• Split YANG model into a server and middlebox version¶

D.5. since draft-ietf-quic-load-balancers-09

• Renamed "Stream Cipher" and "Block Cipher" to "Encrypted Short" and "Encrypted Long"¶

• Added section on per-connection state¶

• Changed "Encrypted Short" to a 4-pass algorithm.¶

• Recommended a random initial nonce when incrementing.¶

• Clarified what SNI LBs should do with unknown QUIC versions.¶

D.6. since draft-ietf-quic-load-balancers-08

• Eliminate Dynamic SID allocation¶

• Eliminated server use bytes¶

D.7. since draft-ietf-quic-load-balancers-07

• Shortened SSCID nonce minimum length to 4 bytes¶

• Removed RSCID from Retry token body¶

• Simplified CID formats¶

• Shrunk size of SID table¶

D.8. since draft-ietf-quic-load-balancers-06

• Added interoperability with DTLS¶

• Changed "non-compliant" to "unroutable"¶

• Changed "arbitrary" algorithm to "fallback"¶

• Revised security considerations for mistrustful tenants¶

• Added Retry Offload considerations for non-Initial packets¶

D.9. since draft-ietf-quic-load-balancers-05

• Added low-config CID for further discussion¶

• Complete revision of shared-state Retry Token¶

• Added YANG model¶

• Updated configuration limits to ensure CID entropy¶

• Switched to notation from quic-transport¶

D.10. since draft-ietf-quic-load-balancers-04

• Rearranged the shared-state retry token to simplify token processing¶

• More compact timestamp in shared-state retry token¶

• Revised server requirements for shared-state retries¶

• Eliminated zero padding from the test vectors¶

• Added server use bytes to the test vectors¶

• Additional compliant DCID criteria¶

D.11. since-draft-ietf-quic-load-balancers-03

• Improved Config Rotation text¶

• Added stream cipher test vectors¶

• Deleted the Obfuscated CID algorithm¶

D.12. since-draft-ietf-quic-load-balancers-02

• Replaced stream cipher algorithm with three-pass version¶

• Updated Retry format to encode info for required TPs¶

• Added discussion of version invariance¶

• Cleaned up text about config rotation¶

• Added Reset Oracle and limited configuration considerations¶

• Allow dropped long-header packets for known QUIC versions¶

D.13. since-draft-ietf-quic-load-balancers-01

• Test vectors for load balancer decoding¶

• Deleted remnants of in-band protocol¶

• Light edit of Retry Offloads section¶

• Discussed load balancer chains¶

D.14. since-draft-ietf-quic-load-balancers-00

• Removed in-band protocol from the document¶

D.15. Since draft-duke-quic-load-balancers-06

• Switch to IETF WG draft.¶

D.16. Since draft-duke-quic-load-balancers-05

• Editorial changes¶

• Made load balancer behavior independent of QUIC version¶

• Got rid of token in stream cipher encoding, because server might not have it¶

• Defined "non-compliant DCID" and specified rules for handling them.¶

• Added psuedocode for config schema¶

D.17. Since draft-duke-quic-load-balancers-04

• Added standard for Retry Offloads¶

D.18. Since draft-duke-quic-load-balancers-03

• Renamed Plaintext CID algorithm as Obfuscated CID¶

• Added new Plaintext CID algorithm¶

• Updated to allow 20B CIDs¶

• Added self-encoding of CID length¶

D.19. Since draft-duke-quic-load-balancers-02

• Added Config Rotation¶

• Added failover mode¶

• Tweaks to existing CID algorithms¶

• Added Block Cipher CID algorithm¶

• Reformatted QUIC-LB packets¶

D.20. Since draft-duke-quic-load-balancers-01

• Complete rewrite¶

• Supports multiple security levels¶

• Lightweight messages¶

D.21. Since draft-duke-quic-load-balancers-00

• Converted to markdown¶

• Added variable length connection IDs¶