HTTP/2 Gzipped Datamatthew@kerwin.net.auhttp://matthew.kerwin.net.au/
Applications and Real-Time
HTTPH2GZIPThis document introduces a new frame type for transporting gzip-encoded data between peers in the
Hypertext Transfer Protocol Version 2 (HTTP/2), and an associated error code for handling
invalid encoding.Note to ReadersThe issues list for this draft can be found at https://github.com/phluid61/internet-drafts/labels/HTTP%2F2%20Gzipped%20DataThe most recent (often unpublished) draft is at http://phluid61.github.io/internet-drafts/http2-encoded-data/This document introduces a mechanism for applying gzip encoding to data
transported between two endpoints in the Hypertext Transfer Protocol Version 2 (HTTP/2) ,
analogous to Transfer-Encoding in HTTP/1.1 .The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this
document are to be interpreted as described in .This document introduces a new HTTP/2 frame type (, Section 11.2),
a new HTTP/2 setting (, Section 11.3),
and a new HTTP/2 error code (, Section 7), to allow the compression
of data.Note that while compressing some or all data in a stream might affect the total length of the
corresponding HTTP message body, the content-length header, if present, should continue to
reflect the total length of the uncompressed data. This is particularly relevant when detecting
malformed messages (, Section 8.1.2.6).SETTINGS_ACCEPT_GZIPPED_DATA (0xTBA) is used to indicate the sender’s ability and
willingness to receive GZIPPED_DATA frames. An endpoint MUST NOT send a GZIPPED_DATA
frame unless it receives this setting with a value of 1.The initial value is 0, which indicates that GZIPPED_DATA frames are not supported. Any
value other than 0 or 1 MUST be treated as a connection error (, Section 5.4.1)
of type PROTOCOL_ERROR.An endpoint may advertise support for GZIPPED_DATA frames and later decide that it no longer
supports them. After sending an ACCEPT_GZIPPED_DATA setting with the value 0, the endpoint
SHOULD continue to accept GZIPPED_DATA frames for a reasonable amount of time to account for
frames that may already be in flight.GZIPPED_DATA frames (type code=0xTBA) are semantically identical to DATA frames
(, Section 6.1), but their payload is encoded using gzip compression.
Significantly: the order of DATA and GZIPPED_DATA frames is semantically significant; and
GZIPPED_DATA frames are subject to flow control (, Section 5.2).
Gzip compression is an LZ77 coding with a 32 bit CRC that is commonly produced
by the gzip file compression program .Any compression or decompression context for a GZIPPED_DATA frame is unique to that frame.
An endpoint MAY interleave DATA and GZIPPED_DATA frames on a single stream.The GZIPPED_DATA frame contains the following fields:Pad Length:
An 8-bit field containing the length of the frame padding in units
of octets. This field is optional and is only present if the
PADDED flag is set.Data:
Encoded application data. The amount of encoded data is the remainder of the frame
payload after subtracting the length of the other fields that are
present.Padding:
Padding octets that contain no application semantic value. Padding
octets MUST be set to zero when sending and ignored when receiving.The GZIPPED_DATA frame defines the following flags:END_STREAM (0x1):
Bit 1 being set indicates that this frame is the last that the
endpoint will send for the identified stream. Setting this flag
causes the stream to enter one of the “half closed” states or the
“closed” state (, Section 5.1).PADDED (0x8):
Bit 4 being set indicates that the Pad Length field is present.A GZIPPED_DATA frame MUST NOT be sent if the ACCEPT_GZIPPED_DATA setting
of the peer is set to 0. See .An intermediary, on receiving a GZIPPED_DATA frame, MAY decode the data and forward it to its
downstream peer in one or more DATA frames. If the downstream peer has not advertised support
for GZIPPED_DATA frames (by sending an ACCEPT_GZIPPED_DATA setting with the value 1) the
intermediary MUST decode the data before forwarding it.If an endpoint detects that the payload of a GZIPPED_DATA frame is not encoded correctly,
for example with an incorrect checksum, the endpoint MUST
treat this as a stream error (see , Section 5.4.2) of type
DATA_ENCODING_ERROR (). The endpoint MAY then choose to immediately send an
ACCEPT_GZIPPED_DATA setting with the value 0.If an intermediary propagates a GZIPPED_DATA frame from the source peer to the destination peer
without modifying the payload or its encoding, and receives a DATA_ENCODING_ERROR from the
receiving peer, it SHOULD pass the error on to the source peer.GZIPPED_DATA frames MUST be associated with a stream. If a GZIPPED_DATA frame is received whose
stream identifier field is 0x0, the recipient MUST respond with a connection error
(, Section 5.4.1) of type PROTOCOL_ERROR.GZIPPED_DATA frames are subject to flow control and can only be sent when a stream is in the
“open” or “half closed (remote)” states. The entire GZIPPED_DATA frame payload is included in flow
control, including the Pad Length and Padding fields if present. If a
GZIPPED_DATA frame is received whose stream is not in “open” or “half closed (local)” state, the
recipient MUST respond with a stream error (, Section 5.4.2) of type
STREAM_CLOSED.GZIPPED_DATA frames can include padding. Padding fields and flags are identical to those defined
for DATA frames (, Section 6.1).The following new error code is defined:DATA_ENCODING_ERROR (0xTBA):
The endpoint detected that its peer sent a GZIPPED_DATA frame with an invalid encoding.This extension is classified as an experiment because it alters the base semantics of HTTP/2;
a change that, if specified insufficiently or implemented incorrectly, could result in data loss
that is hard to detect or diagnose., Section 5.5, mandates that “implementations MUST discard frames that have unknown
or unsupported types”; so if an endpoint or intermediary mishandles GZIPPED_DATA frames, for
example by incorrectly emitting an ACCEPT_GZIPPED_DATA setting or propagating GZIPPED_DATA
frames, and those frames are subsequently discarded, data will be lost. There is no reliable
mechanism to detect such a loss[*].The experiment therefore is to explore the robustness of the HTTP/2 ecosystem in the presence of
such potential failures.[*] For some unreliable mechanisms (i.e. not guaranteed to be in use in all cases, and/or
requiring inspection of HTTP headers) see:Section 8.1.2.6 of , for using the content-length header to detect malformed messages, for HTTP instance digestsFurther to the Use of Compression in HTTP/2 (, Section 10.6),
intermediaries MUST NOT apply compression to DATA frames, or alter the compression of
GZIPPED_DATA frames other than decompressing, unless additional information is available
that allows the intermediary to identify the source of data. In particular, frames that
are not compressed cannot be compressed, and frames that are separately compressed cannot
be merged into a single compressed frame.This document updates the registries for frame types, settings, and error codes in
the “Hypertext Transfer Protocol (HTTP) 2 Parameters” section.This document updates the “HTTP/2 Frame Type” registry
(, Section 11.2). The entries in the
following table are registered by this document.Frame TypeCodeSectionGZIPPED_DATATBDThis document updates the “HTTP/2 Settings” registry
(, Section 11.3). The entries in the
following table are registered by this document.Frame TypeCodeInitial ValueSpecificationACCEPT_GZIPPED_DATATBD0This document updates the “HTTP/2 Error Code” registry
(, Section 11.4). The entries in the
following table are registered by this document.NameCodeDescriptionSpecificationDATA_ENCODING_ERRORTBDInvalid encoding detectedThanks to Keith Morgan for his advice, input, and editorial contributions.GZIP file format specification version 4.3This specification defines a lossless compressed data format that is compatible with the widely used GZIP utility. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind.Key words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Hypertext Transfer Protocol Version 2 (HTTP/2)This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection. It also introduces unsolicited push of representations from servers to clients.This specification is an alternative to, but does not obsolete, the HTTP/1.1 message syntax. HTTP's existing semantics remain unchanged.Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and RoutingThe Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations.Instance Digests in HTTPHTTP/1.1 defines a Content-MD5 header that allows a server to include a digest of the response body. However, this is specifically defined to cover the body of the actual message, not the contents of the full file (which might be quite different, if the response is a Content-Range, or uses a delta encoding). Also, the Content-MD5 is limited to one specific digest algorithm; other algorithms, such as SHA-1 (Secure Hash Standard), may be more appropriate in some circumstances. Finally, HTTP/1.1 provides no explicit mechanism by which a client may request a digest. This document proposes HTTP extensions that solve these problems. [STANDARDS-TRACK]Since -07:define “reliable” in the ‘experimental’ section, and provide pointers
to potential workaroundsremove fragmentation, since the text added no valueSince -06:change document title from “Encoded” to “Gzipped”improve text under GZIPPED_DATA ()clarify that GZIPPED_DATA and DATA can be interleavedexplain experimental status and risks of broken implementationsSince -05:changed ACCEPT_ENCODED_DATA back from a frame to a setting, since it
carries a single scalar value nowSince -04:reduced encoding options to only gzip (suggested by Martin Thomson)remove fragmentation and segment stuff, including reference to ‘http2-segments’ I-Dupdated HTTP/2 reference from I-D to (freshly published) RFC7230Since -03:added ‘identity’ encoding; removed ‘compress’ and ‘zlib’ (suggested by PHK)added SEGMENT flag, for segments that don’t continueclarified that ACCEPT is for a connection, and ENCODED_DATA is for a streamcopied “padding” text from HTTP/2 draftSince -02:moved all discussion of fragmentation and segments to its own sectionSince -01:referenced new draft-kerwin-http2-segments to handle fragmentationSince -00:changed ACCEPT_ENCODED_DATA from a complex setting to a frameimproved IANA Considerations section (with lots of input from Keith Morgan)