Add compression dictionary negotiation and decoding to the fetch processing model

[pmeenan]

What problem are you trying to solve?

Compression dictionary transport provides a mechanism for using fetched resources as a compression dictionary for content-encoding of HTTP responses. The IETF draft for the http-level negotiation and compression is here.

Most of the HTTP-level negotiation is defined in the IETF draft but there are a few places where it intersects with the fetch processing model:

• Dictionary cache, associated metadata, partitioning and clearing.
• Matching an existing dictionary against outbound requests in HTTP-network fetch.
• CORS-readability requirements for storing dictionaries.
• CORS-readability requirements for decoding dictionary-compressed responses.

What solutions exist today?

No response

How would you solve it?

Add the necessary processing steps once we are comfortable that they are appropriate.

Anything else?

Chromium is currently the only browser engine experimenting with compression dictionaries so it might not make sense to fold into the standard quite yet but it is worth having the discussion to make sure that the implementation is something that we can all agree with before it gets too far.

[pmeenan]

From WICG/compression-dictionary-transport#52, the integration needs to specify that the Use-As-Dictionary response header is only processed if it was on the final response in a redirect chain.

[annevk]

I think there is enough implementer interest to proceed here. Apologies for the lack of updates on WebKit/standards-positions#160 until now.

[pmeenan]

I'm working on a PR for Fetch now that incorporates the IETF draft. Looking at it, there are two paths I can take.

1. Add steps to HTTP-network fetch to:
   • select a dictionary partitioning key
   • turn dictionary storage off for navigation requests
   • turn off dictionary support for opaque requests
   • fail opaque responses that used dictionaries
   • and then defer to the IETF document for the processing as part of step 7.5 (and include the dictionary spec)
2. Add a new HTTP-network-dictionary fetch step between HTTP-network-or-cache fetch and HTTP-network fetch that does all of the dictionary matching and storage. The content coding would still happen transparently but all of the processing steps for managing the dictionaries themselves would be in fetch.

Option 2 would spell things out directly but would be duplicating a lot of the IETF draft (and require keeping both in sync as any errata or modifications are made).

@annevk do you have a preference for one over the other?

[annevk]

I'm having a hard time evaluating those options without more context. I think we have to setup the request and do some parts of the response processing as we also want to define the interaction with other HTTP headers and such. Invoking algorithms defined in the IETF specification would be good though where possible.

[pmeenan]

Sorry, thinking through it more, it makes sense for fetch to include the steps for managing the dictionaries like it does with cache entries.

Still a lot of fleshing out to do (and I need to define the actual dictionary storage) but does plugging it in something like this make sense? That moves all of the dictionary processing into a separate layer between cache and network.

Does bypassing the middle step for clients that don't support dictionary compression make sense to include?

I can move it to a work-in-progress PR so I can iterate on it there if the general framework looks like a good way to plug it in.

[annevk]

Without thinking about it too much that looks reasonable. I don't think we need to make support optional.