Roy Fielding’s REST alternative to HTTP cookies

Question

I struggle to understand Roy Fielding’s REST alternative to HTTP cookies, § 6.3.4.2 ‘Cookies’ and § 6.5.4.1 ‘Application State in a Network-Based System’ of his doctoral dissertation Architectural Styles and the Design of Network-Based Software Architectures. How do you interpret it?

6.3.4.2 Cookies

An example of where an inappropriate extension has been made to the protocol to support features that contradict the desired properties of the generic interface is the introduction of site-wide state information in the form of HTTP cookies [73]. Cookie interaction fails to match REST's model of application state, often resulting in confusion for the typical browser application.

An HTTP cookie is opaque data that can be assigned by the origin server to a user agent by including it within a Set-Cookie response header field, with the intention being that the user agent should include the same cookie on all future requests to that server until it is replaced or expires. Such cookies typically contain an array of user-specific configuration choices, or a token to be matched against the server's database on future requests. The problem is that a cookie is defined as being attached to any future requests for a given set of resource identifiers, usually encompassing an entire site, rather than being associated with the particular application state (the set of currently rendered representations) on the browser. When the browser's history functionality (the "Back" button) is subsequently used to back-up to a view prior to that reflected by the cookie, the browser's application state no longer matches the stored state represented within the cookie. Therefore, the next request sent to the same server will contain a cookie that misrepresents the current application context, leading to confusion on both sides.

Cookies also violate REST because they allow data to be passed without sufficiently identifying its semantics, thus becoming a concern for both security and privacy. The combination of cookies with the Referer [sic] header field makes it possible to track a user as they browse between sites.

As a result, cookie-based applications on the Web will never be reliable. The same functionality should have been accomplished via anonymous authentication and true client-side state. A state mechanism that involves preferences can be more efficiently implemented using judicious use of context-setting URI rather than cookies, where judicious means one URI per state rather than an unbounded number of URI due to the embedding of a user-id. Likewise, the use of cookies to identify a user-specific "shopping basket" within a server-side database could be more efficiently implemented by defining the semantics of shopping items within the hypermedia data formats, allowing the user agent to select and store those items within their own client-side shopping basket, complete with a URI to be used for check-out when the client is ready to purchase.

6.5.4.1 Application State in a Network-based System

REST defines a model of expected application behavior which supports simple and robust applications that are largely immune from the partial failure conditions that beset most network-based applications. However, that doesn't stop application developers from introducing features which violate the model. The most frequent violations are in regard to the constraints on application state and stateless interaction.

Architectural mismatches due to misplaced application state are not limited to HTTP cookies. The introduction of "frames" to the Hypertext Markup Language (HTML) caused similar confusion. Frames allow a browser window to be partitioned into subwindows, each with its own navigational state. Link selections within a subwindow are indistinguishable from normal transitions, but the resulting response representation is rendered within the subwindow instead of the full browser application workspace. This is fine provided that no link exits the realm of information that is intended for subwindow treatment, but when it does occur the user finds themself viewing one application wedged within the subcontext of another application.

For both frames and cookies, the failure was in providing indirect application state that could not be managed or interpreted by the user agent. A design that placed this information within a primary representation, thereby informing the user agent on how to manage the hypermedia workspace for a specified realm of resources, could have accomplished the same tasks without violating the REST constraints, while leading to a better user interface and less interference with caching.

• How do we carry user preferences stored in a URI across multiple actions?
  Example. — I assume that Roy Fielding meant that the server should propagate user preferences in the hyperlinks of response representations: from the HTML representation in light mode of the URI /preferences of a website, a user selects a dark theme button which is an hyperlink to the URI /preferences?theme=dark. The server responds with an HTML representation in dark mode with hyperlinks to URIs updated with an additional query component theme=dark. So if the user then selects the website logo which is an hyperlink to the URI /home?theme=dark (not /home), the server responds again with an HTML representation in dark mode with hyperlinks to URIs updated with an additional query component theme=dark.
• How do we embed shopping items in the HTML hypermedia format?
  Example. — I assume that Roy Fielding meant that HTML should define a new <shopping> element for representing shopping items.

  <!DOCTYPE html>
  <html lang="en">
  <title>Shop</title>
  <h1>Our products</h1>
  <shopping name="beer" stock="4" price="5.99" currency="USD">
    <quantity>1<quantity/> Beer (USD 5.99)
  </shopping>
  <shopping name="cider" stock="1" price="3.99" currency="USD">
    <quantity>0<quantity/> Cider (USD 3.99)
  </shopping>
  <shopping name="wine" stock="3" price="9.99" currency="USD">
    <quantity>0<quantity/> Wine (USD 9.99)
  </shopping>
  </html>
  

• How do we store user shopping cart in the client without Javascript?
  Example. — I assume that Roy Fielding meant that if the HTML hypermedia format supported a new <shopping> element for representing shopping items, browsers would understand them natively and therefore would be able to update their application states according to user interactions with the <shopping> elements, instead of relying on client-side scripting.

Answer 1

Just to disambiguate things a bit. "Application State" in REST is used to describe the state of the conversation taking place. So how far are you in your order process, on which page are you in the "wizard" or things like that.

What constitutes "Application State" and just plain "Server State" is sort-of fluid. Your example about "theme" is one of these things. You can design that as "Application State" (to be transient for the server), but it can be easily defined as "Server State", that is, stored an represented on the server (you know, stored on your profile, or something like that).

So there is nuance to this. With that in mind, I'll try to answer some of your questions:

How do we carry user preferences stored in a URI across multiple actions?

You can use both the URI and the content for that. One pretty obvious way is using "hidden" input fields for example, if you're dealing with forms. This was (is?) a pretty common way to deal with "wizard" style multi-page processes.

How do we embed shopping items in the HTML hypermedia format?

Again, you can use URIs and hidden input fields, but you more than likely want some more persistent solution, since you don't want items to disappear when the user hits the 'back' button.

Local storage is also one option. Making it essentially "Client State", which is a third thing separate from "Application State" and "Server State".

The client is also free to have its own state. Like when I use Amazon and I want to buy 2 things, I'll keep the progress in my head and order when I found both items. That is client state.

Again, the thing that is not cool, is to keep "Application State" on the server (or anywhere else other than the representations the client currently sees), since our goal is that the server can process each request completely on its own, independently of any previous messages, and that the client can control the "Application State" exclusively by choosing server provided options in the current representations it sees.

How do we store user shopping cart in the client without Javascript?

You can use URI rewriting, and/or hidden inputs. Or you can make it an "official" server state, and have proper resources for it. Or you can use Javascript and local storage, that is also not against Fielding's REST. It is in fact explicitly mentioned that the server can push code onto the client to extend its capabilities.

What you can't (shouldn't) do according to REST is have a Cookie which is essentially a pointer to a server-side application state, and/or is an opaque application state the client doesn't know about, therefore can't/doesn't control.

I hope I managed to clear up some points.

Answer 2

HTTP was initially designed as a stateless protocol, which means a server should not need to maintain any client-specific session state between requests. Instead all information necessary to process a given request should be included in the request itself. State between requests should only exist on the client side.

The cookies protocol was introduced to make it possible to track client-specific state on the server, for example to maintain a shopping basket or preferences between request. But Roy Fielding argues the design of the cookie protocol is not the best and breaks several of the core design principles behind the web.

So what does he propose instead? He describes two scenarios which are often implemented using cookies and describe how they could be implemented in a way which does not break the core principles.

The first is the use of cookies for user-specific preferences, e.g choice of language or (as in your example) theme. Fielding describes how this can be supported using just URLs.

A web site (or the web as a whole) could be considered a state machine where each state is identified by an URL. The client navigate this state graph by following links. You can always go back to a previous state by using the back button. We usually think of this as navigating pages, but any state change (e.g. selecting a language or theme) could be implemented as following links. Your example with dark theme describes this pretty well.

The second example is a shopping cart. Fielding recognizes that encoding a shopping cart state in URLs is not ideal. He doesn't say why, but I imaging because it is rater counter intuitive that going to a product page, add a product, and then press the back button will remove the product again rather then return to the previous page. Bookmarking a page would also bookmark the exact state of the shopping basket at that point, which would be rather weird.

So there is a need for maintaining state separately from the state encoded in in the URL. But he argues that such a state should be purely client side. The server would not track the current state of the shopping basket per user. Only when the user is ready to order would the whole cart state be passed to the server in a POST request.

But he does not describe in detail how such a technique could be implemented, since his intention in the paper is just to analyze the current state of affairs, not to propose any new protocols. He is not even talking about HTML specifically.

Today a client-side shopping-basket could be implemented using a SPA or using LocalStorage. But neither was realistically available 20 years ago when Fielding wrote the paper. So he is not proposing people stop using cookies, he is just describing some issues with the design. It was still (at least at that point in time) the best solution which was actually available.

How do we store user shopping cart in the client without Javascript?

You can't. There is currently no mechanism which allows you to save state (apart from navigation state) on the client without the use of scripting.

It is possible that Fielding had a hypothetical feature in mind which did not rely on scripting. This would instead require some built-in support in the browser which does not currently exist. But this is pure speculation since he does not describe any concrete solution but only the constraints which such a solution should fulfill.

But Fielding is is not averse to scripting on the client side and does not see JavaScript as in conflict with REST.