Understanding the stateless internet

Question

I'm transitioning from being a desktop developer to a web developer, and I'm having trouble understanding why HTTP is stateless. What are the reasons for it? What are some ways a desktop developer like myself can make the transition to a stateless development environment?

Answer 1

This is the best explanation of the stateless internet that I have seen:

How I Explained REST  to My Wife
http://www.looah.com/source/view/2284

Wife: Who is Roy Fielding?

Ryan: Some guy. He's smart.

Wife: Oh? What did he do?

Ryan: He helped write the first web servers and then did a ton of research explaining why the web works the way it does. His name is on the specification for the protocol that is used to get pages from servers to your browser.

Wife: How does it work?

Ryan: The web?

Wife: Yeah.

Ryan: Hmm. Well, it's all pretty amazing really. And the funny thing is that it's all very undervalued. The protocol I was talking about, HTTP, it's capable of all sorts of neat stuff that people ignore for some reason.

Wife: You mean http like the beginning of what I type into the browser?

Ryan: Yeah. That first part tells the browser what protocol to use. That stuff you type in there is one of the most important breakthroughs in the history of computing.

Wife: Why?

Ryan: Because it is capable of describing the location of something anywhere in the world from anywhere in the world. It's the foundation of the web. You can think of it like GPS coordinates for knowledge and information.

Wife: For web pages?

Ryan: For anything really. That guy, Roy Fielding, he talks a lot about what those things point to in that research I was talking about. The web is built on an architectural style called REST. REST provides a definition of a resource, which is what those things point to.

Wife: A web page is a resource?

Ryan: Kind of. A web page is a representation of a resource. Resources are just concepts. URLs--those things that you type into the browser...

Wife: I know what a URL is..

Ryan: Oh, right. Those tell the browser that there's a concept somewhere. A browser can then go ask for a specific representation of the concept. Specifically, the browser asks for the web page representation of the concept.

Wife: What other kinds of representations are there?

Ryan: Actually, representations is one of these things that doesn't get used a lot. In most cases, a resource has only a single representation. But we're hoping that representations will be used more in the future because there's a bunch of new formats popping up all over the place.

Wife: Like what?

Ryan: Hmm. Well, there's this concept that people are calling Web Services. It means a lot of different things to a lot of different people but the basic concept is that machines could use the web just like people do.

Wife: Is this another robot thing?

Ryan: No, not really. I don't mean that machines will be sitting down at the desk and browsing the web. But computers can use those same protocols to send messages back and forth to each other. We've been doing that for a long time but none of the techniques we use today work well when you need to be able to talk to all of the machines in the entire world.

Wife: Why not?

Ryan: Because they weren't designed to be used like that. When Fielding and his buddies started building the web, being able to talk to any machine anywhere in the world was a primary concern. Most of the techniques we use at work to get computers to talk to each other didn't have those requirements. You just needed to talk to a small group of machines.

Wife: And now you need to talk to all the machines?

Ryan: Yes - and more. We need to be able to talk to all machines about all the stuff that's on all the other machines. So we need some way of having one machine tell another machine about a resource that might be on yet another machine.

Wife: What?

Ryan: Let's say you're talking to your sister and she wants to borrow the sweeper or something. But you don't have it - your Mom has it. So you tell your sister to get it from your Mom instead. This happens all the time in real life and it happens all the time when machines start talking too.

Wife: So how do the machines tell each other where things are?

Ryan: The URL, of course. If everything that machines need to talk about has a corresponding URL, you've created the machine equivalent of a noun. That you and I and the rest of the world have agreed on talking about nouns in a certain way is pretty important, eh?

Wife: Yeah.

Ryan: Machines don't have a universal noun - that's why they suck. Every programming language, database, or other kind of system has a different way of talking about nouns. That's why the URL is so important. It let's all of these systems tell each other about each other's nouns.

Wife: But when I'm looking at a web page, I don't think of it like that.

Ryan: Nobody does. Except Fielding and handful of other people. That's why machines still suck.

Wife: What about verbs and pronouns and adjectives?

Ryan: Funny you asked because that's another big aspect of REST. Well, verbs are anyway.

Wife: I was just joking.

Ryan: It was a funny joke but it's actually not a joke at all. Verbs are important. There's a powerful concept in programming and CS theory called polymorphism. That's a geeky way of saying that different nouns can have the same verb applied to them.

Wife: I don't get it.

Ryan: Well.. Look at the coffee table. What are the nouns? Cup, tray, newspaper, remote. Now, what are some things you can do to all of these things?

Wife: I don't get it...

Ryan: You can get them, right? You can pick them up. You can knock them over. You can burn them. You can apply those same exact verbs to any of the objects sitting there.

Wife: Okay... so?

Ryan: Well, that's important. What if instead of me being able to say to you, "get the cup," and "get the newspaper," and "get the remote"; what if instead we needed to come up with different verbs for each of the nouns? I couldn't use the word "get" universally, but instead had to think up a new word for each verb/noun combination.

Wife: Wow! That's weird.

Ryan: Yes, it is. Our brains are somehow smart enough to know that the same verbs can be applied to many different nouns. Some verbs are more specific than others and apply only to a small set of nouns. For instance, I can't drive a cup and I can't drink a car. But some verbs are almost universal like GET, PUT, and DELETE.

Wife: You can't DELETE a cup.

Ryan: Well, okay, but you can throw it away. That was another joke, right?

Wife: Yeah.

Ryan: So anyway, HTTP--this protocol Fielding and his friends created--is all about applying verbs to nouns. For instance, when you go to a web page, the browser does an HTTP GET on the URL you type in and back comes a web page.

Web pages usually have images, right? Those are separate resources. The web page just specifies the URLs to the images and the browser goes and does more HTTP GETs on them until all the resources are obtained and the web page is displayed. But the important thing here is that very different kinds of nouns can be treated the same. Whether the noun is an image, text, video, an mp3, a slideshow, whatever. I can GET all of those things the same way given a URL.

Wife: Sounds like GET is a pretty important verb.

Ryan: It is. Especially when you're using a web browser because browsers pretty much justGET stuff. They don't do a lot of other types of interaction with resources. This is a problem because it has led many people to assume that HTTP is just for GETing. But HTTP is actually ageneral purpose protocol for applying verbs to nouns.

Wife: Cool. But I still don't see how this changes anything. What kinds of nouns and verbs do you want?

Ryan: Well the nouns are there but not in the right format.

Think about when you're browsing around amazon.com looking for things to buy me for Christmas. Imagine each of the products as being nouns. Now, if they were available in a representation that a machine could understand, you could do a lot of neat things.

Wife: Why can't a machine understand a normal web page?

Ryan: Because web pages are designed to be understood by people. A machine doesn't care about layout and styling. Machines basically just need the data. Ideally, every URL would have a human readable and a machine readable representation. When a machine GETs the resource, it will ask for the machine readable one. When a browser GETs a resource for a human, it will ask for the human readable one.

Wife: So people would have to make machine formats for all their pages?

Ryan: If it were valuable.

Look, we've been talking about this with a lot of abstraction. How about we take a real example. You're a teacher - at school I bet you have a big computer system, or three or four computer systems more likely, that let you manage students: what classes they're in, what grades they're getting, emergency contacts, information about the books you teach out of, etc. If the systems are web-based, then there's probably a URL for each of the nouns involved here: student, teacher, class, book, room, etc. Right now, getting the URL through the browser gives you a web page. If there were a machine readable representation for each URL, then it would be trivial to latch new tools onto the system because all of that information would be consumable in a standard way. It would also make it quite a bit easier for each of the systems to talk to each other. Or, you could build a state or country-wide system that was able to talk to each of the individual school systems to collect testing scores. The possibilities are endless.

Each of the systems would get information from each other using a simple HTTP GET. If one system needs to add something to another system, it would use an HTTP POST. If a system wants to update something in another system, it uses an HTTP PUT. The only thing left to figure out is what the data should look like.

Wife: So this is what you and all the computer people are working on now? Deciding what the data should look like?

Ryan: Sadly, no. Instead, the large majority are busy writing layers of complex specifications for doing this stuff in a different way that isn't nearly as useful or eloquent. Nouns aren't universal and verbs aren't polymorphic. We're throwing out decades of real field usage and proven technique and starting over with something that looks a lot like other systems that have failed in the past. We're using HTTP but only because it helps us talk to our network and security people less. We're trading simplicity for flashy tools and wizards.

Wife: Why?

Ryan: I have no idea.

Wife: Why don't you say something?

Ryan: Maybe I will.

Answer 2

How do you think it would be possible to store the state of billions of billions of billions of billions connections? :) So you only store the state where needed, in sessions.

BTW: HTTP is not connectionless.

Answer 3

As a desktop developer you may be more comfortable with rich UI experiences. Moving to the web can feel like taking a step back. In the web world, there is less freedom of creativity and it can give you a sense of constraint. Don't let that get you down! There are a number of things out there that can help you make the transition and here is a short list of them:

1. State can be shared but is frequently held on the server and referenced using tokens such as session id's, URL parameters, hidden fields or cookie values.
2. Stateless models are well suited for transaction processing. Try to build your model in such a way that can reduce the amount of state required. The ACID principles of transaction processing can help you achieve this.
3. Become familiar with the MVC pattern (if you are not already). This will help improve your design by maintaining a separation of concerns. Some popular frameworks such as Struts (Java) and MVC (.NET) are built around this concept.
4. Consider using a plugin such as Java, Flash or Silverlight for super-rich UI experiences. For semi-rich experiences, consider using popular java-script based libraries such as JQuery or AJAX.

Happy programming!

Answer 4

If you mean bi-directional browsers.

Security reasons.

For example SPAM!.

Taking Bidirectional Communication On The Web To The Next Level

Otherwise internet runs TCP/IP (two protocols) and UDP.

The Transmission Control Protocol (TCP) is one of the core protocols of the Internet Protocol Suite. TCP is one of the two original components of the suite, complementing the Internet Protocol (IP), and therefore the entire suite is commonly referred to as TCP/IP. TCP provides the service of exchanging data directly between two hosts on the same network, whereas IP handles addressing and routing message across one or more networks. In particular, TCP provides reliable, ordered delivery of a stream of bytes from a program on one computer to another program on another computer. TCP is the protocol that major Internet applications rely on, applications such as the World Wide Web, e-mail, and file transfer. Other applications, which do not require reliable data stream service, may use the User Datagram Protocol (UDP) which provides a datagram service that emphasizes reduced latency over reliability.

The Internet Protocol (IP) is the principal communications protocol used for relaying datagrams (packets) across an internetwork using the Internet Protocol Suite. Responsible for routing packets across network boundaries, it is the primary protocol that establishes the Internet. IP is the primary protocol in the Internet Layer of the Internet Protocol Suite and has the task of delivering datagrams from the source host to the destination host solely based on their addresses. For this purpose, IP defines addressing methods and structures for datagram encapsulation. Historically, IP was the connectionless datagram service in the original Transmission Control Program introduced by Vint Cerf and Bob Kahn in 1974, the other being the connection-oriented Transmission Control Protocol (TCP). The Internet Protocol Suite is therefore often referred to as TCP/IP.

Answer 5

Because there was a time where there weren't millions upon millions of webpages. Because there was a time when only universities and research facilities had a couple pages. There was a time when there was not broadband, and http was communicated with 1200 baud modems placed on top of desk telephones. There was a time when "rich web apps" would have required, in their view, a ridiculous amount of bandwidth. And remember, TCP/IP was created because the early internet was very unreliable.

HTTP 1.0 was around in the early 1990s. Think about how the internet of then was, and why they designed it the way they did.

Answer 6

It all sort of evolved. The Internet existed before web browsers and the Web. It was a bubbling pot of ftp, telnet, gopher, ping, finger and a few other bits and bobs. The first web browser, Mosaic (? I think, was a long time ago, 1991 I think, I was at college) acted as a sort of mishmash between ftp and a document viewer. The magic happened in that you could have links in the document which would ftp up a new document.

All the interactivity we have now evolved over the course of the following 20 years. It wasn't a happy evolution either. We had the browser wars, IE and Netscape duked it out for control of standards (Bit of a simplification ;)), and various other 3rd parties started introducing plug ins to allow for rich content. Java was going to be the magic bullet and of course Flash. Does anyone remember the VRML plug ins which promised 3d worlds and delivered exactly half a dozen 3d models of Star Wars models?

I got a bit carried away towards the end, but you get the idea :)

Answer 7

The primary reasons has to do with a combination of what acedemia believed the purpose of HTTP was, and for reasons of scalability. HTML was originally designed to share information or theses accross academic boundaries. It was purely stylized text. It wasn't until the first browser allowed you to serve up pictures that people started thinking beyond that model.

The following considerations solidified the stateless decision:

• The typical interaction was going to be a quick download, and reading. During the delay until the next request, the socket would be sitting idle.
• Sockets take up precious system resources. If we didn't have to maintain a conversation as you would with SMTP, you can do a lot to have one machine handle thousands of clients.
• They already learned precious lessons from managing remote shell accounts, NFS, SMTP and other stateful connection protocols.

As web pages became more complex and included plenty of graphics and stylesheets, HTTP was ammended with the "keep-alive" flag. That would keep the socket live and allow the client to request several resources with the same conversation.

Considering the current useage model of the internet, the original decision is still valid. It can be inconvenient at times, but several small, quantized interactions with a server scales better than idle sockets.

Answer 8

In a desktop application, the user is assumed to be performing some series of tasks, with a defined start and end. In such an application, it makes some sense (not a lot, actually) for users to log into whatever server provides their data, and remain logged in until they're done.

Web interactions don't (typically) follow the same model. In an eCommerce site, for instance, a user may arrive at a product description as the result of a Google search, and immediately leave that page to look at another site's offering of the same product. Or s/he might start the checkout process, then decide that the product is too expensive, and abandon it halfway through. The basic idea of "hypertext" implies the ability and expectation to jump from one location to another.

Permanent connections consume resources. Perhaps just a network socket, perhaps a pool of parsed database queries; it all depends on the application. Given a user that may disappear at any time, it doesn't make a lot of sense to keep those resources committed.

In practice, there's no real need for the user to have a permanent connection. The web application maintains connections to whatever resources (eg, database) it needs, and shares them between all user requests. The web-app framework provides sessions, which are time-limited places to store per-user data for different requests. The only thing that you can't do (easily) is have long-running client-controlled transactions, but that's a bad idea even in an app that maintains connections.

Answer 9

The Internet isn't necessarily stateless - in fact when you look at Java EE - they have Stateful EJBs and Stateless EJBs.

The main reason why developers recommend using a stateless architecture is because of scalability. Imagine trying to keep the state of all your users once you are adding and dropping servers to support your traffic.

It's really not hard to develop a stateless architecture. The main point is keeping as little state as possible (usually a user id - preferably in a cookie) and changing the database as required.

Answer 10

I think it started that way and just continued to be that way. Now that there is so much infrastructure built around it, it is impossible to change it.

Perhaps it started stateless because connections were less reliable at first, and also bandwidth was smaller. If you don't have a lot of active connections, you can handle more traffic more easily.

Please edit or leave a comment if you have better information, or better yet, post your own answer!

Answer 11

Its because servers provide a service (its in the name). You make a request and receive answers - thats all there is to it.

With regards to making a transition to web development I believe ASP.NET Web Forms will do it in a way that will be more understandable to you - but thats only because it hides whats actually happening under layers of abstraction.

Answer 12

A lot can be understood by analyzing the name of HTTP (HyperText Transfer Protocol). It was never designed to be a rich UI protocol. The original idea was to share documents with links between them. I ask you for a document, you reply with a copy of that document.

Originally HTTP only had one verb GET. In that regard, it was designed for static content. Why do you need state when all you're doing is requesting a document that someone is sharing? And that is why HTTP is stateless...because of its origins.