基于 HTTP 长连接的“服务器推”技术

转载地址:http://hi.baidu.com/widebright/item/0fad577cb16189376e29f6a8

在这里看到了所谓的“服务器推“技术
CSDN和《程序员》杂志主办的连续第四届SD2.0（软件开发2.0）大会---盛大资深研究员钱宏武：转换思维可实现更多
http://news.csdn.net/a/20101210/283436.html

文章里面举的例子很有意思，“手机上面的象棋程序是如何做到快捷响应的“。因为我也一直在思考这种应用用上面办法来做计较好，所以认真看了一下。文章大概意思是说http1.1协议服务器是可以在响应客户端之后继续保持这个tcp的连接，不用马上关闭连接，这样服务器端就可以在需要的时候继续往客户端写东西。这样就改变了我们往常的客户端每打开一个网页就新建一个http连接的做法。使用“服务器的推模式“一个明显的好处就是提高这种下棋这类实时应用的客户端的响应，以前如果使用ajax不停的在那里poll的话，超时时间的设置不好控制，大了响应慢，高了服务器压力很大。
当然这种推模式的也会增加服务器端的复杂度。上文提到自己实现http服务器。不过我在网上搜索了一下“http 长连接”资料，想php ，jsp这些可以直接在 php文件通过代码设置页面超时的。还有一个问题，好像说是http1.1规定一个客户端的连接数量只有3 个？ 所以这种 “http长连接“不能开的太多的，不过应该不影响这种办法的应用了。

上面的那篇文章很乱，我找到几篇文章，还也都不错

Comet：基于 HTTP 长连接的“服务器推”技术
http://www.ibm.com/developerworks/cn/web/wa-lo-comet/

Twitter框架学习二Http长连接技术实现server端push功能-Comet的理论与实战
http://tomyz0223.javaeye.com/blog/656218

2011-03-27补充

最初的来源应该是这个文章？
Comet: Low Latency Data for the Browser

http://infrequently.org/2006/03/comet-low-latency-data-for-the-browser/

其他相关的技术和属于有；

HTTP persistent connection

http头部的

Connection: Keep-Alive

在浏览器和服务器端的支持

wiki里面两个词条也都可以看一下：

Comet (programming) http://en.wikipedia.org/wiki/Comet_(programming)

Push technology http://en.wikipedia.org/wiki/Push_technology#HTTP_server_push

其中第二篇提到的几个引用有几个比较有趣的地方：

（1）

XMPP里面一种实现 Bidirectional-streams Over Synchronous HTTP (BOSH) http://xmpp.org/extensions/xep-0124.html

里面说了很多建立这种 “服务器推”的实现技巧

（2）HTML5的两个草案，都是关于实现这个技术的

HTML5 WebSockets http://dev.w3.org/html5/websockets/

HTML5 Server-Sent Events http://dev.w3.org/html5/eventsource/

（3）google 的 ServerPushFAQ文档

http://code.google.com/p/google-web-toolkit-incubator/wiki/ServerPushFAQ

Google的文章里面详细的介绍了 实现机制以及实现的限制和问题。其中就有我想到的多连接时服务器端引起的多线程问题，因为很多http服务器对这种多连接都没有采用特殊处理的话将是很低效的。但文章里面也说，在部分实现里面已经'continuations'这种技术的实现了，可以解决这种情况下的线程太多的问题（多个线程统一挂起，用同一个epoll ？？？）。

Jetty Continuations
Tomcat Advanced I/O support
文章里面也说了除了上面那个实现之外，php python等都还有类似的支持。我正想看看php的“服务器推”呢，看来不应该考虑这个，如果服务器端不支持这个新的技术的话，即使实现也是很低效的。现在网上的例子都是什么不停的重新polling之类的，应该没什么意思。所以这个服务器推最好还是用java来做比较好了，只有这个服务器端有比较好的支持。当然像盛大研究员那样说自己修改http服务器那也算不错。

真正的“服务器推”技术的实现，估计的看Jetty Continuations 或者Tomcat Advanced I/O support 相关的文章了。网上好像也有很多人写了这方面的文章了。比如http://www.javaeye.com/topic/552054 http://cachalot.javaeye.com/blog/118895 http://shilei78.blog.163.com/blog/static/76004198201010155753933/等，都值得一看

这篇文章写的如此的好，我把它完整的摘录下来吧。

What is Server Push?

Web applications communicate using the HTTP protocol. HTTP has no support for allowing a server to notify a client; the model involves a strict request-response model where the client (webbrowser running a GWT app) makes a request to the server (your web server running a servlet, for example), which must then respond with the requested data. There doesn't seem to be any room in this protocol for allowing the server to send a notification to the client.

However, with some magic and creative implementation of the protocol you can 'fake' a notification from server to client, initiated by the server. This is called Server Push. This document will describe how to accomplish this, what the issues are, and how to integrate Server Push into GWT.

A common use case for Server Push is a web chat client. The 'event' of a new chat message needs to be communicated from the server to the client.

Polling as an alternative

Server Push is a more advanced and more efficient replacement for a technique called polling.

Polling Example: A web chat app application may make a request for the 200th line of an open chat conversation by making a request to an API: http://my.chat.app/getChatLine?idx=200.

If this line does not exist yet (because no one is chatting), the server will simply reply "Does Not Exist" and the client must now re-run the same query in a second or two, over and over again, until someone types something. This is polling. It has two significant problems:

Inefficient: If this webapp polls the server every second for new chat lines, and the chat room sees on average 1 new text message very 30 seconds, 29 out of every 30 requests is just 'no new data'. Put differently, to send 1 line of chat, server and client communicate several kilobytes of HTTP content and headers. This is extremely inefficient use of bandwidth. Slow: Imagine for a moment someone types something almost immediatly after one of the clients received a 'no new messages yet' notification. This client will not see the new line for at least another second, as that's how much time will pass before the client will try again. Trying to improve efficiency by polling less often automatically means events get delayed longer.

Server Push avoids both of those problems.

The Basic mechanism of Server Push

Taking the same example of the previous section, your client makes a request for http://my.chat.app/getChatLine?idx=200. However, instead of returning 'does not exist' if this chat line hasn't been typed yet, the server intentionally doesn't respond. It effectively acts like a very slow server would. Once the 200th line is typed by someone, it 'wakes up' and responds with the line. The client doesn't have to keep asking for updates, and there is no poll delay either.

This idea is simple enough, but unfortunately HTTP, and all the tools around the HTTP protocol, such as most web servers, weren't designed with this idea in mind. As a result there are a large number of caveats.

Server Push pitfallsNo Flush

The HTTP protocol has no 'flush'. A normal HTTP connection can get very complicated. A browser might connect to an internal proxy, which connects to an ISP proxy, which ends in your server park at a reverse proxy, and finally connect to your real web server. Any proxy server in the chain, and the web server itself, is allowed to cache responses. In fact, because in HTTP you cannot send an error response once you begin answering, most servlet containers will cache your entire response just in case your code throws an Exception and a response of '500 - Server Error' is warranted.

Because there is no 'flush', this can cause problems. Imagine this chat app. Instead of asking for the '200th' line, it just asks for a live stream. Every time a new chat message arrives on the server, you push this out on the HTTP connection, never actually closing it.

When testing your application this might work fine, but the HTTP protocol does not guarantee that a byte sent out will actually move all the way to the requesting client. Anyone behind a caching proxy will likely see chat messages only in large bursts instead of line by line.

The solution is to always close the connection immediately after you emit any sort of data. In our hypothetical chat app, we would have 2 practical use cases for the http://my.chat.app/getChatLine?idx=200 request:

The 200th line is available. In this case, respond with it (and optionally any other available lines) immediately and close the request. This is similar to any other normal request/response: The client asks something, and you reply with it. The 200th line is NOT available. In this case, you do not respond with anything, but you wait. Once the 200th line is available, you stop waiting, respond with the 200th line, and close the connection.

In either situation, once you reply with anything, don't freeze the connection again. It's the only practical way to avoid flush problems.

Practical Result: Once you send anything in a server push connection, close the connection. The only time you can meaningfully freeze the connection is at the beginning. Once you start sending, don't freeze it again.

NB: There's another reason why this is useful. Internet Explorer basically never allows access to HTTP connections that haven't finished yet. Until you close it, your GWT code can't access the material you've sent so far.

Spinners

Because the browser has no way of knowing the difference between your server being slow, and your server explicitly waiting to respond because it's waiting for an event to occur, browsers usually give an indicator to the user that the page is loading. This can be in the form of a Loading... statement in the status line (Safari does this), or the so-called spinner (the browser logo that animates to indicate the page is still loading) continually spinning. This erroneous indication that the page is not finished loading may confuse some users. In practice there does not seem to be a way to avoid spinners in all browsers. However, XmlHttpRequests, the driving mechanism behind GWT's RPC mechanism, the RequestBuilder class, and the HTTPRequest class, show the least erroneous 'loading' indicators amongst GWT-compatible browsers of all the various ways to create Server Push connections.

Practical Result: Be aware that some users may be a bit confused about your page appearing to still be loading.

TODO: Check for each browser what actually happends and report on it. I can tell with certainty that Safari2 shows a 'loading...' statement in the status bar, which is not actually visible by default, and nothing else.

Timeouts

Because proxies, the web browser running your GWT code, and sometimes even your web server can't tell the difference between a slow or hanging bit of server-side code, and code that is simply waiting for an event, at some point they may assume you're just broken and assume the response is an error.

As a practical matter this means you should never let your Server Push connections last over a minute. As a result you need to employ a tactic called connection refresh:

Every 50 seconds or so, if no events occur during that time, your server should respond 'no events', and your GWT code, upon receiving the 'no events' flag, simply re-establishes the connection. Effectively we still poll, but we only do so once a minute instead of once a second.

Practical Result: Respond with 'no new events' after about 50 seconds or so, and make sure your GWT code understands this response and opens a new connection. Make sure you always use an event tracking number so that your GWT code doesn't miss events that occur in the downtime between this 'connection refresh'. 2 connection limit

All GWT-compatible browsers will only make 2 connections to 1 server. If there are 3 resources to fetch from the same server, the third resource will be queued until one of the first 2 connections is completed. A Server Push connection counts as 1 of these two connections, which poses a problem. For example, two simultaneous Server Push connections to the same server completely block any further attempts by the browser to talk to this server.

One way to alleviate this problem is to make sure only 1 server push connection is open at any point in time. If you need more, multiplex all events onto the same server push channel. (This is effectively what comet and cometd is - it's a multiplex protocol and architecture. In practice Server Push is often equated with the term 'comet' but this isn't technically correct).

Another method that helps is to load as much data as you can from different servers. Not all data can be downloaded from different servers (due to the SameOriginPolicy limitations) but images, CSS, and external javascript files can all be loaded from other servers.

A google maps application could for example load the map images from img.maps.google.com instead of maps.google.com. The browser would perceive the two hosts as different and thus allow 2 connections to each, even if they both resolve to the same IP address (=go to the same physical server machine).

Practical Result: Never create more than 1 server push connection. If possible, load images, CSS, and external javascript from another server, for example from data.myserver.com instead of www.myserver.com. Server Threads

In order to 'freeze' a response, you would usually freeze the thread. In a servlet, you might for example do something like:

synchronized( chatMessages ){
while( chatMessages.size()<200)try{
chatMessages.wait();
}catch(InterruptedException e ){
respond("Server interrupted. Chat Session Closed.");
}
respond(chatMessages.get(200));

where, if anyone types something, you wake any frozen threads by using:

synchronized( chatMessages ){
chatMessages.add(theNewChatLine);
chatMessages.notifyAll();
}

Unfortunately, this means each Server Push connections ties up a thread. Usually web servers (including Tomcat, Jetty and Apache) have a limited number of threads and once they are all in use, any new requests will be ignored because the server thinks it's very busy and can't accept more connections.

You could up the thread pool count in your webserver's configuration, but there are more robust, scalable solutions available, often called 'continuations'.

The idea behind a continuation is that you tell your webserver to freeze the connection. By employing low level Input/Output directives, the webserver can do this without tying up threads.

There is not (yet) a standard way to do this in the servlet spec, but here are the links to the documentation on continuations for the two most often used servlet containers:

Jetty ContinuationsTomcat Advanced I/O support

Note that some other web frameworks simply do not support this. At time of writing, there is no way to avoid tying up threads in PHP, Ruby on Rails, Django (Python), Turbogears (Python), and many others. Web Frameworks that can definitely handle continuations are anything written in Erlang, Twisted (Python), and Seaside (smalltalk).

Practical Result: Use continuations if your web framework supports them. Server Push in GWT

Because of the reasons explained in previous sections, the most scalable, compatible way to implement server push is the Wait, Respond, Close, Re-Open paradigm:

Wait: When the GWT code makes a call to your server for some data that you don't have yet, freeze (wait) Respond: Once the requested data is available, respond with it Close: Then, close the connection. Re-Open: Once your GWT code receives the response, immediately open up a new connection to query for the next event.

You can implement this paradigm using any of the GWT server call mechanisms; GWT-RPC, RequestBuilder and HTTPRequest are all valid options. However, you must keep track of which events your GWT code has received so far. In practice this means keeping a counter and sending it along every request. If you are using GWT-RPC, you should make this counter a field in the POJO object you are sending. If you are using RequestBuilder or HTTPRequest, include this counter in the JSON or XML data that you are sending to the server. Similarly, all events that you send from server to client need to include their 'event number'.

TODO: Add some sample code here for both GWT-RPC and JSON-over-RequestBuilder that shows how to make the call, then check if the server wants us to just re-try (to avoid timeouts), stop bothering the server altogether (stop), or process incoming events and then re-try.