Subject: [hybi] More handshake woes

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Mon, 1 Nov 2010 18:50:24 -0700

In our discussions about starting to update Chrome to -03 framing, we
noticed that although we now have the client sending the Sec-Websocket-Draft
header, the server isn't currently required to echo that parameter back, so
we don't know for sure if we are talking with an upgraded server that
understands this concept or whether we're talking with a -00/76->-02 server.
If no one objects, I would like to make the server echo this value, for as
long as we keep this header in the handshake.

Also, FWIW I would really like to figure out how to close out the handshake
issue. RIght now all of the alternatives seem to have pros and cons, and at
the end of the day we need to move forward.

---

From: John Tamplin <[email-masked]>
Date: Mon, 1 Nov 2010 21:57:44 -0400

> long as we keep this header in the handshake.

No objection on that point, though if we do go with a different handshake
like Adam proposes, it isn't clear the draft header is useful since you have
to know which client you are speaking to before you can read that header.

> the end of the day we need to move forward.

I just finished implementing Adam's proposed handshake, and I will make a
separate post with my experiences on it and suggestions.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Mon, 1 Nov 2010 19:04:14 -0700

> to know which client you are speaking to before you can read that header.

Which is why I said "for as long as we keep this header in the handshake" :)

>
>
>> Also, FWIW I would really like to figure out how to close out the
>> handshake issue. RIght now all of the alternatives seem to have pros and
>> cons, and at the end of the day we need to move forward.
>>
>

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 02 Nov 2010 03:13:08 +0100

>the end of the day we need to move forward.

I note that Eric Rescorla wanted to discuss the three main differences
of his and Adam Barth's proposal compared to what we have in dedicated
threads, but so far we only had one thread for the first issue about
obfuscating traffic past the handshake; whether routing information in
the handshake should be obfuscated in the handshake, and whether the
CONNECT method should be used so far lacks a dedicated thread. It seems
to me there is rough consensus about the first issue, we are unlikely
to gain consensus on the second issue, and lack data on the third issue
(and I also note that the three issues cannot be separated so easily as
far as the overall security considerations go).

Still, I do think it'd make sense to discuss obfuscating routing infor-
mation and using CONNECT in their own separate threads, and I would
appreciate it if the proponents could start threads on those issues as-
well, ideally pointing out the implications of going with obfuscating
the past-handshake traffic and CONNECT but not obfuscating the routing
information in the handshake, and what information on using CONNECT is
needed to make progress on that issue, so we can see about gathering
some data on that (and maybe the implications of using a WEBSOCKET or
other method instead of CONNECT).

(I am also somewhat worried that some people who had previously been
advocates of maximizing simplicity of the frame structure are only just
waiting to see if this is going anywhere while -76 is being deployed,
and will start objecting to the frame structure once there are some
signs of progress; it might be wise to plan for that.)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 02 Nov 2010 10:29:08 -0700

> other method instead of CONNECT).

+1

Scrambling the payload with an XOR is an easy update to Greg's proposal 
with essentially zero overhead, and defends against the NPH/virtual-host 
scenario. I see no major problems with that (other than the debugging 
inconvenience.)

But the Host/URL scrambling proposal is basically not-HTTP using a 
HTTP-like syntax. We really should not be using the HTTP port if we're 
not dispatching Host/URL like HTTP. Existing HTTP servers, for example, 
would need two dispatching mechanisms on the same byte stream. If we 
used a different port that would be fine, but would be a poor design if 
using the same port. (Specifically, it would make the HTTP server more 
complicated even when the application never uses WebSockets.)

-- Scott

---

From: John Tamplin <[email-masked]>
Date: Tue, 2 Nov 2010 13:40:26 -0400

On Tue, Nov 2, 2010 at 1:29 PM, Scott Ferguson <[email-masked]> wrote:
> But the Host/URL scrambling proposal is basically not-HTTP using a HTTP-like
> syntax. We really should not be using the HTTP port if we're not dispatching
> Host/URL like HTTP. Existing HTTP servers, for example, would need two
> dispatching mechanisms on the same byte stream. If we used a different port
> that would be fine, but would be a poor design if using the same port.
> (Specifically, it would make the HTTP server more complicated even when the
> application never uses WebSockets.)

I'm not sure I really see the difference:

draft -03:
=======
GET /foo HTTP/1.1
Connection: Upgrade
Upgrade: WebSocket
Host: foo.org:80
...
\r\n
randomBytes
payload

Adam's proposal:
==============
CONNECT bogusAddress HTTP/1.1
Host: bogusAddress
Sec-WebSocket-Key: xxx
\r\n
maskedHeaders
maskedPayload

In either case, the web server reads a complete HTTP request,
consisting of a method line and headers up to a blank line.  Once it
does that, it has to decide to handle it separately anyway.  The only
advantage of the former is if the web server already supports
Connection: upgrade, it may not have to add as much new funcionality.
However, I don't know many implementations that actually support
Connection: upgrade or how many of those wouldn't need signfiicant
modification for long-lived bidirection connections like WebSockets.

In fact, Adam's proposal is more HTTP compatible than draft -03, as it
doesn't have random bytes after the header not accounted for in a
Content-length header, while CONNECT is supposed to setup a
bidirectional connection for everything that follows.

---

From: Zhong Yu <[email-masked]>
Date: Tue, 2 Nov 2010 13:12:38 -0500

> dispatching mechanisms on the same byte stream. If we used a different port

That has to be done anyway, if a port serves both HTTP and WEBSOCKET.
It would be nice if the two protocols are very different syntactically
and the split
can occur as early as possible.

I don't grasp the idea of disguising WEBSOCKET as HTTP so intermediaries
can be *cheated*. WEBSOCKET has a lot of weight and it should be the
intermediaries
that try to adapt and accommodate. And they are going to. Any complexity
designed into the protocol to confuse intermediaries will soon become obsolete.

If Ian Hickson's original simple design had been accepted a year ago, by now we
would have seen servers and clients deployed, intermediaries fixed, applications
running and users having fun. The imagined obstacles to adoption can't withstand
the demand. However today the delay and uncertainty in the protocol design
have become the real adoption killer.

Zhong Yu

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 02 Nov 2010 12:06:00 -0700

Zhong Yu wrote:
> and the split can occur as early as possible.
>   

The split can occur under application control after normal HTTP dispatch 
using existing HTTP server architecture, taking advantage of existing 
HTTP server capabilities and configuration. That's the main value of 
using HTTP as the handshake protocol.

If we are just using HTTP to save a port, we should use separate ports.

-- Scott

---

From: John Tamplin <[email-masked]>
Date: Tue, 2 Nov 2010 15:22:28 -0400

On Tue, Nov 2, 2010 at 3:06 PM, Scott Ferguson <[email-masked]> wrote:

> using existing HTTP server architecture, taking advantage of existing HTTP
> server capabilities and configuration. That's the main value of using HTTP
> as the handshake protocol.
>

Let's say you are right that existing implementations would require more
effort in order to support CONNECT rather than GET+Upgrade.  What is the
right tradeoff between implementation complexity and resistance to attacks,
both known and unknown?  Does implementation complexity completely trump
security?  If not, what level of security benefits justifies the additional
complexity?

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 02 Nov 2010 12:50:53 -0700

John Tamplin wrote:
> On Tue, Nov 2, 2010 at 3:06 PM, Scott Ferguson <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     The split can occur under application control after normal HTTP
>     dispatch using existing HTTP server architecture, taking advantage
>     of existing HTTP server capabilities and configuration. That's the
>     main value of using HTTP as the handshake protocol.
>
>
> Let's say you are right that existing implementations would require 
> more effort in order to support CONNECT rather than GET+Upgrade.  What 
> is the right tradeoff between implementation complexity and resistance 
> to attacks, both known and unknown?  Does implementation complexity 
> completely trump security?  If not, what level of security benefits 
> justifies the additional complexity?

If you think the HTTP Host/URL is a gaping security hole, use a 
different port and don't use the HTTP handshake.

If you want an early split, using a separate port is a very early split 
and adds zero extra complexity for the plain HTTP requests.

-- Scott

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Nov 2010 08:02:28 +1100

> I'm not sure I really see the difference:

John,

There is a huge difference.

A HTTP request may pass through many layers of handling (on server or
intermediaries) before it gets to some logic that might look for a
WebSocket handshake.       Some of those layers of handling may be
examining host and path information to determine how to route the
request or to simply check that this is a request for this server.

Sending bogus addresses will mean that code which is entirely valid
against the HTTP spec will need to be changed to be made WebSocket
aware.

For example, in the Jetty server, the decision to accept or reject a
websocket connection is ultimately made in application logic within
the servlet container within the HTTP container.   If I receive a HTTP
request for a bogus address, then request might be rejected as not for
this host, or may not be routed to the correct entity within the
servlet container. Thus it will never get passed to the application
logic that could trigger a switch to WebSocket connection handling.

To fix this, I would have to make several parts of the HTTP stack
WebSocket aware (but could do nothing about the user extensible HTTP
handling that can be plugged in below the servlet container) to make
it pass the handshake request.     Currently this code does not need
to be websocket aware, because once the handshake is accepted and the
101 is sent, then the connection is switched to a different set of
handlers that are dedicated to WebSocket.

regards

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 14:29:31 -0700

> handlers that are dedicated to WebSocket.

This is by design.  This code needs to be updated to understand
WebSockets so that an attacker can't exploit it using WebSockets.

Adam

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Nov 2010 11:09:30 +1100

> WebSockets so that an attacker can't exploit it using WebSockets.

But this is HTTP code that should not have to know anything about WebSockets.
It is also aware that URIs, hostname and header fields are user
supplied data and thus cannot be trusted.  Most servers are already
hardened against injection in those fields and the servers that are
not will be vulnerable to XHR attack.

The code may also be in pluggable 3rd party modules or firmware in
appliances, so it cannot be updated by the server provider.

It is not sufficient that the handshake looks-like a HTTP request, it
has to be a HTTP request. Until the websocket connection is
established, we need to be compliant HTTP so that we do not break
existing HTTP infrastructure.

Anyway, I thought the point of encoding the attacker supplied data was
to protect against WebSocket to non-HTTP server attacks as there is no
need to protect HTTP servers from HTTP requests!

Then there is little benefit of making WebSocket any more restrictive
that XHR.  If a non-HTTP server is vulnerable to attacker supplied
values in URIs and header values, then they can be attacked by XHR.

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 17:14:51 -0700

> values in URIs and header values, then they can be attacked by XHR.

You're forgetting that once the handshake completes, the attacker is
freed from having to respect the browser's HTTP invariants.  That
means all these layers that think we're talking HTTP over this socket
are going to get very confused by the attacker's crafted WebSocket
frames, and potentially do bad things.

Adam

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Nov 2010 12:17:51 +1100

> frames, and potentially do bad things.

Adam,

I'm not forgetting and I think you are mixing up various concerns.

For a websocket to HTTP attack to be vulnerable to the "crafted
WebSocket frames", it would have to pass the handshake.   That is why
we are proposing nonces, hashes and Hello frames to be part of the
handshake, so that a none websocket server cannot be tricked into
accepting a websocket connection from a valid websocket client.

The issue with allowing arbitrary content in URIs and host headers is
that perhaps a really silly non-HTTP server could be tricked to
echoing back some of the contents of those attacker provided fields,
and thus tricked into performing the server handshake.   Well firstly
this attack will be thwarted by the nonce being generated AFTER the
attacker has provided the data, so they will not be able to know what
responses to inject.

If we further restrict the content of the URI and header values, then
this attack will be further thwarted because the attacker will not
have the punctuation necessary to form a valid 101 response.

Now there might still be some non-HTTP servers that are vulnerable to
[A-Za-z0-9/_-]* in a HTTP header, but those servers will be just as
vulnerable to XHR and they still will not be able to be tricked into
performing the handshake, so they will never be exposed to ws frames.

regards

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 3 Nov 2010 22:55:20 +0100

> performing the handshake, so they will never be exposed to ws frames.

I agree with Greg here. The handshake issue does not move on because
we're constantly trying to guess what type of imaginary broken server
or intermediary might be flawed by WebSocket. We're at a point where
we almost have to write the broken code ourself to try to prove those
*might* exist, simply forgetting that those supposedly broken servers
are already processing HTTP traffic all the day from valid clients as
well as bots.

Now we're suggesting that since those servers might exist, we should
protect them against their own stupidness by breaking the protocol so
much that it cannot be used at all for the 99.99999% of non-broken
implementations. That is getting ridiculous and counter productive.

Eric suggested that we separate the routing info handling from the
data mangling. It did not appear anyone objected to this. Greg
proposed that the frames make use of a first char that is not compatible
with HTTP so that even the broken servers are not tempted to process the
data as a secondary request. Once again, this is cheap and sensible, we
should do it "just in case". But at one point, we should stop trying to
offer imaginary protection for servers that parse headers with strstr()
on the input buffer without even considering CRLFs. All that we can do
against them can already be done with plain HTTP.

If we talk HTTP, we'll be processed as HTTP is currently processed. What
remains is the issue with indermediaries that might ignore CONNECT or 101.
For them, the proposal of ensuring that ongoing data is not HTTP-compliant
should be enough to break/block the connection. Those which still accept
invalid chars there are certainly already vulnerable to cross-protocol
attacks from other protocols and do not need our help to get fixed.

So now, could we move on towards something that makes sense for real life
and real users ?

Willy