Subject: [Doh] suggested slides for IETF 104 on draft-reid-doh-operator

From: Jim Reid <[email-masked]>
Date: Thu, 14 Mar 2019 16:38:57 +0000

Here’s what I plan to present at the WG. Please comment on the list if you have questions or suggestions.

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 14 Mar 2019 22:19:59 -0400

Thanks for sharing the slides Jim,

(As an aside, the parallels between your summary of the draft and draft-mm-wg-effect-encrypt are striking.)

There are lots of *things* on your list, but maybe it would help to look at a few in more detail to see what they teach us.  I happen to chair a working group that is tasked with improving the captive portal situation, so I thought I'd start there because it's first on your list too.

The first thing I observe is that the story could be different depending on where the remote resolver is configured.  In particular, whether you consider the configuration of an remote, encrypted resolver to be system-wide or per-application.

Captive portal detection is typically done at a system level, and so the system-wide configuration would seem to be most relevant to this particular issue.  That is, unless the network manages to successfully evade the various captive portal detection mechanisms in the hopes that they can somehow present information in the browser, which I'll get back to.

As I've learned, operating systems typically use very different network configuration during the captive portal detection phase, and most won't be applying encrypted DNS policies until they have first validated the network is "usable".  So I would expect that even if the operating system is configured to use an encrypted DNS service other than the one the network provides, it would still use the network-provided service while it is bootstrapping.  For instance, I understand that Android uses the network resolver to discover the IP address of its DoT/DoH resolver (Ben can probably clarify/correct me here).

If the network decides to evade captive portal detection, then it is playing roulette.  Most applications now use authenticated connections to services, and so do many websites.  A DNS redirection is no more effective in these circumstances than a port 80 interception[1].  The application detects the "attack" and throws up errors.  That can lead to the operating system concluding that the network is broken if applications report these errors.  If you are lucky and it is a web browser that first makes a connection and that connection is to a site without HSTS, then the network gets a chance to show a web page.  I shouldn't need to quote stats on app vs. browser usage and proportion of HTTPS on the web to make it clear that this playing this game has increasingly bad prospects.

If this is a per-application configuration, then only those applications that are configured to use remote, encrypted resolvers are affected.  They fail to get the modified DNS results, but all the other problems still exist.

That makes me conclude that captive portals aren't any worse off as a result of operating systems or applications choosing to configure their own encrypted resolver service.  Given that using DNS for captive portals is worse than alternative methods[1], I'd suggest that we not concern ourselves with this particular problem.

Cheers,
Martin

[1] Spoofing DNS results for captive portals poisons the cache, which makes DNS rewriting a bad choice for this use case.  This is - in part - why most captive portals intercept TCP on port 80 and spoof an HTTP 302 instead.  And if you consider DNSSEC-validating endpoints, that would be another reason not to spoof DNS... if those existed in any significant numbers.

---

From: Ben Schwartz <[email-masked]>
Date: Thu, 14 Mar 2019 23:00:56 -0400

On Thu, Mar 14, 2019 at 10:22 PM Martin Thomson <[email-masked]> wrote:

> (As an aside, the parallels between your summary of the draft and
> draft-mm-wg-effect-encrypt are striking.)
>
> There are lots of *things* on your list, but maybe it would help to look
> at a few in more detail to see what they teach us.  I happen to chair a
> working group that is tasked with improving the captive portal situation,
> so I thought I'd start there because it's first on your list too.
>
> The first thing I observe is that the story could be different depending
> on where the remote resolver is configured.  In particular, whether you
> consider the configuration of an remote, encrypted resolver to be
> system-wide or per-application.
>
> Captive portal detection is typically done at a system level, and so the
> system-wide configuration would seem to be most relevant to this particular
> issue.  That is, unless the network manages to successfully evade the
> various captive portal detection mechanisms in the hopes that they can
> somehow present information in the browser, which I'll get back to.
>
> As I've learned, operating systems typically use very different network
> configuration during the captive portal detection phase, and most won't be
> applying encrypted DNS policies until they have first validated the network
> is "usable".  So I would expect that even if the operating system is
> configured to use an encrypted DNS service other than the one the network
> provides, it would still use the network-provided service while it is
> bootstrapping.  For instance, I understand that Android uses the network
> resolver to discover the IP address of its DoT/DoH resolver (Ben can
> probably clarify/correct me here).
>

Yes, if the user has specified a DoT resolver, Android resolves that name
through the network's default DNS server.  Specifically, Android simulates
a parallel version of the network configuration that is used for captive
portal login and DoT bootstrap, but is not accessible to normal apps.  This
version may be used even after login succeeds, in order to handle "captive
portal snapback" (e.g. session expiration).

The Chrome team has also been prioritizing correct functioning with captive
portals on all supported platforms.  I don't foresee DoH implementation
causing any significant regression.

> If the network decides to evade captive portal detection, then it is
> playing roulette.  Most applications now use authenticated connections to
> services, and so do many websites.  A DNS redirection is no more effective
> in these circumstances than a port 80 interception[1].  The application
> detects the "attack" and throws up errors.  That can lead to the operating
> system concluding that the network is broken if applications report these
> errors.  If you are lucky and it is a web browser that first makes a
> connection and that connection is to a site without HSTS, then the network
> gets a chance to show a web page.  I shouldn't need to quote stats on app
> vs. browser usage and proportion of HTTPS on the web to make it clear that
> this playing this game has increasingly bad prospects.
>
> If this is a per-application configuration, then only those applications
> that are configured to use remote, encrypted resolvers are affected.  They
> fail to get the modified DNS results, but all the other problems still
> exist.
>
> That makes me conclude that captive portals aren't any worse off as a
> result of operating systems or applications choosing to configure their own
> encrypted resolver service.  Given that using DNS for captive portals is
> worse than alternative methods[1], I'd suggest that we not concern
> ourselves with this particular problem.
>
> Cheers,
> Martin
>
> [1] Spoofing DNS results for captive portals poisons the cache, which
> makes DNS rewriting a bad choice for this use case.  This is - in part -
> why most captive portals intercept TCP on port 80 and spoof an HTTP 302
> instead.  And if you consider DNSSEC-validating endpoints, that would be
> another reason not to spoof DNS... if those existed in any significant
> numbers.
>
>
> _______________________________________________
> Doh mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/doh
>

---

From: Jim Reid <[email-masked]>
Date: Fri, 15 Mar 2019 07:39:28 +0000

> On 15 Mar 2019, at 02:19, Martin Thomson <[email-masked]> wrote:
> 

I wasn’t aware of that draft Martin. Thanks for the tip.

> ... details snipped ...

> That makes me conclude that captive portals aren't any worse off as a result of operating systems or applications choosing to configure their own encrypted resolver service.

Thanks. That’s good to know. Though I think we need more experience and use cases of what actually happens when DoH clients and captive portals bump into each other.

Somewhat off at a tangent: I run a validating resolver on my laptop and generally use that in preference to whatever DNS service is provided on some guest network. This sometimes mean I bypass that network’s captive portal and end up in a (self-inflicted) world of pain. If widespread use of DoT makes that sort of thing happen for large numbers of unsuspecting users...

Suppose a DoH client has already bootstrapped itself and knows the address of its chosen DoH server. The device moves to a new net that has a captive portal. Will the device try to speak DoH to that server’s address directly instead of doing the captive portal dance? Likewise, what happens what a DoH client is configured to only use DoH and therefore never interacts with the network’s or captive portal’s por 53 resolver?

---

From: Jim Reid <[email-masked]>
Date: Fri, 15 Mar 2019 07:46:42 +0000

> On 15 Mar 2019, at 07:39, Jim Reid <[email-masked]> wrote:
> 
> If widespread use of DoT makes that sort of thing happen for large numbers of unsuspecting users...

Whoops! s/DoT/DoH/

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 15 Mar 2019 09:11:52 +0100 (CET)

On Fri, 15 Mar 2019, Jim Reid wrote:

> Suppose a DoH client has already bootstrapped itself and knows the address 
> of its chosen DoH server. The device moves to a new net that has a captive 
> portal. Will the device try to speak DoH to that server’s address directly 
> instead of doing the captive portal dance?

Then we enter application specific territory. A hypothetical browser might for 
example first verify that it is "captive portal-less" before it activates DoH, 
and it might also feature a config entry that skips that waiting period (a 
wait that is annoying when you restart your browser and have N tabs you'd like 
to reload at once etc).

Yes, validating local resolvers (and others) have the same problem. Captive 
portals in general are a world of pain.

> Likewise, what happens what a DoH client is configured to only use DoH and 
> therefore never interacts with the network’s or captive portal’s por 53 
> resolver?

A browser set to *only* use DoH and never "native" will effectively be locked 
out from the network when it finds itself in a captive portal. (There's at 
least one Firefox bug on this.) Which proves that all pieces work as intended!

In those cases, some users have been known to fire up a *separate* browser to 
resolve the captive situation as then the DoH-using browser can come back to 
the living.

-- 

  / daniel.haxx.se

---

From: "Winfield, Alister" <[email-masked]>
Date: Fri, 15 Mar 2019 09:08:55 +0000

This is obviously a technical user. Somewhat north of, I guess, 70% of normal users would either give up or assume that their local service provider can sort out the problem. Any such support agent may well find it near impossible to help, which client, is that using DoH, is the OS using DoH, ... . From personal experience DNS can be one of the hardest things to diagnose when it is not working right, end-users just don't understand the concepts.

Always remember most users haven't a clue about DNS most wouldn't even know that DNS is a thing to know about. Whatever the default behaviour is it better just work for >99% of use-cases.

Alister Winfield

Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of Sky Limited and Sky International AG and are used under licence.

Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075), Sky Subscribers Services Limited (Registration No. 2340150) and Sky CP Limited (Registration No. 9513259) are direct or indirect subsidiaries of Sky Limited (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in England and Wales and share the same registered office at Grant Way, Isleworth, Middlesex TW7 5QD

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 15 Mar 2019 10:30:46 +0100 (CET)

On Fri, 15 Mar 2019, Winfield, Alister wrote:

>    In those cases, some users have been known to fire up a *separate*
>    browser to resolve the captive situation as then the DoH-using browser
>    can come back to the living.
>
> This is obviously a technical user. Somewhat north of, I guess, 70% of 
> normal users would either give up or assume that their local service 
> provider can sort out the problem.

Which is why I don't think users who *might* end up in a captive portal should 
have DoH-without-fallback enabled unless they know perfectly well what they're 
doing and can get themselves out of it.

A DoH-without-fallback config has several other quirks as well with local host 
names etc that I doubt a lot of users will experience that without having 
asked for it themselves.

-- 

  / daniel.haxx.se

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 15 Mar 2019 10:12:33 +0000

Hiya,

On 15/03/2019 02:19, Martin Thomson wrote:
> result of operating systems or applications choosing to configure
> their own encrypted resolver service.  Given that using DNS for
> captive portals is worse than alternative methods[1], I'd suggest
> that we not concern ourselves with this particular problem.
I think Martin's argument here is convincing.

I'd also note that as a user of stubby (DoT as the system
resolver with my configured choices of recursive), I've
also experienced problems with captive portals that rely
on intercepting DNS, e.g the free wifi in my local airport
does that. The version of stubby I have installed doesn't
have captive portal detection, not sure if that's likely
to be available soon but it'd be a minor improvement.

So basically any encrypted DNS not using the DHCP DNS
as recursive (and possibly also cleartext DNS not using the
DHCP DNS as recursive?) will have an issue, and it's not
a DoH-specific issue.

S.

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 15 Mar 2019 07:26:10 -0400

On Fri, Mar 15, 2019, at 18:39, Jim Reid wrote:
> Suppose a DoH client has already bootstrapped itself and knows the 
> address of its chosen DoH server. The device moves to a new net that 
> has a captive portal. 

If the OS catches the captive portal, everything works nicely once the captive portal is dealt with.  If the captive portal manages to evade detection...

The app might blithely continue.  If the resolver is now blocked by the captive portal (remember, we already have an IP address) then name resolution fails the next time it is needed.

The app that Daniel talks about is more sophisticated.  It attempts to detect the new network and then tries to restart its bootstrapping process.  This is highly unreliable both in detecting the move and in detecting the captive portal (after all the OS missed it too).  So the sophisticated app often ends up looking like the dumb one. 

But this isn't that unique.  It's not just name resolution that fails in this case.  We're back that the "you connected to a broken network" mess I described in my original mail.

The only real take-away then is that evading captive portal detection produces a terrible user experience.

---

From: Thomas Peterson <[email-masked]>
Date: Fri, 15 Mar 2019 12:22:19 +0000

(to those exclusively on the captive-portals list, this email follows on 
a discussion around a presentation discussing implications of DNS over 
HTTP in networks where captive portals are present)

> If the OS catches the captive portal, everything works nicely once the captive portal is dealt with.  If the captive portal manages to evade detection...

As there are numerous folk from browser and OS vendors within this 
mailing list who implement capture portal detection, would there benefit 
in authoring an informational document covering capture portal detection 
methods in the absence of a network's DHCP service not implementing RFC 
7710? Such a document may help describe common methods to inform 
implementers and minimise detection evading capture portals. It may be 
better placed in the capport WG instead of doh.

Regards

---

From: Tommy Pauly <[email-masked]>
Date: Mon, 18 Mar 2019 09:55:56 -0700

> On Mar 15, 2019, at 5:22 AM, Thomas Peterson <[email-masked]> wrote:
> 
> (to those exclusively on the captive-portals list, this email follows on a discussion around a presentation discussing implications of DNS over HTTP in networks where captive portals are present)
> 
> As there are numerous folk from browser and OS vendors within this mailing list who implement capture portal detection, would there benefit in authoring an informational document covering capture portal detection methods in the absence of a network's DHCP service not implementing RFC 7710? Such a document may help describe common methods to inform implementers and minimise detection evading capture portals. It may be better placed in the capport WG instead of doh.

Agreed that CAPPORT is a good place for this discussion.

As far as providing a document to describe OS-vendor-specific mitigations, I'm not sure if the benefit would be that large. It may be useful as an appendix in our Captive Portal Architecture document? The problem is that captive portals that want to whitelist OS probes for portals always can some way or another. Captive portals that do want to play nice don't whitelist these probes, and the detection generally works fine.

Thanks,
Tommy
 
> Captive-portals mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/captive-portals