Subject: [Doh] New: draft-livingood-doh-implementation-risks-issues

From: "Livingood, Jason" <[email-masked]>
Date: Sat, 9 Mar 2019 01:23:56 +0000

FYI that this document has posted. I have requested 10 mins of agenda time in the DOH WG at IETF 104. 

Thanks and have a nice weekend,
Jason
        
On 3/8/19, 7:50 PM, "[email-masked]" <[email-masked]> wrote:

            A new version of I-D, draft-livingood-doh-implementation-risks-issues-01.txt
            has been successfully submitted by Jason Livingood and posted to the
            IETF repository.
            
            Name:		draft-livingood-doh-implementation-risks-issues
            Revision:	01
            Title:		Centralized DNS over HTTPS (DoH) Implementation Issues and Risks
            Document date:	2019-03-08
            Group:		Individual Submission
            Pages:		24
            URL:            https://www.ietf.org/internet-drafts/draft-livingood-doh-implementation-risks-issues-01.txt
            Status:         https://datatracker.ietf.org/doc/draft-livingood-doh-implementation-risks-issues/
            Htmlized:       https://tools.ietf.org/html/draft-livingood-doh-implementation-risks-issues-01
            Htmlized:       https://datatracker.ietf.org/doc/html/draft-livingood-doh-implementation-risks-issues
            Diff:           https://www.ietf.org/rfcdiff?url2=draft-livingood-doh-implementation-risks-issues-01
            
            Abstract:
               The DNS over HTTPS (DoH) protocol is specified in RFC8484.  This
               document considers Centralized DoH deployment, which seems one likely
               way that DoH may be implemented, based on recent industry discussions
               and testing.  This describes that implementation model, as well the
               potential associated risks and issues.  The document also makes
               recommendations pertaining to the implementation of DoH, as well as
               recommendations for further study prior to widespread adoption.

            
            Please note that it may take a couple of minutes from the time of submission
            until the htmlized version and diff are available at tools.ietf.org.
            
            The IETF Secretariat

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sat, 9 Mar 2019 19:28:57 +0100

On Sat, Mar 09, 2019 at 01:23:56AM +0000,
 Livingood, Jason <[email-masked]> wrote 

> agenda time in the DOH WG at IETF 104.

A few remarks (summary: a real issue but the draft obscures it, by
piling a lot of questionable arguments, and by lacking rigor in
separating DoH from the general issue of public DNS resolvers).

> Separating the Protocol from Implementation Issues

There have been already discussions about the need to discuss
separately DoH-the-protocol and DoH-the-deployment. For the latter,
Bert Hubert
<https://blog.powerdns.com/2019/02/07/the-big-dns-privacy-debate-at-fosdem/>
suggested DoC (DNS-over-Cloud).

> Network operators, ranging from ISPs to enterprises, schools, and
> others work hard to provide outstanding DNS and network performance,

Nice but clearly false. One of the reasons why many users switch to
public DNS resolvers is because many local networks and ISP do a lousy
job (specially in some parts of the world). Not to mention those who
simply announce 8.8.8.8 trough DHCP...

> In addition, most also provide DNS-based services such as opt-in
> parental controls for consumers or malware/security protection in
> enterprises, content filtering in schools, etc.

"most"? I doubt it. Source?

> the current Centralized DoH implementation model does not appear to
> make it possible for these operators to continue to play a value
> added role in the delivery of network services

Some people may consider that this is not something we want from the
ISPs. "value added" is often the code word for violations of network
neutrality.

> with 30 companies being responsible for 30% of global traffic

A very questionable metric. If you care only about bytes, video
providers will always win. But other metrics could be used: is a video
of a cute kitten more important than a PDF document which is a
multi-million dollars contract?

> The Dyn attack provides a vivid illustration of how DNS
> infrastructure vulnerabilities - and DNS space concentration - can
> wreak havoc on the stability of the Internet."

Is it really relevant, since it was an attack on authoritative DNS
servers? Having lot of local resolvers would not have changed its
consequences.

> DNS blocklists, which are one of the primary and most effective ways
> to protect a network and its users against malware, phishing, spam,
> DDoS attacks, etc.

Can you explain how a blocklist on the DNS resolver protects against
spam and dDoS?

> Loss of Parental Controls or other Content Controls

The draft seems to assume that lying DNS resolvers are a good thing
(politically and technically) but I don't think there is an IETF
consensus here.

> Disruption of Legally-Mandated National-Level DNS Blocks

It think it is a feature of DoH, not a bug. In Europe, for instance,
many users switch to public DNS resolvers precisely to be able to
bypass the lying resolver.

> Smaller DNS Labor Market

Certainly we, at the IETF, don't work just to preserve our jobs!

Really, you pile way too many arguments to kill "centralized DoH", and
some are really weak.

> This document makes the following recommendations

The most important is missing: help developers to create DoH servers
that can be easily installed and managed so, instead of a few DoH
providers, we can have many of them. (Currently, there is no easy way
to download and install a DoH server.)

> ICANN legal department assessment:
> ICANN currently required Generic Top Level Domain (gTLD) operators
> to meet certain technical criteria, as noted in Specification 6,
> Part 1.1 of their Registry Agreement [48].  ICANN may need to
> consider whether those DNS operators should or must add support for
> DoT and/or DoH in the future

Besides the fact that ICANN already suffers from a serious mission
creep, gTLD operators operate authoritative servers so I don't see why
you mention DoH or DoT, which are currently only for the
stub-to-resolver link.

> Develop Centralized DoH Data Privacy Guidelines/Frameworks

draft-reid-operator-doh
draft-dickinson-bcp-op
draft-dickinson-doh-dohpe

---

From: "Ralf Weber" <[email-masked]>
Date: Sat, 09 Mar 2019 22:57:53 +0100

Moin!

> simply announce 8.8.8.8 trough DHCP...
While it is true that there are ISPs that give you 8.8.8.8 as a resolver
there are also a lot of ISPs that invest in their DNS service and offer
a service that (due to the network nature) is better than what a public
resolver can provide. So stating this as false also clearly is false.

It is as with in most service oriented business, you have good and you
have bad ones, to simplify does not work here.

> "most"? I doubt it. Source?
It depends on the region, but e.g in the UK all of the big providers
offer such a service. Also working for a company (Nominum now part of
Akamai) that builds software for such DNS based service I can tell you
that we sold the software around the world and we are not the only
company in that space.

> servers? Having lot of local resolvers would not have changed its
Well given that as a result of it a lot of work has been done to use
resolvers to protect against such a authoritative outage, e.g
	draft-ietf-dnsop-serve-stale
so the it certainly is relevant for the next attack.

> Can you explain how a blocklist on the DNS resolver protects against
Easy. Spam is often send by bots. DNS resolvers that e.g block or
redirect MX lookups or stop the communication of the bots with its
masters by blocking the C&C domain.

I have seen networks coming of spam list just by deploying these
technologies.

> bypass the lying resolver.
Well again not sure how you quantify many, but those users and even
more so these public resolver service providers, who usually also
do business in these countries, are breaking the law. I never
understood why this has no consequences for them, while when I
worked at a European multi national ISP we had to deliver the
mandated blocks per country of the user and not of the server
for the any cast network we had, which often served users out
of country.

> to download and install a DoH server.)
There is some server or proxy software out there already, and I know
there are others working on it. But to deploy it you need money and
a business case, and I can not find a primary business case to run
a DoH server. There are secondary business cases like deliver fast
Internet for the ISPs or gather more user data for Ad networks or
better map users for CDNs. Bur the relative cost to different
players is different, as for examples ISP would have to scale up
siginpificantly from there current DNS service to deliver DoH,
while for ad networks or CDNs the additional cost for the DNS
service is low.

I think the only way to get lots of DoH providers is to help the
ISPs to do it as they are the natural decentralised player on the
internet.

So long
-Ralf
—--
Ralf Weber

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sun, 10 Mar 2019 08:33:06 +0100

On Sat, Mar 09, 2019 at 10:57:53PM +0100,
 Ralf Weber <[email-masked]> wrote 
 a message of 93 lines which said:

> > > Network operators, ranging from ISPs to enterprises, schools,
> > > and others work hard to provide outstanding DNS and network
> > > performance,
> >
> > Nice but clearly false. One of the reasons why many users switch
> > to public DNS resolvers is because many local networks and ISP do
> > a lousy job (specially in some parts of the world). Not to mention
> > those who simply announce 8.8.8.8 trough DHCP...
>
> While it is true that there are ISPs that give you 8.8.8.8 as a
> resolver there are also a lot of ISPs that invest in their DNS
> service and offer a service that (due to the network nature) is
> better than what a public resolver can provide. So stating this as
> false also clearly is false.

I never said that "_all_ ISPs are this or that". The draft
did. Sometimes, the service provided by the public resolver is better,
sometimes it is not. Saying that all ISPs "work hard to provide
outstanding DNS" is simply advertisment.

> masters by blocking the C&C domain.

OK. It does not protect you against spam, it protects other users
against the spam you send. That's quite different.  If the "regular"
DNS resolver blocs C&C, spam-sending bots won't use it and will switch
to other systems: alternative name resolution systems or simply
DNS-over-somethng (I agree here with Warren Kmmari
<https://mailarchive.ietf.org/arch/msg/doh/pXt1njCBlBrI_fRfPp6He1oybic>).

> those users and even more so these public resolver service
> providers, who usually also do business in these countries, are
> breaking the law.

You are not a lawyer and neither I am but I think it is more
complicated. At least in France but probably in many other countries,
legal rulings by court are to be enforced only by organisations which
have been named in the rulings. Most of them don't name Google Public
DNS or Cloudflare 1.1.1.1 or Yandex DNS. For the censorship which does
not go through courts, such as the blacklist of the governement in
France, the list is not public and therefore public DNS resolvers
cannot follow it, even if they want (at least one not-for-profit
organisation explicitely asks for this list and was denied). So, no, I
don't think they are breaking the law (yes, IANAL, but nobody here is).

> There is some server or proxy software out there already,

I know, I wrote one. But most of them are experimental, hard to
install, not something that you can install as easily as a HTTP server
or a regular DNS resolver.

> But to deploy it you need money and a business case, and I can not
> find a primary business case to run a DoH server.

The will to provide an alternative? In France, at least two
not-for-profit organisations operate a public DNS resolver precisely
for that. There is not only business in life.

> ISPs to do it as they are the natural decentralised player on the

Clearly, we disagree here, but it seems more a political disagreement
than a technical issue with the protocol of this WG.

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 10 Mar 2019 10:42:20 +0000

Hiya,

> time in the DOH WG at IETF 104.

You'll be doing well to cover even the relevant parts of
that in 10 minutes! (And yes it may well make more sense
to discuss these topics in dnsops.)

I say "relevant parts" because I think the draft has a
mixture of arguments about DoH and ones that are fairly
clearly independent of DoH.

For example: AMP and smart-speakers are technologies that
could also be criticised as enabling mega-companies to bypass
systems operated by many others (such as the DNS and/or the
web); a bunch of arguments are fairly generic "confidentiality
mechanisms change who can see what", and lastly, the ability
of current web-browser/service-provider combinations to
quickly change network traffic patterns affects lots more
than just DNS.

I think I also agree with some of Stephane's criticism,
(of this and also of draft-reid) e.g. the recommendations
described here don't seem to me to be justified by the
arguments provided - some I'd agree with, others not, but
the current text just does not justify making that set of
recommendations.

I guess there isn't time before the submission cut-off, and
it depends whether or not you really want this draft to end
up as an RFC, but a serious re-structuring and merging with
draft-reid could help a lot if the authors did want that text
to end up in an RFC.

Cheers,
S.

---

From: Eliot Lear <[email-masked]>
Date: Sun, 10 Mar 2019 12:32:42 +0100

Hi Stephen,

I think this is a great discussion to have.  Both drafts are provocative and, at least to me, important.  I don’t agree with everything in them, but I especially like how draft-livingood is organised, because one can essentially break out each paragraph into its own exploration to determine if one agrees or not, both in terms of issues raised and recommendations.

Is this a DoH thing?  I guess that boils down to whether the IETF views its standards as enabling technology.  In as much as we do, then we should discuss the ramifications of our decisions, especially if someone sees architectural risks.  Two examples of how this broadens out:
Centralization/concentration isn’t limited to DoH but DoH could well act as a catalyst.  it depends if the mechanism is benefiting from non-preexisting relationships.
Trust and accountability, as relates to a service. This is a whopper of a topic, but clearly worthy of exploration.  Anyone who thinks this is a simple topic is fooling themselves.
I’d be up for a side meeting on all of this in Prague if others are interested.

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 10 Mar 2019 12:27:11 +0000

Hiya,

On 10/03/2019 11:32, Eliot Lear wrote:
> Centralization/concentration isn’t limited to DoH but
> DoH could well act as a catalyst. 

With one caveat, I agree: the mozilla/CF/DoH experiment (and
that google could do similarly) has caused a reaction all
right;-)

I don't think that it's the protocol or RFC itself that's the
catalyst though, it's the combination of the RFC with all the
rest of the above.

Cheers,
S.

---

From: "Livingood, Jason" <[email-masked]>
Date: Mon, 11 Mar 2019 01:49:31 +0000

[JL] It is my experience as an operator that it is indeed *not* false, else I'd not have written it nor would many ISPs with similar concerns not agreed with the statement. The key perhaps is that I didn't say *all* of course - but it sure seems like we're nit picking here. I don't disagree that in some parts of the world and in some networks that DNS operations could be improved. But I certainly think you can concede generally speaking that networks (of all types) strive to maximize performance & reliability, and this usually includes good DNS services. 

[JL] It is not what is intended in the document but your perspective on that issue is noted.

   > If you care only about bytes, video

[JL] Yes, it suggests that when a provider handles rather a lot of a particular protocol that an outage or attack on that provider can have a significant impact. The quoted paper uses it as an example of the larger issue. 

    spam and dDoS?

[JL] Quite a lot of spam and DDoS traffic emanates from botted hosts. To the extent that you can prevent malware infection, detect and remediate infection, or detect and/or interrupt command & control traffic then you can have an impact on spam and DDoS. 

[JL] I totally understand the 'lying resolver' perspective. But we need to be realistic about how technology is ultimately used. Many people have young children and may want to filter what content they have access to at home. And schools for young children may similarly want to block certain content from school. Or want to ensure their devices aren't talking to well-known malware C&C servers, etc.

Anyway - thanks for your review and sharing your thoughts!

Jason

---

From: "Livingood, Jason" <[email-masked]>
Date: Mon, 11 Mar 2019 02:01:20 +0000

[JL] Perhaps so. See Section 2 @ https://tools.ietf.org/html/draft-livingood-doh-implementation-risks-issues-01#section-2. In any case, months ago I started to raise some of these (non-protocol but implementation) issues on the DoH WG list and it was suggested I document them and bring it back to the WG, which is what I have done.

---

From: "Livingood, Jason" <[email-masked]>
Date: Mon, 11 Mar 2019 02:07:22 +0000

On 3/10/19, 6:42 AM, "Doh on behalf of Stephen Farrell" <[email-masked] on behalf of [email-masked]> wrote:
>  (And yes it may well make more sense

[JL] Happy to go to whatever venue ultimately makes sense. :-)

[JL] Yeah, that is fair. It's rather a long list and some parts are probably more certain whilst others more speculative. Given available time I thought getting the different points out for comment was best - after all that's what I-Ds are really anyway - just a doc to spur comment/discussion. 

[JL] Dunno - we shall see. I'm not super fanatical about the doc itself. If merging it with others makes sense, or moving it to a different WG, or whatever - that is fine, I am quite flexible on this.

---

From: "Ralf Weber" <[email-masked]>
Date: Mon, 11 Mar 2019 09:47:03 +0100

Moin!

> don't think they are breaking the law (yes, IANAL, but nobody here is).
As you say we are both not lawyers and law is a thing that still is
highly country dependant, so it might be different in different countries.
Most jurisdictions though have something from the old roman law “Ignorantia
legis non excusat” meaning not knowning the law doesn’t protect you from
punishment. So while you may be fine now, because your volume is so low
that lawmakers don’t care, you might not be in the future.

> for that. There is not only business in life.
Good, but even not for profit need money from someone and I doubt that we
can supply DoH service for large portion of the Internet using non profit
organisations.

> than a technical issue with the protocol of this WG.
You didn’t mention an alternative or do you really think a lot of
different non profit organisations will provide resolving for the
Internet?

Also the DoH protocol and subsequent actions of application providers
triggered these discussions. So were should we discuss them if not here.

So long
-Ralf
—--
Ralf Weber

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 12 Mar 2019 16:09:15 +0100

On Mon, Mar 11, 2019 at 09:47:03AM +0100,
 Ralf Weber <[email-masked]> wrote 
 a message of 48 lines which said:

> punishment.

« Nul n'est censé ignorer la loi » in french.

In the cases I mentioned, the people know the law and never pretended
they were not aware of it. But there is a difference between the law,
which applies automatically to all residents, and the court decisions,
which apply only to the parties involved.

Also, in the case of the police blacklist, as I said, it is secret,
even if people want to enforce it, they cannot.

> Good, but even not for profit need money from someone and I doubt
> that we can supply DoH service for large portion of the Internet
> using non profit organisations.

But the goal is not to provide services to "a large portion". There is
no point in creating a "not-for-profit GAFA". Quite the contrary. What
I would like to see is a lot of small DoH resolvers, each with its own
group of users.

> different non profit organisations will provide resolving for the

Yes. Like they provide most of the interesting stuff on the
Internet.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 12 Mar 2019 16:16:39 +0100

On Mon, Mar 11, 2019 at 01:49:31AM +0000,
 Livingood, Jason <[email-masked]> wrote 
 a message of 64 lines which said:

> But I certainly think you can concede generally speaking that
> networks (of all types) strive to maximize performance &
> reliability, and this usually includes good DNS services.

I certainly hope it is the case. But not all the things I hope for
materialize.

> Many people have young children and may want to filter what content
> they have access to at home.

And some have children and never deployed any sort of technical
parental control. (Or not using the DNS.)

It seems to me there is a more profound issue here: should we (the
IETF), when creating new protocols, ensure that *all* previous usages
continue to work? Even if we never condoned them?

Replying Yes to this question would put a severe burden on IETF
shoulders; there are so many "creative" uses of the TCP/IP
technologies that it is impossible to guarantee that everything will
continue to work as before. Creating new protocols or new variants of
old protocols will certainly disrupt things. Is it always bad? I don't
think so.

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 12 Mar 2019 16:55:29 +0000

[JL] The way you frame it, most people would like to say NO of course. But it is not a binary yes/no IMO. In this case we’re not saying consider every single corner case / use, but consider one of the primary use cases. To ignore how a protocol is used by large numbers of implementers when updating/enhancing/replacing the protocol seems silly - we at the IETF can do better.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 12 Mar 2019 18:09:10 +0100

> On 12 Mar 2019, at 16:16, Stephane Bortzmeyer <[email-masked]> wrote:
> 

Wrong questions.  Right questions:

Should the IETF consider the world in which we live and how the protocols we are creating might change that world?
Should we propose mitigations for none, some, or all of the changes?

Eliot