Subject: [weirds] Bootstrapping: should it be a client or server -side problem ?

From: "Carlos M. Martinez" <[email-masked]>
Date: Mon, 10 Jun 2013 10:47:30 -0300

... or both ?

We've been talking a bit about bootstrapping, and for both proposed
approaches the issue of client complexity (whether it lies in issuing
complex DNS queries or parsing XML files) has been mentioned as a drawbacks.

Thinking about this, if we were to move the problem of bootstrapping to
the server, then we can do whatever we want (even parse ASN.1 if you are
into sadomasochism :-D ). The client complexity issue becomes moot.

Bootstrapping clients has the advantage of avoiding having poorly
written clients hitting the same RDAP server over and over again, even
if they get 301 answers most of the time.

Now, if we could agree on creating a 'pool' of RDAP servers, just like
NTP.org does with NTP servers, we could perhaps avoid the issue
mentioned above.

It would work something like this:

-- zone 'rdapservers.org' or even better 'rdap-servers.arpa':

pool	IN	A	<lacnic rdap server ip>
pool	IN	A	<arin rdap server ip>
pool	IN	A	<ripencc rdap server ip>
pool	IN	A	<apnic rdap server ip>
pool	IN	A	<afrinic rdap server ip>
pool	IN	A	<com TLD rdap server ip>
pool	IN	A	<net TLD rdap server ip>

... same for AAAA records and adding all those who want to chip in ...

If we make this decision early on (like, now) before most clients are
written and distributed with operating systems, and if we do a good PR
work on this, I think we can get a nicely bootstrapped system and we can
share the load.

Note that having this scheme in place does not prevent the existence of
'smart' clients who want/prefer to bootstrap themselves in order to
avoid redirects.

cheers!

~Carlos

---

From: Andy Newton <[email-masked]>
Date: Mon, 10 Jun 2013 14:39:26 +0000

I like the idea, especially since we know people are gonna write dumb
clients.

-andy

On 6/10/13 9:47 AM, "Carlos M. Martinez" <[email-masked]> wrote:

>_______________________________________________
>weirds mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/weirds
>

---

From: "John Levine" <[email-masked]>
Date: 10 Jun 2013 21:43:36 -0000

>NTP.org does with NTP servers, we could perhaps avoid the issue

For numbers, that seems entirely workable.  There are only five RIRs,
they're all of similar size, and they have existing working
relationships.

But for names, it's out of the question.  Here's some query numbers
from the Feb 2013 monthly reports:

TLD	Queries

.COM	8.1G
.NET	714M
,INFO	162M
.ORG	116M
.BIZ	83M
.XXX	3.9M
,AERO	1.6M
.POST	1.1M
.COOP	475K
.MUSEUM	67K

(I don't understand why .POST got 1.1M queries, since the domain
appeared to have only 8 active names.  Squatters getting ready for
sunrise, perhaps.)

There are more WHOIS queries to Verisign than there are to everyone
else put together, and the long tail will be very long, with most
domains getting only a few thousand queries a day.

The .COM WHOIS servers are getting about 3000 queries per second.
There's no way anyone is going to share that.

Given the amount of traffic, the vast differences in query rates, and
the fact that few of the TLDs have relationships with any other TLDs,
the only plausible way to distribute bootstrap info for names is
through the DNS.  I hope we can resolve this in Berlin.

R's,
John

---

From: Gavin Brown <[email-masked]>
Date: Mon, 10 Jun 2013 23:36:37 +0100

+1. This was my idea if rdap.org ever got too much traffic for my little VM to handle. I would be happy to donate the domain for this purpose, but wouldn't rdap.arpa be a better name?

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Wed, 12 Jun 2013 15:18:03 -0300

Thanks John, that's useful insight. Are those numbers public ? Is there
a source we can reference ?

As a possible way forward, we could split the pools into names and
numbers pools:

--- zone 'rdap-servers.arpa'

pool.numbers	IN	A	<lacnic rdap server>
pool.numbers	IN	A	<arin rdap server>
pool.numbers	IN	A	<ripencc rdap server>
pool.numbers	IN	A	<apnic rdap server>
pool.numbers	IN	A	<afrinc rdap server>
...
pool.names	In	A	<.com rdap server #1>
pool.names	In	A	<.com rdap server #2>
pool.names	In	A	<.com rdap server #3>
pool.names	In	A	<.net rdap server>
pool.names	In	A	<.org rdap server>

--- END zone

Under this scheme:

- names and numbers servers would redirect to the 'head' pool of the
other camp: <rir> rdap server, when receiving a query for a name would
redirect to pool.names.rdap-servers.arpa, <tld> rdap server, when
receiving a query for a number would redirect to
pool.numbers.rdap-servers.arpa

- each camp can decide how to better share the load, as, given the
numbers provided by John, I believe it will be easier to find common
ground within each camp rather than try to unify everything under the
same umbrella

- .com for example could provide more than one entry in order to capture
an adequate proportion of the whole queries

- quasi-smart clients, while not doing a full bootstrap, can choose
which pool to direct their queries to, based on the type of the object
they're querying

Thoughts ?

~Carlos

---

From: "John R Levine" <[email-masked]>
Date: 12 Jun 2013 15:05:50 -0400

> a source we can reference ?

They're public, from the monthly registry reports on the ICANN web site.

> As a possible way forward, we could split the pools into names and

No, a names pool doing http forwarding is completely out of the question, 
unless you expect all of the servers in the pool to be run by Verisign. 
Someone getting 3000 queries/day can't realistically offer to share load 
with someone getting 3000 queries/second.  All you need is for one or two 
busy clients to pick your IP to get DDoS'ed.  And beyond that, there are 
no existing relationships between ccTLDs and gTLDs to set up the pool.

>> through the DNS.  I hope we can resolve this in Berlin.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Chris Wright <[email-masked]>
Date: Thu, 13 Jun 2013 13:23:22 +1000

I still maintain my original pov from the conversation and presentation done way back at the start of this working group, that bootstrapping with SRV or NAPTR records in the TLD zone is the right way to go for domain name weirds queries.

It could be done for numbers to, using the reverse lookup zones, but alas, as others have said there is only a few of them and they can redirect to each other if that's the way they choose to work.

The TLD operator, if required, could redirect to WEIRDS servers for domains further down their own tree, they need to interact with those people to get name server information anyway, so they could get the WEIRDS server information as well.

The argument that "we want to write clients in javascript and we cant do that if we have to do a DNS lookup" is a red herring and irrelevant, as javascript running as a system program (AKA node.js) is perfectly able to do DNS lookups, and javascript running in a browser can call back to the server that served the javascript to do the DNS lookup server side (or indeed be seeded with the lookup URLs from each TLD when it is served to the browser if the website operator deemed it necessary).

It technically works without a problem and doesn't rely on us creating a centralised single source of failure - it is the best solution of any presented so far and I still don't understand the resistance to it.

Thanks

Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of John R Levine
Sent: Thursday, 13 June 2013 5:06 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [weirds] Bootstrapping: performance issues

> Thanks John, that's useful insight. Are those numbers public ? Is 
> there a source we can reference ?

> As a possible way forward, we could split the pools into names and 

No, a names pool doing http forwarding is completely out of the question, unless you expect all of the servers in the pool to be run by Verisign. 
Someone getting 3000 queries/day can't realistically offer to share load with someone getting 3000 queries/second.  All you need is for one or two busy clients to pick your IP to get DDoS'ed.  And beyond that, there are no existing relationships between ccTLDs and gTLDs to set up the pool.

>> through the DNS.  I hope we can resolve this in Berlin.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY "I dropped the toothpaste", said Tom, crestfallenly.

---

From: "John R Levine" <[email-masked]>
Date: 13 Jun 2013 00:33:26 -0400

> I still maintain my original pov from the conversation and presentation 
> done way back at the start of this working group, that bootstrapping 
> with SRV or NAPTR records in the TLD zone is the right way to go for 
> domain name weirds queries.

I must be dim, since I could have sworn we drove a stake through this one 
last year.  Could you explain in detail why it is preferable to the 
alternative of A or AAAA or CNAME labels at <tld>.rdap.arpa?

Javascript in web browsers and shell scripts can trivially do the 
A/AAAA/CNAME lookup by sticking the TLD into the URL they are fetching, 
but have to do backflips to do SRV or NAPTR lookups.  What useful 
abilities will clients have with SRV or NAPTR to justify the extra work? 
As I recall, you proposed that Javascript clients do an http callback to 
whatever server provided the Javascript to turn the SRV or NAPTR into a 
name.  Why is this preferable to a much faster DNS lookup?  A DNS lookup 
can be secured with a verifiable chain of trust using DNSSEC; what is the 
equivalent for a callback to some random web server?  Keep in mind that 
since there are no existing RDAP servers so there is no need for backward 
compatibility to find RDAP servers on nonstandard ports or at peculiar 
URIs.

Also, section 2.2.3.3 of the appliant guidebook for ICANN's new TLDs 
specifically lists the records allowed in the new TLD zones, and neither 
SRV nor NAPTR is on the list.  It says:

   Applicants should be aware that a service based on use of less-common
   DNS resource records in the TLD zone, even if approved in the registry
   services review, might not work as intended for all users due to lack
   of application support.

How much work do you think it will be to get the rules changed?  Are you 
offering to negotiate with ICANN to do it?

R's,
John

---

From: Pieter Vandepitte <[email-masked]>
Date: Thu, 13 Jun 2013 11:05:18 +0200

I liked the idea of naptr records and consequently being free to choose
your own host, port, scheme and base path as specified in
http://tools.ietf.org/html/draft-ietf-weirds-rdap-query-05#section-3
And it would be sad to give up that freedom... Ok, it comes at a price
of clients having to know how to handle naptr records, but i think it's
a minimal requirement for developers who write software for the domain
name community to know how to handle them... no?

On javascript clients: i don't know the numbers at other tld's but at
.be the port 43 queries currently outnumber the web queries. it's imo
not the ultimate goal of this WG to develop a protocol for javascript
clients in the browser (correct me if i'm wrong). And if needed, one can
write a server side proxy client or do some other tricks (like 'abusing'
a redirect enabled rdap server which redirects - by querying itself for
naptr records - the client to the authoritative server) :-)

Pieter

-- 
*Pieter Vandepitte*
*Software Engineer*
+32 16 29 89 27
*www.dnsbelgium.be* <http://www.dnsbelgium.be/>

---

From: Andy Newton <[email-masked]>
Date: Thu, 13 Jun 2013 10:35:26 +0000

>naptr records - the client to the authoritative server) :-)

Our REST queries surpassed our port 43 queries last year. Keep in mind
that "web" queries are not the same thing as REST. We've had web queries
for years and years and they have never been as popular as port 43. But
after the introduction of REST, it only took one year for those queries to
climb above port 43.

And when we did the analysis, Javascript clients are in the majority. So
taking them into account is very important.

I agree with John, NAPTR/SRV would be the death of bootstrapping in RDAP.
It isn't successful and its time the IETF admitted that. I would be loath
to implement it in my NicInfo client, and I say that having implemented
S-NAPTR and U-NAPTR client code in the past (and having been co-author of
the S-NAPTR RFC).

-andy

---

From: Pieter Vandepitte <[email-masked]>
Date: Thu, 13 Jun 2013 13:13:21 +0200

Andy, thanks for your comments

Is this valuable analysis somewhere available for us (usage of web 
queries, rest queries, port 43, ...)? It would be great to have and 
would avoid some discussions on the list :-)

Kind regards

Pieter

---

From: Frederico A C Neves <[email-masked]>
Date: Thu, 13 Jun 2013 09:53:00 -0300

> alternative of A or AAAA or CNAME labels at <tld>.rdap.arpa?

As a "salomonical solution" solving, the single point of failure, the
inability of redirects to/from unrelated parties, the difficulties
involved with the use of SRV class of records, the non use of another
TXT record at the apex of the zones and the possibility of name
collisions. Use a hexhash("rdap").TLD.

 % python
 >>> import hashlib
 >>> hashlib.sha1("rdap").hexdigest()
 'cc749546f2f7320a64ea61ab4a125be59d490b96'

cc749546f2f7320a64ea61ab4a125be59d490b96.TLD IN [A|AAAA|CNAME]

Fred

---

From: Gavin Brown <[email-masked]>
Date: Thu, 13 Jun 2013 20:33:34 +0100

I think an HTTP based bootstrapping solution is the only one that will be acceptable to client implementers. Scaling the server side is clearly a problem, but I would like to propose a solution.

Let's say we have a well-known bootstrap URI (such as http://rdap.arpa/) which contains a domain which resolves to one or more addresses. These addresses could be anycasted, and advertised by multiple operators, in much the same way that the AS112 system works. The DNS for the domain could also be anycasted.

Participation would be voluntary, and operational policy would be based on consensus among the operators (a la the root servers). The HTTP service would be based on open source code.

The number of instances could be tuned so that smaller operators don't carry a disproportionate volume of traffic. Large operators already have anycast networks that could be used for this purpose. Large consumers of RDAP data could run a local instance to take load off the public operators.

Anycasting of TCP services used to be problematic, but I know of a few that exist and work fairly well, especially for short-lived and easily cacheable services.

I would prefer to use TLS (ie https://rdap.arpa/) but it's not clear to me how this could scale to multiple operators without using multiple TLSA records. Naturally, rdap.arpa would be signed anyway.

I think the above is cracking a walnut with a sledgehammer, but thought it worth discussing.

-- 
Gavin Brown
Chief Technology Officer
CentralNic Ltd
Innovative Registry Services for ccTLD and gTLD registries
https://www.centralnic.com/

CentralNic Ltd is a company registered in England and Wales with company
number 4985780. Registered Offices: 35-39 Moorgate, London, EC2R 6AR.

---

From: "John Levine" <[email-masked]>
Date: 13 Jun 2013 22:55:23 -0000

>S-NAPTR and U-NAPTR client code in the past (and having been co-author of

Thanks.

Anyone who thinks that an HTTP bootstrap for names is plausible really
needs to do the arithmetic.  When I said 3000 queries/sec for Verisign
and 3000 queries/day for small domains, those are real numbers, based
on the monthly gTLD reports for February 2013 available on the ICANN
web site.  If you disagree with them, please show your work.

That is a difference in query rates of roughly 100,000 to 1.  DNS
based load sharing, which is what a http bootstrap would use, sends
similar amounts of traffic to each address.  There isn't any way to
send 99.99% of the traffic to one place and 0.001% to each of ten
others.  (No, you can't use 100,000 A records.  Don't even think of
proposing stunt DNS servers that serve up some names 100,000 more
often than others.)  

If we wave our hands and imagine against experience that gTLDs and
ccTLDs would cooperate closely enough to put different people's
servers behind some kind of load sharing system, please describe an
actual load sharing system that will split the work that
asymmetrically.  If you think you'll do it with anycast, same
question, plus a bonus question about why you think that anycast works
with TCP.

As I said, only Verisign can handle that query rate, and I don't think
anyone thinks a design that only works for Verisign would be
acceptable.  Again, if you disagree, please explain how your plan will
work and handle the 100,000:1 ratio in query rates between large and
small TLDs.

A DNS A/AAAA/CNAME bootstrap like <TLD>.rdap.arpa is entirely
workable.  Yes, it is a single point of failure, but I think it is
reasonable to assume that rdap.arpa would be hosted on the same
servers as in-addr.arpa which is served from six places around the
world, maybe more if they use anycast.  Everybody I know thinks that
in-addr.arpa is plenty reliable.

The suggestion to use a reserved name <something>.tld, whether a
pseudorandom hex name or something else would work technically, but it
runs into the same problem as _srv.tld: the ICANN rules for new gTLDs
don't allow non-glue A or AAAA records in the TLD zone.  I also
believe from experience that trying to reserve any host name in all
TLDs, even a funny looking one that no TLD is currently using, will
run into severe opposition in the IETF.

R's,
John

PS: I understand CentralNIC's desire to make uk.com look like a
pseudo-TLD without Verisign's cooperation, but that doesn't strike me
as a problem that it is our job to solve, any more than it would be
for my watkins-glen.ny.us without Neustar's help.  If it works for all
the TLDs that talk to IANA, i.e., ccTLDs, gTLDs, and the legacy TLDs,
we've succeeded, and the policy boundaries are below the TLDs are
local issues.

---

From: Byron Ellacott <[email-masked]>
Date: Mon, 17 Jun 2013 02:29:59 +0000

Hi John,

>question, plus a bonus question about why you think that anycast works

Bonus answer: dig +tcp txt chaos hostname.bind @k.root-servers.net ?  The
routing system is more stable than you may be giving it credit for :-)
This is an irrelevant tangent; I agree with your point.

>in-addr.arpa is plenty reliable.

Thanks for the vote of confidence in our reliability!

The current operators of ?.in-addr-servers.arpa are ICANN and the five
RIRs.  APNIC carries a peak of ~1,800 qps on e.in-addr-servers.arpa,
anycast across three locations (tangentially, < 50qps on TCP) - I don't
think serving ~1,000 qps for rdap.arpa bootstrap would be a problem for
us, though I can't speak for ICANN or the other RIRs of course.  You'd
presumably be looking at an IANA process to maintain the zone file?

  Byron
(Speaking predominantly for myself, but in my capacity as APNIC's
Technical Director when stating that APNIC could handle rdap.arpa DNS
queries)

---

From: "John R Levine" <[email-masked]>
Date: 16 Jun 2013 23:21:44 -0400

> presumably be looking at an IANA process to maintain the zone file?

I expect that ordinary DNS caches will make the DNS query rate 
considerably less than the 3000 whois/sec that .COM sees.  It is my 
impression that most of the load comes from relatively few clients who 
send a lot of queries and so will get most of their DNS answers from their 
local cache.  If someone from Verisign is listening, the DNS query rate 
for whois.verisign-grs.com would be a reasonable estimate of the load.

All of the TLDs already have relationships with IANA to manage their name 
servers and contact info, and line 8b of the template already asks for the 
name of their WHOIS server.  I'd expect the name (for CNAME) or addresses 
(A/AAAA) of the RDAP server could be another item added to the existing 
database.

If you're hinting that it would be a good idea to check with IANA to 
ensure that this presents no unexpected administrative problems, I 
completely agree.

Regards,

---

From: George Michaelson <[email-masked]>
Date: Mon, 17 Jun 2013 14:24:33 +1000

I took Byron's comments in the sense that a discovery anchor which has to
handle internet-wide load is a good fit for anycast services, which we
already cooperate in as a delagete under IANA, alongside ICANN investment
in an anycast mesh for in-addr.arpa. There is an investment in systems, and
cooperation already in place.

When you consider that the number space is a significant part of the RDAP
information space, it makes sense for us to self-organize to manage this
discovery load, and it seems like a reasonable place to ask: is this
something we might do for wider community benefit, as a 'for the internet'
type thing?

-George

On Mon, Jun 17, 2013 at 1:21 PM, John R Levine <[email-masked]> wrote:

> impression that most of the load comes from relatively few clients who send
> a lot of queries and so will get most of their DNS answers from their local
> cache.  If someone from Verisign is listening, the DNS query rate for
> whois.verisign-grs.com would be a reasonable estimate of the load.
>
> ______________________________**_________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/weirds<https://www.ietf.org/mailman/listinfo/weirds>
>

---

From: "John R Levine" <[email-masked]>
Date: 17 Jun 2013 00:49:09 -0400

> something we might do for wider community benefit, as a 'for the internet'

I don't entirely understand what you're suggesting, but it seems to me 
that numbers can bootstrap from a pool of RIR RDAP servers, and names can 
boostrap from <tld>.domain.rdap.arpa, so there's no need for anything more 
complicated.

If in-addr.arpa is anycasted and rdap.arpa can piggyback on that 
infrastructure, that's great, but please let's avoid the black hole of 
overgenerality.

If you look around the IETF, you'll find that the graveyard is full of 
excessively general discovery mechanisms.  The current webfinger effort is 
a good bad example, with what was supposed to be a lightweight way to 
autoconfigure mail clients now groaning under the weight of far too much 
stuff you can ask about via overgeneralized URLs.

Regards,

---

From: Andy Newton <[email-masked]>
Date: Mon, 17 Jun 2013 10:05:52 +0000

On 6/17/13 12:49 AM, "John R Levine" <[email-masked]> wrote:

>excessively general discovery mechanisms.

I couldn't agree more. That's why I think the IANA registries approach is
the best solution. It is simple, and it can be the basis for the more
complicated rdap.arpa or rdap-servers.org solution. And it allows those
interested in writing higher performance clients to skip that first hop.

-andy