Subject: [idn] Another solution?

From: C C Magnus Gustavsson <[email-masked]>
Date: Tue, 19 Mar 2002 13:39:03 +0100 (CET)

I've been thinking about a not yet mentioned (AFAIK) solution for a
while but dismissed it as too ambitious. Yet, the longer we keep
going without getting there, the more reasonable it seems to me. I
suppose the respons will be "too much work, too late" but it could be
worth considering at least.

We could consider wiping the slate clean and design a new protocol.
(Bear with me just a little while longer.) Let's call it e g INS (for
Internet Name System or International Names System) to distinguish it
from DNS and ask IANA to assign, say, port 40 for it's use. We would
not only be able to introduce non-ASCII domain names using UTF-8 in a
nice, clean fashion and in accordance with RFC 2277. We could double
(or triple) the maximal allowed length for domain names and have four
times the number of flags. We could introduce a version number in the
protocol to make future changes easier and use experience with EDNS to
make it easier to expand in the future if needed. We could replace SRV
records (RFC 2782) with a cleaner and better solution, introducing
QSERV and QPROTO fields in the queries. And so on. All in all, we
could clean up and improve the current DNS standard significantly.

It seems we have a golden opportunity here. Nobody will use a new
protocol to get SRV records or longer domain names, but perhaps to
get international domain names. Does something like this really have
to take that long time to get consensus on, especially compared to the
time already used without reaching consensus so far? If we stick to
basic DNS principles and simply change the format of RRs, I think we
already have a pretty good idea (gained from the past years) about
what needs to be done; as long as we are careful and don't try to
build a completely new system but focus on cleaning up the old one.

In contrast to IPv6, this could very well be used in parallell with
the old one and handled by the same software.

Magnus

---

From: Paul Robinson <[email-masked]>
Date: Tue, 19 Mar 2002 18:39:21 -0800

On Mar 19, "D. J. Bernstein" <[email-masked]> wrote:

> Paul Robinson writes:
> > Something *should* be done, but your argument has a hint of
> > 'I never want anything done, ever' about it, which is putting people
off.
>
> I have put a huge amount of effort into evaluating the costs of various
> IDN proposals. Please read http://cr.yp.to/proto/idnc3.html before you
> make any more comments about what I'm trying to do.

I stand by my original statement. Something should be done, but your
document make you look like a typical whiner - you point out all the
problems, but offer no solutions to some of the problems you raise. The
solutions you do offer will take at least 4 years IMHO to be effective, and
your phase 2 will actually INTRODUCE more problems, as well as allowing the
continuation of other problems you raise.

> You may be aware that I'm the author of the mail software with the
> Internet's largest increase in SMTP-server deployment over the past 18
> months (reaching #2 in total deployment, behind only Sendmail), and the
> DNS software with the Internet's largest increase in domain-name
> deployment over the past 18 months (reaching #2 in total deployment,
> behind only BIND). My programs relay 8-bit data without trouble, even
> though certain people obviously don't understand why this is important.

I know who you are. I know your software. I have spent many hours looking at
your code. It's why I'm suspicious of anything you ever write or say. Don't
get me wrong, I'm not lacking in respect for some of the things you've done,
but I think you're shooting yourself in the foot with the way you'er going
about this. I don't want to have an argument with you, but you're
approaching this from the wrong angle. All very wrong. It's what you do, and
it makes sense to you, but shouting at everybody THAT THEY'RE ALL WRONG AND
IT MUST ALL STOP RIGHT NOW OR ELSE THE WORLD WILL END AND MAIL WILL BREAK is
not the way to get IDNA fixed. Perhaps expanding out IDNC3 to something more
than 'these pieces of software need to be 8-bit clean and then we start
registration' might be somewhere to head next...

> I want internationalized domain names (and mailbox names and so on) to
> work. But I can't support the IDNA proposal; IDNA is a disaster. I also
> can't accept having _any_ protocol move forward over so many objections.
> Even the strongest desire to _do something_ is less important than the
> requirement to obtain consensus for any change.

Don't you understand that to get what you want, all Unix boxen have to be
*completely* 8-bit OK, all MS boxen have to be *completely* 8-bit OK and a
good 20% of sites out there will just have to shut down ops permanently? How
are you proposing to display alpha-ol.com on a VT100?

Had you not considered (and excuse me if I'm being dumb here), that just
modifying a DNS resolver to understnad PunyCode might just be a nice short
cut? That perhaps, just maybe, software that recognises e-mail addresses as
being any continuous text with @ in them, and URLs as beginning http:// is
going to be able to just throw a string down to a resolver that knows that
it's doing? And that maybe replacing the DNS resolver on all the machine s
out there to be able to do lookups with PunyCode might be a TAD more
realistic than trying to get EVERYTHING, EVERYWHERE to be good with 8-bit?

I must be missing something here... I've *got* to be missing something
here... please, somebody tell me what it is...

--
Paul Robinson

---

From: Paul Robinson <[email-masked]>
Date: Tue, 19 Mar 2002 20:57:33 +0000

On Mar 19, "D. J. Bernstein" <[email-masked]> wrote:

> Go sell a Greek user an ``internationalized domain name'' with a delta,
> Pete. Then tell him that most of his correspondents will see the delta
> as incomprehensible gobbledygook rather than a delta. See what he says.

OK, scenario 1:

You tell him that although it's gobbledygook to people without greek
alphabet support, it will still work. It's not convenient, but it WILL work.
Guaranteed. For his business colleagues and friends in Greece, who DO have
the latest and greatest software, it will display as a delta. His ISP hasn't
had to upgrade, and everybody in the world can use his domain - eventually
they will see it as a delta as well, but for now they see it as an encoded
string they can still use no problem.

Scenario 2:

Oops, sorry, our mistake, it's NOT gobbledygook, it's prefectly fine. For
everybody in Greece. Unfortunately, his bank in the UK can't understand his
e-mail address because the S/360 coders haven't got time to upgrade all the
systems and applications software. His family won't be able to send him mail
through systems that are running proprietary or legacy mail applications
because they don't understand this 8-bt stuff. When he's abroad, his website
and e-mail address may be useless. But it's OK, because it's a CLEAN
implementation and a great protocol, and everybody else will catch up
sometime in the next 4-10 years. Until then, he has to get a 'normal' domain
to see himself over.

> Of course, display failures are not as intolerable as interoperability
> failures. But they're still failures.

And they are failures for OS developers and application developers. Not the
IETF. Not for the IDNA WG. Not for anybody who wants to get IDNs through.
Not for the people who don't want to have to re-write the MTA on the PDP
they have running in the back office. Not for people who want to have to
deal with another SMTP spec change. The only problem as I see it, is that
until software that deals with IDN knows how to display PunyCode properly,
people will see some crap on the screen. What you are proposing IS
introducing an interoperability failure, which through your own admission is
worse than a display failure.

> Surely you agree that bounced mail is serious!

Which of these is easier to implement:

1. An updated DNS resolver
2. Making every piece of software and display device that might ever have to
deal with IDNs capable of handling UTF-8?

If you were IT director of a large firm, and you had a choice as to which to
roll out, which would you choose?

--
Paul Robinson

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 19 Mar 2002 23:24:29 -0500

I guess the fundamental question is whether ACE is really "IDN"?
Is not ACE just an ugly mask slapped on to the DNS to allow clients to fake
out the user in that they are getting a multilingual domain name, but in
fact, all they got from the DNS operators is a string of uninteligible ASCII
characters?
Also remember that with the introduction of ACE, we are effectively creating
an "alternative system" to the DNS!
Cause now we are saying that (for the following example imagine, xx--ace = a
multilingual label <ML>)
- xx--ace is NOT really xx--ace as you see it, but it really is <ML>
- OR <ML> is the same as xx--ace
Either way, it means that we have fundamentally changed the concept of a
"unique" DNS name.
All the while risking that the solution is not even solving the IDN problem.
So what do you tell the customer they got?  An ASCII String? or a
Multilingual Domain name?  Or do you say its a Buy one Get one Free?!

Here is my thought: ACE is an easier route for server operators (transport),
and therefore is a good intermediary solution towards IDN for servers; Just
use 8-bit is an easier route for users and client apps (display), and
therefore is a good intermediary solution towards IDN for clients.  Why cant
we go both routes and layout a roadmap for an eventual convergence with a
protocol change that utilizes 8bit data?

Edmon

---

From: "Bruce Thomson" <[email-masked]>
Date: Wed, 20 Mar 2002 13:14:55 +0900

(Deleting ccs to iesg, etc. I really don't think we should be bombarding
their inbox.)

> roll out, which would you choose?

Frankly, if I was the IT director of a large firm, I wouldn't roll it out in the near
future, in either scenario. There's just too much missing that won't be filled
in for a few years. We need a transition period in which the multilingual-capable
people of the world who also like to try out leading-edge technology play around
with it for their own amusement, presumably while using legacy domains for their
main mail/websites.

e-mail itself only went mainline within the past very few years. Before that,
people would say things like, "well, I would like to use e-mail maybe, but
if I pass out my e-mail address on my business card, I would have to check
my mail every day and I can't be bothered." The pioneers played with it
(some quite profitably, no doubt) for years and the mainstream benefitted
much much later. This is reasonable and natural.

Look at where IPv6 and multicast are, reasonably well defined but quite a
ways away from being commonplace. Do you really expect idns to be
different?

That's why I tend to take a different view from "if I was the IT manager".
Unlike that manager, we are making something that we will have to live
with for a long time, and the idea that we can have a relatively quick,
painless transition for the mainstream customer is probably getting more
credibility than it deserves.

Bruce

---

From: "D. J. Bernstein" <[email-masked]>
Date: 20 Mar 2002 04:32:50 -0000

Paul Robinson writes:
> alphabet support, it will still work. It's not convenient, but it WILL
> work. Guaranteed.

False. IDNA does _not_ work. IDNA causes interoperability failures. Mail
will bounce, for example, in situations where ASCII domain names would
have worked fine. IDNA coauthor Adam Costello has admitted this.

> And that maybe replacing the DNS resolver on all the machines out
> there to be able to do lookups with PunyCode might be a TAD more 

Here you are assuming that the only problem is the DNS resolver---that
the conversion between the local character encoding and the IDNA
character encoding can be handled entirely by the DNS resolver.

That assumption is false. Consider, for example, an MTA configured to
accept mail for pi.cr.yp.to, with a Greek pi. The MTA compares the
incoming domain name to pi.cr.yp.to. That doesn't involve the resolver.

People who say that IDN is purely a DNS issue are confused.

> Making every piece of software and display device that might ever have
> to deal with IDNs capable of handling UTF-8?

Here you're being simultaneously inconsistent and shortsighted. Fixing
bad displays is part of the cost of IDNs. In the context of UTF-8, you
agree with me that this is a cost; in the context of IDNA, you ignore
the cost completely.

In fact, the cost of fixing UTF-8 displays is much _smaller_ than the
cost of fixing IDNA displays. UTF-8 has been around for many years, has
built up incredible momentum (as illustrated by RFC 2277), and already
works in a huge number of programs.

The extra programs hurt by IDNA aren't just UTF-8-aware clients. Fixing
the IDNA display failures also means changing web servers, mail servers,
DNS servers, etc., so that the sysadmin can put a properly displayed IDN
into his server configuration files. Think about the above pi.cr.yp.to
example again.

> The solutions you do offer will take at least 4 years IMHO to be
> effective

Let's suppose 4 years is right, and let's compare the results to IDNA
after 4 years.

IDNC3 requires 8-bit fixes to some widely deployed programs, certainly.
But IDNA needs _much larger_ changes in _many more_ programs. So, after
the same 4 years, only a fraction of the IDNA work will be done. IDNA
will still have an incredible number of display failures, plus the
interoperability failures and all the other IDNA problems.

Even worse, IDNA doesn't do _anything_ to fix the other half of the
email problem. Do you seriously believe that Chinese users will be
satisfied with email addresses where the domain part can contain Chinese
characters but the box part is still required to be ASCII? It's obvious
how to fix this with UTF-8; how, pray tell, do we fix it with IDNA?

I presume that you're not one of the 7-bit-forever crackpots. How do you
propose migrating from IDNA to UTF-8? This is much more costly than
moving directly to UTF-8, because it needs a compatibility period during
which everyone supports two different encodings of the same character.
Doesn't it bother you that the IDNA documents don't discuss this at all?

What makes your position particularly shameful is the fact that people
proposed requiring 8-bit transparency _eleven years ago_. If it hadn't
been for Paul Vixie et al. making your ``it'll take years!'' argument
back then, we would have had 8-bit transparency today. Do you want to be
facing the same stupid bugs in another eleven years?

> Until then, he has to get a 'normal' domain to see himself over.

Correct. Your example Greek user has an ASCII domain name that's always
displayed with an ASCII d instead of the truly desirable Greek delta. 

Now, please explain why the same user should prefer a domain name that's
_occasionally_ displayed with the desired delta but _usually_ displayed
as incomprehensible gobbledygook.

Your answer, of course, will be something like this: ``The gobbledygook
is a temporary problem. In twenty years, after the massive IDNA upgrade
is complete, everyone will see a delta!''

In short, you're looking at the long-term IDNA benefits (never mind the
interoperability failures and all the other problems) but refusing to
look at the long-term UTF-8 benefits. Inconsistent once again.

> Something should be done, but your document make you look like a
> typical whiner - you point out all the problems, but offer no
> solutions to some of the problems you raise.

False. http://cr.yp.to/proto/idnc3.html explains how IDNC3 offers
solutions to every one of the IDNA problems that it points out:

   * interoperability failures;
   * inconsistent displays of the same name;
   * unnecessary implementation and deployment costs;
   * multiple semantically similar names;
   * identical displays of different names; and
   * typing failures.

Each solution is listed right next to the problem, so I can't imagine
how you missed this.

> What you are proposing IS introducing an interoperability failure,

False. Every step in http://cr.yp.to/proto/idnc3.html preserves
interoperability.

> How are you proposing to display alpha-ol.com on a VT100?

There are several options. One option is to work around the hardware
limitations in software, displaying something like

          |
   /\/ /\ |   /^ /\ /\ /\
   \/\ \/ | * \_ \/ | | |

Another, much more popular, option is to move your email reading, web
browsing, etc. from your 1970s-vintage VT100 to a graphics terminal.
Have you considered the VT340, for example? Or an IBM PC, model 5150?

> a good 20% of sites out there will just have to shut down ops permanently

Get a grip, Paul.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: "Adam M. Costello" <[email-masked]>
Date: Wed, 20 Mar 2002 07:36:03 +0000

C C Magnus Gustavsson <[email-masked]> wrote:

> Let's call it e g INS (for Internet Name System or International Names
> System) to distinguish it from DNS...

Edmon Chung <[email-masked]> wrote:

> Is not ACE just an ugly mask slapped on to the DNS...
>
> Why cant we go both routes and layout a roadmap for an eventual
> convergence with a protocol change that utilizes 8bit data?

You are both focusing too much on DNS.  DNS is just one of many
protocols that carries domain names.  Others include SMTP, RFC 822,
IMAP, HTTP, URIs, SSL certificates, and on and on...  The problem of how
to stuff non-ASCII characters into domain names applies equally to every
protocol that uses domain names; DNS isn't really that special.  Making
DNS accomodate non-ASCII characters in domain names doesn't solve the
problem for any other protocol.  Each one would need to be updated.

And the harder part of the problem is not how to preserve the
characters, but how to compare the labels: knowing that e-with-acute
is the same as e followed by combining-acute.  Simply making protocols
8-bit clean doesn't give you that.

> Is not ACE just an ugly mask slapped on to the DNS to allow clients
> to fake out the user in that they are getting a multilingual domain
> name, but in fact, all they got from the DNS operators is a string of
> uninteligible ASCII characters?

Yes!  (Except that it can be slapped onto any protocol, not just DNS.)

> Cause now we are saying that (for the following example imagine,
> xx--ace = a multilingual label <ML>)

It's the latter: <ML> is the same as xx-ace (they are equivalent domain
labels).

> Either way, it means that we have fundamentally changed the concept of
> a "unique" DNS name.

It's not a fundamental change.  Even now, foo and Foo and FOO are
equivalent labels, even though they are not identical strings.  Software
needs to be slightly clever when comparing labels for equivalence.  IDNA
does not introduce a fundamental change, but merely an evolutionary one:
the cleverness needed to compare IDN labels is greater, because now <ML>
and xx--ace are equivalent, but it's the same fundamental concept.

> Multilingual Domain name?  Or do you say its a Buy one Get one Free?!

I'm sure registrars will put all sorts of marketing spin on it, but I'd
say you got one domain name which can be represented by a variety of
strings, some of which are ASCII-only and some of which aren't.

AMC

---

From: C C Magnus Gustavsson <[email-masked]>
Date: Wed, 20 Mar 2002 11:46:34 +0100 (CET)

On Wed, 20 Mar 2002, Adam M. Costello wrote:

> IMAP, HTTP, URIs, SSL certificates, and on and on...

Yes, I know. Considering how many times this mantra has been repeated
on the list, it seems unlikely anyone wouldn't be aware of it. ;-)

I get the feeling we've been here before. I hope you don't mind me
saying so, but as soon as anyone suggests anything but ACE you claim
this person is focusing too much on DNS. I think we all see the
advantages of ACE, but I'm not so sure everyone see the drawbacks.

> Each one would need to be updated.

Yes, or at least looked over; that's the unfortunate reality. The real
question is whether it's worth it or not, and I suppose that's what
the discussions in this wg with few exceptions have always been about.

I wrote some time ago, when this list was recently set up (and quoting
Linus Torvalds) that when some things need to be fixed, they really
need to be fixed. We have come a long way since but I still see this
as one such case. Perhaps even more so. We then had the choice between
a quick and dirty fix that wouldn't solve all the problems and a more
painful solution that would be better in the long run. We still do.

In this matter, I have to agree with Dan Bernstein, Dan Oscarsson and
others; I still really think we should do it right from the beginning.
We have a five story building on quick sand here and I'm not so sure
we should add another five stories on top of it without partially
tearing down the old one to build a more solid foundation first.

You can hardly claim my suggestion will break anything. It will take
more effort and more time for sure and my suggestion is obviously the
most ambitious. But it will solve the largest number of problems as
well. Again, the question is: Is it worth it? I take it yours and Mats
Dufbergs answer is "no" but I'm not so sure about the rest of the wg.
This is not going to be an easy change-over for the Internet no matter
what solution we choose, so why not do the foundation right? This is,
as said, a golden opportunity to fix a number of other old problems.

> 8-bit clean doesn't give you that.

Yes. That's exactly my point. This effort has taken many years. By now
we have to realize that it's not going to be an easy solution. So, are
we sure we don't want to do it right? $100 is expensive for a pizza
but when buying a house for $150,000, adding $1000 isn't much.

Magnus

---

From: "Edmon Chung" <[email-masked]>
Date: Wed, 20 Mar 2002 09:06:47 -0500

Hi Adam,

Your argument seems to be a bit odd...

From: "Adam M. Costello" <[email-masked]>
> protocols that carries domain names.

So imagine you are teaching a kid to play soccer, and he totally missed a
kick... What you just said is the same as telling him, you are focusing too
much on the ball, there are many other players in your team, and there are
the opponents as well, you need to look at them or you wont be able to kick
the ball!

Of course we understand that there is a big implication, all the more reason
to focus on DNS and make it right first.  Then we can consider the other
protocols.  Just like in my example, you need to teach the kid how to kick
the ball right before you teach him strategies of the game.

> needs to be slightly clever when comparing labels for equivalence.

Again this is quite a stretch in your argument.  I am fairly confident that
there will be no dispute in saying that foo=FOO and xx--ace=<ML> is a
totally different concept!  If you think that xx--ace=<ML> is not
fundamentally changing the concept of name matching in DNS, then what is the
difference of saying "dog"=<DOGinCHINESE> is equivalent?!  This is a very
dangerous road you are taking.

> strings, some of which are ASCII-only and some of which aren't.

So you are effectively endorsing the buy one get one free approach...

Edmon

---

From: tedd <[email-masked]>
Date: Thu, 21 Mar 2002 10:17:26 -0500

>>  as incomprehensible gobbledygook rather than a delta. See what he says.

Okay, I'm not Greek, but as the registered owner of .com, I'll tell you*.

I expect that some windoze users will display it (.com) as a box dot 
com; mac users will display it as a delta dot com; and everyone else 
will display it as a bq--eida.com (RACE), or dq--uuyg.com (DUDE), or 
zq---h9g.com (AMZ_C), or punycode, or whatever -- AND, they will 
continue to do so until they have the appropriate software on their 
machines to handle the conversion. That's what I expect.

On the other hand, I do not expect that the IDN will eventually 
produce something that will transparently produce the appropriate 
glyphs on millions of users machines. In my mind, that's simply not 
possible because I think it requires that every machine on this 
planet have a complete copy of UNICODE installed and available.

Granted, I am clueless about a lot of this IDN stuff and I may be 
"hubris" (as Mr. Paul Hoffman once informed me) in my offering my 
opinion to the "Gods of the Internet", but I do think I see the 
distinction between which side of the modem one is on and what 
responsibility lies in each camp.

><-snip>

I would vote for scenario 1. Besides, IMHO (in my hubris opinion) 
that's the only way it's going to work.

tedd

* The Greek version of "delta" registers a internal error -- it's the 
Math language set that provides an acceptable "delta".
-- 
http://sperling.com

---

From: tedd <[email-masked]>
Date: Thu, 21 Mar 2002 10:17:26 -0500

tedd

---

From: Simon Josefsson <[email-masked]>
Date: Mon, 25 Mar 2002 21:33:02 +0100

> could clean up and improve the current DNS standard significantly.

I think many people have been thinking about this.  However, people
seems to fear that designing a new protocol would increase the speed
of namespace fragmentation.  foo.com would be available in DNS and not
in INS and vice versa.  I believe the "namespace fragmentation"
argument is a poor one -- namespace fragmentation isn't a protocol
issue, it is a political issue.  Put pressure on organizations
handling the namespace to not fragment it and they won't fragment it.
If the pressure isn't there, they will continue to do whatever they
can get away with.