Subject: Re: [idn] Moving Towards UTF8 vs ASCII(ACE) Forever

From: "John Stracke" <[email-masked]>
Date: Thu, 21 Mar 2002 17:47:54 -0500

>> Provably false: well-coded applications know the limitations of domain
>> names, and do not even attempt to make requests for non-ASCII names.
>
>First of all, I disagree with the "well-coded" part because I believe a
>well-coded application will do the dns request as is and allow the dns
>response to determine the validity of a domain name.

Nope.  You can almost certainly give your user a better error message if 
you validate the input yourself.

For that matter, a well-designed application will not even make it 
possible to enter anything but ASCII in an input field (whatever) for a 
domain name.  Basic rule of usability: making user mistakes impossible is 
better than catching them after the fact.

>Secondly, if this is
>the case, then the plugin should be intercepting the input to the app and
>not the output.  It is still a possibility.

No, because that input is highly application-specific.

/========================================================\
|John Stracke                    |Principal Engineer     |
[email-masked]   |Incentive Systems, Inc.|
|http://www.incentivesystems.com |My opinions are my own.|
|========================================================|
|Diplomacy: The art of letting someone else have your way|
\========================================================/

---

From: "D. J. Bernstein" <[email-masked]>
Date: 21 Mar 2002 23:50:07 -0000

John Stracke writes:
> domain name.

That's incredibly bad design.

You're violating the basic principles of information hiding articulated
by Parnas in the early 1970s. Instead of isolating the name-existence
decision in one place (the DNS server's database), you're spreading the
decision across a huge number of programs on a huge number of machines.
Changing a decision becomes extremely expensive.

We're seeing the economic consequences of this in BIND's res library.
That library is the most widely used UNIX DNS-lookup mechanism, and is
one of the largest sources of UTF-8 failures; see http://pi.cr.yp.to. It
has to be fixed on a huge number of machines.

(Note that this cost, together with the other costs of making UTF-8 IDNs
work, is only a tiny fraction of the costs of making IDNA work.)

If you think that the 8-bit problems in res are an example of people
agreeing with your design ideas, you're mistaken. The change was made in
a panic in 1996, when CERT announced that several programs had security
flaws based on careless use of DNS PTR results. Of course, anyone who
thinks about the problem for a moment can see that unusual DNS A _input_
has no relevance to the security issue, but people in a security-related
panic often don't stop to think about the damage they're causing.

> Basic rule of usability: making user mistakes impossible is 

You obviously aren't achieving that goal. You're catching typos if and
only if they involve non-ASCII characters. What about ASCII typos? What
is the basis for your assertion that the no-such-name message should, as
a matter of UI design, be different between these two situations?

Even worse, what about ASCII typos that produce valid domain names?

Basic rule of usability: Have the computer copy the data so that the
user doesn't have an opportunity to make a mistake. Saves time, too.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: Dave Crocker <[email-masked]>
Date: Thu, 21 Mar 2002 18:11:43 -0600

At 11:50 PM 3/21/2002 +0000, D. J. Bernstein wrote:
>ohn Stracke writes:

A UI that actually enforces the syntactic rules of the underlying service?

Yes, indeed, that is truly a terrible idea.

It would be much better to let users enter strings that are not valid so 
that some time in the future -- they can never be sure when -- the user's 
string will be found to be invalid and cause an error return to the user -- 
again, sometime later.

Absolutely a superior outcome!

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: YangWoo Ko <[email-masked]>
Date: Fri, 22 Mar 2002 12:20:38 +0900

> Absolutely a superior outcome!

in that sense, current design of ms ie is very bad.
it just sends non ldh input to their seach engine !

-- 
/*------------------------------------------------
YangWoo Ko : [email-masked]
------------------------------------------------*/

---

From: YangWoo Ko <[email-masked]>
Date: Fri, 22 Mar 2002 13:26:23 +0900

> it just sends non ldh input to their seach engine !

i withdraw above wrong statement.

yves corrected me that "if the address looks like a domain name, 
ie first does a dns lookup of the domain name after encoding it 
in UTF-8"

thanks to yves.

i am still worndering what "looks like a domain name" means.

---

From: "Bruce Thomson" <[email-masked]>
Date: Fri, 22 Mar 2002 13:46:54 +0900

The IE feature that translates domain names to UTF-8 is nice,
but it's still some bugs and is not production level code.

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 22 Mar 2002 19:04:45 +0900

----- Original Message -----
From: "Bruce Thomson" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; "IETF idn working group" <[email-masked]>
Sent: Friday, March 22, 2002 6:29 PM
Subject: Re: [idn] URL encoding in html page

> > What if all the html viewable text is in english, but, only the href url contains
> > legacy (korean) encoded hostnames?  chinese visitors would see clean english homepage,
> > but fail to click through the korean link.
> >
> Well, that could happen, but a META tag would solve that so easily. Personally
> I often use a simple text editor to deal with HTML, and would find it easier to
> use legacy encodings or UTF-8 than cut-and-paste ACE from somewhere.
> Of course the user could do it either way and it would work.

Yes. Charset META tags help. But, many homepages  have assumptions on the main audience's
default char encodings and very often omit the  META tag for the encoding like :
  <meta http-equiv="Content-Type" content="text/html; charset=euc-kr">

Moreover, IDN url would be used in a pure FRAMESET document that defines frame URLs
and contains no viewable texts. Such FRAMESET documents often omit charset META tags.
 (look into the html source of http://www.freeway.co.kr/ )

AFIAK, 99.99999% of korean homepages have implicit/explicit
legacy korean encoding (KS_C_5601-1987 or euc-kr). So do most japanese/chineses homepages.
UTF8/UCS-2 encodings are rarely used in global WEB publishing.  Legacy encodings
will dominates even in the future, because it is compact and inexpensive.

IF we want to make IDN truly internationally interoperable, all IDN-aware webbrowsers/applications
should contain libaries of all kinds of legacy-to-Unicode conversion routines. It will burden
too much memory load on handheld devices like PDA.

Moreover, legacy encodings are revised separately from unicode. We may face with as toughest
versioning problems as we did in stringprep/nameprep versioning problems for newly added unicode points.
How to guarantee  stability and intergrity of IDN operations in the all combinations of  numerous kinds and versions of iDN-aware
applications and legacy encodings?

Soobok Lee

---

From: "Bruce Thomson" <[email-masked]>
Date: Fri, 22 Mar 2002 18:29:15 +0900

> MS IE 6.0 already know IDNA or UDNS?  Otherwise , do you mean that IE determines
> the encoding correctly if <META> charset tags are present in appropirate section
> in html pages ?
> 
Right, except for the base URL bug I mentioned, it works fine.

> Is HTTP/1.2 being planned for IDN  HOST: values ?
> If not, HTTP/1.2 HOST: values should not contain any legacy/utf8 encoding.
> 
Well, depending on how you want it to work, version 1.1 might be OK. It allows
%-escaped UTF-8 I believe. And the Apache web server takes it OK.
Of course if the browser sent ACE that would work too, and doesn't exactly
violate 1.1, but in that case you have to make sure that all your virtual site
definitions at the server are in ACE or that the server does the translation.

Would using ACE here be a change to the HTTP spec, or would an out-of-band
agreement be OK?

Bruce

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 22 Mar 2002 17:34:19 +0900

----- Original Message ----- 
From: "Bruce Thomson" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; "IETF idn working group" <[email-masked]>
Sent: Friday, March 22, 2002 5:14 PM
Subject: Re: [idn] URL encoding in html page

> Encodings of anything on an HTML page are an issue as well. Various
> methods are already defined for the codeset to be determined, so
> whether UTF-8, legacy encodings, or ACE are used, there shouldn't
> be a problem. If uses a different encoding for the URL from the rest
> he will have a problem, 

> but that's no different than if he tried to mix
> two different legacy encodings in his text without intervening META
> tags.
> 
> By the way idns in HTML already work pretty well, except that IE can't handle
> an idn in its base for relative URLS (unless of course you specify
> explicit ACE in your BASE directive). Until Microsoft fixes this, just
> use absolute URLs.

> 
> IDNA probably doesn't need to make a recomendation here. What is more
> critical is how the browser sends the data in the HTTP headers.

Soobok Lee

> 
> Bruce
> 
> > 
> > If a simple HTML page contains the following tag,
> >   <a href=http://www.<ML>.com>Hello World!</a>
> >  in which, <ML> maybe in a native legacy encoding or utf8 encoding, it is easy to imagine that
> >  some vistors who click that link may be led to wrong sites or nowhere.
> > 
> > When <ML> encoding is not specified by HTML <meta> charset tags  and 
> > the author and visitors have different default char encodings,
> > interoperability problems will surge on , no matter which architure we 
> > choose between IDNA and UTF8-based IDN. IDN-non-aware and even IDN-aware 
> > Web Browsers would not be able to decide which encoding was used by the author.
> > 
> > IDNA-non-aware browsers will always fail to resolve native encoded Web Hostmaname in the URL, 
> > even when the html page has specified its charset encoding in the <head> section.
> > In this case, IDNA backward compatibility cannot save the old browser from failures,
> > without dns/webserver workarounds for on-the-fly native-to-ACE heuristic encodings.
> >  
> > With LDH-only URL, we had no such problems and headaches.
> > 
> > someone may argue that  the html URL should have to contain ACEed URL like this:
> > 
> >   <a href=http://www.bq--blahblah.com>Hello World!</a>
> > 
> > Should IDNA recommend all HTML authors to use such ACEed URL for backward compatilbility
> > and error-free fast deployment?
> > 
> > 
> > 
> > 
> > 
> >

---

From: "Bruce Thomson" <[email-masked]>
Date: Fri, 22 Mar 2002 17:14:48 +0900

he will have a problem, but that's no different than if he tried to mix

Bruce

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 21 Mar 2002 23:59:05 -0800

Soobok Lee <[email-masked]> wrote:

> Should IDNA recommend all HTML authors to use such ACEed URL
> for backward compatilbility and error-free fast deployment?

It certainly seems like a sensible approach.  Users wouldn't have to see
the ACE (unless they go poking around in the HTML source), and any
browser, even old versions, would be able to resolve it.

-Doug Ewell
 Fullerton, California

---

From: "James Seng" <[email-masked]>
Date: Fri, 22 Mar 2002 15:40:35 +0800

This is a good question, one which is related to IDN & URL.

Check this out: http://www.i-d-n.net/draft/draft-masinter-url-i18n-07.txt

-James Seng

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 22 Mar 2002 16:25:04 +0900

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 22 Mar 2002 07:18:09 -0800

Compliant browsers already have to handle Unicode, since NCRs (e.g.
&#x1234; ) are always Unicode code points. All XML parsers also have
to handle Unicode (UTF-8 and UTF-16).

> Legacy encodings

While I do expect the transition to Unicode to take some time, once
some of the older browsers die off it may shift more rapidly than we
think.

Mark
—————

Γνῶθι σαυτόν — Θαλῆς
[For transliteration, see http://oss.software.ibm.com/cgi-bin/icu/tr]

http://www.macchiato.com

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "IETF idn working group" <[email-masked]>
Sent: Friday, March 22, 2002 02:04
Subject: Re: [idn] URL encoding in html page

> How to guarantee  stability and intergrity of IDN operations in the
all combinations of  numerous kinds and versions of iDN-aware

---

From: "Soobok Lee" <[email-masked]>
Date: Sat, 23 Mar 2002 01:16:59 +0900

----- Original Message -----
From: "Mark Davis" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; "IETF idn working group" <[email-masked]>
Sent: Saturday, March 23, 2002 12:18 AM
Subject: Re: [idn] URL encoding in html page

> to handle Unicode (UTF-8 and UTF-16).

Right, Already.
MS IE and NEtscape  already have been supporting  UNICODE
from serveral year ago, but still most homepages are in legacy encodings.
MS WORD (already unicode based) have features to produce  (from
unicode-based .doc files)   legacy encoded .html files  for web publishing

Korean/Japanese/Chinese texts in UTF8 are 50% bigger than legacy ones.
50% more disk space and bandwidth  will be required.
Each Cyrillic alhpabet in legacy code occupy one octet, while in UTF8,
it requires 3 octets. 200% more space is needed.
I cannot imagine the entire Russians make transition to UTF8.
Legacy encnodings are more space efficient than UNICODE.

legacy-to-legacy conversions like BIG5->KSX1001 are really being implemented
as two steps of BIG5->UNICODE  and UNICODE->KSX1001. UNICODE
are actively used  as such intermediate encodings, but still not  be  used and entered
directly by  end users so actively. Rather, UNICODE  may be a hub to facilitate  interchange
of informations in different legacy encodings or  font sharing for differently legacy-encoded chars.

I  regard UNICODE as a substrate (not as a competitor) upon which legacy encodings are built.

> some of the older browsers die off it may shift more rapidly than we

I am not UNICODE expert nor character expert. But, everyday, i  feel
the strong inertia toward legacy encodings in our local language communties.
language-tagging-enabled text format like HTML will lengthen the lifespan
of legacy encodings by great amounts  and allow legacy-coded HTML texts
are internationally interchanged without problems.

Soobok Lee

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 22 Mar 2002 10:21:05 -0800

From my experience talking with customers in the field, the main reason that people are not serving
up UTF-8 pages is not the bandwidth, it is the fact that there are still some browsers out in the
field that do not yet handle it correctly. While they are dying off fairly quickly, it is not quite
at the point where people are willing to write them off.

As far as size goes, it is worthwhile looking at some data samples. The following are from a page on
the Unicode site that is translated into different languages, so it has essentially the same
information on each page.
 Size Page
            8882 s-chinese.html
            8946 t-chinese.html
            9347 esperanto.html
            9498 maltese.html
            9739 icelandic.html
            9833 czech.html
            9944 welsh.html
            10064 danish.html
            10109 swedish.html
            10127 polish.html
     Size Page
            10219 interlingua.html
            10221 italian.html
            10297 spanish.html
            10308 portuguese.html
            10312 lithuanian.html
            10329 german.html
            10376 romanian.html
            10401 korean.html
            10506 french.html

     Size Page
            10726 japanese.html
            10953 hebrew.html
            11192 arabic.html
            13292 greek.html
            13870 russian.html
            13892 persian.html
            14549 hindi.html
            15337 georgian.html
            15853 deseret.html

So the best case is about 50% of the worst case. Some of this is due to the encoding, and some is
due to different languages just using different numbers of characters. However, when you look at web
pages in general use, the amount of text (in bytes) is really swamped by graphics, Javascript, HTML
code, and so on. So fundamentally, even the variations above are not that important in practice.

BTW This is getting way off topic.

Mark
—————

Γνῶθι σαυτόν — Θαλῆς

http://www.macchiato.com

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 22 Mar 2002 23:52:55 +0000

Soobok Lee <[email-masked]> wrote:

> in which, <ML> maybe in a native legacy encoding or utf8 encoding, it
> is easy to imagine that some vistors who click that link may be led to
> wrong sites or nowhere.

Very easy to imagine indeed, because the HTML spec says that the href
attribute must contain a URI, and the URI spec says that the host must
contain only ASCII letters, digits, hyphens, and dots (or it may be a
bracket-enclosed IPv6 address literal).

> Should IDNA recommend all HTML authors to use such ACEed URL for
> backward compatilbility and error-free fast deployment?

Not necessary, since the HTML and URI specs already limit the host to
ASCII letters, digits, hyphens, and dots.

Bruce Thomson <[email-masked]> replied:

> Well, depending on how you want it to work, version 1.1 might be OK.
> It allows %-escaped UTF-8 I believe.

HTTP 1.1 says that the Host: header field must contain the <authority>
part of the URI (that is, <host>[:<port>] ), and the URI spec forbids
%-escapes in <authority>.

> Would using ACE here be a change to the HTTP spec

No.  ACE host labels are honest-to-goodness valid ASCII host labels, so
you can use them wherever traditional ASCII host labels are allowed.
You don't need any special permission or invitation.

AMC

---

From: "Soobok Lee" <[email-masked]>
Date: Sat, 23 Mar 2002 09:38:02 +0900

Dear Mark,

UNICODE wil get more and more popularity as time goes by.
But, that does not mean that legacy encodings will disappear or will be obsoleted by UTF8.
There are at least 2 reasons why legacy encodings will be forever.

 1.  most legacy codes are standardized by local *governmants* that are best qualified to
       find and reflect  local communities's character needs.
      For example, Korean GOV has been constantly revising its KSX100? local legacy codes
        to include new Graphic characters and new rarely-used Chinese letters , even before
        UNICODE decided to include them.
      In other words, legacy codes are under control of  their language communites. But UNICODE
        are not, and has its own schedules and principles and motivations.
      It may be *politically* impossible for legacy codes to be obsoleted by UNICODE.
      Can we imagine  Korean Gov publish its laws and rules documents in UNICODE, not in KSX100x ?

 2. legacy encodings are already  internationally interoperable in popular HTML/MIME contents.
     There is no reason why  KSX100x-encoded homepage owners/message senders
       should abandon legacy encodings and make transitions into UTF8    at the cost of additional
       space and operational inefficiency    now and even in the forseeable future.

 I believe UNICODE is now everywhere and  will be everywhere even in the future. In the same time,
  UNICODE has provided legacy encodings/codes with more opportunities to be interoperable with
    minimum costs.

 Soobok Lee

  ----- Original Message -----
  From: Mark Davis
  To: Soobok Lee ; IETF idn working group
  Sent: Saturday, March 23, 2002 3:21 AM
  Subject: OT - Re: [idn] URL encoding in html page

  From my experience talking with customers in the field, the main reason that people are not serving up UTF-8 pages is not the
bandwidth, it is the fact that there are still some browsers out in the field that do not yet handle it correctly. While they are
dying off fairly quickly, it is not quite at the point where people are willing to write them off.

  As far as size goes, it is worthwhile looking at some data samples. The following are from a page on the Unicode site that is
translated into different languages, so it has essentially the same information on each page.
    Size Page
              8882 s-chinese.html
              8946 t-chinese.html
              9347 esperanto.html
              9498 maltese.html
              9739 icelandic.html
              9833 czech.html
              9944 welsh.html
              10064 danish.html
              10109 swedish.html
              10127 polish.html
       Size Page
              10219 interlingua.html
              10221 italian.html
              10297 spanish.html
              10308 portuguese.html
              10312 lithuanian.html
              10329 german.html
              10376 romanian.html
              10401 korean.html
              10506 french.html

       Size Page
              10726 japanese.html
              10953 hebrew.html
              11192 arabic.html
              13292 greek.html
              13870 russian.html
              13892 persian.html
              14549 hindi.html
              15337 georgian.html
              15853 deseret.html

  So the best case is about 50% of the worst case. Some of this is due to the encoding, and some is due to different languages just
using different numbers of characters. However, when you look at web pages in general use, the amount of text (in bytes) is really
swamped by graphics, Javascript, HTML code, and so on. So fundamentally, even the variations above are not that important in
practice.

  Mark
  —————

  Γνῶθι σαυτόν — Θαλῆς

  http://www.macchiato.com

---

From: "Soobok Lee" <[email-masked]>
Date: Sat, 23 Mar 2002 10:10:17 +0900

----- Original Message ----- 
From: "Adam M. Costello" <[email-masked]>
To: "IETF idn working group" <[email-masked]>
Sent: Saturday, March 23, 2002 8:52 AM
Subject: Re: [idn] URL encoding in html page

> bracket-enclosed IPv6 address literal).

Right!  
Under Current URI and HTTP spec, native-encoded IDN hostnames
are just illegal inputs and they should be passed to some exception
handling mechanisms , for example, search engines like  search.netscape.com
or auto.search.msn.com.

> ASCII letters, digits, hyphens, and dots.

We experts already knew this. But, many ML.com registrants don't know  about this
poor destiny of ML.com. They want to use native ML.com in their HTML homepage.

If we want to have interoperable URI supporting native IDN, we should revise
URI spec and HTTP spec BOTH. But, native IDN supports accompany potential
legacy code versioning and code interoperablility problems.
Would anyone provide indepth analysis on this caveat  ?

Soobok Lee

---

From: "Bruce Thomson" <[email-masked]>
Date: Sat, 23 Mar 2002 15:00:30 +0900

> You don't need any special permission or invitation.

Well the HTTP spec doesn't forbid the characters used in ACE, to be sure.
But you are tunnelling a new form of information through the legacy protocol,
right? As an implementor of a web server, how do I know how to interpret
the "host:" field in the HTTP header? If I see something that looks like ACE,
I suppose I would need to decode it to the original UTF-8 to compare with
my virtual domain definitions. As a WG member, I could probably figure out
that I need to do that, but I don't see where that is implied by the specification.

Bruce

---

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 23 Mar 2002 22:54:08 +0000

Bruce Thomson <[email-masked]> wrote:

> Well the HTTP spec doesn't forbid the characters used in ACE, to be
> sure.  But you are tunnelling a new form of information through the
> legacy protocol, right?

Sort of.  The ACE label has always been a valid host label.  The ACE
label itself is not a new form of information.  What's new is that this
traditional ASCII host label is now considered to be equivalent to a
non-ASCII label.  But even if you don't know that, you can still treat
it as a traditional ASCII host label (which it is), and everything will
work, except that humans looking at it won't find it meaningful.

> As an implementor of a web server, how do I know how to interpret the
> "host:" field in the HTTP header?  If I see something that looks like
> ACE, I suppose I would need to decode it to the original UTF-8 to
> compare with my virtual domain definitions.

No, you wouldn't.  If you don't know anything about IDNs, then in your
mind all host names are ASCII, so you would do an ASCII comparison
between the Host: field (which is ASCII) and your virtual domain
definitions (which are also ASCII), and that would work just fine.

On the other hand, if you know about IDNA, then your virtual domain
definitions might be non-ASCII, but you know that the proper way to
compare two domain names is to convert both to ASCII and then do an
ASCII comparison.  (That's IDNA rule 3.)

In neither case do you need to decode the ACE.  The only time you'd have
a reason to do that is when you're trying to hide the ACE from humans.

AMC

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 24 Mar 2002 21:36:43 +0900

----- Original Message ----- 
> Would anyone provide indepth analysis on this caveat  ?

 Even if we stay with current HTTP/1.1 which allows only ASCII host: header values,
 still we could revise  URI spec to allow native (utf8 or legacy encoding) IDN in URI.

 1) With IDNA and HTTP/1.1 , the web browser can encode Native IDN in URI into ACE one , and
 then open HTTP 1.1 session into the ACEed hostname with ACE host: value.

 2) With IDNA and revised HTTP with utf8 host support,  the web browser can encode 
 utf8 IDN in URI into ACE one, and  then open HTTP session into ACE hostname with utf8 host: value.

 3) With UTF8-based IDN and revised HTTP with utf8 host support, it can check whether 
 the native IDN is in utf8, and, if not, convert the iDN into utf8 , and then open
 HTTP session into utf8 webhost with utf8 host: value.

 2) and 3) may be infeasible due to HTTP's lack of capability negotiation feature like that of ESMTP,
 because the new web browser with native IDN URI support  can't decide whether the web server supports 
 native IDN or supports only ASCII(ACE) host in HOST: value   before trying that twice with both forms 
  of host: value (utf8 first, and then ACE if needed). Using ACE host: value is always  safe in 1) and 2).

 BTW, in 1) and 2), we cannot avoid legacy versioning problems because 
  most ACE conversion would be done by "ACE(NFKC(CaseFold(legacy-to-Unicode(native label))))".
  Most homepages in east asia are in legacy encodings and that monopoly (near 100%) won't change
   in the forseeable future.

 new legacy codes may be created after IDN-aware browsers are distributed.
 old legacy codes may get new code points for newly added characters.
 If IDN-aware browsers/applications are not upgraded with new legacy-to-Unicode mappings,
  they will occasionally fail to convert  legacy-encoded IDN into UNICODE one.
  That kind of IDN failure had  never seen in LDH DNS.  

Soobok Lee

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 24 Mar 2002 23:28:13 +0900

----- Original Message ----- 
From: "Soobok Lee" <[email-masked]>
To: "IETF idn working group" <[email-masked]>
Sent: Sunday, March 24, 2002 9:36 PM
Subject: Re: [idn] URL encoding in html page

>  2) and 3) may be infeasible due to HTTP's lack of capability negotiation feature like that of ESMTP,

  s/and 3)//    :-)     In 3), the webserver surely support native utf8 host: value.

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 25 Mar 2002 07:20:34 -0800

It is of course possible that some areas do not accept it, much as the
United States has not accepted the metric system (except for
scientific work, and the important realm of soft-drink bottles). It is
difficult to predict the speed of adoption of any technology, but I
suspect you will be surprised at the situation in 5 or 10 years.

Mark
—————

Γνῶθι σαυτόν — Θαλῆς

http://www.macchiato.com

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "Mark Davis" <[email-masked]>; "IETF idn working group"
<[email-masked]>
Sent: Friday, March 22, 2002 16:38
Subject: Re: OT - Re: [idn] URL encoding in html page

>   From my experience talking with customers in the field, the main
reason that people are not serving up UTF-8 pages is not the
> bandwidth, it is the fact that there are still some browsers out in
the field that do not yet handle it correctly. While they are
>   As far as size goes, it is worthwhile looking at some data
samples. The following are from a page on the Unicode site that is
>   So the best case is about 50% of the worst case. Some of this is
due to the encoding, and some is due to different languages just
> using different numbers of characters. However, when you look at web
pages in general use, the amount of text (in bytes) is really
> swamped by graphics, Javascript, HTML code, and so on. So
fundamentally, even the variations above are not that important in
>   [For transliteration, see
http://oss.software.ibm.com/cgi-bin/icu/tr]
>   > > [For transliteration, see
http://oss.software.ibm.com/cgi-bin/icu/tr]
>   > > > To: "Soobok Lee" <[email-masked]>; "IETF idn working
group"
>   > > <[email-masked]>
>   > > > > > What if all the html viewable text is in english, but,
only the
>   > > href url contains
>   > > > > > legacy (korean) encoded hostnames?  chinese visitors
would see
>   > > clean english homepage,
>   > > > > Well, that could happen, but a META tag would solve that
so
>   > > easily. Personally
>   > > > > I often use a simple text editor to deal with HTML, and
would find
>   > > it easier to
>   > > > Yes. Charset META tags help. But, many homepages  have
assumptions
>   > > on the main audience's
>   > > > default char encodings and very often omit the  META tag for
the
>   > > encoding like :
>   > > > Moreover, IDN url would be used in a pure FRAMESET document
that
>   > > defines frame URLs
>   > > > and contains no viewable texts. Such FRAMESET documents
often omit
>   > > charset META tags.
>   > > > legacy korean encoding (KS_C_5601-1987 or euc-kr). So do
most
>   > > japanese/chineses homepages.
>   > > > UTF8/UCS-2 encodings are rarely used in global WEB
publishing.
>   > > > IF we want to make IDN truly internationally interoperable,
all
>   > > IDN-aware webbrowsers/applications
>   > > > should contain libaries of all kinds of legacy-to-Unicode
conversion
>   > > routines. It will burden
>   > > > Moreover, legacy encodings are revised separately from
unicode. We
>   > > may face with as toughest
>   > > > versioning problems as we did in stringprep/nameprep
versioning
>   > > problems for newly added unicode points.

---

From: [email-masked] <[email-masked]>
Date: Tue, 26 Mar 2002 01:52:53 +0800

Dear Yves Arrouye:
     The following is the message from Mr. YangWoo Ko ([email-masked]),
but something is not the same in our IE environments.
> i am still worndering what "looks like a domain name" means.
     As I  know in Taiwan and China Area, the input Chinese character string
with non-protocol-head in Chinese Domain name form  typed into input  bar of
IE , the browser will intercept these string and convert it to a UTF-8 %nn
string as one of parameters to auto.search.msn.com , then it will be passed
to Realname's server. The server of Realname will convert the dotted IDN
domain name to an UTF-8 form and re-post it in the input bar with an error
reply to hold it in client.
     The server of Realname never to do dns lookup and not try to  re-direct
the client browser to do dns lookup in replied UTF-8 form.
     That is very strange , why MicroSoft/Realname do not support UTF-8 IDN
resolving  as she have proposed ?
In win2K system , the IE can not display all UTF-8 string in input bar of IE
, but it can passed it to resolver and resolve it correctly. Although we
know the cross visiting of different native language based web server will
caused some problems in relative addressed gif file  downloaded correctly
from visited web server, but it is work well in win2K.
     So, why RealName server suspend the redirection of client with re-post
UTF-8 IDN to stop win2K to do further resolving in client ?

Can you help me to understand it ?

L.M.Tseng

---

From: "A. Vine" <[email-masked]>
Date: Mon, 25 Mar 2002 17:11:44 -0800

FYI, browsers and many other apps already have massive conversion
libraries.

IF we want to make IDN truly internationally
interoperable, all IDN-aware webbrowsers/applications

Soobok Lee

---

From: "Soobok Lee" <[email-masked]>
Date: Tue, 26 Mar 2002 10:21:56 +0900

----- Original Message -----
  From: A. Vine
  To: Soobok Lee
  Cc: IETF idn working group
  Sent: Tuesday, March 26, 2002 10:11 AM
  Subject: Re: [idn] URL encoding in html page

    FYI, browsers and many other apps already have massive conversion libraries.

Yes. But, legacy encodings are often revised or there may be new born legacy encoding.
What and how much does that affect the stable operation of  IDN  in networks of various versions of
IDN-aware applications?
That is my questions  on which I still has no answer.
Soobok Lee
        IF we want to make IDN truly internationally interoperable, all IDN-aware webbrowsers/applications

should contain libaries of all kinds of legacy-to-Unicode conversion routines. It will burdentoo much memory load on handheld
devices like PDA.Moreover, legacy encodings are revised separately from unicode. We may face with as toughestversioning problems as
we did in stringprep/nameprep versioning problems for newly added unicode points.How to guarantee  stability and intergrity of IDN
operations in the all combinations of  numerous kinds and versions of iDN-awareapplications and legacy encodings?Soobok Lee

---

From: Martin Duerst <[email-masked]>
Date: Thu, 28 Mar 2002 16:30:25 +0900

>two different legacy encodings in his text without intervening META

Just to avoid further misunderstandings:

Any single HTML document is in a single encoding. Mixing encodings
is not possible. Using multiple <meta> tags to try to do this
won't work.

Regards,   Martin.

#-#-#  Martin J. Du"rst, I18N Activity Lead, World Wide Web Consortium
#-#-#  mailto:[email-masked]   http://www.w3.org/People/D%C3%BCrst

---

From: Martin Duerst <[email-masked]>
Date: Thu, 28 Mar 2002 15:29:42 +0900

At 10:21 02/03/26 +0900, Soobok Lee wrote:
>Answering just for the Web case:
>
>An old browser will not understand a new encoding. So it won't be
>able to display the page, nor it won't be able to resolve natively
>encoded domain names. To use a new encoding, people will have to
>upgrade to a new browser version.
>
>Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 28 Mar 2002 17:03:42 +0900

>Check this out: http://www.i-d-n.net/draft/draft-masinter-url-i18n-07.txt

please check out the latest version, at
http://www.ietf.org/internet-drafts/draft-masinter-url-i18n-08.txt

We are currently working on an update, please send in your
comments as soon as possible.

Also, please have a look at
http://www.ietf.org/internet-drafts/draft-ietf-idn-uri-01.txt
for the link between the work on IRIs and on IDN. Please note
that this draft is a WG draft, and of course any comments are
appreciated. Please note that the current version of the draft
doesn't say anything about ACE, and the main reason for this
is that I didn't want to put in too much as long as it was not
clear which ACE would be used.

Regards,    Martin.