Subject: [Ietf-languages] Northern Thai Variants

From: Richard Wordingham <[email-masked]>
Date: Thu, 10 Jan 2019 09:08:59 +0000

I have been looking at localisation for northern Thai (language code
nod) and it seems to me that, simplifying, for any speech variety there
are three basic varieties of writing system:

1) Writing in the Tai Tham script (subtag Lana)
2) Writing in the Thai script (subtag Thai) with consonants having
the same phonetic values as the corresponding Tai Tham consonants.
3) Writing in the Thai script with stop consonants having
the same phonetic values as in Standard Thai.

My idea for approaching this is to use the labels:

nod-Lana
nod-Thai-etymo
nod-Thai-phonetic

However, there are several issues with this approach.

The first is whether nod-etymo and and nod-phonetic would then be
allowed as identifiers of Northern Thai Thai-script writing systems.
nod-Lana-etymo would not make sense, and I'm not sure how to word the
subtag record to exclude, or at least discourage, it.

The second is the possibility that Scheme 2, but not Scheme 3, should
be treated as a transform of Scheme 1, and therefore scheme 2 should
be nod-Lana-t-Thai, which would remove the need for variants at this
level.  My most cogent objection is that the transformation depends on
phonetic and etymological information - ambiguities in some
orthographies for Scheme 1 are removed in Scheme 2, and word-final
<SIGN OA BELOW, RA> is converted according to etymology, (generally)
with <O ANG, RO RUEA> in words of Pali/Sanskrit origin and <O ANG, NO
NU> in native words.

Users of Northern Thai learn Standard Thai in school, and will
generally be more fluent in writing Standard Thai than in writing
Northen Thai.

Can I please have advice.

Richard.

---

From: Mark Davis ☕️ <[email-masked]>
Date: Thu, 10 Jan 2019 12:07:27 +0100

A few mechanics:
1. nod-Lana-etymo : The prefix values are only a recommendation, so there
is nothing preventing any variant with any prefix. You can have
fr-basiceng for example. In practice that is no problem, just as ja-Thai-AQ
is not a problem.
2. nod-Lana-t-Thai : would be nod-Thai-t-nod-Lana (or nod-Thai-t-und-Lana),
meaning nod in the Thai script that has been transformed from Lana.

Mark

On Thu, Jan 10, 2019 at 10:09 AM Richard Wordingham <richard.wordingham=
[email-masked]> wrote:

> _______________________________________________
> Ietf-languages mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-languages
>

---

From: John Cowan <[email-masked]>
Date: Thu, 10 Jan 2019 10:57:18 -0500

On Thu, Jan 10, 2019 at 4:09 AM Richard Wordingham <richard.wordingham=
[email-masked]> wrote:

> My idea for approaching this is to use the labels:

The first is of course fine and needs no registration.

I am firmly opposed to the second and third: the names are far too generic,
and people would use them for all sorts of things in all sorts of random
ways.  We would begin to see "en-US-phonetic" for foe-NET-ik
ree-SPELL-ingz, something which would have absolutely nothing to do with
Northern Thai phonetic use of Thai script.  This is a principle we have
fairly consistently stuck with: variant tags are meant to make sense only
in specific contexts, with a very few exceptions like 'fonipa', since the
IPA is inherently generic.

If there are no names for these conventions in Northern Thai or Central
Thai, try to come up with some.  Perhaps the names of the people who
established or promoted these orthographies?  That's what we've done with
sl-bohoric and sl-dajnko as well as be-tarask.  Dates can also be helpful,
as in ru-luna1918, the Russian orthography published by the committee
headed by A. A. Lunacharsky in 1918.

> subtag record to exclude, or at least discourage, it.

You can't: as Mark Davis points out, Preferred-Value for a variant subtag
is only a suggestion.   But as he also says, that doesn't matter much.
What's important is to keep Northern Thai in context somehow.

> level.

I think that should only be applied to a document whose original was in
Lana script and has been actually transliterated (by hand or machine, it
doesn't matter) to another script.  It would be suitable for a
Hebrew-letter version of the Syriac Bible (something I have seen done), but
not for documents originally written in nod-Thai.

> generally be more fluent in writing Standard Thai than in writing

A more broadly scoped question: given the difficulties in reading Northern
Thai, even in Thai script, at all, is it really necessary to mark this
distinction clearly?  Are there spelling checkers, for instance, or is the
barrier between the two variants high enough to make life difficult for
readers of one to handle the other?  Is there any significant
standardization (even de facto, like standard English) around either
variant?

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 10 Jan 2019 22:25:54 +0000

On Thu, 10 Jan 2019 10:57:18 -0500
John Cowan <[email-masked]> wrote:

> On Thu, Jan 10, 2019 at 4:09 AM Richard Wordingham
> <richard.wordingham= [email-masked]> wrote:  
> 
> I am firmly opposed to the second and third: the names are far too
> generic, and people would use them for all sorts of things in all
> sorts of random ways.  We would begin to see "en-US-phonetic" for
> foe-NET-ik ree-SPELL-ingz, something which would have absolutely
> nothing to do with Northern Thai phonetic use of Thai script.  This
> is a principle we have fairly consistently stuck with: variant tags
> are meant to make sense only in specific contexts, with a very few
> exceptions like 'fonipa', since the IPA is inherently generic.

Would a Thai term like _thapsap_ (ทับสัพท์ 'transliterate, transcribe')
be open to the same objection?  It's the nearest I've seen to a name,
but I fear it's totally ambiguous between the two.

> If there are no names for these conventions in Northern Thai or
> Central Thai, try to come up with some.  Perhaps the names of the
> people who established or promoted these orthographies?  That's what
> we've done with sl-bohoric and sl-dajnko as well as be-tarask.  Dates
> can also be helpful, as in ru-luna1918, the Russian orthography
> published by the committee headed by A. A. Lunacharsky in 1918.

Orthography?!?  I was deliberately vague when I said 'varieties of
writing system'. I shall ask around, but I'm not confident I will get
useful answers.

> > The second is the possibility that Scheme 2, but not Scheme 3,
> > should be treated as a transform of Scheme 1, and therefore scheme
> > 2 should be nod-Lana-t-Thai, which would remove the need for
> > variants at this level.  

> I think that should only be applied to a document whose original was
> in Lana script and has been actually transliterated (by hand or
> machine, it doesn't matter) to another script.  It would be suitable
> for a Hebrew-letter version of the Syriac Bible (something I have
> seen done), but not for documents originally written in nod-Thai.

Well, there is at least some material, albeit probably mostly
translated, in 'nod-Thai-etymo' in the Wikipedia Northern Thai
incubator.

Neither nod-Thai-t-nod-Lana nor nod-Thai-t-und-Lana gives any clue as
to the conversion (or correspondence) method.  Looking at the Russian
to Latin script example in rfc6469, I don't see how one would
distinguish the results of transforms for English and French.  I will
ditch the idea of naming the writing systems as transformations.

> A more broadly scoped question: given the difficulties in reading
> Northern Thai, even in Thai script, at all, is it really necessary to
> mark this distinction clearly?  Are there spelling checkers, for
> instance, or is the barrier between the two variants high enough to
> make life difficult for readers of one to handle the other?  Is there
> any significant standardization (even de facto, like standard
> English) around either variant?

Surprisingly enough, printed Northern Thai seems easier to read in the
Lanna script than in the Thai script.  The Thai script does not mark
syllable boundaries as well as the Lanna script does.

Spelling checkers face one major hurdle - guessing word boundaries.  If
the user is prepared to mark word boundaries, then it is fairly simple
to use one on Firefox, and it is possible to use one on LibreOffice -
it will probably be much easier when I get round to using a SIL
packaging tool.  I've been using one for the Lanna script based on a
single dictionary.  Unfortunately, I haven't been able to start talking
about licensing the word list, so the dictionary's copyright stops me
sharing the spell-checker, or more precisely the Hunspell database. I'm
hoping there's no copyright in the vocabulary of a translation of the
New Testament.

For the Lanna script, a major use of spell-checkers is to ensure that
characters are in the right order - otherwise searching text will be
very difficult.  If the appearance of the text was the primary
concern, I would need different spell-checkers for different spelling
schemes.

For the Thai script, spell-checkers are less important - unless they
can be used to fight Thai line-breaking, which is a significant
annoyance for users of minority languages.  Unfortunately, victory
could be marred by searchers not ignoring WJ.

For spell-checking, the fewer unused words a spell-checker accepts, the
better.  Therefore, it will help to segregate spellings between the two
systems.  At least 12% of the vocabulary is different because words
starting with the voiced unaspirated Indic letters in the Lanna script
are different between to the two Thai script systems.  One quickly sees
which system one is reading; I suspect more rapidly than one detects
the difference between 'nod-Thai-etymo' and the standard Thai language.

One difference between the two Thai script spellings is the degree of
Sanskritisation of the spellings.  The 'nod-Thai-phonetic' is more
heavily Sanskritised, with 'dharma' being ธรรม (the usual Thai form) as
opposed to the Pali-derived ธัมม์, which is no longer allowed for Thai
by the Royal Institute dictionary, though it retains the longer form
ธัมมะ).

One problem with having a user interface in a mixed nod-Thai is that
the user would have to keep switching reading convention, which would be
likely to be confusing.  

Thank you for your comments.

Richard.

---

From: John Cowan <[email-masked]>
Date: Thu, 10 Jan 2019 17:44:58 -0500

On Thu, Jan 10, 2019 at 5:26 PM Richard Wordingham <richard.wordingham=
[email-masked]> wrote:

> by the Royal Institute dictionary, though it retains the longer form

Perhaps use "dharma" and "dhamma" as the tags?  Weird, but probably unique.