Subject: [ogpx] type-system : binary elements in JSON serializations

From: Meadhbh Hamrick <[email-masked]>
Date: Sun, 28 Mar 2010 10:33:46 -0700

i'm not sure if this was the recommendation made at the f2f meeting
last tuesday, so let me try to define it.

the issue is that JSON does not provide a mechanism to differentiate
between a string that contains textual data and a string that contains
the base64 encoding of binary data. the current technique for avoiding
this ambiguity was to encode binary data as an array of unsigned 8 bit
numbers. many people do not like this technique.

the suggestion at the f2f was, if i remember it correctly, to simply
BASE64 encode the binary data and carry the encoding in a string.

this got me thinking...  section 2.1.9 does not include a generic
conversion from a string to a binary. maybe we should put this
conversion in the LLSD section instead of the JSON encoding section.
if someone is expecting an element to be a binary, it is likely to
attempt to access it as a binary anyway. by moving this conversion
into the LLSD section, it allows us to punt on tryign to id binaries
in the encoding and allows users of other serialization schemes to be
lazy about the conversion.

so i'm suggesting we make a change like this to section 2.1.9:

"2.1.9. Binary

   Data of type Binary contains a sequence of zero or more octets.  The
   default Binary is a sequence of zero octets.

   Conversions:

   String  The contents of the string are interpreted as a BASE64 encoding
      and converted following the rules in RFC XXXX with <insert alphabet here>
      and <insert statement about ignoring line breaks here>."

comments?

also, if we have a conversion from string to binary, must we have a
conversion the other way? it would be simple to define, just curious
if peeps think we MUST do it.

-cheers
-meadhbh
--
meadhbh hamrick * it's pronounced "maeve"
@OhMeadhbh * http://meadhbh.org/ * [email-masked]

---

From: "Hurliman, John" <[email-masked]>
Date: Mon, 29 Mar 2010 12:05:48 -0600

I think supporting conversion to and from string and binary at the LLSD level is a good idea, and I don't see any downsides to this approach.

John

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Meadhbh Hamrick
> Sent: Sunday, March 28, 2010 10:34 AM
> To: ogpx
> Subject: [ogpx] type-system : binary elements in JSON serializations
> 
> _______________________________________________
> ogpx mailing list (VWRAP working group)
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ogpx

---

From: David W Levine <[email-masked]>
Date: Mon, 29 Mar 2010 14:13:19 -0400

[email-masked] wrote on 03/29/2010 02:05:48 PM:

> [image removed] 
> 
> Re: [ogpx] type-system : binary elements in JSON serializations
> 
> Hurliman, John 
> 
> to:
> 
> Meadhbh Hamrick, ogpx
> 
> 03/29/2010 02:06 PM
> 
> Sent by:
> 
> [email-masked]
> 
> I think supporting conversion to and from string and binary at the 
> LLSD level is a good idea, and I don't see any downsides to this 
approach.
> John
+1

---

From: Meadhbh Hamrick <[email-masked]>
Date: Mon, 29 Mar 2010 11:24:29 -0700

coolies. i'll implement it and open a trac ticket for it.

also... do we want to have a string to binary conversion?
--
meadhbh hamrick * it's pronounced "maeve"
@OhMeadhbh * http://meadhbh.org/ * [email-masked]

On Mon, Mar 29, 2010 at 11:13 AM, David W Levine <[email-masked]> wrote:
>> LLSD level is a good idea, and I don't see any downsides to this approach.
>>
>> John
>>
> +1
>

---

From: "Hurliman, John" <[email-masked]>
Date: Mon, 29 Mar 2010 13:19:20 -0600

I think the conversion should go both ways assuming the string is base64. A string of "hello world" would produce zero binary octets, but "aGVsbG8gd29ybGQ=" would produce 11 binary octets.

John

> From: Meadhbh Hamrick [mailto:[email-masked]]
> Sent: Monday, March 29, 2010 11:24 AM
> To: David W Levine
> Cc: Hurliman, John; ogpx; [email-masked]
> Subject: Re: [ogpx] type-system : binary elements in JSON
> serializations
>

---

From: Meadhbh Hamrick <[email-masked]>
Date: Mon, 29 Mar 2010 13:13:17 -0700

okay. so we could say that a valid BASE64 encoding would be converted
into a string of octets, but an invalid BASE64 encoding would return
the default binary.

---

From: Joshua Bell <[email-masked]>
Date: Tue, 30 Mar 2010 10:22:54 -0700

in 4.2. (JSON Serialization) draft-hamrick-vwrap-type-system-00 calls out:

Binary  LLSD 'Binary' values are represented as a JSON 'JSONArray'.
       That is, they follow the ECMA-262 [ECMA262r5] 'JSONArray' non-
       terminal whose members are integer numbers representing each
       octet of the binary array.

This thread seems predicated on conversion between Binary and String as a
convenience for both JSON and XML serialization. However, the current JSON
serialization of Binary is an array of octets, and there are no conversions
defined for arrays.

I believe an unstated implication of this thread (and F2F discussion) is
that we should change the JSON serialization of Binary from octet array to
base64 string. Otherwise, after a memory->JSON->memory round-trip, the
base64->string conversion isn't very useful.

(My apologies if it was stated but I missed it.)

On Mon, Mar 29, 2010 at 1:13 PM, Meadhbh Hamrick <[email-masked]>wrote:

> > I think the conversion should go both ways assuming the string is base64.
> A string of "hello world" would produce zero binary octets, but
> "aGVsbG8gd29ybGQ=" would produce 11 binary octets.
> >
> > John
> >

---

From: "Hurliman, John" <[email-masked]>
Date: Tue, 30 Mar 2010 12:42:55 -0600

> Joshua Bell
> Sent: Tuesday, March 30, 2010 10:23 AM
> To: ogpx
> This thread seems predicated on conversion between Binary and String as
> a convenience for both JSON and XML serialization. However, the current
> JSON serialization of Binary is an array of octets, and there are no
> conversions defined for arrays.
> 
> I believe an unstated implication of this thread (and F2F discussion)
> is that we should change the JSON serialization of Binary from octet
> array to base64 string. Otherwise, after a memory->JSON->memory round-
> trip, the base64->string conversion isn't very useful.
> 

Yes, I think that was stated earlier in the thread but thank you for clarifying that point. It can get confusing when the conversations move between face to face meetings and e-mail threads.

To summarize, this change would alter sections "2.1.5 String", "2.1.9 Binary", and "4.2 JSON Serialization".

John