Subject: [ldapext] OS-dependent LDIF 'attr:< file://filename' handling

From: Hallvard B Furuseth <[email-masked]>
Date: Sun, 03 Jun 2007 20:52:32 +0200

How do LDAP implementations handle "attr:< file:///foo/bar" on non-Unix
filesystems?

If the file is opened in binary mode, the value may get zero-padded at
the end (to fill a complete block in the filesystem).

But if it is opened in text mode, end-of-line markers in the file will
be replaced with LF, a byte which is handled like an end-of-file marker
(like ^Z on windows) will truncate the file, and on some types of
filesystems spaces at the end of lines may be removed.

I suppose servers could ignore trailing zero-bytes in attribute values
if that would fix a syntax error when parsing the value...

Or we could define 'attr:<< URL' or something to mean the URL provides
base64-encoded data.

-- 
Regards,
Hallvard

_______________________________________________
Ldapext mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ldapext

---

From: "Michael Ströder" <[email-masked]>
Date: Sun, 3 Jun 2007 23:55:55 +0200

On 8:52:32 pm 2007-06-03 Hallvard B Furuseth <[email-masked]>
wrote:
> How do LDAP implementations handle "attr:< file:///foo/bar" on
> non-Unix filesystems?
>

I don't understand the problem. The OS takes care of it. Could you please
give an example of an OS which returns zero-padded data to an application
when opening a file in binary mode?

> But if it is opened in text mode,

I'd never do this.

Ciao, Michael.

---

From: Kurt Zeilenga <[email-masked]>
Date: Sun, 3 Jun 2007 20:30:06 -0700

> How do LDAP implementations handle "attr:< file:///foo/bar" on non- 

RFC 2849 says:
     The contents of the referenced file are to be included verbatim
     in the interpreted output of the LDIF file.

 > If the file is opened in binary mode

If by "binary mode", you mean some mode in causes the content to be
processed without translation, then that would seem like the right thing
for an LDIF parser to do.

-- Kurt

---

From: Hallvard B Furuseth <[email-masked]>
Date: Mon, 4 Jun 2007 13:22:07 +0200

Michael Ströder writes:
>> If the file is opened in binary mode, the value may get zero-padded
>> at the end (to fill a complete block in the filesystem).
>
> I don't understand the problem.  The OS takes care of it.

If it can.  There are or have been filesystems where file sizes are
not stored in number of bytes but number of blocks.  That's why ^Z was
needed in text files on DOS or whatever - to get better granularity
than blocks it needed a character to work like an EOF mark.

(There are other kinds of filesystems or file types - e.g. files with
fixed-length records instead of streams of bytes.  When used for text
files, they might use one record per line and space-pad it, hence
the "space may be removed at the end of lines in text files".)

> Could you please give an example of an OS which returns zero-padded
> data to an application when opening a file in binary mode?

Me?  No.  Well, DOS if I remember that example correctly.  But I'm a
Unix person, which is why I asked about non-Unix.

>> But if it is opened in text mode,

That's a relief:-)

-- 
Regards,
Hallvard

---

From: Howard Chu <[email-masked]>
Date: Mon, 04 Jun 2007 04:56:08 -0700

>>> But if it is opened in text mode,
Well, there's two possibilities here - on those oddball systems, either 
a solution exists, or it doesn't. Any system that can't provide binary 
byte-stream file semantics is going to have trouble with all kinds of 
URLs in all kinds of software, not just LDAP.

As for DOS - that EOF marker isn't actually needed, since the directory 
structure records file sizes in bytes. Just a holdover from DOS 1.x or 
somesuch (CP/M perhaps?).

re: record-oriented files - mainframes have a great variety of these. It 
seems the file: URL syntax doesn't accommodate these details, so anyone 
trying to use those files as anything other than a byte-stream is SOL. A 
bit of a shame, but then again, they have other utilities available for 
repacking a structured file into a byte-stream if necessary. Having 
implemented structured file support in FTP for our mainframes a couple 
decades ago, and seeing how few other systems cared, I'd say it's no 
great loss.

re: fixed-length records for text - that would be pretty unusual since 
OS's with record-oriented files always provide both variable and fixed 
options. Anybody who deliberately chose to use a fixed-length format for 
text already knows there will be pad bytes, and probably expects them to 
be propagated. Really it doesn't seem that any of this is worth worrying 
about.
-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

---

From: Hallvard B Furuseth <[email-masked]>
Date: Mon, 4 Jun 2007 16:04:59 +0200

Howard Chu writes:
> Well, there's two possibilities here - on those oddball systems,
> either a solution exists, or it doesn't. Any system that can't provide
> binary byte-stream file semantics is going to have trouble with all
> kinds of URLs in all kinds of software, not just LDAP.

Well, they'd need to maintain such info about their URLs in separate
files instead of having the filesystem take care of it.  And a local
fix could be to implement some other URL scheme than file:// - one
which knows of those separate files.

> somesuch (CP/M perhaps?).

I guss I'll survive without porting full LDIF to CP/M:-)

> re: record-oriented files - mainframes have a great variety of
> these. (...) Having implemented structured file support in FTP for our
> mainframes a couple decades ago, and seeing how few other systems
> cared, I'd say it's no great loss.

That's good.  For me this is all mostly a theoretical concern.  Living
almost purely in a Unix word I didn't know how much it was a practical
concern somewhere as well.

-- 
Regards,
Hallvard