[clean-list] UNIX (or POSIX) I/O

[Maks Verver]

Hi Donn and others,

I experienced some of the same problems with Clean under Windows, especially
the memory leaks in the StdFile module which fill up the heap in no time.
The problems with 'fend' are annoying too; you probably just shouldn't use
stdio for anything but user input (on the console); input redirection
generates random garbage after the end of file marker as you probably
noticed already. Personally I don't understand what's so hard about
implementing 'fend' for stdio too.

Regarding the implementation of POSIX-compliant low-level I/O operations: it
shouldn't be that hard to wrap them, but you'd probably end up with similar
functions as already provided by the StdFile module (except that your
functions would operate on an even lower level). Personally I'd prefer if
someone would fix the StdFile module instead. If it would work correctly, I
wouldn't want to use the low-level I/O operations instead (just as I don't
use them from C, except for special/non-file descriptors).

I wonder how other people process data from external files beyond the size
of few kilobytes? Do you simply increase the heap memory up to a few
gigabytes and rely on the operating system to swap away the leaked heap
memory? Or is there a way to work around the StdFile memory leaks?

Kind regards,
Maks Verver.


-----Original Message-----
From: clean-list-admin@cs.kun.nl [mailto:clean-list-admin@cs.kun.nl] On
Behalf Of Donn Cave
Sent: donderdag 20 mei 2004 2:27
To: clean-list@cs.kun.nl
Subject: [clean-list] UNIX (or POSIX) I/O

I downloaded Clean yesterday and wrote my first program today.
It doesn't work, and from what I read in the archives of this
list, perhaps it can't.  I'm on MacOS X, but I assume it would
be the same for any other UNIX platform.

I wanted to try a program I'd previously written in Haskell,
that got into serious trouble with 300 Mb text input.  But to
start with I simply read in a line, from stdin.

When input is empty, I get "Heap full".  If I add a check for
end of file, I get an IO error stating that "fend" is not allowed
for stdio and stderr.  I understand from the archives that this
may be by design, in a manner of speaking.  So I or anyone who
would want to write a normal UNIX application will need some
alternative to StdFile.

I don't see any POSIX functions at all, so my question(s) -

- How hard would it be to "wrap" these functions?  open, read
   write, close, lseek, pipe, dup, dup2, ...   Is there a reasonably
   documented foreign function interface, including whatever is
   needed for I/O functions?

- Then I would want to add the buffered I/O functionality in
   a Clean module, which I'm going to assume would be feasible.
   So that's not a question, unless anyone feels differently.

Thanks,

	Donn Cave, donn@u.washington.edu



_______________________________________________
clean-list mailing list
clean-list@cs.kun.nl
http://www.cs.kun.nl/mailman/listinfo/clean-list