[clean-list] Reading columnar data

Siegfried Gonzi siegfried.gonzi@kfunigraz.ac.at
Wed, 05 Sep 2001 12:04:43 +0200


Enclosed a program which let you handle files of the following scheme:

Wir Oesterreicher sind die besten auf der ganzen Welt.
Unsere Kulturleistungen sind der Musikantenstadel
und der Villacher-Fasching
Value1    Value2 Value 3 ....
1.23        23.45     123.34
34.56    23.45        23.445
...

The program first counts the number of lines. You only have to specify the name of the
file, the number of columns (in the above example 3) and the lines of the header (in the
above example 4). The program then reads in the columnar double-values into an array
which becomes created.

Another notice. The line

(line,fileNextLine) = sfreadline file

in ReadArray is essential, because otherwise ReadArray would fill up the array always
only with first line of columns (I didn't find a better way to move to the next line).
The overhead is negligible.

I think the execution time is comparable to C.

S. Gonzi
module ReadColumns
import StdEnv, StdFile


ReadArray:: {*{#Real}} !File -> {*{#Real}}
ReadArray marray file = ReadZeilen 0 marray file
where
     ReadZeilen:: !Int {*{#Real}} !File -> {*{#Real}}
     ReadZeilen i marray file
           | i == (size marray) = marray
           = ReadZeilen (i+1) (ReadSpalten 0 marray file) fileNextLine
           where
                (line,fileNextLine) = sfreadline file
                ReadSpalten:: !Int {*{#Real}} !File -> {*{#Real}}
                ReadSpalten j marray file
                 | j ==  (size marray.[0]) = marray
                 #! (b,double,file) = sfreadr file
                 = ReadSpalten (j+1) { marray & [i,j] = double } file


TakeOne:: {*{#Real}} -> !Real
TakeOne marray = marray.[0,2]

CountLines:: !File -> !Int
CountLines file = ReadLines 0 file
where
      ReadLines:: !Int !File -> !Int
      ReadLines nLines file
            | sfend file = nLines
            #! (line,filerest) = sfreadline file
            = ReadLines (nLines + 1) filerest

ReadNLines:: !Int !File -> !File
ReadNLines n file
         | n == 0 = file
         #! (line, file) = sfreadline file
         = ReadNLines (n - 1) file

LinesArray:: !Int !Int !Int -> !Int
LinesArray ni NHeader nLines
         | (ni - NHeader) <= 0 = abort("Negative array index")
         = (ni - NHeader)

MakeArray:: !Int !Int -> {*{#Real}}
MakeArray ni nj
             | ni <=0 || nj <=0 = abort("Negative array index")
             =  {{0.0\\x <- [0..(nj-1)]}\\y <- [0..(ni-1)]}

Extract:: !String  !*Files -> ({*{#Real}},!*Files)
//Extract:: !String  !*Files -> (!Real,!*Files)
Extract inputfile file
         #! (readok, infile,file) = sfopen inputfile FReadText file
         | not readok = abort ("Cannot read")
         #! nLines = (CountLines infile)
         #! Ni = LinesArray (nLines-1) NHeader nLines
         #! marray = MakeArray Ni Nj
         #! infile = ReadNLines NHeader infile
         = ((ReadArray marray infile),file)


 // File name
inputfile :== "aer.txt"
// Number of header lines
NHeader:==3
// Number of columns
Nj:==5

Start:: *World  -> ({*{#Real}},*World)
//Start:: *World  -> (!Real,*World)
Start world  = accFiles (Extract inputfile ) world