Trying to get a NN code going

[John van Groningen]

Richard A. O'Keefe wrote:
>As an example for a 1 week C course I taught recently ("for", not "in";
>I only thought of it just afterwards, but _next_ time...) I took a
>simple Neural Network program written by someone who is an expert in
>the field and speeded it up by a factor of 2.5, basically by writing
>the kind of clean C any APL fan would produce (:-).
>
>It occurred to me:  why not rewrite it in Clean?  It ought to go rather
>nicely.  The improved C version gets its performance by
> - massive (automatic) inlining (C compilers have got quite smart)
> - *amazing* loop unrolling and the resulting
> - scheduling to take advantage of superscalarity
>Now, a Clean compiler ought to be able to beat the pants off any C
>compiler at inlining, but it is not at all clear to me that it's likely
>to do the kind of loop unrolling, or even scheduling, that is needed.

The Clean compiler does not automatically inline functions, except for some=
 lambda expressions that are uncurried by the compiler. To inline a=
 function, rewrite it as a macro. Macros can not be recursive, but a=
 recursive function can still be written as a macro by using a recursive=
 local function in the macro, see for example foldl and foldr.
The compiler does not unroll loops and does not have an instruction schedule=
r.

>My strategy was
> 1. Develop list based translation with the test data in the source code.
> 2. Revise it to read the data from a file at run time.
> 3. Revise it again to use {#Real} instead of [Real] for vectors.
>
>However, I'm stuck at step 1.
>On a SPARCstation 5 running Solaris 2.6.1, Clean 1.3 is happy with the
>program, even the test data (which is about 1400 lines containing
>9320 numbers).  I compiled it with -h 80M -s 4M.  The result ran for
>1 second of user time, 12 seconds of system time, and died with a
>segmentation violation.
>
>What do UNIX Clean programmers do when they get a segmentation
>violation?  How do I track this down to its cause?

When a Clean program dies with a segmentation violation there are 2 possible=
 causes: a stack overflow or an incorrecly used array index.

Array indices can be checked at runtime by enabling the Code Generator=
 option Check Indices in the CleanIDE, or by using -ci with clm. The=
 CleanIDE will automatically recompile all modules when this option is=
 enabled, clm will not.

On the Mac stack overflows can be detected with the Check Stacks option. On=
 other platforms I usually run with a small heap and -st, which prints the=
 stack sizes before collecting garbage.

>I hoped that the Macintosh version might be a bit better, but when
>I try to _compile_ the program on a PowerMac 7600/120 running MacOS 8.1,
>an Errors window pops up, containing
>
>    Fatal Error in comsupport:TH_Alloc "Insufficient Memory"
>
>Trouble is, I can't tell whether this means the Clean _IDE_ needs to
>be resized or the Clean _compiler_, nor by how much.  I note that the
>compiler is still running (it's in the switcher menu) and have no idea
>what I'm supposed to do about that.

The CleanCompiler needs to be resized with the Finder or ResEdit (SIZE=
 resource). You can quit the CleanCompiler by pressing any key, or by=
 quiting the CleanIDE.

>Replacing the timing test data with toy (XOR) data,
>the program gets a 'rule X in module Y does not match' error.
>This applies to both the SPARCstatation and the PowerMac.
>I know what the immediate cause is (calling an elementary matrix
>update function with arguments of the wrong sizes), but I don't
>know _where_ it is happening in the program or what the sizes
>actually are.

You can get a stack trace of the top of the stack on the PowerMac or Windows=
 by enabling the time profiler and running the program again.
This stack trace can be written to the file 'Messages' by enabling 'Write=
 stderr to file' in the Application options dialogue.

>This is the first Clean program I've had where the compiler hasn't told
>me about all the bugs, and I'm wondering how other Clean programmers go
>about debugging their code.  In SML, I'd just whack in a few print
>commands and be away laughing.

You can insert print statements using the functions in StdDebug. For example=
 by adding a guard with trace_t

f arg1 arg2
        | trace_t "f called\n"
        =3D arg1+arg2           =20

or by adding a new first alternative:

f arg1 arg2
        | not (trace_t "f called\n")
                =3D f arg1 arg2
f arg1 arg2
        =3D arg1+arg2

to prevent "Function may fail" warnings.

John van Groningen