[clean-list] More realistic benchmark (I)
Philippos Apolinarius
phi500ac at yahoo.ca
Fri Oct 16 08:05:48 MEST 2009
Dear Dr. Simon.
I believe that I oversimplified the benchmark. However, since the suggestions improved my program a lot, I am sending you a more realistic version. Here are my comments on the link you have sent me:
1 --- The functional approach does not work for the real stuff, because I use random processes to access the array. I am sending you a more realistic program in the next email.
2 --- Although my previous example works with unboxed arrays, what I am really storing are Trees, that represent electronic circuits.
3 --- I would rather not use unsafeRead and unsafeWrite. I don't use them in Clean, and I think I should not use them in Haskell.
Here is the first program, where I replaced Double by a Tree (Windows). I will explain why I did this in a separate email, to prevent confusion. Anyway, now Haskell is 23 times slower than Clean, as you can check from my numbers, or repeating the benchmark. BTW, I tried to use unsafeRead, and unsafeWrite, but I got a segmentation fault when I tried to run the program.
/* Clean version
C:\Clean 2.2\geneticprog\bench>population.exe
280000000
Execution: 0.07 Garbage collection: 0.18 Total: 0.26
C:\Clean 2.2\geneticprog\bench>population.exe
280000000
Execution: 0.07 Garbage collection: 0.17 Total: 0.25
*/
module population
import StdEnv
:: Op = AND | OR | NOT;
:: Tree= L Real | T Op [Tree];
fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
# (L n)= b
# a= {a&[i]= L (n+3.0)}
# (c, a)= a![i]
# (L x)= c
= fn (i-1) a (x+acc)
Start= fn 2000000 vt 0.0
where
vt:: .{Tree}
vt = createArray 2000001 (L 137.0)
{-# Haskell version #-}
{-# OPTIONS -fvia-C -optc-O3 -fexcess-precision -optc-msse3 #-}
import Control.Monad.ST
import Data.Array.ST
import Data.Array.Base
data Op = AND | OR | NOT;
data Tree= L Double | T Op [Tree]
main = print $ runST
(do arr <- newArray (1,2000000)
(L 137.0) :: ST s
(STArray s
Int Tree)
go arr 2000000 0.0 )
go :: STArray s Int Tree -> Int -> Double -> ST s Double
go !a i !acc
| i < 1 = return acc
| otherwise=do
b <- readArray a i
writeArray a i (setDouble ((getDouble b)+3.0))
c <- readArray a i
go a (i-1) (acc+ (getDouble c))
getDouble (L r)= r
getDouble _ = 0.0
setDouble r= L r
{-# D:\Programs\ghc-programs\bench>ghc -O2 --make \
bingo.hs -fvia-C -optc-O3 -optc-msse3 -o bingo.exe
D:\ghc-programs\bench>bingo.exe +RTS -sstderr -K32000000
bingo.exe +RTS -sstderr -K32000000
2.8e8
72,392,664 bytes allocated in the heap
118,966,296 bytes copied during GC
37,490,648 bytes maximum residency (4 sample(s))
416,176 bytes maximum slop
66 MB total memory in use (1 MB lost due to fragmentation)
Generation 0: 119 collections, 0 parallel, 1.42s, 1.42s elapsed
Generation 1: 4 collections, 0 parallel, 0.14s, 0.16s elapsed
INIT time 0.03s ( 0.00s elapsed)
MUT time 0.06s ( 0.06s elapsed)
GC time 1.56s ( 1.58s elapsed)
EXIT time 0.00s ( 0.02s elapsed)
Total time 1.66s ( 1.64s elapsed)
%GC time 94.3% (96.2% elapsed)
Alloc rate 772,188,416 bytes per MUT second
Productivity 3.8% of total user, 3.8% of total elapsed
#-}
--- On Thu, 10/15/09, Simon Peyton-Jones <simonpj at microsoft.com> wrote:
From: Simon Peyton-Jones <simonpj at microsoft.com>
Subject: RE: [Fwd: Re: [clean-list] Clean versus Haskell]
To: "Adrian Hey" <ahey at iee.org>, "clean-list at science.ru.nl" <clean-list at science.ru.nl>
Received: Thursday, October 15, 2009, 1:22 AM
I submitted Philippos as a performance bug to GHC's trac, here:
http://hackage.haskell.org/trac/ghc/ticket/3586
If you follow the link you'll see lots of commentary, including a version that generates code twice as fast as Clean's, and is purely functional.
That said, I think it's v bad that a straightforward program runs so slowly, and it's certainly true that this is an area we could pay more attention to. (Trouble is, there are so many such areas!)
Meanwhile, I'm curious: are the arrays in Philippos's program strict? Or lazy? If strict, that's a pretty big difference. (The "STU" arrays mentioned in the above link are strict and unboxed; that's what the "U" means.)
Simon
| -----Original Message-----
| From: clean-list-bounces at science.ru.nl [mailto:clean-list-bounces at science.ru.nl] On
| Behalf Of Adrian Hey
| Sent: 15 October 2009 07:33
| To: clean-list at science.ru.nl
| Subject: [Fwd: Re: [clean-list] Clean versus Haskell]
|
|
| -------- Original Message --------
| Subject: Re: [clean-list] Clean versus Haskell
| Date: Thu, 15 Oct 2009 06:41:20 +0100
| From: Adrian Hey <ahey at iee.org>
| To: Philippos Apolinarius <phi500ac at yahoo.ca>
| References: <8400.82204.qm at web58803.mail.re1.yahoo.com>
|
| Hello Philippos,
|
| GHC has a long standing performance bug for garbage collection and
| mutable arrays:
|
| http://hackage.haskell.org/trac/ghc/ticket/650
|
| For some reason they haven't (can't?) fixed it. I guess the
| authors of Haskell/ghc shootout entries are aware of this so don't
| use native mutable arrays in their entries (at least not those that
| show haskell/ghc to be "fast" :-)
|
| Regards
| --
| Adrian Hey
|
| Philippos Apolinarius wrote:
| > I wrote a very simple program to check whether Haskell improved its array
| processing libraries or not. Here is how to compile and run the program arr.hs in
| Haskell (I have used the GHC compiler):
| >
| >> ghc -O arr.hs -o arr.exe
| >
| > $ time arr.exe +RTS -K32000000
| > 2.8e8
| >
| > real 0m3.938s
| > user 0m0.031s
| > sys 0m0.000s
| >
| > The same program in Clean:
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01 Garbage collection: 0.01 Total: 0.03
| >
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01 Garbage collection: 0.01 Total: 0.03
| >
| > This means that Clean is 390 times faster than Haskell in this particular problem.
| These results makes me worder whether Haskell is safer than Clean. It turns out that
| Haskell checks index out of range at runtime, exactly like Clean. Larger problems
| make the difference between Clean and Haskell even worse. For instance, neural
| networks like the one described in Schmidtt et al. run 400 times faster in Clean.
| >
| > Haskell seems to be slow, and not safe. For instance, GHC compiler does not at a
| program trying to write into a closed handle.
| >
| > module Main where
| > import System( getArgs )
| > import IO
| >
| > main = do
| > args <- getArgs
| > if (length args /= 2)
| > then putStr "Usage: f1a f2a <n>"
| > else (do
| > fromHandle <- openFile (head args) ReadMode
| > contents <- hGetContents fromHandle
| > toHandle <- openFile (head (tail args)) WriteMode
| > hClose toHandle -- Comment this line
| > hPutStr toHandle contents
| > hClose toHandle
| > putStr "Done.")
| >
| > The Clean equivalent program is somewhat smaller. In my opinion it is easier to
| understand. What is more important, Clean compiler balks at closed handles.
| >
| > module cleancopy
| > import StdEnv, ArgEnv
| >
| > Start w
| > # argv= getCommandLine
| > | size argv < 3 = abort "Usage, etc."
| > # (ok, f, w)= fopen argv.[1] FReadText w
| > (contents, f)= freads f 64000
| > (ok, f, w)= fopen argv.[2] FWriteText w
| > f= fwrites contents f
| > = fclose f w
| >
| > Below you will find the array examples. You can check that Clean is really much
| faster than Haskell. I wonder why the Benchmarks Game site does not report such a
| large difference between Haskell and Clean performances. I believe that people who
| wrote Haskell benchmarks for the Benchmarks Game site cheated in using foreign
| pointers to access arrays.
| >
| > -- arr.hs
| > import Control.Monad.ST
| > import Data.Array.ST
| > main = print $ runST
| > (do arr <- newArray (1,2000000)
| > 137.0 :: ST s
| > (STArray s
| > Int Double)
| > a <- readArray arr 1
| > b <- readArray arr 1
| > fn 2000000 arr 0.0 )
| >
| >
| > fn i a acc | i < 1 = do (return acc)
| > fn i a acc= do
| > b <- readArray a i
| > writeArray a i (b+3.0)
| > c <- readArray a i
| > fn (i-1) a (acc+c)
| >
| > //Clean version
| > module arraytest
| > import StdEnv
| > fn i a acc | i<1 = acc
| > fn i a=:{[i]=b} acc
| > # a= {a&[i]= b+3.0}
| > # (c, a)= a![i]
| > = fn (i-1) a (c+acc)
| >
| > Start= fn 2000000 vt 0.0
| > where
| > vt:: .{#Real}
| > vt = createArray 2000001 137.0
_______________________________________________
clean-list mailing list
clean-list at science.ru.nl
http://mailman.science.ru.nl/mailman/listinfo/clean-list
__________________________________________________________________
Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now
http://ca.toolbar.yahoo.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.science.ru.nl/pipermail/clean-list/attachments/20091015/a1acccc5/attachment.html
More information about the clean-list
mailing list