1118856Sphk$FreeBSD$ 2118856Sphk 3118856SphkA small tool to do the statistics legwork on benchmarks etc. 4118856Sphk 5118856SphkPrepare your data into two files, one number per line 6118856Sphkrun 7118856Sphk ./ministat data_before data_after 8118856Sphk 9118856Sphkand see what it says. 10118856Sphk 11118856SphkYou need at least three data points in each data set, but the more 12118856Sphkyou have the better your result generally gets. 13118856Sphk 14118856SphkHere are two typical outputs: 15118856Sphk 16118856Sphkx _1 17118856Sphk+ _2 18118856Sphk+--------------------------------------------------------------------------+ 19118856Sphk|x + x+ x x x + ++ | 20118856Sphk| |_________|______AM_______________|__A___________M_______________|| 21118856Sphk+--------------------------------------------------------------------------+ 22118856Sphk N Min Max Median Avg Stddev 23118856Sphkx 5 36060 36138 36107 36105.6 31.165686 24118856Sphk+ 5 36084 36187 36163 36142.6 49.952978 25118856SphkNo difference proven at 95.0% confidence 26118856Sphk 27118856SphkHere nothing can be concluded from the numbers. It _may_ be possible to 28118856Sphkprove something if many more measurements are made, but with only five 29118856Sphkmeasurements, nothing is proven. 30118856Sphk 31118856Sphk 32118856Sphkx _1 33118856Sphk+ _2 34118856Sphk+--------------------------------------------------------------------------+ 35118856Sphk| + | 36118856Sphk| x + +| 37118856Sphk|x x x x + +| 38118856Sphk| |_______________A_____M_________| |_M___A____| | 39118856Sphk+--------------------------------------------------------------------------+ 40118856Sphk N Min Max Median Avg Stddev 41118856Sphkx 5 0.133 0.137 0.136 0.1354 0.0015165751 42118856Sphk+ 5 0.139 0.14 0.139 0.1394 0.00054772256 43118856SphkDifference at 95.0% confidence 44118856Sphk 0.004 +/- 0.00166288 45118856Sphk 2.95421% +/- 1.22812% 46118856Sphk (Student's t, pooled s = 0.00114018) 47118856Sphk 48118856SphkHere we have a clearcut difference, not very big, but clear and unambiguous. 49118856Sphk 50118856Sphk 51