[gecode-users] Using Gecode for benchmarking

[Christian Schulte]

Dear all,

Over the years Gecode has become a popular platform for conducting
experiments and benchmarks. We of course encourage you to use Gecode for
benchmarking, regardless of whether Gecode shines or sucks at a certain
task.

However, we kindly ask to take the following suggestions into account:

Always use the latest version of Gecode (it is pretty pointless to use a
three year old version and then draw conclusions about the state of the
art).

Use a binary package created by us or compile Gecode with its default
settings. For example, measurements with compiler optimizations switched off
do not make any statement about Gecode. This is like assessing the top-speed
of a Porsche with flat tires and handbrake on.

If you compare models among different constraint programming systems, make
sure that you compare apples with apples. The following paper discusses
common pitfalls: http://www.springerlink.com/content/gn1h43w83721m667/

Then, if you are unsure and see behavior that is difficult to explain, just
ask us! We are always willing to help.

All the best
Christian


--
Christian Schulte, www.ict.kth.se/~cschulte/