Benchmarks

Objectives

The information provided here clarifies that Gecode is competitive with respect to runtime and memory usage. However, efficiency is but one aspect of a constraint programming system. The most important criterion in Gecode is a simple implementation and a clean and powerful API. This allows for powerful propagators for global constraints as the key issue for CP systems.

The benchmarks used here are not representative for real-life problems. Only models are compared that produce the same search tree on all used systems. The comparison does not answer the question which system is best for solving a particular problem, as other models may be more appropriate for a specific solver.

The benchmark programs
are exactly those that you find in the example folder of the Gecode source distribution. For the examples used in this comparison, we provide an overview page including details about propagators and branchings used, and statistics about the resulting search tree.

Runtime
has been measured as the arithmetic mean of 20 runs for each problem, with a coefficient of deviation below 2%.

Memory consumption
has been measured using the memusage utility from glibc. The numbers represent the peak heap size allocated from the operating system.

Memory benchmarking is difficult, and all systems provide one way or another to tune memory usage. We did not tune anything (which is very much to the disadvantage of Gecode as it supports efficient recomputation). The memory benchmarks called baseline have been obtained by solving an empty problem. They give an idea of the basic memory requirements of the respective system.

System
All benchmarks were carried out on a 2.8GHz P IV with 1GB of RAM, running Linux 2.6.18. The compiler used is gcc 4.2.1, without platform specific optimizations.

Select benchmarks

SystemComparison
Gecode 2.1.0 Runtime
ILOG Solver Memory
SICStus Prolog
Mozart/Oz