Krzysztof Krawiec


Home

Research:

edit SideBar

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks.

@INPROCEEDINGS { McDermott:2012:GPN:2330163.2330273,
    AUTHOR = { McDermott, James and White, David R. and Luke, Sean and Manzoni, Luca and Castelli, Mauro and Vanneschi, Leonardo and Jaƛkowski, Wojciech and Krawiec, Krzysztof and Harper, Robin and De Jong, Kenneth and O'Reilly, Una-May },
    TITLE = { Genetic programming needs better benchmarks },
    BOOKTITLE = { Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference },
    SERIES = { GECCO '12 },
    YEAR = { 2012 },
    ISBN = { 978-1-4503-1177-9 },
    LOCATION = { Philadelphia, Pennsylvania, USA },
    PAGES = { 791--798 },
    NUMPAGES = { 8 },
    URL = { http://doi.acm.org/10.1145/2330163.2330273 },
    DOI = { 10.1145/2330163.2330273 },
    ACMID = { 2330273 },
    PUBLISHER = { ACM },
    ADDRESS = { New York, NY, USA },
    KEYWORDS = { genetic programming, benchmarks },
    ABSTRACT = { Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks. },
}


Powered by PmWiki