IEEE Transactions on Evolutionary Computation Special Issue on
Benchmarking Sampling-Based Optimization Heuristics: Methodology and Software (BENCH)

TEVC logo

NEWS


The submission deadline has been extended to September 15, 2021.

Official Link


The official call is available as PDF on the TEVC journal web page. It is mostly identical to this web page. In any case, please do not hesitate to get in touch if you have any questions about the scope of the special issue, the submission, or the review process.

Motivation


Benchmarking plays an important role in the study of evolutionary computation methods and other optimization algorithms. Among other benefits, benchmarking helps us analyze strengths and weaknesses of different techniques -- knowledge that can be used to design more efficient optimization approaches. Core to benchmarking is a well-designed experimental setup, which ranges from the selection of algorithms, problem instances, and performance metrics over efficient experimentation to a sound evaluation of the benchmark data. To assist researchers and users of evolutionary computation methods, a number of different tools addressing the various different aspects of benchmarking are available. However, most of them are developed in isolation, without a possible integration to already existing software in mind. This hinders knowledge transfer between the different research groups and between academic and industrial practitioners of evolutionary computation methods.

The goal of this special issue is to provide an overview of state-of-the-art software packages, methods, and data sets that facilitate sound benchmarking of evolutionary algorithms and other optimization techniques. By providing an overview of today's benchmarking landscape, new synergies will be laid open, helping the community to converge towards a higher compatibility between tools, towards better reproducibility and replicability of our research, a better use of resources, and, ultimately, towards higher standards in our benchmarking practices.

Topics


We welcome submissions on the following topics. We are particularly interested in submissions that present methods, software, and data collections that are applicable beyond a specific use case and that relate to problem classes of wider impact and interest.
Availability of code, software, and data in an open-source format is strongly encouraged, but not formally required. They must be well documented and made available for public download or in special cases (in particular for real-world problems) upon request.
We are particularly interested in submissions that discuss the role of the contributed techniques to the existing benchmarking landscape; this includes a discussion of interfaces with other existing software and/or the challenges in implementing these.
No restriction is made on the type of optimization problems that are being analyzed; contributions on constrained/unconstrained, noisy/ non-noisy, static/dynamic, single-/ multi-objective, combinatorial/continuous/mixed-integer problems, etc. are equally welcome.
Algorithm comparisons are as much in the scope as are methods to analyze problem characteristics or any other component of algorithm benchmarking. We also welcome submissions which propose comparisons between sampling-based heuristics and other optimization techniques.
NOT in the scope of this special issue are reports which mostly focus on technical aspects such as descriptions of software architecture or user manuals. That is, submission shall focus on the contribution that the proposed method, data collection, or software makes towards better benchmarking practices of our community.

Submission


Manuscripts should be prepared according to the ``Information for Authors'' section of the journal and submissions should be made through the journal submission website, by selecting the Manuscript Type ``BENCH Special Issue Papers'' and clearly adding ``Benchmarking Special Issue Paper'' to the comments to the Editor-in-Chief.
Submission of a manuscript implies that it is the authors' original unpublished work and is not being submitted for possible publication elsewhere.

Important Dates



Guest Editors


Page last modified: August 24, 2021.