A Runtime Analysis of Bias-invariant Neuroevolution and Dynamic Fitness Evaluation

In the field of neuroevolution (NE), evolutionary algorithms are used to update the weights, biases and topologies of artificial neural networks (ANNs). A recent theoretical work presented the first runtime analysis of NE in a simple setting, considering a single neuron and intuitive benchmark function classes. However, this work was limited by the unrealistic settings with regard to activation functions and fitness measurements.In this paper, we extend upon this first work by overcoming the two shortcomings. Firstly, we consider a more realistic setting in which the NE also evolves a third parameter, termed the bend, allowing the previous benchmark function classes to be solved efficiently even in the fixed bias case. This setting mimics rectified linear unit activation functions, which are common in real-world applications of ANNs. Secondly, we consider a dynamic fitness function evaluation paradigm where the weights and biases are updated after each new sample. Experimental results in both cases support the presented theoretical results.