240 T. Devendra et al. 26.5.1 Population Initialisation Stanley and Miikkulainen [12] have given comprehensive justification as to why it is beneficial to start the entire population with the simplest network possible. The initialisation used in this paper follows this. The simplest network is defined as one that only has connections between each of the input features and the output node(s). 26.5.2 Genome Evaluation and Fitness Function To calculate the fitness of a genome, the prediction error is calculated using the cross entropy loss function (L). L=−ylog ˆy −(1−y)log 1− ˆy (26.1) where ˆy is defined as the prediction and y as the true label in the classification problem. Once the loss is calculated, the negative value is taken as the fitness (−L). This is done to reward a lower cost and therefore, a higher fitness. 26.5.3 Tracking Innovation of Topology Innovation tracking refers to the tracking of new structural additions to a network. When a new connection is formed, an associated innovation number is attributed to the pair of nodes between which the connection exists. If the connection has occurred before, it simply takes on the innovation number of the first instance of it occurring. 26.5.4 Genetic Operators Mutations occur when the population reproduces to create the next generation. It is through these mutations that the population moves towards the “optimal” solution. Each mutation operation has an associated percentage chance to occur. The weight mutation involves the perturbation of a connection weight by a random value, chosen from a standard Gaussian distribution. The mechanism of a mutation involves the chance to reset the connection weight or simply reset all the connection weights within the genome in order to prevent against local minima in the backpropagation learning. The add connection mutation simply creates a connection between two already existing nodes. The connection is initialised with a random weight. Connections that can also bypass multiple layers can be created. Simply, the addition of a node involves splitting an existing connection by inserting a node in its place. The old connection is disabled and is replaced by two new connections. Removal of nodes and connections are done with the limitation that at least one path from an input feature to the output must exist. This is to ensure forward and backpropagation is possible. Crossover combines two parents genomes to create a new one. The two genomes are lined up and compared using the innovation numbers. Individual genes can then be identified as matching, disjoint or excess, with respect to their innovation number [12]. The child genome is created by inheriting genes depending on how each gene is classified. 26.5.5 Speciation Speciation of the population is a concept introduced to protect innovation in the model [12]. This property is especially important, as often, networks need time to adjust to new nonlinearities introduced by topological structural additions; this can lead to poor initial fitness of the genome. A compatibility function is defined that can be used as a measure for similarity between two genomes: δ = c1E N + c2D N +c3 ¯W (26.2)
RkJQdWJsaXNoZXIy MTMzNzEzMQ==