3. Gene

A gene is the smallest unit of encoded information.

Depending on the problem, a gene can represent:

• A binary decision (0 or 1)
• A real number (continuous variable)
• A position in sequence (permutation encoding)

Although genes are simple individually, their interaction determines the overall quality of the solution.

This is called the building block hypothesis in GA theory. Good partial gene combinations tend to survive and combine into better solutions.

4. Encoding Methods 

Binary encoding represents a chromosome as a sequence of 0s and 1s. Each bit corresponds to a specific decision variable or feature.

Example: 101101

This method is simple, easy to implement, and mathematically convenient. It is one of the earliest and most widely used encoding techniques in theoretical genetic algorithm models.

Common Applications:

• Feature selection
• Knapsack problems
• Theoretical GA research

However, binary encoding may require conversion when solving real-valued or continuous optimisation problems, which can reduce efficiency.

5. Real-Valued Encoding

In real-valued encoding, chromosomes are represented using decimal (floating-point) numbers instead of binary digits.

Example: [1.25, -0.89, 3.14]

This method is especially useful for continuous optimisation problems where precision is important.

Advantages:

• Higher numerical precision
• Faster convergence
• No need for binary-to-decimal conversion

Widely Used In:

1. Hyperparameter tuning
2. Neural network optimisation
3. Function optimisation

• Permutation Encoding

Permutation encoding represents a solution as an ordered sequence of numbers, where each number appears only once.

Example: [3, 1, 4, 2]

This encoding is ideal for ordering or sequencing problems.

Common Applications:

• Travelling Salesman Problem (TSP)
• Scheduling
• Routing problems

Because order matters, this encoding requires special crossover operators (such as Order Crossover – OX1). Standard crossover techniques may disrupt the sequence and produce invalid solutions.

The fitness function evaluates how good a chromosome is.

It:

• Guides evolution
• Determines survival probability
• Is problem-specific

Higher fitness = better solution.

It can be:

• Maximization problem
• Minimization problem

In machine learning:

• Fitness may be accuracy
• Or error reduction
• Or validation score

6. Fitness Function

The fitness function is one of the most important parts of a Genetic Algorithm. It measures how good a chromosome (solution) is at solving the given problem. In simple terms, it tells the algorithm which solutions are strong and which ones should be eliminated.

Without a fitness function, evolution cannot happen, because the algorithm would not know which solutions deserve to survive.

The fitness function evaluates how good a chromosome is.

It:

• Guides evolution
• Determines survival probability
• Is problem-specific

Higher fitness = better solution.

It can be:

• Maximization problem
• Minimization problem

In machine learning:

• Fitness may be accuracy
• Or error reduction
• Or validation score

6. Termination Criteria in Genetic Algorithms

A Genetic Algorithm (GA) does not run forever. It continues evolving solutions only until a predefined stopping condition, known as the termination criteria, is met. Proper termination is important because it prevents unnecessary computation and ensures the algorithm stops when a satisfactory solution is found.

Below are the most common termination conditions explained clearly:

• Maximum Generations Reached

This is the simplest stopping rule.
The algorithm is allowed to run for a fixed number of generations (iterations).

1. Example: Stop after 100 generations.
2. Even if the perfect solution is not found, the process ends.

Useful when computational resources are limited.
Risk: The algorithm may stop before reaching the best possible solution.

• Desired Fitness Achieved

The GA stops when a solution reaches or exceeds a predefined fitness value.

1. Example: Stop when model accuracy reaches 95%.
2. For minimisation problems, stop when the error falls below a threshold.

Ensures the algorithm stops once the objective is achieved.
Efficient when a clear performance target exists.

• No Improvement Over Time (Convergence)

If the fitness value does not improve for several consecutive generations, the algorithm assumes it has converged.

1. Example: No improvement in best fitness for 20 generations.
2. Indicates the population may be stuck in a local optimum.

Prevents wasting time when progress stalls.
Risk: May stop early if diversity is low.

• Time Limit Exceeded

The algorithm runs only for a predefined amount of time.

1. Example: Stop after 10 minutes.
2. Useful in real-time or production systems.

Practical for large-scale or computationally expensive problems.

Why Proper Termination Matters

Choosing the right termination condition balances:

1. Solution quality
2. Computational cost
3. Execution time

A well-defined stopping rule ensures that the genetic algorithm in machine learning delivers efficient and optimised results without overusing resources.

7. Termination Criteria

Selection mimics “survival of the fittest.”

The goal is not to always pick the best individual, because that would reduce diversity, but to probabilistically favour better individuals.

• Roulette Wheel Selection

What the Image Shows

• A circular wheel is divided into slices.
• Each slice size is proportional to an individual's fitness.
• A pointer spins and selects one slice.
• Larger slices = higher probability of selection.

Conceptual Explanation

Imagine a casino roulette wheel:

• Individuals with higher fitness occupy larger sections.
• When the wheel spins, individuals with larger sections are more likely to be selected.

Mathematically:

Where:

1. fif_ifi​ = fitness of individual i
2. Probability is proportional to fitness

Advantages

1. Simple to implement
2. Works well when fitness differences are moderate

Limitation

1. Very high fitness individuals may dominate
2. Sensitive to scaling of fitness values

• Tournament Selection

What the Image Shows

1. A small random group of individuals is selected.
2. Among them, the one with the highest fitness is chosen.
3. Process repeats to select more parents.

Conceptual Explanation

Steps:

1. Randomly select k individuals.
2. Compare their fitness.
3. Choose the best.
4. Repeat until enough parents are selected.

The parameter k controls selection pressure:

1. Large k → Stronger competition → Faster convergence
2. Small k → More randomness → More diversity

Advantages

1. Easy to implement
2. Not affected by fitness scaling
3. More stable than roulette selection

This is one of the most widely used methods in practical genetic algorithm machine learning systems.

• Stochastic Universal Sampling

What the Image Shows

• A roulette wheel similar to standard roulette.
• Instead of one pointer, multiple equally spaced pointers are used.
• All selections happen in a single spin.
• Conceptual Explanation

Unlike roulette selection:

1. SUS ensures individuals are selected proportionally but more evenly.
2. Reduces randomness bias.
3. Ensures fair sampling of the population.

Advantages

1. Lower variance
2. More consistent selection
3. Maintains diversity better than roulette

8. Crossover

Crossover, also known as recombination, is a genetic operator used in Genetic Algorithms to combine the genetic information of two parent chromosomes in order to produce new offspring solutions.

In simple terms, crossover mixes parts of two good solutions to create potentially better ones.

Just like biological reproduction combines traits from two parents, crossover blends selected genes from both parents in hopes of passing strong characteristics to the next generation.

Purpose of Crossover

1. Exploit good gene combinations: If both parents have strong traits, combining them may produce an even stronger solution.
2. Create new solution patterns: Mixing genes introduces new combinations that may not have existed before.
3. Accelerate convergence: By combining high-quality solutions, the algorithm can reach optimal solutions faster.

• One-Point Crossover

What the Image Shows

1. Two parent chromosomes
2. One random split point
3. Exchange gene segments after the split

Example:

Parent 1: 11001 | 010
Parent 2: 10111 | 111

Child 1: 11001 111
Child 2: 10111 010

Interpretation

1. Maintains the structure of gene blocks
2. Simple and efficient
3. Works well for binary encoding

• Multi-Point Crossover

What the Image Shows

1. Two or more cut points
2. Segments alternated between parents

Example:

Parent 1: 11 | 001 | 01
Parent 2: 10 | 111 | 11

Child: 11 111 01

Interpretation

1. Increases gene mixing
2. Produces higher diversity
3. Better for complex search spaces

• Davis Order Crossover (OX1)

What the Image Shows

• Two random cut points chosen in Parent 1
• Segment copied directly to child
• Remaining positions filled from Parent 2 in order

Why It Is Special

Normal crossover breaks permutation validity.
OX1 preserves:

• Order
• Position constraints
• No repetition

Used In:

• Travelling Salesman Problem
• Scheduling
• Routing problems
• Uniform Crossover

What the Image Shows

• For each gene, a random decision is made.
• A coin flip determines whether the gene comes from Parent 1 or Parent 2.

Example:

Parent 1: 110010
Parent 2: 101101

Mask: 101011

Child: 1(from P1) 0(from P2) 0(from P1) 1(from P2) 1(from P2) 0(from P1)

Interpretation

• Maximum mixing
• Very high diversity
• Less preservation of gene blocks

Genetic Algorithm Steps (Step-by-Step Explanation)

Here are the genetic algorithm steps explained clearly and sequentially:

Step 1: Define the Problem and Fitness Function

First, clearly define the problem you want to solve and decide how you will measure solution quality. This measurement is called the fitness function. It determines how good each solution is.

Step 2: Initialise the Population

Generate an initial population of random candidate solutions (chromosomes). Each chromosome represents a possible answer to the problem.

Step 3: Evaluate Fitness

Calculate the fitness score for each chromosome using the defined fitness function. Higher fitness means a better solution (for maximisation problems).

Step 4: Selection

Select the best-performing chromosomes to act as parents. Better fitness increases the probability of selection.

Step 5: Crossover (Recombination)

Combine selected parents to create new offspring. This mixes genetic information to form improved solutions.

Step 6: Mutation

Apply small random changes to some offspring. Mutation maintains diversity and prevents premature convergence.

Step 7: Create New Generation

Replace the old population with the new offspring population.

Step 8: Check Termination Condition

If stopping criteria are met (maximum generations, target fitness, etc.), stop. Otherwise, repeat from Step 3.

This cycle continues until an optimal or near-optimal solution is found.

Application of Genetic Algorithm in Machine Learning

The application of the genetic algorithm in machine learning is mainly focused on optimisation tasks where traditional methods struggle. One of the most common uses is feature selection, where GA helps identify the most relevant input features, reducing dimensionality and improving model performance.

Another major application is hyperparameter tuning. Instead of manually testing parameter combinations, genetic algorithms search the space efficiently to find optimal learning rates, tree depths, or network architectures. This improves accuracy while saving time.

Genetic algorithms are also used in neural network optimisation, including evolving weights or network structures. In complex problems such as rule-based systems and symbolic regression, GA can automatically discover patterns without relying on gradient-based methods.

Because genetic algorithms explore multiple solutions simultaneously, they are particularly useful for non-linear, multi-modal, or large search-space problems in machine learning, where conventional optimisation techniques may get stuck in local optima.

Conclusion

Genetic Algorithms are powerful evolutionary optimisation techniques inspired by natural selection. In genetic algorithm machine learning, they are widely used to search complex solution spaces where traditional optimisation methods may fail. By maintaining a population of candidate solutions and applying operators like selection, crossover, and mutation, GA balances exploration and exploitation effectively. 

Selection ensures the survival of fitter individuals, while crossover and mutation introduce diversity and innovation. Proper tuning of population size and genetic operators significantly impacts convergence and solution quality. Overall, genetic algorithms provide a flexible, robust, and global search approach for solving optimisation, feature selection, scheduling, and neural network problems.