Implement a simple neural network in C# .NET – Part 1

Last article “function of a neuron“, we saw how an artificial neuron is functioning with manual training on AND gate data. We saw that after 6th iteration a simple network learned and is now capable of predicting. Let’s put together the learning in the form of code and do it programmatically. I will divide this post into two parts:

  1. Create the neural network structure
  2. Train with AND gate data using backpropagation algorithm

The idea of this building from scratch is to get to know more via code and not the intention to build another deep learning library. There are many libraries which do the job for you, and we will talk about those later by building more complex network and train with lots of data (CSV and images). The idea of this post of to make the concept more clear which will help build more knowledge with the already available libraries for deep learning.

Before going into the code and divide it into various classes and function, let see the below diagram again. We will use some notion from the below diagram while building your own first neural network program.

Pulse

A pulse is an electric signal passing through the dendrite of neuron which forms the basis of data (value stored in double datatype). Below is the simple form of Pulse which one or more characters which in this case is a Value

class Pulse
{
    public double Value { get; set; }
}

 

Dendrite

A simple projection of a neuron that receives the pulse input and feeds into the neuron itself for computation. I have defined three properties for a Dendrite

InputPulse: The input signal

SynapticWeight: The connection strength between the dendrite and synapsis of two neurons.

Learnable: Just a bool used during training to adjust the SynapticWeight value

{
  public Pulse InputPulse { get; set; }

  public double SynapticWeight { get; set; }

  public bool Learnable { get; set; } = true;
}

Neuron

The neuron itself combined with its core function “Fire” and Dendrite. The workflow of a neuron is it receives the input values, do the weighted sum of all the input signals from all dendrites and pass them to the activation function which in this case is Step activation function. The output of the Activation is assigned to OutputPulse which will travel through the Axon of the neuron.

class Neuron
{
  public List<Dendrite> Dendrites { get; set;}

  public Pulse OutputPulse { get; set;}
  
  private double Weight;
  
  public Neuron()
  {
    Dendrites = new List<Dendrite>();
    OutputPulse = new Pulse();
  }

  public void Fire()
  {
    OutputPulse.Value = Sum();

    OutputPulse.Value = Activation(OutputPulse.Value);
  }
  
  public void UpdateWeights(double new_weights)
  {
    foreach (var terminal in Dendrites)
    {
      terminal.SynapticWeight = new_weights;
    }
  }
  
  private double Sum()
  {
    double computeValue = 0.0f;
    foreach (var d in Dendrites)
    {
      computeValue += d.InputPulse.Value * d.SynapticWeight;
    }
    
    return computeValue;
  }
  
  private double Activation(double input)
  {
    double threshold = 1;
    return input >= threshold ? 0 : threshold;
  }
}

 

NeuralLayer

As you see the below image, we stack a bunch of neuron in the form of layers to build a complex network, I have created another class which will create a layer first.

class NeuralLayer
{
  public List<Neuron> Neurons { get; set; }

  public string Name { get; set;}

  public double Weight { get; set;}

  public NeuralLayer(int count, double initialWeight, string name = "")
  {
    Neurons = new List<Neuron>();
    for(int i=0;i<count;i++)
    {
      Neurons.Add(new Neuron());
    }
    
    Weight = initialWeight;
    
    Name = name;
  }

  public void Optimize(double learningRate, double delta)
  {
    Weight += learningRate * delta;
    foreach (var neuron in Neurons)
    {
      neuron.UpdateWeights(Weight);
    }
  }

  public void Log()
  {
    Console.WriteLine("{0}, Weight: {1}", Name, Weight);
  }
}

 

NetworkModel

The NetworkModel is the complete architecture with layers of neuron and has Train functionality.

class NetworkModel
{
  public List<NeuralLayer> Layers { get; set; }

  public NetworkModel()
  {
    Layers = new List<NeuralLayer>();
  }

  public void AddLayer(NeuralLayer layer)
  {
    int dendriteCount = 1;

    if (Layers.Count > 0)
    {
      dendriteCount = Layers.Last().Neurons.Count;
    }

    foreach (var element in layer.Neurons)
    {
      for (int i = 0; i < dendriteCount; i++)
      {
        element.Dendrites.Add(new Dendrite());
      }
    }
  }

  public void Build()
  {
    int i = 0;
    foreach (var layer in Layers)
    {
      if (i >= Layers.Count - 1)
      {
        break;
      }

      var nextLayer = Layers[i + 1];
      CreateNetwork(layer, nextLayer);

      i++;
    }
  }

  public void Train(NeuralData X, NeuralData Y, int iterations, double learningRate = 0.1)
  {
    //Implement Training Method
  }

  public void Print()
  {
    
    DataTable dt = new DataTable();
    dt.Columns.Add("Name");
    dt.Columns.Add("Neurons");
    dt.Columns.Add("Weight");
    
    foreach (var element in Layers)
    {
      DataRow row = dt.NewRow();
      row[0] = element.Name;
      row[1] = element.Neurons.Count;
      row[2] = element.Weight;
      
      dt.Rows.Add(row);
    }
    
    ConsoleTableBuilder builder = ConsoleTableBuilder.From(dt);
    builder.ExportAndWrite();
  }
  
  private void CreateNetwork(NeuralLayer connectingFrom, NeuralLayer connectingTo)
  {
    foreach (var to in connectingTo.Neurons)
    {
      foreach (var from in connectingFrom.Neurons)
      {
        to.Dendrites.Add(new Dendrite() { InputPulse = from.OutputPulse, SynapticWeight = connectingTo.Weight });
      }
    }
  }
}

Invoke the NetworkModel from the Main method in a Console Project like this below.  This will create a network with three layers with initial weights.

NetworkModel model = new NetworkModel();
model.Layers.Add(new NeuralLayer(2, 0.1, "INPUT"));
model.Layers.Add(new NeuralLayer(2, 0.1, "HIDDEN"));
model.Layers.Add(new NeuralLayer(1, 0.1, "OUTPUT"));
  
model.Build();
model.Print();

 

The code itself is self-explanatory, so I don’t bored you guys with lots of text. Next article will implement the training method using the backpropagation algorithm and run a sample training.

2 thoughts on “Implement a simple neural network in C# .NET – Part 1

  1. Tayyab Hussain says:

    Hi there, code is definitely self explanatory but it will help if you add couple of lines of comments especially before functions for beginners to understand the motive of the function.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.