Computations in the middle layer-convert m inputs to n outputs

As a calculation in the middle layer in deep learning, we will explain how to convert m inputs to n outputs.

In deep learning, multiple inputs and multiple outputs are continuously connected in multiple stages to obtain the final output.

At this time, it is necessary to obtain n outputs from m inputs.

For example, for 4 inputs, you get 5 outputs. We will get 3 outputs from 5 inputs.

(x0, x1, x2, x3)
↓
(y0, y1, y2, y3, y4)
↓
(z0, z1, z2)

Get 3 outputs from 2 inputs

Now, as a simple example, let's describe how to get three outputs from two inputs. First, write it in Perl code.

use strict;
use warnings;

my $x0 = 0.5;
my $x1 = 0.8;

my $w00 = 0.1;
my $w01 = -0.4;
my $w10 = 0.2;
my $w11 = -0.5;
my $w20 = 0.3;
my $w21 = 1.3;

my $b0 = 0.3;
my $b1 = -0.9;
my $b2 = 0.5;

my $y0 = ($w00 * $x0) + ($w01 * $x1) + $b0;
my $y1 = ($w10 * $x0) + ($w11 * $x1) + $b1;
my $y2 = ($w20 * $x0) + ($w21 * $x1) + $b2;

print "($y0, $y1, $y2)\n";

The output result is as follows.

(0.0299999999999999, -1.2, 1.69)

By performing such a calculation, we were able to obtain three outputs ($y0, $y1, $y2) from two input values ($x1, $x2).

Convert the output with the activation function

Deep learning requires that the output be transformed by some rule before passing it to the next input. The reason this conversion is necessary is that learning will not proceed without this conversion.

Such a function is called an activation function, and one of the activation functions is a function called ReLU.

#ReLU function
sub relu {
  my ($num) = @_;
  
  return $num> 0: $num: 0;
}

This function returns its value if the input is greater than or equal to 0, and 0 if it is less than or equal to 0.

Let's apply the activation function to the previous sample. You don't have to write it as a function, so write it in the source code.

use strict;
use warnings;

my $x0 = 0.5;
my $x1 = 0.8;

my $w00 = 0.1;
my $w01 = -0.4;
my $w10 = 0.2;
my $w11 = -0.5;
my $w20 = 0.3;
my $w21 = 1.3;

my $b0 = 0.3;
my $b1 = -0.9;
my $b2 = 0.5;

my $y0 = ($w00 * $x0) + ($w01 * $x1) + $b0;
my $y1 = ($w10 * $x0) + ($w11 * $x1) + $b1;
my $y2 = ($w20 * $x0) + ($w21 * $x1) + $b2;

$y0 = $y0> 0? $y0: 0;
$y1 = $y1> 0? $y1: 0;
$y2 = $y2> 0? $y2: 0;

print "($y0, $y1, $y2)\n";

The output result is as follows. If the value is negative, it has changed to 0.

(0.0299999999999999, 0, 1.69)

Generalization of the process of converting m inputs to n outputs

Let's generalize the process of converting m inputs to n outputs. Please take a closer look at the process below.

use strict;
use warnings;

my $x0 = 0.5;
my $x1 = 0.8;

my $w00 = 0.1;
my $w01 = -0.4;
my $w10 = 0.2;
my $w11 = -0.5;
my $w20 = 0.3;
my $w21 = 1.3;

my $b0 = 0.3;
my $b1 = -0.9;
my $b2 = 0.5;

my $y0 = ($w00 * $x0) + ($w01 * $x1) + $b0;
my $y1 = ($w10 * $x0) + ($w11 * $x1) + $b1;
my $y2 = ($w20 * $x0) + ($w21 * $x1) + $b2;

$y0 = $y0> 0? $y0: 0;
$y1 = $y1> 0? $y1: 0;
$y2 = $y2> 0? $y2: 0;

print "($y0, $y1, $y2)\n";

Let's write out all the necessary information.

The number of inputs is two.

The number of outputs is three.

How many $w parameters do you need? This is "number of inputs" 2 "* number of outputs" 3 "= 6".

How many $b parameters do you need? Same as the number of outputs "3".

Input is

[$x0, $x1]

The parameter $w is

[
  $w00, $w01,
  $w10, $w11,
  $w20, $w21,
]

The parameter $b is

[
  $b0,
  $b1,
  $b2,
]

The output you receive is

[$y0, $y1, $y2]

am.

Arrangement

Let's code it. The first is arraying. If you can get the same output as the first example, that's the correct answer.

Keep an eye out and look at the law.

use strict;
use warnings;

my $x = [0.5, 0.8];
my $y = [0, 0, 0];
my $x_len = @$x;
my $y_len = @$y;

my $w = [
  0.1, -0.4,
  0.2, -0.5,
  0.3, 1.3,
];;

my $b = [
  0.3,
  -0.9,
  0.5,
];;

$y->[0] = ($w->[$x_len * 0 + 0] * $x->[0]) + ($w->[$x_len * 0 + 1] * $x->[ 1]) + $b->[0];
$y->[1] = ($w->[$x_len * 1 + 0] * $x->[0]) + ($w->[$x_len * 1 + 1] * $x->[ 1]) + $b->[1];
$y->[2] = ($w->[$x_len * 2 + 0] * $x->[0]) + ($w->[$x_len * 2 + 1] * $x->[ 1]) + $b->[2];

$y->[0] = $y->[0]> 0? $y->[0]: 0;
$y->[1] = $y->[1]> 0? $y->[1]: 0;
$y->[2] = $y->[2]> 0? $y->[2]: 0;

print "($y->[0], $y->[1], $y->[2])\n";

The output results are the same below.

(0.661503159202952, 0.802183888558582, 0.755838899094797)

Looping

After the array is finished, the next step is looping. You can apply the loop algorithm by using a data structure called an array. Let's rewrite the process in a double loop. If you can get the same output as the first example, that's the correct answer.

use strict;
use warnings;

my $x = [0.5, 0.8];
my $y = [0, 0, 0];
my $x_len = @$x;
my $y_len = @$y;

my $w = [
  0.1, -0.4,
  0.2, -0.5,
  0.3, 1.3,
];;

my $b = [
  0.3,
  -0.9,
  0.5,
];;

for (my $y_index = 0; $y_index <$y_len; $y_index ++) {
  my $total = 0;
  for (my $x_index = 0; $x_index <$x_len; $x_index ++) {
    $total + = ($w->[$x_len * $y_index + $x_index] * $x->[$x_index]);
  }
  $total + = $b->[$y_index];
  $y->[$y_index] = $total> 0? $Total: 0;
}

print "($y->[0], $y->[1], $y->[2])\n";

The output results are the same below.

(0.0299999999999999, 0, 1.69)

Now, even if m is a number other than 2, or n is a number other than 3, it can be handled.

We have finished learning how to convert m inputs to n outputs and apply the activation function among the techniques required for deep learning.

Perl AI Deep Learning Tutorial