Perl AI Deep Learning Tutorial

softmax function

The softmax function is a function that transforms the output into a probability distribution with a total of 100%.

use strict;
use warnings;

#softmax function
sub softmax {
  my ($nums) = @_;
  
  my $exp_total = 0;
  for (my $i = 0; $i <@$nums; $i ++) {
    $exp_total + = exp($nums->[$i]);
  }
  
  my $nums_out = [];
  for (my $i = 0; $i <@$nums; $i ++) {
    $nums_out->[$i] = exp($nums->[$i]) / $exp_total;
  }
  
  return $nums_out;
}

my $outputs = [0.07, 0.14, 0.24];

my $softmax_outputs = softmax ($outputs);

# 0.306954386271124 0.329211090547647 0.363834523181229
print "@$softmax_outputs\n";

The output of the softmax function is computable even if the output contains 0s.

my $outputs = [0, 0.14, 0.24];

my $softmax_outputs = softmax ($outputs);

# 0.292267512114761 0.336187661442797 0.371544826442441
print "@$softmax_outputs\n";

loss function cross entropy error contains 0 in the argument value If it is, it will be log(0), so it will be impossible to calculate, but it can be avoided by using the softmax function as the activation function of the output layer.

Derivative of softmax function

The derivative of the softmax function is difficult to express.

However, the derivative of the softmax function and the composite function of the cross entropy error is easy to express, so it can be used for the inverse mispropagation method.

• Derivatives of the softmax function and the composite function of the cross entropy error