Softmax.hpp File Reference

Stable column-wise softmax. More...

#include <Eigen/Dense>

Include dependency graph for Softmax.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces
namespace	FlexNN
	Namespace for the FlexNN neural network library.
namespace	FlexNN::Activations
namespace	FlexNN::Activations::detail

Functions
Eigen::MatrixXd	FlexNN::Activations::detail::softmax_forward (const Eigen::MatrixXd &Z)
	Stable column-wise softmax forward.
Eigen::MatrixXd	FlexNN::Activations::detail::softmax_backward_hidden (const Eigen::MatrixXd &s, const Eigen::MatrixXd &upstream) noexcept
	Softmax hidden backward: dZ = J * upstream per column.

Detailed Description

Stable column-wise softmax.

Forward is exp(Z - max(col)) / sum(exp(...)) per column (batch). Backward for hidden Softmax is J * upstream where J = diag(s) - s s^T. Last-layer Softmax + cross-entropy is fused as A - Y in NeuralNetwork::backward() and never calls the hidden path (see LLD §4.3).