LeakyReLU.hpp File Reference

Leaky ReLU with configurable alpha (default 0.01). More...

#include <Eigen/Dense>

Include dependency graph for LeakyReLU.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces
namespace	FlexNN
	Namespace for the FlexNN neural network library.
namespace	FlexNN::Activations
namespace	FlexNN::Activations::detail

Functions
Eigen::MatrixXd	FlexNN::Activations::detail::leaky_relu_forward (const Eigen::MatrixXd &Z, double alpha=kLeakyAlpha)
	LeakyReLU forward: A = Z>0 ? Z : alpha*Z.
Eigen::MatrixXd	FlexNN::Activations::detail::leaky_relu_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &Z, const Eigen::MatrixXd &, double alpha=kLeakyAlpha) noexcept
	LeakyReLU backward: dZ = dA * (Z>0 ? 1 : alpha).

Variables
constexpr double	FlexNN::Activations::detail::kLeakyAlpha = 0.01

Detailed Description

Leaky ReLU with configurable alpha (default 0.01).

alpha is the slope for z <= 0. Historic fixed value 0.01 (v0.1) is kept as kDefaultLeakyAlpha in ActivationParameters.hpp; this header now takes alpha as a parameter so layers can store per-layer ActivationParameters.