Functions
Eigen::MatrixXd	forward (Activation act, const Eigen::MatrixXd &Z, const ActivationParameters &params=ActivationParameters{})
	Elementwise forward for a single activation.

Eigen::MatrixXd	backward (Activation act, const Eigen::MatrixXd &dA, const Eigen::MatrixXd &Z, const Eigen::MatrixXd &A, const ActivationParameters &params=ActivationParameters{}) noexcept
	Elementwise backward for a single activation (hidden layers).

Eigen::MatrixXd	softmax_forward (const Eigen::MatrixXd &Z)
	Stable column-wise softmax forward.

Eigen::MatrixXd	softmax_backward_hidden (const Eigen::MatrixXd &s, const Eigen::MatrixXd &upstream) noexcept
	Softmax hidden backward: `dZ = J * upstream` per column.

Eigen::MatrixXd	leaky_relu_forward (const Eigen::MatrixXd &Z, double alpha=kLeakyAlpha)
	LeakyReLU forward: A = Z>0 ? Z : alpha*Z.

Eigen::MatrixXd	leaky_relu_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &Z, const Eigen::MatrixXd &, double alpha=kLeakyAlpha) noexcept
	LeakyReLU backward: dZ = dA * (Z>0 ? 1 : alpha).

Eigen::MatrixXd	none_forward (const Eigen::MatrixXd &Z)
	Identity forward: A = Z.

Eigen::MatrixXd	none_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &, const Eigen::MatrixXd &) noexcept
	Identity backward: dZ = dA.

Eigen::MatrixXd	relu_forward (const Eigen::MatrixXd &Z)
	ReLU forward: A = max(0, Z) elementwise.

Eigen::MatrixXd	relu_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &Z, const Eigen::MatrixXd &) noexcept
	ReLU backward: dZ = dA * (Z > 0).

Eigen::MatrixXd	sigmoid_forward (const Eigen::MatrixXd &Z)
	Sigmoid forward: A = 1/(1+exp(-clamp(Z))) elementwise.

Eigen::MatrixXd	sigmoid_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &, const Eigen::MatrixXd &A) noexcept
	Sigmoid backward: dZ = dA * A * (1 - A).

Eigen::MatrixXd	tanh_forward (const Eigen::MatrixXd &Z)
	Tanh forward: A = tanh(Z) elementwise.

Eigen::MatrixXd	tanh_backward (const Eigen::MatrixXd &dA, const Eigen::MatrixXd &, const Eigen::MatrixXd &A) noexcept
	Tanh backward: dZ = dA * (1 - A^2).

Variables
constexpr double	kLeakyAlpha = 0.01

Function Documentation

◆ backward()

Eigen::MatrixXd FlexNN::Activations::detail::backward	(	Activation	act,
		const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	Z,
		const Eigen::MatrixXd &	A,
		const ActivationParameters &	params = `ActivationParameters{}`
	)

noexcept

Elementwise backward for a single activation (hidden layers).

Computes ‘dZ = f’(Z) ⊙ dAwheredAis upstream gradient. For Softmax hidden this is **not** fused — usesoftmax_backward_hiddeninstead. For the last-layer Softmax + cross-entropy, the caller (NeuralNetwork::backward) fuses and computesdZ = (A - Y)/m` directly, never calling this.

Note: noexcept: allocates MatrixXd; bad_alloc terminates via std::terminate (acceptable for host; no heap in MCU runtime which does not use this).

Parameters

act	Activation that was used in forward
dA	Upstream gradient (same shape as Z/A)
Z	Pre-activation from forward
A	Post-activation from forward (needed for Sigmoid/Tanh to avoid recompute)
params	Same params as forward (leakyAlpha for LeakyReLU)

Returns: Gradient w.r.t. Z, same shape as Z

◆ forward()

Eigen::MatrixXd FlexNN::Activations::detail::forward	(	Activation	act,
		const Eigen::MatrixXd &	Z,
		const ActivationParameters &	params = `ActivationParameters{}`
	)

Elementwise forward for a single activation.

Pure function on MatrixXd (double). Used by Layers::<Concrete>::forward() via switch(act). Header-only per-activation impls are in include/activations/<Act>.hpp — this dispatcher keeps layer code short. params is only used when act==LeakyReLU (others ignore it).

Parameters

act	Activation to apply
Z	Pre-activation (linear output)
params	Hyperparameters (e.g., leakyAlpha for LeakyReLU)

Returns: Post-activation matrix A, same shape as Z

◆ leaky_relu_backward()

Eigen::MatrixXd FlexNN::Activations::detail::leaky_relu_backward	(	const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	Z,
		const Eigen::MatrixXd &	,
		double	alpha = `kLeakyAlpha`
	)

inlinenoexcept

LeakyReLU backward: dZ = dA * (Z>0 ? 1 : alpha).

Parameters

dA	Upstream gradient
Z	Pre-activation from forward
alpha	Same alpha used in forward

◆ leaky_relu_forward()

Eigen::MatrixXd FlexNN::Activations::detail::leaky_relu_forward	(	const Eigen::MatrixXd &	Z,
		double	alpha = `kLeakyAlpha`
	)

inline

LeakyReLU forward: A = Z>0 ? Z : alpha*Z.

Parameters

Z	Pre-activation
alpha	Slope for z<=0 (0 < alpha < 1, default 0.01)

◆ none_backward()

Eigen::MatrixXd FlexNN::Activations::detail::none_backward	(	const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	,
		const Eigen::MatrixXd &
	)

inlinenoexcept

Identity backward: dZ = dA.

◆ none_forward()

Eigen::MatrixXd FlexNN::Activations::detail::none_forward ( const Eigen::MatrixXd & Z )

inline

Identity forward: A = Z.

◆ relu_backward()

Eigen::MatrixXd FlexNN::Activations::detail::relu_backward	(	const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	Z,
		const Eigen::MatrixXd &
	)

inlinenoexcept

ReLU backward: dZ = dA * (Z > 0).

Note: uses Z, not A, so the dead-neuron boundary (Z==0) is 0 as in the original lib/Layer.cpp:70 implementation. This matches the subgradient choice for ReLU and avoids a branch on A.

◆ relu_forward()

Eigen::MatrixXd FlexNN::Activations::detail::relu_forward ( const Eigen::MatrixXd & Z )

inline

ReLU forward: A = max(0, Z) elementwise.

◆ sigmoid_backward()

Eigen::MatrixXd FlexNN::Activations::detail::sigmoid_backward	(	const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	,
		const Eigen::MatrixXd &	A
	)

inlinenoexcept

Sigmoid backward: dZ = dA * A * (1 - A).

Uses A (already computed) to avoid recomputing exp — matches the standard d sigmoid / dz = s * (1-s) form.

◆ sigmoid_forward()

Eigen::MatrixXd FlexNN::Activations::detail::sigmoid_forward ( const Eigen::MatrixXd & Z )

inline

Sigmoid forward: A = 1/(1+exp(-clamp(Z))) elementwise.

Clamps Z to [-15,15] before exp to avoid overflow (exp(15) ~ 3.2M).

◆ softmax_backward_hidden()

Eigen::MatrixXd FlexNN::Activations::detail::softmax_backward_hidden	(	const Eigen::MatrixXd &	s,
		const Eigen::MatrixXd &	upstream
	)

inlinenoexcept

Softmax hidden backward: dZ = J * upstream per column.

Softmax hidden backward per column: dZ = J * upstream.

For hidden Softmax (disallowed in v0.1 but tested), the Jacobian is J = diag(s) - s s^T where s = softmax(Z) per column. This helper applies J to the upstream gradient column-wise. Last-layer Softmax never calls this — it uses the fused A - Y path.

Note: noexcept: allocates MatrixXd; bad_alloc terminates.

Parameters

s	Softmax output per column (from forward)
upstream	Upstream gradient (W_next^T * dZ_next) per column

Returns: Gradient w.r.t. Z

For each column, s is the softmax output for that column and up is the upstream gradient column (W_next^T * dZ_next for that sample). The Jacobian is J = diag(s) - s * s^T, so J*up = s ⊙ (up - s^T up). Implemented without forming J explicitly for O(n) per column.

Parameters

s	Softmax output per column [out × batch]
upstream	Upstream gradient [out × batch]

Returns: Gradient w.r.t. Z, same shape

◆ softmax_forward()

Eigen::MatrixXd FlexNN::Activations::detail::softmax_forward ( const Eigen::MatrixXd & Z )

inline

Stable column-wise softmax forward.

Implements exp(Z - max(Z, col)) / sum(exp(...)) per column for numerical stability. Used by detail::forward(Softmax, Z) and directly in tests.

Parameters

Z	Pre-activation [out × batch] (each column is a sample)

Returns: Softmax output, same shape, columns sum to 1

◆ tanh_backward()

Eigen::MatrixXd FlexNN::Activations::detail::tanh_backward	(	const Eigen::MatrixXd &	dA,
		const Eigen::MatrixXd &	,
		const Eigen::MatrixXd &	A
	)

inlinenoexcept

Tanh backward: dZ = dA * (1 - A^2).

Uses A = tanh(Z) from forward, so no second tanh call.

◆ tanh_forward()

Eigen::MatrixXd FlexNN::Activations::detail::tanh_forward ( const Eigen::MatrixXd & Z )

inline

Tanh forward: A = tanh(Z) elementwise.

Variable Documentation

◆ kLeakyAlpha

constexpr double FlexNN::Activations::detail::kLeakyAlpha = 0.01

inlineconstexpr

Functions

Variables

Function Documentation

◆ backward()

◆ forward()

◆ leaky_relu_backward()

◆ leaky_relu_forward()

◆ none_backward()

◆ none_forward()

◆ relu_backward()

◆ relu_forward()

◆ sigmoid_backward()

◆ sigmoid_forward()

◆ softmax_backward_hidden()

◆ softmax_forward()

◆ tanh_backward()

◆ tanh_forward()

Variable Documentation

◆ kLeakyAlpha