|
FlexNN 1
Fully connected neural network built from scratch with flexible n-layer design and multiple activations.
|
Type-erased layer — holds one concrete layer type. More...
#include <Layer.hpp>
Public Member Functions | |
| Layer (Dense d) | |
| Layer (Conv1D c) | |
| Layer (BatchNorm1D b) | |
| Layer (MaxPool1D m) | |
| Layer (AvgPool1D a) | |
| LayerType | type () const noexcept |
| Layer kind (Dense, Conv1D, ...). | |
| Activations::Activation | activation () const noexcept |
| Activation for this layer. | |
| Activations::ActivationParameters | activationParams () const noexcept |
| Activation hyperparameters (e.g., LeakyReLU alpha). | |
| std::pair< Eigen::MatrixXd, Eigen::MatrixXd > | forward (const Eigen::MatrixXd &input) const |
| Forward pass for this layer (dispatch via visit). | |
| Eigen::MatrixXd | backward (const Eigen::MatrixXd &nextW, const Eigen::MatrixXd &nextdZ, const Eigen::MatrixXd &currZ) const |
| Backward pass for this layer (dispatch via visit). | |
| Eigen::MatrixXd | backward (const Eigen::MatrixXd &upstream, const Eigen::MatrixXd &currZ) const |
| Backward with already-propagated upstream (W_next^T*dZ_next). | |
| void | updateWeights (const Eigen::MatrixXd &dW, const Eigen::VectorXd &db, double lr) noexcept |
SGD update — dispatches to concrete update. | |
| void | update (const Eigen::MatrixXd &dW, const Eigen::VectorXd &db, double lr) noexcept |
| Eigen::MatrixXd | weightsMatrix () const |
| Weight matrix for backward — Dense/Conv1D. | |
| Eigen::MatrixXd | propagate (const Eigen::MatrixXd &dZ) const |
| Propagate gradient to previous layer (W^T * dZ or col2im). | |
| std::pair< Eigen::MatrixXd, Eigen::VectorXd > | grad (const Eigen::MatrixXd &dZ, const Eigen::MatrixXd &input) const |
| Compute dW/db for this layer given dZ and input. | |
| const auto & | variant () const noexcept |
| Direct access to the underlying variant (for ModelIO). | |
| auto & | variant () noexcept |
| const Dense * | asDense () const noexcept |
| Dense * | asDense () noexcept |
| const Conv1D * | asConv1D () const noexcept |
| Conv1D * | asConv1D () noexcept |
| const BatchNorm1D * | asBatchNorm1D () const noexcept |
| BatchNorm1D * | asBatchNorm1D () noexcept |
| const MaxPool1D * | asMaxPool1D () const noexcept |
| MaxPool1D * | asMaxPool1D () noexcept |
| const AvgPool1D * | asAvgPool1D () const noexcept |
| AvgPool1D * | asAvgPool1D () noexcept |
Type-erased layer — holds one concrete layer type.
In PR-05 the variant holds Dense and Conv1D. Later PRs expand it:
std::variant<Dense, Conv1D, BatchNorm1D>std::variant<Dense, Conv1D, BatchNorm1D, MaxPool1D, AvgPool1D>Each concrete type implements the same interface: LayerType type() const noexcept Activations::Activation activation() const noexcept pair<MatrixXd,MatrixXd> forward(const MatrixXd&) const MatrixXd backward(const MatrixXd& nextW, const MatrixXd& nextdZ, const MatrixXd& currZ) const void update(const MatrixXd& dW, const VectorXd& db, double lr) noexcept
The wrapper forwards to the active alternative via std::visit.
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inlinenoexcept |
Activation for this layer.
|
inlinenoexcept |
|
inlinenoexcept |
|
inlinenoexcept |
|
inlinenoexcept |
|
inlinenoexcept |
|
inlinenoexcept |
|
inlinenoexcept |
|
inline |
Backward pass for this layer (dispatch via visit).
For the last layer, NeuralNetwork::backward fuses Softmax+CE and never calls this for the last layer. For hidden layers this applies ‘f’(Z) ⊙ (W_next^T * dZ_next)`.
Kept for dense_test compatibility (takes nextW/nextdZ).
|
inline |
Backward with already-propagated upstream (W_next^T*dZ_next).
Used by NeuralNetwork::backward after layers[i+1].propagate(dZ_next) so the caller handles Conv1D transpose via col2im.
|
inline |
Forward pass for this layer (dispatch via visit).
| input | Batch input [in_dim × batch] |
[out_dim × batch]
|
inline |
Compute dW/db for this layer given dZ and input.
For Dense: dW = dZ * X^T / batch, db = rowMean(dZ). For Conv1D: dW/db via Conv1D::grad. Returns pair (dW, db) as two matrices (db as column vector promoted to MatrixXd).
|
inline |
Propagate gradient to previous layer (W^T * dZ or col2im).
Used by NeuralNetwork::backward to compute upstream for hidden layers: upstream = layers[i+1].propagate(dZ_next). For Dense this is W^T * dZ; for Conv1D this is col2im(W^T * dZ) via Conv1D::propagate.
|
inlinenoexcept |
|
inlinenoexcept |
SGD update — dispatches to concrete update.
Kept as updateWeights for compatibility with legacy Layer::updateWeights signature used by NeuralNetwork::updateWeights. New code may call update.
|
inlinenoexcept |
Direct access to the underlying variant (for ModelIO).
Lets ModelIO do std::visit without public variant exposure of concrete types. Kept const — mutation goes via updateWeights.
|
inlinenoexcept |
|
inline |
Weight matrix for backward — Dense/Conv1D.
For Dense this is W [out×in]; for Conv1D this is W [outCh×inCh*K]. Kept for ModelIO inspection; for backward prefer propagate() which handles Conv1D col2im correctly. Not noexcept (allocates on copy).