Neuron

The monomial is of the form: f(x) = c · x^m + b

• x: input to monomial
• m: order of monomial, neurons, and MLP
• c: coefficient of x^m
• b: constant (bias)
• Both c and b are trainable parameters.

Here the b term is intentionally added to represent the bias as used in neural networks.

This modification is done to understand the nature of non-linearity over the linear nature of MLP, the direct impact of non-linearity on results, optimization of weights, and variation compared to standard MLP.

Derivative of Monomial: f'(x) = m · c · x^m-1

In standard MLP, this monomial replaces the linear neuron y = w · x + b.

Neural Network

Two types of neural networks are defined based on input: MNN and MNN2D.

• MNN takes 1D vector input.
• MNN2D takes 2D matrix input.
• Both produce a 1D output vector.

Both networks have similar mechanisms and weight structures, with gradient storing per layer.

For 1D i/o, it has 1D product and activations per layer. For 2D i/o, it has 2D product and activations per layer; output is calculated via Mean/Max/Weighted Mean pooling in MNN2D.

2D bias matrix for each weight matrix

An MLP uses a bias vector element-wise added to the output obtained by the product of previous activation and current weight. For monomial neurons this can amplify or restrain values dramatically, so each weight value has its own bias in the 2D case.

A gradient factor (α ≥ 0.8) is utilized so coefficients absorb the major change while biases absorb the minor part.

Notes

• Experimental: use caution when training (monomials can explode for large inputs or high orders).
• Tuning of initialization, learning rate, and gradient scaling is important.