Neural Network Architectures

This page lists the neural network models and components that are available in the library.

Regular Architectures

MultiLayerPerceptron(dim_in, dim_hidden, n_layers, dim_out=1, activ=nn.ReLU, final_activ=nn.Identity, geometric_init=False)

Bases: ImplicitModel

A simple customizable Multi-layer Perceptron (MLP)

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
activ	Module	Activation function of the network. Defaults to torch.nn.ReLU.	ReLU
final_activ	Module	Final activation of the network, to be applied to the output before returning the value. Defaults to torch.nn.Identity.	Identity
geometric_init	bool	whether to initialize the parameter following the SAL method. Defaults to False.	False

References

• On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes, Davies et al., 2021
• SAL: Sign Agnostic Learning of Shapes from Raw Data, Atzman and Lipman, 2020

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

SirenNet(dim_in, dim_hidden, n_layers, dim_out=1, w0=6.0, w0_first_layer=30.0, residual=False, init_type='siren', activation_type='siren')

Bases: ImplicitModel

Implementation of a SIREN network, that is a multilayer perceptron with periodic activation functions [1].

Alongside the base implementation from [1], several improvements published over the years are also available. They consist of either an initialization scheme, or a different activation function.

Activation functions available are

• "siren": the original SIREN activation [1]
• "finer": sin((|x|+1)x) activation proposed by FINER [2]
• "lsa": learnable spectral activation [3]
• "id" : identity (no activation, usually used in the last layer)

Initialization schemes available are

• "siren": the original SIREN initialization scheme [1]
• "geometric": geometric initialization scheme [4]
• "mfgi": multi-frequency geometric initialization scheme [5]
• "winner": initialization proposed by WINNER [6]

References

[1] Implicit Neural Representations with Periodic Activation Functions, Sitzmann et al., 2020
[2] FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions, Liu et al., 2023
[3] Learnable Spectral Activations, Shor et al., 2026 [4] DiGS : Divergence guided shape implicit neural representation for unoriented point clouds, Y.Ben-Shabat et al., 2022
[5] Improving Accuracy and Efficiency of Implicit Neural Representations: Making SIREN a WINNER, H.Chandravamsi et al., 2026

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
w0	float	Frequency of the activation function. Defaults to 6.	6.0
w0_first_layer	float	Frequency of the activation function in the first layer. Defaults to 30.	30.0
residual	bool	Whether to include residual connections between each layer. Defaults to False.	False
init_type	str	Initialization scheme to consider for the network's weights (case insensitive). Choices are "siren", "geometric", "mfgi", "winner".	'siren'
activation_type	str	Which activation to choose for the network (case insensitive). Choices are "siren", "finer", "lsa", "id". Defaults to "siren".	'siren'

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

TailedMultiLayerPerceptron(dim_in, dim_hidden, n_layers)

Bases: Module

Tailed Multilayer Perceptron architecture. A MLP where the output is a sum of all layer activations up to some specified depth. Acts as a natural LoD.

References

[1] T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation, Yang et al., 2025
[2] SAND: Spatially Adaptive Network Depth for Fast Sampling of Neural Implicit Surfaces, Yang et al., 2026

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
activ	Module	Activation function of the network. Defaults to torch.nn.ReLU.	required

QuaNet(dim_in, dim_hidden, n_layers, dim_out=1, activation=nn.Softplus, residual=False)

Bases: Module

Quadratic deep network implementation.

Code adapted from https://github.com/Galaxeaaa/HotSpot/blob/main/models/Net.py

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
activation	Callable	Activation function to be used between each layer. Defaults to nn.Softplus.	Softplus
residual	bool	whether to consider residual connections between layers. Defaults to False.	False

References

Universal Approximation with Quadratic Deep Networks, Fan et al., 2019

Lipschitz Architectures

Lipschitz neural networks are neural networks where the Lipschitz constant is controlled (equal or inferior to a given value). Recall that a function \(f\) is \(K\)-Lipschitz if for any points \(x,y \in \mathbb{R}^n\), we have:

\[|f(x) - f(y)| \leqslant K\,||x -y||.\]

The signed distance function is notoriously a 1-Lipschitz function, and using 1-Lipschitz neural implicit enables SDF approximations with guarantees as well as the use of different loss functions (see for instance the hKRTrainer class).

DenseLipAOL(dim_in, dim_hidden, n_layers, dim_out=1, coeff_lip=1.0, activation=nn.ReLU())

Bases: ImplicitModel

Neural network made of Semi-Definite Programming neural layers, as proposed by [1]. Using a square matrix \(W \in \mathbb{R}^{k \times k}\) and a bias vector \(b \in \mathbb{R}^k\), each layer is defined as:

\[x \mapsto \sigma(WT^{-1/2}x + b)\]

where \(T\) is a diagonal matrix of size \(\mathbb{R}^{k \times k}\):

\[ T_{ii} = \sum_{j=1}^k \left| (W^T W)_{ij}\right|\]

and \(\sigma(x) = \max(0, x)\) is any 1-Lipschitz function.

References

[1] Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks, Prach and Lampert, 2022

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
coeff_lip	float	Lipschitz constant. Defaults to 1.	1.0
activation	Module	Activation function to consider. Defaults to nn.ReLU().	ReLU()

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

DenseLipBjorck(dim_in, dim_hidden, n_layers, dim_out=1, group_sort_size=2, bias=True, k_coeff_lip=1.0)

Bases: ImplicitModel

A Lipschitz neural architecture based on Björck's orthonormalization. The implementation is based on the SpectralLinear layer of the deel-torchlip library.

References

• Sorting out Lipschitz function approximation, Anil et al., 2019
• https://github.com/deel-ai/deel-torchlip

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
group_sort_size	int	Size of the GroupSort activation function. If set to zero, the activation will be a FullSort. Defaults to 2 (minimum).	2
bias	bool	whether to include bias vectors in the layers. Defaults to True.	True
k_coeff_lip	float	Lipschitz constant of the network. Defaults to 1.	1.0

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

DenseLipCPL(dim_in, dim_hidden, n_layers, dim_out=1, coeff_lip=1.0, activation=nn.ReLU(), with_group_sort=True)

Bases: ImplicitModel

Neural network made of Convex Potential layers, as proposed by [1]. Using a square matrix \(W \in \mathbb{R}^{k \times k}\) and a bias vector \(b \in \mathbb{R}^k\), each layer is defined as:

\[x \mapsto x - \frac{2}{||W||_2} W^T \sigma(Wx + b)\]

where \(\sigma(x) = \max(0, x)\).

References

[1] A Dynamical System Perspective for Lipschitz Neural Networks, Meunier et al., 2022

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
coeff_lip	float	Lipschitz constant. Defaults to 1.	1.0
activation	Module	Activation function to consider. Defaults to nn.ReLU().	ReLU()
with_group_sort	bool	whether to add a GroupSort2 between each layer to slightly increase accuracy. Defaults to True	True

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

DenseLipSDP(dim_in, dim_hidden, n_layers, dim_out=1, coeff_lip=1.0, activation=nn.ReLU(), with_group_sort=True)

Bases: ImplicitModel

Neural network made of Semi-Definite Programming neural layers, as proposed by [1]. Using a square matrix \(W \in \mathbb{R}^{k \times k}\), a bias vector \(b \in \mathbb{R}^k\) and an additional vector \(q \in \mathbb{R}^k\) as parameters, each layer is defined as:

\[x \mapsto x - 2WT^{-1} \sigma(W^Tx + b)\]

where \(T\) is a diagonal matrix of size \(\mathbb{R}^{k \times k}\):

\[ T_{ii} = \sum_{j=1}^k \left| (W^T W)_{ij} \,\exp(q_j - q_i) \right|\]

and \(\sigma(x) = \max(0, x)\) is the rectified linear unit (ReLU) function.

References

[1] A Unified Algebraic Perspective on Lipschitz Neural Networks, Araujo et al., 2023
[2][https://github.com/deel-ai/orthogonium/blob/main/orthogonium/layers/conv/SLL/sll_layer.py](https://github.com/deel-ai/orthogonium/blob/main/orthogonium/layers/conv/SLL/sll_layer.py)

Parameters:

Name	Type	Description	Default
dim_in	int	dimension of the input vector. Usually 2 or 3 for neural implicits.	required
dim_hidden	int	dimension of the hidden layers.	required
n_layers	int	number of hidden layers.	required
dim_out	int	dimension of the output layers. Defaults to 1.	1
coeff_lip	float	Lipschitz constant. Defaults to 1.	1.0
activation	Module	Activation function. This architecture is proven to be 1-Lipschitz for ReLU, sigmoid and tanh. Defaults to ReLU.	ReLU()
with_group_sort	bool	whether to add a GroupSort2 between each layer to slightly increase accuracy. Defaults to True	True

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

parameter_singular_values(model)

Computes the minimal and maximal singular values of each parameter matrices of the input model.

Parameters:

Name	Type	Description	Default
model	ImplicitModel	the input implicit model	required

Returns:

Name	Type	Description
dict	dict	a dictionnary that assigns a tuple (s_min, s_max) to every layer name with a matrix of parameters

Positional encodings

Positional encodings are fixed transformations applied to the input data before being fed into a neural network. They increase the model's accuracy by tackling low-frequency biases. When defining a neural model, simply use a torch.nn.Sequential object to add your encoding before your neural network. Note that the encoding size and the input dimension of the network should match:

# Add an encoding made of Fourier Features to a simple multi-layer perceptron
model = torch.nn.Sequential(
    RandomFourierEncoding(geometry,1000),
    MultiLayerPerceptron(1000, 128, 4)
)

GaussianEncoding(points, stdv=1.0, trainable=False)

Bases: PointDistanceEncoding

Variant of the PointDistanceEncoding where distance are then fed into a Gaussian kernel:

\[x \mapsto e^{ - \frac{||x-p||^2}{\sigma}}\]

Parameters:

Name	Type	Description	Default
points	Tensor	The points consider for encoding	required
stdv	float	standard deviation of the Gaussian kernel. Defaults to 1.	1.0
trainable	bool	whether the points can be trained as model parameters. Defaults to False.	False

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

HyperplaneEncoding(points, normals, trainable=False)

Bases: ImplicitModel

Encoding that considers points with normals sampled on the geometry and applies the signed distance to each corresponding hyperplanes:

\[x \mapsto \langle n, x-p \rangle\]

where \((p,n)\) are points and normals.

Parameters:

Name	Type	Description	Default
points	Tensor	the points to consider	required
normals	Tensor	the normals associated with each point. Will be normalized.	required
trainable	bool	whether the points and normals can be trained as model parameters. Defaults to False.	False

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

PointDistanceEncoding(points, trainable=False)

Bases: ImplicitModel

Encodes the input point x with its l2 distance to sampled points \(p\) in space:

\[x \mapsto ||x-p||_2\]

Parameters:

Name	Type	Description	Default
points	Tensor	The points consider for encoding	required
trainable	bool	whether the points can be trained as model parameters. Defaults to False.	False

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

RandomFourierEncoding(input_dim, encoded_dim, stdv=1.0)

Bases: ImplicitModel

Encodes a given position into different frequencies of trigonometric functions :

\[x \mapsto [\cos(2\pi b x), \sin(2 \pi b x)]\]

where \(b\) is sampled from a normal distribution of zero mean (and provided standard deviation)

Parameters:

Name	Type	Description	Default
input_dim	int	dimension of the input points (usually 2 or 3).	required
encoded_dim	int	size of the final encoded vector. Should be an even number (as both sin and cos of every frequency is computed). the tensor b will have shape (geometry.dim, dim_encoded//2)	required
stdv	float	Standard variation of the normal distribution used to compute b. Defaults to 1..	1.0

Raises:

Type	Description
ValueError	Fails if the encoding size is not an even number.

References

• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains, Tancik et al. (2020) : https://arxiv.org/abs/2006.10739
• https://github.com/jmclong/random-fourier-features-pytorch/tree/main

__init_subclass__()

This code is called every time a children class is defined. Will wrap the init function of the children class so that its arguments are stored in the _config attribute.

load(path, map_location='cpu') classmethod

Loads an implicit model from a file on the disk.

Parameters:

Name	Type	Description	Default
path	str	path to the saved file.	required
map_location	str	map_location parameter for torch.load. Defaults to "cpu".	'cpu'

Raises:

Type	Description
Exception	Fails if the loaded file is not an implicitlab model file.

Returns:

Name	Type	Description
ImplicitModel		the loaded model with correct type, architecture and weights.

save(path)

Saves the model to the disk in a file prescribed by path. This methods saves both the model's weights as well as its configuration, i.e. its type as a subclass of ImplicitModel and the arguments of its constructor. That way, the model can be re-instanciated automatically provided.

Parameters:

Name	Type	Description	Default
path	str	path to the file into which the model will be saved. We use .inr extension as a convention.	required

Utilities

count_parameters(model, with_grad=True)

Counts the number of optimizable parameters inside a pytorch neural network.

Parameters:

Name	Type	Description	Default
model	Module	the neural model to consider	required
with_grad	bool	whether to count all the parameters in the model's tensors, or only optimizable parameters (with requires_grad=True). Defaults to True.	True

Returns:

Name	Type	Description
int	int	number of optimizable parameters in the model

parameter_singular_values(model)

Computes the minimal and maximal singular values of each parameter matrices of the input model.

Parameters:

Name	Type	Description	Default
model	ImplicitModel	the input implicit model	required

Returns:

Name	Type	Description
dict	dict	a dictionnary that assigns a tuple (s_min, s_max) to every layer name with a matrix of parameters