Access to this article is restricted until 24 months after publication by request of the publisher. Restriction lift date: 2026-12-12
Adaptive multilayer extreme learning machines
Loading...
Files
Date
2024-12-12
Authors
Filelis-Papadopoulos, Christos K.
Morrison, John P.
O’Reilly, Philip
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier Ltd.
Published Version
Abstract
Extreme learning machines is a neural network type that has been utilized in tasks such as regression and classification, due to their efficient training process, which is based on pseudoinverse matrices and randomized weights, avoiding the computationally intensive backpropagation. In order to further improve their performance and reduce their complexity with respect to number of required hyperparameters, especially in the case of multiple layer architectures, a novel multilayer adaptive approach, based on residual networks, is proposed. This approach constructs the network iteratively with respect to error minimization and parsimony using a recursive pseudoinverse matrix framework. A new block approach, using mixed precision arithmetic and Graphics Processing Units (GPU) is proposed. The proposed technique is coupled with a new adaptive penalty criterion to ensure adequate numbers of neurons are included in each layer, while avoiding highly correlated basis. Adaptive regularization, along with scaling, is also incorporated to ensure Symmetric Positive Definiteness (SPD) of the Gram matrix. Several random number distributions for the proposed approach are examined and discussed. Handling of large datasets is discussed and a new batch variant is proposed. The proposed scheme is evaluated for regression and classification tasks in a multitude of datasets and is compared with other neural network architectures.
Description
Keywords
Recursive pseudoinverse matrix , Modeling , Multilayer Extreme Learning Machine , Graphics Processing Unit
Citation
Filelis-Papadopoulos, C. K., Morrison, J. P. and O’Reilly, P. (2024) 'Adaptive multilayer extreme learning machines', Mathematics and Computers in Simulation, 231, pp.71-98. https://doi.org/10.1016/j.matcom.2024.12.004