Hybrid Series/Parallel All-Nonlinear Dynamic-Static Neural Networks: Development, Training, and Application to Chemical Processes

This paper presents the development of data-driven hybrid nonlinear static-nonlinear dynamic neural network models and addresses the challenges of optimal estimation of parameters for such hybrid networks. A parallel static-dynamic neural network and two variants of series networks, specifically, nonlinear static-nonlinear dynamic and nonlinear dynamic-nonlinear static networks, are investigated in this work. Performances of the proposed fully nonlinear hybrid series and parallel network models are compared with the existing state-of-the-art data-driven models like long–short-term memory networks and gated recurrent unit models as well as DABNet-type linear-dynamic-nonlinear-static neural network. Algorithms are developed for training the series and parallel hybrid networks where the static and dynamic networks can be trained independently by different algorithms. These algorithms offer flexibility for incorporating different types of static and dynamic network architectures and their training algorithms thus offering tradeoff between computational expense and accuracy for highly nonlinear systems. In addition to the typical training objective that minimizes some squared error, an objective function is considered that penalizes overfitting and uncertainty in parameter estimates. Performances of the proposed algorithms are evaluated for three nonlinear example problemsa two-tank pH neutralization reactor, the widely used Van de Vusse reactor, and a pilot plant for postcombustion CO₂ capture using monoethanolamine solvent. Computational expense and convergence performance of the proposed network architectures and their training algorithms are presented.