Deep Reservoir Computing: A Novel Class of Deep Recurrent Neural Networks

In this thesis we propose a novel class of deep Recurrent Neural Networks (RNNs) explicitly extending the Reservoir Computing framework to the Deep Learning paradigm. Thereby, we introduce the Deep Echo State Network (DeepESN) model characterized by a hierarchy of randomized recurrent layers. The introduction of randomized deep RNNs has provided tools to analyze deep recurrent models separately from learning algorithms aspects. The analysis and the experimental assessments conducted on DeepESNs highlighted that layering in deep RNNs is intrinsically able to develop hierarchical, distributed temporal features. We evaluated our approach on controlled scenarios and challenging real-world tasks. Overall, DeepESN models allowed us to design extremely efficient deep RNNs that obtained performance competing with state-of-the-art approaches.