ReservoirComputing.jl

ReservoirComputing.jl provides an efficient, modular and easy to use implementation of Reservoir Computing models such as Echo State Networks (ESNs), liquid state machines, and conceptors. For information on using this package please refer to the stable documentation. Use the in-development documentation to take a look at not yet released features.

Features

ReservoirComputing.jl provides layers, models, and functions to help build and train reservoir computing models. More specifically the software offers:

• Base layers for reservoir computing model construction such as ReservoirComputer and ReservoirChain.
• Additional, lower level building blocks for custom reservoir computers, such as Collect, ESNCell, DelayLayer, NonlinearFeaturesLayer, MemoryESNCell, LinearReadout, SVMReadout, and more
• Fully built models:
  • ESN variations:
    • Echo state networks ESN
    • Echo state networks with delays DelayESN/InputDelayESN/StateDelayESN
    • Edge of stability echo state networks ES2N
    • Euler state networks EuSN
    • Hybrid echo state networks HybridESN
    • Neuromorphic reservoir computing EIESN/AdditiveEIESN
    • Support vector echo-state machine SVESM
    • Residual echo state networks ResESN
    • Reservoir memory networks RMNCell/RMNESN/RMNResESN
    • Deep echo state networks DeepESN
    • Local information flow echo state network LIFESN
  • Liquid state machines LSM
  • Conceptors Conceptor
  • Next generation reservoir computing NGRC
  • Reservoir computing with cellular automata RECA
• Wrappers:
  • Deep reservoirs DeepReservoir
  • Local information flow LocalInformationFlow
• Continuous time reservoirs:
  • SciMLProblemReservoir
  • Continuous-time echo state networks ContinuousESN
• 20+ reservoir initializers and 5+ input layer initializers
• 5+ reservoir states modification algorithms
• Sparse matrix computation through SparseArrays.jl
• StateSpaceSet inputs and outputs for train and predict through StateSpaceSets.jl
• Multiple training algorithms via LIBSVM.jl and MLJLinearModels.jl
• Ridge training via LinearSolve.jl (QRFactorization default; other algorithms selectable)

Installation

ReservoirComputing.jl can be installed using either of

julia> ] # press the closing square bracket to enter Pkg mode
pkg> add ReservoirComputing

or

using Pkg
Pkg.add("ReservoirComputing")

Quick Example

To illustrate the workflow of this library we will showcase how it is possible to train an ESN to learn the dynamics of the Lorenz system. You can find the same example fully explained in the getting started page.

using LuxCore: setup
using OrdinaryDiffEqAdamsBashforthMoulton
using Plots
using Random
using ReservoirComputing

Random.seed!(42)
rng = MersenneTwister(17)

function lorenz(du, u, p, t)
    du[1] = p[1] * (u[2] - u[1])
    du[2] = u[1] * (p[2] - u[3]) - u[2]
    du[3] = u[1] * u[2] - p[3] * u[3]
end

prob = ODEProblem(lorenz, [1.0f0, 0.0f0, 0.0f0], (0.0, 200.0), [10.0f0, 28.0f0, 8/3])
data = Array(solve(prob, ABM54(); dt=0.02))
shift = 300
train_len = 5000
predict_len = 1250

input_data = data[:, shift:(shift + train_len - 1)]
target_data = data[:, (shift + 1):(shift + train_len)]
test = data[:, (shift + train_len):(shift + train_len + predict_len - 1)]

esn = ESN(3, 300, 3; init_reservoir=rand_sparse(; radius=1.2, sparsity=6/300),
    state_modifiers=NLAT2)

ps, st = setup(rng, esn)
ps, st = train(esn, input_data, target_data, ps, st)
output, st = predict(esn, predict_len, ps, st; initialdata=test[:, 1])

plot(transpose(output)[:, 1], transpose(output)[:, 2], transpose(output)[:, 3];
    label="predicted")
plot!(transpose(test)[:, 1], transpose(test)[:, 2], transpose(test)[:, 3];
    label="actual")

Contributing

See CONTRIBUTING.md.

Citing

If you use this library in your work, please cite:

@article{martinuzzi2022reservoircomputing,
  author = {Francesco Martinuzzi and Chris Rackauckas and Anas Abdelrehim and Miguel D. Mahecha and Karin Mora},
  title = {ReservoirComputing.jl: An Efficient and Modular Library for Reservoir Computing Models},
  journal = {Journal of Machine Learning Research},
  year = {2022},
  volume = {23},
  number = {288},
  pages = {1--8},
  url = {http://jmlr.org/papers/v23/22-0611.html}
}

Acknowledgements

This project was possible thanks to initial funding through the Google summer of code 2020 program. Francesco M. further acknowledges ScaDS.AI and RSC4Earth for supporting additional progress on the library. Current developments are possible thanks to research funding for Francesco M. provided by MPIPKS.