This is a PyTorch-based implementation of my project S-ANFIS: State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling (2022). S-ANFIS is an generalization of Jang's ANFIS: adaptive-network-based fuzzy inference system (1993). The implemenation can easliy be used to fit an ANFIS network.
S-ANFIS is a simple generalization of the ANFIS network, where the input to the premise and the consequence part of the model can be controlled separately. As general notation, I call the input the premise part "state" variables s and the input of the consequence part "input" or "explanatory" variables x.
For an in-depth explaination, check out our paper.
This package is intended to be installed on top of PyTorch, so you need to do that first.
Make sure to consider the correct operating system: Windows, macOS (Intel / Apple Silicon) or Linux. Everything is explained on the developer's website.
To ensure that PyTorch was installed correctly, verify the installation by running sample PyTorch code:
import torch
x = torch.rand(5, 3)
print(x)sanfis can be installed via pip:
pip install sanfisFirst let's generate some data! The given example is an AR(2)-process whoose AR-parameters depend on the regime of two independent state variables:
# Load modules
import numpy as np
import torch
from sanfis import SANFIS, plottingtools
from sanfis.datagenerators import sanfis_generator
# seed for reproducibility
np.random.seed(3)
torch.manual_seed(3)
## Generate Data ##
S, S_train, S_valid, X, X_train, X_valid, y, y_train, y_valid, = sanfis_generator.gen_data_ts(
n_obs=1000, test_size=0.33, plot_dgp=True)
Set a list of membership functions for each of the state variables that enter the model:
# list of membership functions
membfuncs = [
{'function': 'sigmoid',
'n_memb': 2,
'params': {'c': {'value': [0.0, 0.0],
'trainable': True},
'gamma': {'value': [-2.5, 2.5],
'trainable': True}}},
{'function': 'sigmoid',
'n_memb': 2,
'params': {'c': {'value': [0.0, 0.0],
'trainable': True},
'gamma': {'value': [-2.5, 2.5],
'trainable': True}}}
]The given example uses two sigmoid functions for each state variable.
Now create the model, fit and evaluate:
# make model / set loss function and optimizer
fis = SANFIS(membfuncs=membfuncs, n_input=2, scale='Std')
loss_function = torch.nn.MSELoss(reduction='mean')
optimizer = torch.optim.Adam(fis.parameters(), lr=0.005)
# fit model
history = fis.fit([S_train, X_train, y_train], [S_valid, X_valid, y_valid],
optimizer, loss_function, epochs=1000)
# eval model
y_pred = fis.predict([S, X])
plottingtools.plt_prediction(y, y_pred,
save_path='img/sanfis_prediction.pdf')
# plottingtools.plt_learningcurves(history)
The implementation allows a very flexible usage of membership functions. For each input variable that enters the premise-part of the model, the type and number of membership functions can be flexibly chosen. As of today, three possible membership functions are implemented:
The Gaussian is described by 2 parameters, mu for the location and sigma for the wideness.
# Example
gaussian_membfunc = {'function': 'gaussian',
'n_memb': 3, # 3 membership functions
'params': {'mu': {'value': [-2.0, 0.0, 1.5],
'trainable': True},
'sigma': {'value': [1.0, 0.5, 1.0],
'trainable': True}}
}In this example, three membership functions are considered.
The general bell-shaped function is described by three parameters, a (wideness), b (shape) and c (location).
bell_membfunc = {'function': 'bell',
'n_memb': 2,
'params': {'c': {'value': [-1.5, 1.5],
'trainable': True},
'a': {'value': [3.0, 1.0],
'trainable': False},
'b': {'value': [1.0, 3.0],
'trainable': False}}
}The sigmoid is described by two parameters: c (location) and gamma (steepness).
sigmoid_membfunc = {'function': 'sigmoid',
'n_memb': 2,
'params': {'c': {'value': [0.0, 0.0],
'trainable': True},
'gamma': {'value': [-2.5, 2.5],
'trainable': True}}
}Remember to add a list of membership functions as membfunc argument when creating the SANFIS oject, e.g.:
MEMBFUNCS = [gaussian_membfunc, bell_membfunc, sigmoid_membfunc]
model = SANFIS(MEMBFUNCS, n_input=2)
model.plotmfs(bounds=[[-2.0, 2.0], # plot bounds for first membfunc
[-4.0, 2.0], # plot bounds for second membfunc
[-5.0, 5.0]], # plot bounds fo third membfunc
save_path='img/membfuncs.pdf')
Tensorboard provides visualization needed for machine learning experimentation. Further information can be found here
pip install tensorboardTensorboard functionality can be added via arguments in the fit() function, e.g.
history = model.fit( ...
use_tensorboard=True,
logdir='logs/tb',
hparams_dict={}
)Note that hparams_dict is an optional argument where you can store additional hyperparameters of you model, e.g. hparams_dict={'n_input':2}.
tensorboard --logdir=logs/tb
To use the plain vanilla ANFIS network, simply remove the state variables s from the training (fit()). This automatically sets the same input for premise and consequence part of the model.
# Set 4 input variables with 3 gaussian membership functions each
MEMBFUNCS = [
{'function': 'gaussian',
'n_memb': 3,
'params': {'mu': {'value': [-0.5, 0.0, 0.5],
'trainable': True},
'sigma': {'value': [1.0, 1.0, 1.0],
'trainable': True}}},
{'function': 'gaussian',
'n_memb': 3,
'params': {'mu': {'value': [-0.5, 0.0, 0.5],
'trainable': True},
'sigma': {'value': [1.0, 1.0, 1.0],
'trainable': True}}},
{'function': 'gaussian',
'n_memb': 3,
'params': {'mu': {'value': [-0.5, 0.0, 0.5],
'trainable': True},
'sigma': {'value': [1.0, 1.0, 1.0],
'trainable': True}}},
{'function': 'gaussian',
'n_memb': 3,
'params': {'mu': {'value': [-0.5, 0.0, 0.5],
'trainable': True},
'sigma': {'value': [1.0, 1.0, 1.0],
'trainable': True}}},
]
# generate some data (mackey chaotic time series)
X, X_train, X_valid, y, y_train, y_valid = datagenerator.gen_data(data_id='mackey',
n_obs=2080, n_input=4)
# create model
model = SANFIS(membfuncs=MEMBFUNCS,
n_input=4,
scale='Std')
optimizer = torch.optim.Adam(params=model.parameters())
loss_functions = torch.nn.MSELoss(reduction='mean')
# fit model
history = model.fit(train_data=[X_train, y_train],
valid_data=[X_valid, y_valid],
optimizer=optimizer,
loss_function=loss_functions,
epochs=200,
)
# predict data
y_pred = model.predict(X)
# plot learning curves
plottingtools.plt_learningcurves(history, save_path='img/learning_curves.pdf')
# plot prediction
plottingtools.plt_prediction(y, y_pred, save_path='img/mackey_prediction.pdf')
- AnfisTensorflow2.0 by me
- bare-bones implementation of ANFIS (manual derivatives) by twmeggs
- PyTorch implementation by James Power
- simple ANFIS based on Tensorflow 1.15.2 by Santiago Cuervo
I am very thankful for feedback. Also, if you have questions, please contact gregor.lenhard92@gmail.com
If you use my work, please cite it appropriately:
Gregor Lenhard and Dietmar Maringer, "State-ANFIS: A Generalized Regime-Switching Model for Financial Modeling", 2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CIFEr), 2022, pp. 1-8, doi: 10.1109/CIFEr52523.2022.9776208.
BibTex:
@INPROCEEDINGS{lenhard2022sanfis,
author={Gregor Lenhard and Dietmar Maringer},
booktitle={2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics ({CIFEr})},
title={State-{ANFIS}: A Generalized Regime-Switching Model for Financial Modeling},
year={2022},
pages={1--8},
doi={10.1109/CIFEr52523.2022.9776208},
organization={IEEE}
}