mcfly package

Submodules

mcfly.find_architecture module

Summary: This module provides the main functionality of mcfly: searching for an optimal model architecture. The work flow is as follows: Function generate_models from modelgen.py generates and compiles models. Function train_models_on_samples trains those models. Function find_best_architecture is wrapper function that combines these steps. Example function calls can be found in the tutorial notebook (https://github.com/NLeSC/mcfly-tutorial)

mcfly.find_architecture.find_best_architecture(X_train, y_train, X_val, y_val, verbose=True, number_of_models=5, nr_epochs=5, subset_size=100, outputpath=None, model_path=None, metric='accuracy', class_weight=None, **kwargs)[source]

Tries out a number of models on a subsample of the data, and outputs the best found architecture and hyperparameters.

Parameters:
  • X_train (numpy array) – The input dataset for training of shape (num_samples, num_timesteps, num_channels)
  • y_train (numpy array) – The output classes for the train data, in binary format of shape (num_samples, num_classes)
  • X_val (numpy array) – The input dataset for validation of shape (num_samples_val, num_timesteps, num_channels)
  • y_val (numpy array) – The output classes for the validation data, in binary format of shape (num_samples_val, num_classes)
  • verbose (bool, optional) – flag for displaying verbose output
  • number_of_models (int, optiona) – The number of models to generate and test
  • nr_epochs (int, optional) – The number of epochs that each model is trained
  • subset_size (int, optional) – The size of the subset of the data that is used for finding the optimal architecture
  • outputpath (str, optional) – File location to store the model results
  • model_path (str, optional) – Directory to save the models as HDF5 files
  • class_weight (dict, optional) – Dictionary containing class weights (example: {0: 0.5, 1: 2.})
  • metric (str, optional) – metric that is used to evaluate the model on the validation set. See https://keras.io/metrics/ for possible metrics
  • **kwargs (key-value parameters) – parameters for generating the models (see docstring for modelgen.generate_models)
Returns:

  • best_model (Keras model) – Best performing model, already trained on a small sample data set.
  • best_params (dict) – Dictionary containing the hyperparameters for the best model
  • best_model_type (str) – Type of the best model
  • knn_acc (float) – accuaracy for kNN prediction on validation set
mcfly.find_architecture.kNN_accuracy(X_train, y_train, X_val, y_val, k=1)[source]

Performs k-Neigherst Neighbors and returns the accuracy score.

Parameters:
  • X_train (numpy array) – Train set of shape (num_samples, num_timesteps, num_channels)
  • y_train (numpy array) – Class labels for train set
  • X_val (numpy array) – Validation set of shape (num_samples, num_timesteps, num_channels)
  • y_val (numpy array) – Class labels for validation set
  • k (int) – number of neighbors to use for classifying
Returns:

accuracy – accuracy score on the validation set
Return type:

float
mcfly.find_architecture.store_train_hist_as_json(params, model_type, history, outputfile, metric_name='accuracy')[source]

This function stores the model parameters, the loss and accuracy history of one model in a JSON file. It appends the model information to the existing models in the file.

Parameters:
  • params (dict) – parameters for one model
  • model_type (Keras model object) – Keras model object for one model
  • history (dict) – training history from one model
  • outputfile (str) – path where the json file needs to be stored
  • metric_name (str, optional) – name of metric from history to store
mcfly.find_architecture.train_models_on_samples(X_train, y_train, X_val, y_val, models, nr_epochs=5, subset_size=100, verbose=True, outputfile=None, model_path=None, early_stopping=False, batch_size=20, metric='accuracy', class_weight=None)[source]

Given a list of compiled models, this function trains them all on a subset of the train data. If the given size of the subset is smaller then the size of the data, the complete data set is used.

Parameters:
  • X_train (numpy array of shape (num_samples, num_timesteps, num_channels)) – The input dataset for training
  • y_train (numpy array of shape (num_samples, num_classes)) – The output classes for the train data, in binary format
  • X_val (numpy array of shape (num_samples_val, num_timesteps, num_channels)) – The input dataset for validation
  • y_val (numpy array of shape (num_samples_val, num_classes)) – The output classes for the validation data, in binary format
  • models (list of model, params, modeltypes) – List of keras models to train
  • nr_epochs (int, optional) – nr of epochs to use for training one model
  • subset_size – The number of samples used from the complete train set
  • verbose (bool, optional) – flag for displaying verbose output
  • outputfile (str, optional) – Filename to store the model training results
  • model_path (str, optional) – Directory to store the models as HDF5 files
  • early_stopping (bool) – Stop when validation loss does not decrease
  • batch_size (int) – nr of samples per batch
  • metric (str) – metric to store in the history object
  • class_weight (dict, optional) – Dictionary containing class weights (example: {0: 0.5, 1: 2.})
Returns:

  • histories (list of Keras History objects) – train histories for all models
  • val_metrics (list of floats) – validation accuraracies of the models
  • val_losses (list of floats) – validation losses of the models

mcfly.modelgen module

mcfly.modelgen.generate_CNN_hyperparameter_set(min_layers=1, max_layers=10, min_filters=10, max_filters=100, min_fc_nodes=10, max_fc_nodes=2000, low_lr=1, high_lr=4, low_reg=1, high_reg=4)[source]

Generate a hyperparameter set that define a CNN model.

Parameters:
  • min_layers (int) – minimum of Conv layers
  • max_layers (int) – maximum of Conv layers
  • min_filters (int) – minimum number of filters per Conv layer
  • max_filters (int) – maximum number of filters per Conv layer
  • min_fc_nodes (int) – minimum number of hidden nodes per Dense layer
  • max_fc_nodes (int) – maximum number of hidden nodes per Dense layer
  • low_lr (float) – minimum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_lr (float) – maximum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • low_reg (float) – minimum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_reg (float) – maximum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
Returns:

hyperparameters – parameters for a CNN model
Return type:

dict
mcfly.modelgen.generate_CNN_model(x_shape, class_number, filters, fc_hidden_nodes, learning_rate=0.01, regularization_rate=0.01, metrics=['accuracy'])[source]

Generate a convolutional neural network (CNN) model.

The compiled Keras model is returned.

Parameters:
  • x_shape (tuple) – Shape of the input dataset: (num_samples, num_timesteps, num_channels)
  • class_number (int) – Number of classes for classification task
  • filters (list of ints) – number of filters for each convolutional layer
  • fc_hidden_nodes (int) – number of hidden nodes for the hidden dense layer
  • learning_rate (float) – learning rate
  • regularization_rate (float) – regularization rate
  • metrics (list) – Metrics to calculate on the validation set. See https://keras.io/metrics/ for possible values.
Returns:

model – The compiled Keras model
Return type:

Keras model
mcfly.modelgen.generate_DeepConvLSTM_hyperparameter_set(min_conv_layers=1, max_conv_layers=10, min_conv_filters=10, max_conv_filters=100, min_lstm_layers=1, max_lstm_layers=5, min_lstm_dims=10, max_lstm_dims=100, low_lr=1, high_lr=4, low_reg=1, high_reg=4)[source]

Generate a hyperparameter set that defines a DeepConvLSTM model.

Parameters:
  • min_conv_layers (int) – minimum number of Conv layers in DeepConvLSTM model
  • max_conv_layers (int) – maximum number of Conv layers in DeepConvLSTM model
  • min_conv_filters (int) – minimum number of filters per Conv layer in DeepConvLSTM model
  • max_conv_filters (int) – maximum number of filters per Conv layer in DeepConvLSTM model
  • min_lstm_layers (int) – minimum number of Conv layers in DeepConvLSTM model
  • max_lstm_layers (int) – maximum number of Conv layers in DeepConvLSTM model
  • min_lstm_dims (int) – minimum number of hidden nodes per LSTM layer in DeepConvLSTM model
  • max_lstm_dims (int) – maximum number of hidden nodes per LSTM layer in DeepConvLSTM model
  • low_lr (float) – minimum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_lr (float) – maximum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • low_reg (float) – minimum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_reg (float) – maximum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
Returns:

hyperparameters – hyperparameters for a DeepConvLSTM model
Return type:

dict
mcfly.modelgen.generate_DeepConvLSTM_model(x_shape, class_number, filters, lstm_dims, learning_rate=0.01, regularization_rate=0.01, metrics=['accuracy'])[source]

Generate a model with convolution and LSTM layers. See Ordonez et al., 2016, http://dx.doi.org/10.3390/s16010115

Parameters:
  • x_shape (tuple) – Shape of the input dataset: (num_samples, num_timesteps, num_channels)
  • class_number (int) – Number of classes for classification task
  • filters (list of ints) – number of filters for each convolutional layer
  • lstm_dims (list of ints) – number of hidden nodes for each LSTM layer
  • learning_rate (float) – learning rate
  • regularization_rate (float) – regularization rate
  • metrics (list) – Metrics to calculate on the validation set. See https://keras.io/metrics/ for possible values.
Returns:

model – The compiled Keras model
Return type:

Keras model
mcfly.modelgen.generate_base_hyper_parameter_set(low_lr=1, high_lr=4, low_reg=1, high_reg=4)[source]

Generate a base set of hyperparameters that are necessary for any model, but sufficient for none.

Parameters:
  • low_lr (float) – minimum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_lr (float) – maximum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • low_reg (float) – minimum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_reg (float) – maximum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
Returns:

hyperparameters – basis hyperpameters
Return type:

dict
mcfly.modelgen.generate_models(x_shape, number_of_classes, number_of_models=5, metrics=['accuracy'], model_type=None, cnn_min_layers=1, cnn_max_layers=10, cnn_min_filters=10, cnn_max_filters=100, cnn_min_fc_nodes=10, cnn_max_fc_nodes=2000, deepconvlstm_min_conv_layers=1, deepconvlstm_max_conv_layers=10, deepconvlstm_min_conv_filters=10, deepconvlstm_max_conv_filters=100, deepconvlstm_min_lstm_layers=1, deepconvlstm_max_lstm_layers=5, deepconvlstm_min_lstm_dims=10, deepconvlstm_max_lstm_dims=100, low_lr=1, high_lr=4, low_reg=1, high_reg=4)[source]

Generate one or multiple untrained Keras models with random hyperparameters.

Parameters:
  • x_shape (tuple) – Shape of the input dataset: (num_samples, num_timesteps, num_channels)
  • number_of_classes (int) – Number of classes for classification task
  • number_of_models (int) – Number of models to generate
  • metrics (list) – Metrics to calculate on the validation set. See https://keras.io/metrics/ for possible values.
  • model_type (str, optional) – Type of model to build: ‘CNN’ or ‘DeepConvLSTM’. Default option None generates both models.
  • cnn_min_layers (int) – minimum of Conv layers in CNN model
  • cnn_max_layers (int) – maximum of Conv layers in CNN model
  • cnn_min_filters (int) – minimum number of filters per Conv layer in CNN model
  • cnn_max_filters (int) – maximum number of filters per Conv layer in CNN model
  • cnn_min_fc_nodes (int) – minimum number of hidden nodes per Dense layer in CNN model
  • cnn_max_fc_nodes (int) – maximum number of hidden nodes per Dense layer in CNN model
  • deepconvlstm_min_conv_layers (int) – minimum number of Conv layers in DeepConvLSTM model
  • deepconvlstm_max_conv_layers (int) – maximum number of Conv layers in DeepConvLSTM model
  • deepconvlstm_min_conv_filters (int) – minimum number of filters per Conv layer in DeepConvLSTM model
  • deepconvlstm_max_conv_filters (int) – maximum number of filters per Conv layer in DeepConvLSTM model
  • deepconvlstm_min_lstm_layers (int) – minimum number of Conv layers in DeepConvLSTM model
  • deepconvlstm_max_lstm_layers (int) – maximum number of Conv layers in DeepConvLSTM model
  • deepconvlstm_min_lstm_dims (int) – minimum number of hidden nodes per LSTM layer in DeepConvLSTM model
  • deepconvlstm_max_lstm_dims (int) – maximum number of hidden nodes per LSTM layer in DeepConvLSTM model
  • low_lr (float) – minimum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_lr (float) – maximum of log range for learning rate: learning rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • low_reg (float) – minimum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
  • high_reg (float) – maximum of log range for regularization rate: regularization rate is sampled between 10**(-low_reg) and 10**(-high_reg)
Returns:

models – List of compiled models
Return type:

list
mcfly.modelgen.get_learning_rate(low=1, high=4)[source]

Return random learning rate 10^-n where n is sampled uniformly between low and high bounds.

Parameters:
  • low (float) – low bound
  • high (float) – high bound
Returns:

learning_rate – learning rate
Return type:

float
mcfly.modelgen.get_regularization(low=1, high=4)[source]

Return random regularization rate 10^-n where n is sampled uniformly between low and high bounds.

Parameters:
  • low (float) – low bound
  • high (float) – high bound
Returns:

regularization_rate – regularization rate
Return type:

float

mcfly.storage module

Summary: Functions to save and store a model. The current keras function to do this does not work in python3. Therefore, we implemented our own functions until the keras functionality has matured. Example function calls in ‘Tutorial mcfly on PAMAP2.ipynb’

mcfly.storage.loadmodel(filepath, modelname)[source]

Load model + weights from json + npy file, respectively

Parameters:
  • filepath (str) – directory where the data will be stored
  • modelname (str) – name of the model to be used in the filename
Returns:

model – reproduced model
Return type:

Keras object
mcfly.storage.savemodel(model, filepath, modelname)[source]

Save model to json file and weights to npy file

Parameters:
  • model (Keras object) – model to save
  • filepath (str) – directory where the data will be stored
  • modelname (str) – name of the model to be used in the filename
Returns:

  • json_path (str) – Path to json file with architecture
  • numpy_path (str) – Path to npy file with weights

Module contents