Hopular: Modern Hopfield Networks for Tabular Data
Bernhard Schäfl1, Lukas Gruber1, Angela Bitto-Nemling1, 2, Sepp Hochreiter1, 2
1 ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria 2 Institute of Advanced Research in Artificial Intelligence (IARAI)
Detailed blog post on this paper at this link.
The software was developed and tested on the following 64-bit operating systems:
- Rocky Linux 8.5 (Green Obsidian)
- macOS 12.4 (Monterey)
The recommended way to install the software is to use
$ pip3 install git+https://github.com/ml-jku/hopular
Hopular has two modes of operation:
listfor displaying various information.
optimfor optimizing Hopular using specified hyperparameters.
More information regarding the operation modes is accessible via the
-h flag (or, alternatively, by
$ hopular -h
$ hopular <mode> -h
To display all available datasets, the
--datasets flag has to be specified in the
$ hopular list --datasets
Optimizing a Hopular model using the default hyperparameters is achieved by specifying the corresponding dataset in the
$ hopular optim --dataset <dataset_name>
To optimize a Hopular model on the
GlassIdentificationDataset using the default hyperparameters, only the dataset
name itself needs to be specified. More details on the default values are available in the
console interface implementation.
$ hopular optim --dataset "GlassIdentificationDataset"
Optimizing a smaller Hopular model on the
GlassIdentificationDataset utilizing only
4 modern Hopfield networks,
iterative refinement blocks, and a scaling factor of
10 is achieved by manually specifying the respective
$ hopular optim --dataset "GlassIdentificationDataset" --num_heads 4 --num_blocks 2 --scaling_factor 10
The datasets, which are part of this repository, are publicly available and may be licensed differently. Hence, the LICENSE of this repository does not apply to them. More details on the origin of the datasets are available in the accompanying paper.
This repository is MIT-style licensed (see LICENSE), except where noted otherwise.