DeDLOC

This is a supplementary code for an anonymous paper submission for NeurIPS'21. The code is divided into several sections, matching the corresponding evaluations in the original paper.

• src contains the common DeDLOC code that is required by all other sections
• albert is the supplementary code for controlled experiments with ALBERT-large on Wikitext103
• swav is for training SwAV on ILSVRC data, similarly in similar conditions.
• sahajbert contains the code used to conduct public collaborative experiment for Bengali language.
• p2p is a step-by-step tutorial that explains decentralized NAT traversal and circuit relays

Installation

Before running any experiments, one must install the library from src section as such:

• Prepare an environment with python 3.7-3.9
  • Anaconda is recommended, but not required
• Ensure that your machine has a recent version of Golang (1.15 or higher)
  • To install Go, follow the instructions on the official website.
• Install the library: cd ./src && python setup.py install

For all distributed experiments, the installation procedure must be repeated on every machine that participates in the experiment. We recommend using machines with at least 2 CPU cores, 16GB RAM and, when applicable, a low/mid-tier NVIDIA GPU.

We recommend running albert experiments first: other experiments build on top of that starter and may reqire more careful setup (e.g. for public participation). Furthermore, for this experiment, we provide a script for launching experiments using cloud preemptible GPUs.