InfScale

Prerequisites

Python 3.10+ is needed. We recommend to use pyenv to set up an environment.

pyenv install 3.10.12
pyenv global 3.10.12

Note that Python 3.10+ needs openssl1.1.1 and make sure openssl1.1.1+ is installed in your system.

Dependencies and Version

• PyTorch version: 2.4.0
• PyMultiWorld version: >= 0.2.1
• Transformers version: 4.45.2

Installation

Run the following under the top folder (infscale):

pip install .

This will install dependencies as well as InfScale package.

Patch Transformers

Locate the patch file for Transformers 4.52.2 by running python -c "import site; print(site.getsitepackages())". The command will produce a list of site-packages paths of a Python environment (e.g., ['/home/user1/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages']). Let SITE_PACAKGES denote one of the paths. Then, run the following command:

cd SITE_PACKAGES
patch -p1 < infscale/patch/transformers-v4.45.2.patch

Running development code

This is useful during local development. As a prerequisite, dependencies should be resolved. Thus, it is necessary to install InfScale once (see Installation). Once dependencies are resolved, under infscale (top folder), run the following command:

python -m infscale

This command will print out the following (for example):

Usage: python -m infscale [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  start       Start agent / controller / job.
  stop        Stop job.
  update      Update job.

Quickstart

For minimal execution of InfScale, one controller and one agent are needed. Run controller first and then agent, each on a separate terminal.

python -m infscale start controller

python -m infscale start agent id123

After the agent and controller started successfully, a job can be started by running the following command:

python -m infscale start job examples/resnet152/single_rep.yaml

In the above example, examples/resnet152/single_rep.yaml is an example for specifying a job specification file. A job specification file contains a field called job_id. The field can be used to stop a job (see stop command for more details).

To see some log messages, add INFSCALE_LOG_LEVEL=DEBUG before each of the above command.

To Run Tests

python -m pytest

License

Apache License 2.0.