Installation

🚧

Uploading pipelines to Pipeline Catalyst works best in Python 3.9. We strongly recommend you use Python 3.9 because the pipeline-ai library is still in beta and is known to cause opaque errors when pipelines are serialised from a non-3.9 environment.

We recommend installing Pipeline using a Python virtual environment manager such as conda, poetry, pipenv or virtualenv/venv.

Linux, Mac (intel)

pip install -U pipeline-ai

Mac (arm/M1)

Due to the ARM architecture of the M1 core it is necessary to take additional steps to install Pipeline, mostly due to the transformers library. We recomend running inside of a conda environment as shown below.

1. Make sure Rosetta2 is disabled.
2. From terminal run:

xcode-select --install

3. Install Miniforge, instructions here: https://github.com/conda-forge/miniforge or follow the below:
   1. Download the Miniforge install script here: https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
   2. Make the shell executable and run
   Shell

   sudo chmod 775 Miniforge3-MacOSX-arm64.sh
   ./Miniforge3-MacOSX-arm64.sh
   

4. Create a conda based virtual env and activate:

conda create --name pipeline-env python=3.9
conda activate pipeline-env

5. Install tensorflow

conda install -c apple tensorflow-deps
python -m pip install -U pip
python -m pip install -U tensorflow-macos
python -m pip install -U tensorflow-metal

6. Install transformers

conda install -c huggingface transformers -y

7. Install pipeline

python -m pip install -U pipeline-ai

Versions

We strongly recommend installing the latest stable version of pipeline, but if you think you need to install a specific version, specify the version, such as:

pip install -U "pipeline-ai==0.4.6"

Find the available release versions in the PyPI release history.

Installing for development

This project is made with poetry, so firstly setup poetry on your machine.

Once that is done, run:

sh setup.sh

With this you should be good to go. This sets up dependencies, pre-commit hooks and pre-push hooks.

You can manually run pre commit hooks with:

pre-commit run --all-files

To run tests manually, run:

pytest