Uploading pipelines to Pipeline Catalyst works best in Python 3.9. We strongly recommend you use Python 3.9 because the pipeline-ai library is still in beta and is known to cause opaque errors when pipelines are serialised from a non-3.9 environment.
We recommend installing Pipeline using a Python virtual environment manager such as
Linux, Mac (intel)
pip install -U pipeline-ai
Due to the ARM architecture of the M1 core it is necessary to take additional steps to install Pipeline, mostly due to the transformers library. We recomend running inside of a
conda environment as shown below.
- Make sure
- From terminal run:
Miniforge, instructions here: https://github.com/conda-forge/miniforge or follow the below:
- Download the
Miniforgeinstall script here: https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
- Make the shell executable and run
sudo chmod 775 Miniforge3-MacOSX-arm64.sh ./Miniforge3-MacOSX-arm64.sh
- Download the
- Create a
condabased virtual env and activate:
conda create --name pipeline-env python=3.9 conda activate pipeline-env
conda install -c apple tensorflow-deps python -m pip install -U pip python -m pip install -U tensorflow-macos python -m pip install -U tensorflow-metal
conda install -c huggingface transformers -y
python -m pip install -U pipeline-ai
We strongly recommend installing the latest stable version of
pipeline, but if you think you need to install a specific version, specify the version, such as:
pip install -U "pipeline-ai==0.4.6"
Find the available release versions in the PyPI release history.
Installing for development
This project is made with poetry, so firstly setup poetry on your machine.
Once that is done, run:
With this you should be good to go. This sets up dependencies, pre-commit hooks and pre-push hooks.
You can manually run pre commit hooks with:
pre-commit run --all-files
To run tests manually, run:
Updated about 1 month ago