Uploading pipelines to Pipeline Catalyst works best in Python 3.9. We strongly recommend you use Python 3.9 because the pipeline-ai library is still in beta and is known to cause opaque errors when pipelines are serialised from a non-3.9 environment.
We recommend installing Pipeline using a Python virtual environment manager such as
pip install -U pipeline-ai
Due to the ARM architecture of the M1 core it is necessary to take additional steps to install Pipeline, mostly due to the transformers library. We recomend running inside of a
conda environment as shown below.
- Make sure
- From terminal run:
Miniforge, instructions here: https://github.com/conda-forge/miniforge or follow the below:
- Download the
Miniforgeinstall script here: https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
- Make the shell executable and run
sudo chmod 775 Miniforge3-MacOSX-arm64.sh ./Miniforge3-MacOSX-arm64.sh
- Download the
- Create a
condabased virtual env and activate:
conda create --name pipeline-env python=3.9
conda activate pipeline-env
conda install -c apple tensorflow-deps
python -m pip install -U pip
python -m pip install -U tensorflow-macos
python -m pip install -U tensorflow-metal
conda install -c huggingface transformers -y
python -m pip install -U pipeline-ai
We strongly recommend installing the latest stable version of
pipeline, but if you think you need to install a specific version, specify the version, such as:
pip install -U "pipeline-ai==0.4.6"
Find the available release versions in the PyPI release history.
This project is made with poetry, so firstly setup poetry on your machine.
Once that is done, run:
With this you should be good to go. This sets up dependencies, pre-commit hooks and pre-push hooks.
You can manually run pre commit hooks with:
pre-commit run --all-files
To run tests manually, run:
Updated 10 months ago