pipeline.Pipeline
Pipeline - class
pipeline.objects.pipeline.Pipeline(
new_pipeline_name: str,
compute_type: str = "gpu",
min_gpu_vram_mb: int = None,
)
Description
The Pipeline
class is used to either start a new context manager session to create a graph, or to interact with graph objects currently stored in Pipeline
. As it acts as both a context manager and object store there are various static and non static functions used in both situations.
When the context manager is closed, the resultant computational graph is stored inside of the Pipeline
object and must be retrieved.
Parameters
new_pipeline_name
(str
) - the name of the pipeline graph to be created, this is used to obtain the graph later.compute_type
(str
, optional) - the compute resource required for this pipeline, eithercpu
orgpu
. This is only used when running the pipeline on an external service.min_gpu_vram_mb
(int
, optional) - the minimum required amount of GPU memory required to run the pipeline. This is only used when running the pipeline on an external service.
Functions
output(self, *outputs: Variable) -> None
output(self, *outputs: Variable) -> None
Description
Adds Variable
objects to the output list for the pipeline. When the pipeline graph finishes executing via the run call, the real value of these Variable
objects that have been computed from the graph are returned.
Parameters
*outputs
(Variable
) - an arbitrary number ofVariable
objects.
get_pipeline(graph_name: str) -> Graph
get_pipeline(graph_name: str) -> Graph
Description
Retrieves a pipeline graph based on it's name. Typically this will be used shortly after the context manager.
Parameters
graph_name
(str
) - the name of the pipeline.
add_variable(variable: Variable) -> None
add_variable(variable: Variable) -> None
Description
Adds a variable to the graph being constructed inside of the context manager. this function cannot be called outside of the context manager.
Parameters
variable
(Variable
) - the variable to be added.
add_variables(*variables: Variable) -> None
add_variables(*variables: Variable) -> None
Description
The same as add_variable
but accepts an arbitrary number of inputs.
Parameters
*variables
(Variable
) - an arbitrary list ofVariable
objects to be added.
add_function(function: Function) -> None
add_function(function: Function) -> None
Description
This should not be used by a user and is called automatically when a function is called in the context manager to add it to the graph.
Parameters
function
(Function
) - the function object to be added to the graph.
add_graph_node(graph_node: GraphNode) -> None
add_graph_node(graph_node: GraphNode) -> None
Description
This is should not be used by a user and is called automatically when a function is called in the context manager as the input, output, and function are all used to construct the node.
Parameters
graph_node
(GraphNode
) - theGraphNode
object to be added to the graph.
Examples
Basic usage of Pipeline
context manager.
from pipeline import Pipeline, Variable, pipeline_function
@pipeline_function
def add_numbers(a: int, b: int) -> int:
return a + b
with Pipeline("add-numbers") as pipeline:
a = Variable(type_class=int, is_input=True)
b = Variable(type_class=int, is_input=True)
pipeline.add_variables(a, b)
c = add_numbers(a, b)
pipeline.output(c)
Updated 10 months ago