Skip to main content

Table 2 PaPy's core components (classes) and their roles

From: A lightweight, flow-based toolkit for parallel and distributed bioinformatics pipelines

Component Description & function
Piper, Worker The core components (processing nodes) of a pipeline. User-defined functions (or external programs) are wrapped as Worker instances; a Piper wraps a Worker and, in conjunction with numap, further species the mode of evaluation (serial/parallel, local/remote, etc.); these key pipeline elements also provide exception-handling, logging, and produce/spawn/consume functionality.
Dagger Defines the data-flow pipeline in the form of a DAG; allows one to add, remove, connect pipers, and validate topology. Coordinates the starting/stopping of NuMaps.
Plumber High-level interface to run & monitor a pipeline: Provides methods to save/load pipeline code, alter and monitor state (initiate/run/pause/stop/etc.), and save results. (See Additional file 1 §3.2 for more information on the subtle differences between the Plumber and Dagger classes.)
NuMap Implements a process/thread worker-pool. Allows pipers to evaluate multiple, nested map functions in parallel, using a mixture of threads or processes (locally) and, optionally, remote RPyC servers.