`parallel` tasks¶

A Parallel task is defined by an array of other tasks to be run concurrently.

[tool.poe.tasks]
check.parallel = ["mypy", "pylint"]

By default the contents of the array are interpreted as references to other tasks (actually an inline ref task). However, this behaviour can be altered by setting the default_item_type option locally on the parallel task.

Subtask outputs are forwarded to the console with a prefix identifying the task, in either streaming or buffered mode.

Note

Because subtask output is captured via a pipe rather than written directly to the terminal, tools that detect terminal support by calling isatty() (such as Ruff, Behave, and others) will disable colored output when run as parallel subtasks.

The standard workaround is to set the FORCE_COLOR environment variable on the affected subtask, which tells tools to force colored output regardless of TTY detection. Poe also respects this variable for its own output:

[tool.poe.tasks.check]
parallel = [
  { cmd = "ruff check" },
  { cmd = "mypy" },
]
env = { FORCE_COLOR = "1" }

Alternatively, set FORCE_COLOR at the global or project level if all tasks should use it.

Available task options¶

parallel tasks support all of the standard task options with the exception of use_exec and capture_stdout.

The following options are also accepted:

ignore_failbool | Literal["return_zero", "return_non_zero"] 📖: If set to something other than false then the failure (or non-zero return value) of one task in the parallel group does not abort the execution.
default_item_typestr 📖: Change the task type that is applied to string array items in this parallel group.
output_modeLiteral["stream", "buffer"] 📖: Controls how subtask stdout is handled. The default stream mode outputs each line as it arrives, while buffer keeps each task’s stdout grouped together and prints it when the task subprocess completes.
prefixstr 📖: Set the prefix applied to each line of output from subtasks. By default this is the task name.
prefix_maxint 📖: Set the maximum width of the prefix. Longer prefixes will be truncated. Default is 16 characters.
prefix_templatestr 📖: Specifies a template for how the prefix is applied after truncating it to the prefix_max length. The default prefix_template is {color_start}{prefix}{color_end} |

Continue on subtask failure¶

A failure (non-zero result) will result in any remaining subtasks being cancelled, unless the ignore_fail option is set on the task like so:

[tool.poe.tasks]
attempts.parallel = ["task1", "task2", "task3"]
attempts.ignore_fail = true

If you want to run all the subtasks to completion but return non-zero result in the end of the parallel group if any of the subtasks have failed you can set ignore_fail option to the return_non_zero like so:

[tool.poe.tasks]
attempts.parallel = ["task1", "task2", "task3"]
attempts.ignore_fail = "return_non_zero"

Changing the default item type¶

If you want strings in the array to be interpreted as a task type other than ref you may specify then default_item_type option like so:

[tool.poe.tasks]
check.parallel = [
  "linters:run_mypy(all=True)",
  "linters:run_pylint",
  "linters:run_flake8",
]
default_item_type = "script"

Alternatively you can declare other task types inline like so:

[tool.poe.tasks]
check.parallel = [
  { script = "linters:run_mypy(all=True)" },
  { script = "linters:run_pylint" },
  { script = "linters:run_flake8" },
]

Hint

An array within a parallel task is interpreted as a sequence task, so you can run certain parallel subtasks with strict ordering.

Forwarding free arguments to subtasks¶

By default, free arguments passed to a parallel task are not forwarded to any of its subtasks. However, they can be forwarded selectively by referencing the special $POE_EXTRA_ARGS environment variable in individual subtask definitions. Only subtasks that explicitly reference $POE_EXTRA_ARGS will receive them.

See the Forwarding free arguments via $POE_EXTRA_ARGS section of the args guide for details and examples.

Buffer subtask output¶

By default a parallel task forwards each line of task output directly to the console with the configured prefix, so output lines from different tasks are interleaved.

If you prefer the output from each task subprocess to print as one contiguous block after it finishes, set output_mode to "buffer":

[tool.poe.tasks]
test-matrix.parallel = ["test-py310", "test-py311"]
test-matrix.output_mode = "buffer"

In buffered mode each flushed line still receives the configured prefix, but stdout is held back until the leaf task exits, fails, or is cancelled.

To avoid unbounded memory use, Poe keeps up to approximately 4MiB of stdout per leaf task in memory. If adding another complete line would exceed that limit then the current buffered lines are flushed early and buffering continues with later output. Normally the buffer preserves whole lines from the source. However in case a single output line exceeds the 4MiB limit, the line will be broken.

Redirect a subtask’s output to a file¶

Set capture_stdout on an individual subtask to send its output to a file instead of the console:

[tool.poe.tasks.checks]
parallel = [
  "lint",
  { cmd = "pytest", capture_stdout = "pytest.log" },
]

The captured subtask writes its stdout to the file and is excluded from the prefixed console output, while the other subtasks stream to the console as usual.

Customize output prefixing¶

When running multiple tasks in parallel a prefix is applied to each line of output to identify the origin. By default the prefix includes the task name and has a distinct color applied to it (6 colors in rotation).

Example output:

task_1 | task_1 output line 1
task_2 | task_2 output line 1
task_3 | task_3 output line 1
task_1 | task_1 output line 2
task_4 | task_4 output line 1
task_3 | task_3 output line 2
task_5 | task_5 output line 1
task_6 | task_6 output line 1

This behavior is customizable with the following options:

Setting the prefix changes the prefix content: default {name}. The {index} tag is also available which indicates the index of the subtask within the parallel array.
Setting the prefix_max changes the maximum width of the prefix: default 16
Setting the prefix_template changes how the prefix is formatted: default {color_start}{prefix}{color_end} |

For example:

[tool.poe.tasks]
parallel = ["build", "test", "deploy-to-prod"]
prefix = "{index}:{name}"
prefix_max = 10
prefix_template = "[{prefix}] "

will result in output rendered like:

[1:build] first task output line
[2:test] second task output line
[3:deploy_…] third task output line

Note that:

When the {prefix} portion of the tempalate is longer that the configured prefix_max then it is truncated with ellipsis
omitting the {color_start} and {color_end} tags from the template disables prefix coloring.