parallel tasks

A Parallel task is defined by an array of other tasks to be run concurrently.

[tool.poe.tasks]
parallel = ["mypy", "pylint"]

By default the contents of the array are interpreted as references to other tasks (actually an inline ref task). However, this behaviour can be altered by setting the default_item_type option locally on the parallel task.

Subtask outputs are streamed to the console as they arrive with a prefix identifying the task.

Available task options

parallel tasks support all of the standard task options with the exception of use_exec and capture_stdout.

The following options are also accepted:

ignore_failbool | str 📖

If true then the failure (or non-zero return value) of one task in the parallel group does not abort the execution.

default_item_typestr 📖

Change the task type that is applied to string array items in this parallel group.

prefixstr 📖

Set the prefix applied to each line of output from subtasks. By default this is the task name.

prefix_maxstr 📖

Set the maximum width of the prefix. Longer prefixes will be truncated. Default is 16 characters.

prefix_templatestr 📖

Specifies a template for how the prefix is applied after truncating it to the prefix_max length. The default prefix_template is {color_start}{prefix}{color_end} |

Continue on subtask failure

A failure (non-zero result) will result in any remaining subtasks being cancelled, unless the ignore_fail option is set on the task like so:

[tool.poe.tasks]
attempts.parallel = ["task1", "task2", "task3"]
attempts.ignore_fail = true

If you want to run all the subtasks to completion but return non-zero result in the end of the sequence if any of the subtasks have failed you can set ignore_fail option to the return_non_zero like so:

[tool.poe.tasks]
attempts.parallel = ["task1", "task2", "task3"]
attempts.ignore_fail = "return_non_zero"

Changing the default item type

If you want strings in the array to be interpreted as a task type other than ref you may specify then default_item_type option like so:

parallel = [
  "linters:run_mypy(all=True)",
  "linters:run_pylint",
  "linters:run_flake8",
]
default_item_type = "script"

Alternatively you can declare other task types inline like so:

[tool.poe.tasks]
parallel = [
  { script = "linters:run_mypy(all=True)" },
  { script = "linters:run_pylint" },
  { script = "linters:run_flake8" },
]

Hint

An array within a parallel task is interpreted as a sequence task, so you can run certain parallel subtasks with strict ordering.

Customize output prefixing

When running multiple tasks in parallel a prefix is applied to each line of output to identify the origin. By default the prefix includes the task name and has a distinct color applied to it (6 colors in rotation).

Example output:

task_1 | task_1 output line 1
task_2 | task_2 output line 1
task_3 | task_3 output line 1
task_1 | task_1 output line 2
task_4 | task_4 output line 1
task_3 | task_3 output line 2
task_5 | task_5 output line 1
task_6 | task_6 output line 1

This behavior is customizable with the following options:

  • Setting the prefix changes the prefix content: default {name}. The {index} tag is also available which indicates the index of the subtask within the parallel array.

  • Setting the prefix_max changes the maximum width of the prefix: default 16

  • Setting the prefix_template changes how the prefix is formatted: default {color_start}{prefix}{color_end} |

For example:

[tool.poe.tasks]
parallel = ["build", "test", "deploy-to-prod"]
prefix = "{index}:{name}"
prefix_max = 10
prefix_template = "[{prefix}] "

will result in output rendered like:

[1:build] first task output line
[2:test] second task output line
[3:deploy_…] third task output line

Note that:

  1. When the {prefix} portion of the tempalate is longer that the configured prefix_max then it is truncated with ellipsis

  2. omitting the {color_start} and {color_end} tags from the template disables prefix coloring.

Hint

If capture_stdout is set on a subtask then its output will of course be excluded.