AX2544

Processing tasks in parallel

Each Scheduler job can have multiple tasks. By default, each task in the job is processed sequentially, in the order that the tasks are listed in the job.

If desired, you can configure tasks so that they are processed concurrently (in parallel) instead of sequentially. If appropriate, this may speed up the processing of the job.

Configuring tasks for parallel processing

In order to process tasks in parallel, the tasks must be configured to run as subordinate jobs (sub-jobs). To do this, edit the following settings in the Task Control section for each task:

• Select Create a Subordinate Job for this Task.
• Ensure that the following setting is not selected: Wait for all Subordinate Jobs to complete before proceeding to the next Task.

In the following example, if all four tasks are configured to be run as subordinate jobs, then they can be run in parallel (depending on the available Scheduler threads).

Scheduler task configured to run as a subordinate job to enable parallel processing

How parallel processing works

When a task is configured to execute as a subordinate job, then it is not processed within the "parent" job. Instead, a sub-job is created for the task. The sub-job joins the Scheduler queue and is eligible for processing according to the normal Scheduler processing rules. For more information, see Processing priority for scheduled jobs.

For example, imagine that you have a job with four tasks, and these tasks are not dependent on each other. If you use the default settings, Scheduler takes the first task in the list and starts processing. The second task is not started until the first task is complete, and so on.

If instead you configure each task as a sub-job, then when the "parent" job is processed, it will create four sub-jobs. If two Scheduler threads are available for processing, then two of the sub-jobs are processed at the same time. If four Scheduler threads are available, then all four sub-jobs are processed at the same time. Once all of the sub-jobs are complete, the parent job is completed, and its status reflects the overall status of all of the sub-jobs.

If tasks are dependent on each other, then you should not process them as sub-jobs, or you should use the Wait setting as appropriate. For example, imagine that the first four tasks in the job can be run in any order, but the fifth task must be processed last. In that case, you can configure the first four tasks to run as sub-jobs, but on the fourth task you must enable Wait for all Subordinate Jobs to complete before proceeding to the next Task. This will cause Scheduler to wait for all sub-jobs to finish before it proceeds to the fifth, final task.

In the following example, the file processing tasks are configured as sub-jobs so that they can be run in parallel. The last file processing task is configured to wait, so that all of the file processing tasks will be finished before the file collection task begins.

Scheduler task configured to wait for all subordinate jobs to complete

Using an Echo task to control jobs with sub-jobs

By default, if a sub-job fails, this failure status will not prevent the parent job from continuing to process tasks. If you want the parent job to stop processing on sub-job failure, you can use an Echo task to do this.

If the Task Control option Wait for all Subordinate jobs to complete before proceeding to the next task is enabled for an Echo task, then the Echo task will inherit the status of the sub-jobs when they are completed. If one of the sub-jobs fails, then the Echo task will inherit that failed status. By default, this means that the job will not process any further tasks due to the sub-job failure.

In the previous example, if one of the file processing tasks being run as a sub-job fails, this will not stop the final file collection task from being processed. To work around this, you can place an Echo task after the file processing tasks and configure it to wait, and then if any of the file processing sub-jobs fail the Echo task will fail. This task failure within the parent job will stop the final file collection task from being processed. (When using this configuration, the fourth file processing task does not need to be configured to wait, because the job will wait on the Echo task instead.)