Queues

Q: Does Nextflow Tower support the use of multiple AWS Batch queues during a single job execution?

Yes. Even though you can only create/identify a single work queue during the definition of your AWS Batch compute environment in Tower, you can spread tasks across multiple queues via your pipeline configuration when your job is sent to Batch for execution.

Adding the following snippet, either to your nextflow.config or the Advanced Features > Nextflow config file field of Tower Launch UI, will cause processes to be distributed across two AWS Batch queues, depending on the assigned queue name.

# nextflow.config 

process {
  withName: foo {
    queue: `TowerForge-1jJRSZmHyrrCvCVEOhmL3c-work`
  }
}

process {
  withName: bar {
    queue: `custom-second-queue`
  }
}