Adding a Workflow

Adding a Workflow

While Workbench provides thousands of community-authored workflows through external repositories, you may need to add your own private workflows. This offers several benefit:

1. Custom Logic Definition:

   • Allows you to define custom logic that may not be available in publicly accessible repositories.

2. Rapid Iteration:

   • Enables rapid iteration on your workflows before making them publicly available.

3. Compliance History:

   • Helps meet compliance needs by maintaining a history of all past versions of a workflow.

4. Secure Storage:

   • Provides a secure platform to store workflows, especially those containing potentially sensitive information.

   • (Note: Workflows should never contain PHI - Protected Health Information.)

5. Proprietary Development:

   • Facilitates the development of proprietary workflows without exposing the source code.

Upload Process

1. Navigate to the Workflows page

2. Click "Add Workflow" in the top right corner

3. Upload workflow files by either:

   • Dragging and dropping files

   • Using the file picker

When uploading files, consider your workflow structure:

• Single file uploads preserve only the filename and strip preceding paths

• For complex workflows with nested directories, either:

  • Upload as a directory to maintain structure

  • Combine files into a ZIP archive

• Preserving file paths ensures proper workflow import resolution

After adding all files, click "Upload" to proceed to metadata configuration.

Required Metadata

Configure your workflow with the following information:

Name Choose a meaningful name for easy future reference.

Version Name your first version. While semantic versioning is recommended, any meaningful name is acceptable. The name will be converted to an ID by removing illegal characters and spaces.

Entrypoint Specify the starting WDL file for workflow execution. Begin typing in the entrypoint input to see a filtered list of your uploaded files. This file should contain a single WDL workflow that represents the start of execution.

Description (Optional) Add markdown-supported documentation, instructions, or notes. This field can be edited after workflow creation.

After completing metadata, click "Save" to view your workflow's version page.

Additional Workflow Metadata

Workflow Naming

When adding a workflow through the user interface, a workflow name is mandatory. However, when using the command line interface (CLI), the name can be omitted. In cases where the workflow name is not provided in the CLI, it will be extracted directly from the Workflow Description Language (WDL).

For instance, consider the following workflow:

workflow hello_world {

}

the value hello_world will be extracted and set as the workflow name, if no other name was provided.

Workflow Authors

Adding Authors to Your Workflow

When adding a workflow in Workbench, you have the option to specify authors directly within the workflow definition, ensuring that essential metadata remains associated with your workflow. In the main workflow's meta section, you can provide either a single author or a list of authors.

Here's how you can add a single author:

meta {
  author: "George Clement"
}

And for multiple authors:

meta {
  authors [
    "George Clement",
    "Suzy Shrier"
  ]  
}

If no author is provided directly in the workflow, Workbench will automatically set you as the author.

Troubleshooting Workflow Creation

Workflow syntax is not correct

Only workflows which are syntactically correct will be saved to your account. Prior to saving a workflow, it is recommended that you use a tool to perform static type checking and syntax validation. On such tool is MiniWDL, a command line utility and execution engine for WDL. If running the miniwdl check command fails the workflow is unlikely , and you are still not seeing your workflow in Workbench, please contact DNAstack support

miniwdl check path/to/your/workflow.wdl

Missing files

When adding files in the User Interface of Workbench, make sure all WDL files referenced in import statements are properly uploaded with the appropriate path's. If you have a complex workflow structure you should upload your workflow files as a zip folder to ensure the relative paths are preserved

Wrong entrypoint

The entrypoint tells workbench which file contains the main workflow code. If the entrypoint is wrong validation will like fail or the resulting workflow that is submitted to an execution engine will be incorrect.