Working with JSON Data

Many commands used by the Workbench CLI expect their inputs to be JSON or JSON-like. In order to provide a flexible and powerful mechanism to input JSON Data, the CLI has adopted and extended a syntax defined originally by HTTPie. This syntax gives users the ability to define arbitrarily complex JSON directly from the command line with minimal need to escape special characters.

Literal JSON

When building scripts it is often useful to read JSON into a program, manipulate it in memory, and then output the resulting JSON directly without saving it to a file. The Workbench CLI supports this use case by allowing you to define literal JSON objects and arrays.

omics workbench runs submit --url ${WORKFLOW_URL} \
  --workflow-params '{"hello_world.greeting": "Hello", "hello_world.people": [ {"name": "Paullete", "age": 42}, { "name": "Heinrich", "age": 32}]}'

# You can also output the literal JSON by running a command and pass the contents to the flag 
omics workbench runs submit \
    --url ${WORKFLOW_URL} \
    --workflow-params "$(mutateJson input.json)" 

Load JSON from a file

Rather than passing raw JSON directly, the @ operator can be used to prefix the path to a local file containing JSON. This operator will cause the CLI to load the contents of the file and interpret them as JSON.

For example, if there was a local file called inputs.json with the following contents:

{
  "hello_world.greeting":"Hello", 
  "hello_world.people": [ 
    {
      "name": "Paullete", 
      "age": 42
    }, 
    { 
      "name": "Heinrich",
       "age": 37
    }
  ]
}

You could easily reference the entire file from the command line by specifying @inputs.json as the value passed to the relevant command flag. This would result in the file contents being loaded, interpreted as JSON, and passed to the command.

omics workbench runs submit \
    --url ${WORKFLOW_URL} \
    --workflow-params @inputs.json

Build Complex JSON with Key:Value Pairs

The CLI allows you to build complex JSON objects using a combination of keys, operators, and values. When building complex values for a single parameter, each key:value pair should be separated with a single comma , except in the case of input overrides used by the runs submit command, which uses white space instead.

The basic form used for building complex objects is <Keys><Operator><Value>

Keys

JSON objects are constructed by sequentially merging all the keys defined into a single object, building nested objects and arrays as required. A key defined on the command line corresponds to the literal JSON key in the resulting JSON object. Keys can contain any valid characters (including spaces) except square brackets, which must be escaped with a backslash \in order to be used.

Square brackets are used to denote a nested object or an array. Assume you have the following key:value pair: hello_world[i]=Goodbye

If i can be interpreted as a number (such as 0), then the value Goodbye will be set to the ith position in an array with the whole array assigned to the key hello_world in the resulting object. Empty brackets may also be used to indicate a JSON array, with the value simply being appended to the end, instead of a fixed index.

{
  "hello_world": [ "Goodbye" ]
}

If i cannot be interpreted as a number (such as greeting) then the value Goodbye will be assigned to a key i in a nested object which is in turn assigned to the hello_world key in the resulting object.

{
  "hello_world": {
    "greeting": "Goodbye"
  }
}

You can use any number of square brackets following the first initial key to describe arbitrarily deep JSON structures. Additionally, omitting the initial key and simply defining square brackets with a number allows you to define a root-level array.

[][greeting]="Hello"
[1][greeting]="Goodbye"
[][greeting]="Farewell"

[
  {
    "greeting": "Hello"
  },
  {
    "greeting": "Goodbye"
  },
  {
    "greeting": "Farewell"
  }
]

Operators

• = - assign the value as a string to the given key.

• := - interpret the value as JSON literal. This operator would be used for setting any JSON primitive other than a string (number, boolean, null), an object, or an array.

• :=@ - Load the contents of a file and interpret them as a JSON literal. This is helpful if you have snippets of ??? you would like to reuse that are saved to disk.

• =@ - Load the contents of a file as a string without interpreting them as JSON.

Examples

Consider the following set of examples demonstrating how to use the different operators.

hello_world.greeting=hello                                    # String
hello_world.people[0][name]=Paulette                          # String
hello_world.people[0][age]:=42                                # Literal JSON - interpret as a number
hello_world.people[1]:[email protected]                         # Load a JSON file
hello_world.people[2]:='{"name": "Katherine", "age": 53}'     # Literal JSON - interpret as an object
[email protected]                              # Load RAW File contents

If we were to pass these values into the Workbench CLI as workflow params, we would expect the resulting object to look like the following.

{
  "hello_world.greeting": "Hello",
  "hello_world.people": [
    {
      "name": "Paulette",
      "age": 42
    },
    {
      "name": "Heinrich",
      "age": "37",
    },
    {
      "name": "Katherine",
      "age": 53
    } 
  ],
  "hello_world.goodbye": "Goodbye!"
}