All chapters
On this page

Chapter 16 / Help and reference

16. Reference

16.1 Command line

The relay binary is /usr/local/bin/relay (macOS), /usr/bin/relay (Ubuntu), or relay-daemon.exe in C:\Program Files\North Shore Automation\Relay\ (Windows). Service-affecting verbs need sudo.

Run relay with no arguments to display normal help. relay help, relay --help, relay -help, and relay -h display the same help. To read about one command, use relay help COMMAND or relay COMMAND --help. This is also safe for an internal helper when its command name is known. relay --version prints the product name and version. relay version keeps the plain version output used by automation.

Normal help lists the operator commands below. relay help --all also lists internal helpers. Internal helpers do not appear in normal help.

Command Purpose
relay version / relay --version Print the version.
relay status [--data-dir] JSON: service_state (running/stopped/installing/updating), license_state (active/grace/expired/inactive/unknown), activation_ref, version, service_last_exit_code, telemetry_state (on/off/stalled/unknown), telemetry_last_sent. on means the node is switched on and has somewhere to send to. off means the customer switched it off, and nothing else ever produces that word, so a node reported as off was chosen, not broken. stalled means the node is switched on but has no address to send to, so it is sending nothing; it says nothing about why. unknown means Relay cannot say: the service is stopped, or the node's stored switch could not be read, or the daemon answered with a word this version of relay does not know.
relay install [--license-code CODE] --data-dir DIR --service-user USER Register the service and store the license code. Prefer RELAY_LICENSE_CODE in the environment; the command-line option can expose the code in history and process listings.
relay deactivate [--data-dir] Release the license seat; service and data stay.
relay update [--to VERSION] [--apply] [--catalog-url URL] Check for, or apply, an update from the catalog. Needs CATALOG_BASE_URL in the environment or --catalog-url, or it reports that no catalog is configured and does nothing. Not available for Ubuntu, which is not in the catalog.
relay recover [--data-dir] [--exe-path] Restore the previous binary and data snapshot after a bad update.
relay uninstall [--purge] [--data-dir] Release the seat and unregister the service; --purge also deletes the data directory.
relay webhook addr / addr set HOST:PORT / addr --reset Show, set, or reset the webhook listen address.
relay support-bundle [--data-dir DIR] [--out PATH] Write a diagnostic bundle.
relay telemetry [on|off] [--data-dir DIR] Show whether this node is reporting. Use on or off to change the reporting switch on the running node. stalled means it is switched on but has no address to send to.
relay start Ask the registered platform service manager to start the Relay service.

Exit codes: 0 OK; 1 update or recover failed; 10 contract mismatch; 11 not installed; 12 activation failed; 13 busy; 64 usage.

Service integration

These commands support the installed Relay service and platform integration.

Command Purpose
relay serve Run the service in the foreground (used by launchd/systemd; not for interactive use).
relay service-status / relay service-start / relay service-stop / relay service-restart Control the background service. JSON result with state and outcome. Not supported on Windows: manage that service through Windows Services.

How long a service stop can take. Most stops end within a few seconds. A stop can take up to one minute, because Relay finishes every write it has started before it exits. Relay's service settings give it that minute: on macOS and Ubuntu the service manager waits 60 seconds before it forces Relay to quit, and on Windows Relay gets 60 seconds at a machine shutdown. The installer writes these settings each time it installs or updates Relay. Do not force-quit Relay during a stop.

Stopping the service with a managed Iconik connection. Available in the internal managed Iconik release. Older installers and evaluation builds do not offer this feature. While the managed Iconik connection is active, a service stop first finishes each Iconik click Relay is confirming with the job queue. This can add up to 20 seconds. The stop then hands the job queue back to the cloud service, so that the next start can take it at once. This adds up to 10 seconds more. If the cloud service does not answer in that time, Relay stops anyway, and the next start waits up to about 16.5 minutes before the job queue resumes (15.14). If Relay stops during connection setup, the next start continues the same setup request (15.13).

16.2 File locations

See chapter 2.7 for the per-platform tables. Inside the data directory: relay.json (configuration, including the stored license code), identity.json (service account), relay-node.lock (the process-lifetime guard that prevents two Relay daemons from using one data directory), store/relay.db (workflows, settings, job history, protected Test evidence, and encrypted pending-request recovery state), store/request-bodies/ (encrypted request bodies retained only for pending recovery), credentials.sealed (sealed secrets), credstore.key on macOS and Ubuntu or credstore.dpapi on Windows (the matching key that seals secrets and protects pending recovery), license-grace.json (offline grace state and the last known workflow limit of a Limited license), logs/relay.log, run/relay.sock (editor connection, macOS/Ubuntu), update-probation (present only during an update), and any .bak snapshot a data upgrade wrote (chapter 13.6). A previous Relay version may have left intake/ here. Current Relay neither reads nor creates that folder. You may delete it yourself only after confirming it is no longer needed. The data directory is sensitive. It is owned by the Relay service account and readable by members of that account's group, which on most nodes holds nobody else; it is not readable by everyone on the machine. Do not edit the journal, delete the node lock, delete recovery rows or request bodies, or separate them from the matching key file. Do not copy or share the data directory as a diagnostic bundle.

The managed Iconik permission check also uses store/relay.db.iconik-action-probe.lock to prevent overlapping checks. The empty lock file can remain after a check finishes. Do not delete it while Relay is running. The internal managed Iconik release offers the guided connection on Settings > Iconik (12.10).

Workflow folders, folder membership, and Created and Modified dates also live in store/relay.db. All consoles connected to that node share them. Backups, workflow exports, and support bundles do not include this organization or its dates. Imported workflows receive dates on the destination node. Workflows imported from the Workflows page go into the selected folder, or to the top level. Workflows imported from Settings > Backups go to the top level. Sort choices stay with each console and node (4.7).

Connect Iconik (internal managed Iconik release; 12.10) keeps its progress in the iconik.setup_journal setting in store/relay.db. Relay clears it when the account is linked. It holds Iconik item IDs, names, and times, never a token, the approval link, the account email, or the tenant name. Customer backups and support bundles do not carry it. Do not edit it.

Two more Iconik settings (internal managed Iconik release) live in store/relay.db. Do not edit them. Backups do not carry either one.

  • iconik.sample names the Relay Connection Test Iconik Job and its workflow, and keeps the proof of the current cloud connection: when the proof was armed, the I ran it time, the run that proved it, and why Relay could not create the test job, if it could not. It holds IDs, times, and a reason code only. Support bundles keep it.
  • iconik.connection_display keeps the tenant name and the account email of the linked Iconik account, for the card on Settings > Iconik. Each Reconnect replaces it. Support bundles leave it out.

S3 multipart recovery records also live in store/relay.db. They store destination details, upload IDs, and credential references, without file contents or credential secrets. Removing job history does not remove these records. Each destination uses a lock file named store/relay.db.s3-multipart-<destination hash>.lock. An empty lock file can remain after a transfer ends. Do not delete these files while Relay is running or edit recovery records. Keep the data directory and referenced credentials available for cleanup.

If startup reports relay node is already running for this data directory, do not remove the lock file. Stop the other Relay service or process that uses the same data directory, confirm it has exited, then start Relay again.

One pending request can retain at most 64 MiB of non-body recovery state. All pending recovery state and encrypted request bodies share a 1 GiB node-wide private storage budget, in addition to each action's normal body and value limits. Relay refuses work that cannot reserve the required space before its first send. It does not evict waiting jobs. Terminal or cancelled work releases its references, and startup removes only unreferenced request-body files.

On Windows, an update or recovery can leave .update-probation.relay-delete-<random identifier> if cleanup cannot remove it. This small file records a completed removal of the update marker. Relay excludes it from snapshots; it does not contain workflow media.

16.3 Network

Direction What Default
Outbound License validation Cryptlex licensing service, HTTPS, about hourly. Ask NSA support for the current firewall allowlist.
Outbound Release catalog and downloads releases.northshoreautomation.com, HTTPS
Outbound Telemetry North Shore Automation's monitoring ingress, HTTPS, hourly. The address comes from the licensing system; ask NSA support for the current firewall allowlist. Blocking it costs nothing: the node keeps running workflows (chapter 12.6).
Outbound SQS polling the queue's regional SQS endpoint, HTTPS
Outbound Managed Iconik connection (internal managed Iconik release) api.controlroom.media, HTTPS port 443, during setup, to keep the connection, and to register each custom action that Relay manages in Iconik, and the managed queue's regional SQS endpoint, HTTPS. Relay takes the proxy for these calls from the node's environment, as for native Iconik actions.
Outbound Native Iconik actions, S3 Transfer, SCP transfer, HTTP Request as configured per action or workflow connection. Native Iconik actions share one pool of open connections and use HTTP/2 when Iconik offers it. Relay takes the proxy it uses for Iconik from the node's environment, so set HTTPS_PROXY and NO_PROXY for the Relay service if Iconik is reached through a proxy.
Outbound CatDV actions and the CatDV credential test the server address in the CatDV credential, HTTPS, or HTTP where the server offers only that (10.7)
Outbound Iconik custom actions (internal managed Iconik release) https://app.iconik.io, HTTPS port 443, to create, change, and remove the custom actions that Relay manages in Iconik. Relay uses the node's Iconik connection and the same proxy settings as native Iconik actions.
Outbound Connect Iconik (internal managed Iconik release) https://app.iconik.io, HTTPS port 443, from the node that you connect. Relay asks Iconik once a second whether you approved, for up to 15 minutes, then sets up the application and token. Your browser opens the approval page on app.iconik.io from your own computer.
Outbound Iconik domain logo (internal managed Iconik release) https://app.iconik.io, HTTPS port 443: GET /API/settings/v1/system/current/ with the managed Iconik connection, then one download of the logo from the address that Iconik gives, over HTTPS only and with no Iconik key, when the Iconik settings page opens or the connection changes. The address and all redirects must resolve only to public servers; private and reserved network addresses are refused. Relay accepts only a PNG or JPEG image of at most 256 KiB, and each request times out after 10 seconds. A logo is kept in memory for 1 hour; no logo or a failure for 5 minutes. A changed or removed connection clears it. Relay takes the proxy from the node's environment, as for native Iconik actions.
Outbound Iconik credential test https://app.iconik.io, HTTPS port 443, one read-only request per test, through the same proxy settings as native Iconik actions (10.8)
Outbound Started by name read (internal managed Iconik release) https://app.iconik.io, HTTPS port 443: GET /API/users/v1/users/<user ID>/ after each recorded Iconik Job click, with the managed Iconik connection. A found name is kept in memory for 10 minutes, a failure for 1 minute, so repeated clicks by one user cost one read. At most 2 reads run at a time. Each read times out after 5 seconds. A run waits for the name for at most 2 seconds. After a 401, one more read of /API/users/v1/users/current/ checks whether Iconik refuses the key itself (8.10).
Inbound Webhooks 127.0.0.1:8787; set in Settings → Webhooks
Inbound Remote node management SSH, port 22, from the managing editor

Cloud outages with a managed Iconik connection. Available in the internal managed Iconik release. Older installers and evaluation builds do not offer this feature. Relay holds a cloud session for the managed job queue that lasts one hour and is renewed while the cloud service answers. If the cloud service stops answering, Relay keeps taking jobs for the rest of the current session: never less than 45 minutes, and at most about one hour. It then stops taking jobs until the cloud service answers, and resumes on its own (15.14). A restart during an outage starts with no session, so Relay takes no jobs until the cloud service answers.

Managed job queue calls. Available in the internal managed Iconik release. Older installers and evaluation builds do not offer this feature. Relay reads Iconik clicks from the managed job queue at the queue's regional SQS endpoint over HTTPS, with the credentials of its current cloud session. It makes four kinds of call: it asks for up to 10 clicks at a time and waits up to 20 seconds for one to arrive; it renews a click's hidden time every 45 seconds while it works on the click; it gives a click back to the queue; and it removes a click after it saved the job. Relay asks for clicks only when it has room for them (15.18). It does not use AWS settings from the environment or from profile files for these calls.

Iconik custom action calls and request pace. Available in the internal managed Iconik release. Older installers and evaluation builds do not offer this feature. Relay makes outbound HTTPS calls to app.iconik.io to manage custom actions in Iconik. Relay sends one request at a time for the whole node. It sends at most 2 requests each second. Each request times out after 15 seconds. When Iconik reports fewer than 100 requests remaining, Relay waits 20 seconds before the next request. When Iconik answers 429 (too many requests), Relay waits for the time that Iconik gives, up to 5 minutes. If Iconik gives no time, Relay picks a random wait between half of a limit and the full limit. The first limit is 5 seconds, so the first wait is 2.5 to 5 seconds. The limit doubles after each repeat, up to 60 seconds, so the longest waits are 30 to 60 seconds. Relay uses the same wait after a server error, a network failure, or an answer that does not arrive in full. Relay shares the Iconik token, and the Iconik rate limit, with Iconik workflow steps on the same node. A 429 seen by a workflow step also slows custom action calls, and the other way round. Because of this pace, a large set of changes can take several minutes to appear in Iconik. Relay calls Iconik for custom actions only when an Iconik Job or the destination label changes, again while a change is not finished (first after 15 seconds, then at longer gaps of up to 5 minutes), and for the checks after a create that Relay could not confirm (1 minute, 10 minutes, 1 hour, and 24 hours later). It never polls Iconik for changes that administrators make.

16.4 Job statuses

Queued · Running · Waiting to retry · Succeeded · Completed with warnings · Failed · Cancelled · Interrupted · Blocked · Indeterminate. Chapter 11.1.

16.5 License states and displayed statuses

Canonical states: Active · Grace · Expired · Inactive · Unknown. Shown on Settings → About under License. The editor can derive the displayed statuses Expiring and Expiration passed from the license dates. Chapter 13.2.

Limited license (chapter 13.8): the license can cap how many workflows are on at one time. A license without a cap has no limit. A cap value that is not a whole number of 1 or more makes the state Inactive, with the reason code entitlement-invalid and the reason The license has an invalid workflow limit. Contact your reseller.

16.6 Node types at a glance

Kind Nodes
Triggers Watch folder · Schedule · Manual · Webhook · SQS Queue · Start Workflow
Triggers - internal managed Iconik release Iconik Job (16.10). Older installers and evaluation builds do not offer this trigger.
Actions - file Copy file · Move file · Rename file · Delete file · Extract file metadata
Actions - transfer S3 Transfer · SCP transfer
Actions - CatDV CatDV Get Clip · CatDV Update Clip
Actions - Iconik Find Asset · Get Asset · Create Asset · Update Asset · Delete Asset · Add Asset to Collection · Remove Asset from Collection · DLR Lookup (exits Found, No match, Error; Update a clip is off by default; No match must be connected). Find Asset and Get Asset accept assets with no custom metadata; no custom fields appear for those assets.
Actions - utility Log message · Set variables · Extract JSON values · Script
Actions - system HTTP Request (None or Basic authentication) · Run shell command
Actions - flow Start Workflow
Control Switch · For Each
Terminals Complete · Error

Action state: regular actions can be enabled or disabled with their power button or Step actions menu. Disabled actions keep their settings and connections, pass input unchanged through Success, and create no outputs. A reached disabled action shows Skipped: disabled in Test and job detail. Triggers, controls, and terminals cannot be disabled this way. The selected node must support disabled actions. See 4.2 and chapter 6.

S3 upload limits: on an S3 step set to Upload, files below 100 MiB use one request; files of 100 MiB or more use multipart upload. Chunk size (MiB) defaults to 64 and accepts whole numbers from 5 to 5120. Relay increases the effective size to stay within 10,000 parts, with up to four part requests at once. The last part can be smaller. Endpoint limits also apply. Failure cleanup has a separate 30-second budget. Downloads do not use the chunk setting. See S3 Transfer and the multipart setup guide.

Variable processing: variable-capable text fields offer Regex and Treat as filename. The Regex flask tests pasted text on the selected node, through the existing editor connection and SSH bridge, with no new port. Limits are 64 KiB input, 16 KiB regex, and 256 KiB output in UTF-8 bytes. Samples remain temporary. See chapter 8.3 for syntax, filename parts, and errors.

16.7 Keyboard shortcuts

Command-K search · Command-Shift-K action palette · Command-S save · Command-Z / Shift-Command-Z undo / redo · Command-0 to Command-4 select All nodes or a machine · drag, middle-drag, or Space-drag pan · Shift-drag select · Delete removes the selection · Esc close · ? shortcut list. The list names Command on a Mac and Ctrl elsewhere. Chapter 4.8.

16.8 Script limits

Script uses profile relay-script/es2022-v2. Each execution has a 256 KiB source limit, 10 MiB input and result limits, a 128 MiB JavaScript heap limit, a 256 MiB worker process limit, and a 1 GiB total Script-worker limit on the node. Timeout seconds defaults to 30 and accepts 1 to 600. Response limit bytes defaults to 10 MiB and accepts 1 KiB to 32 MiB. Four workers are available, with one reserved for editor checks, so no more than three live Script executions run together. Within each Script, at most four HTTP requests can be pending. Their response limits total at most 32 MiB, and their request data totals at most 10 MiB. The default response setting allows three pending requests; the maximum allows one.

16.9 Glossary

Action - a step that does work. Available data - the panel listing every value a step can use, including nested fields observed in structured outputs. Carried file - ${item.path}, the file a job is working on. CatDV connection - the one CatDV credential a workflow selects in its header and lends to every CatDV step in it. CatDV seat - one CatDV license sign-in; a job holds one seat from its first CatDV step until the job ends. Complete / Error - terminals that end a job succeeded or failed. Control Room - NSA's free desktop app that installs, licenses, updates, and opens Relay. Credential - a sealed secret used by actions. Draft - a saved workflow that is not valid to run in production. Error handling - the canvas that runs when an action fails under the workflow default policy. Error output - an action's optional Error port. Executable - a saved workflow that passes validation. External queue - an SQS connection this node polls. For Each - a control that sends every item in one JSON array through a repeated path before continuing through Done. Hidden value - a Workflow Test value omitted by evidence protection; the step gives the reason. Older Tests can retain omissions from earlier protection rules. Job - one production run or accepted Workflow Test. JSON-bearing output - a declared output that may hold ordinary text or one complete JSON object or array; recorded structured values expose nested fields. Named CatDV value - one addressable field of a clip CatDV Get Clip read, inserted from Available data under Built-in fields or User-defined fields. Named Iconik value - one addressable field of an Iconik result inserted from Available data; basic Get and Find results contain Asset and Metadata, while the additional-details profile also contains Files and Other details. A DLR Lookup's Matched DLR contains Asset and Metadata only. Node - one machine running the Relay service. Node bench - the panel below the canvas for configuring a step and viewing Test evidence. Replay input - one of the five newest protected trigger inputs retained for another Workflow Test. Retry window - how long Relay keeps retrying an action after a temporary failure, counted from the first request and including its retry waits, but not initial admission queue time. Script connection - a named saved credential available to Script as public fields and protected request-only handles. Script profile - the released Script runtime a script was authored against; a node running a different profile cannot activate that workflow. Seat - one activation of a license code. Source screening - the check that refuses credential-looking literals in Script source. Slug - a step's identifier, steps.<slug>. Tenant guard - the Iconik System ID check on queue messages. Trigger - what starts a job. Waiting to retry - a live job whose saved request is waiting for backoff or recovery capacity without using an active processing slot. Workflow Test - one real run of the current draft, marked Test and excluded from production Stats. Workflow errors - the source node of the Error handling canvas.

16.10 Iconik Jobs

Available in the internal managed Iconik release. Older installers and evaluation builds do not offer this feature.

Trigger: Iconik Job (chapter 5.8), with ${trigger.iconik.job_type}, ${trigger.iconik.context}, ${trigger.iconik.payload}, and ${trigger.iconik.user_name} (8.10). Not-started errors: NOT_STARTED (Unsupported payload size), NOT_STARTED (Iconik Job deleted), NOT_STARTED (Iconik Job disabled), NOT_STARTED (No workflow attached), and NOT_STARTED (Workflow not runnable): <detail> (9.7, 15.15). From the managed Iconik connection also: NOT_STARTED (Unknown Iconik Job), NOT_STARTED (Request rejected): <reason>, NOT_STARTED (Relay busy), NOT_STARTED (Click not confirmed in time), and the Blocked job SKIPPED (Duplicate click): same click as job <first job> (11.7, 15.15). Custom action problems and waiting notes: 15.16. Iconik Job names: 1 to 100 characters, unique on the node ignoring case, including accented and other non-English letters (Café and CAFÉ are the same name). Job type keys: lowercase letters, digits, and hyphens, 1 to 64 characters, unique. Destination label: 0 to 40 characters (12.9). Request body limit: 256 KiB.

Managed queue intake limits. These values are fixed. They are not settings.

Limit Value
Request body Up to 256 KiB (262,144 bytes). A larger request gets NOT_STARTED (Unsupported payload size).
New clicks Relay works on at a time 5
Clicks asked for in one call Up to 10, with a wait of up to 20 seconds
Time a click is hidden from the queue while Relay works on it 120 seconds, renewed every 45 seconds, at most 15 minutes for each hand-out
Retries while a workflow's waiting line is full After 5, 10, and 20 seconds, then every 30 seconds
Hold before Relay gives a busy click back to the queue About 13.5 minutes
NOT_STARTED (Relay busy) On the click's fourth hand-out, about one hour after the first
Duplicate click window 5 minutes, fixed, opened only by a click that queued a job
Copies of one click recognized At least 14 days
Cloud queue keeps a click 7 days
Hand-outs before the cloud moves a click to the dead-letter queue 5
NOT_STARTED (Click not confirmed in time) 8 days after Relay saved the click

Relay Connection Test. The test job proves the cloud connection (12.10, and the managed Iconik connection guide). After I ran it, Relay waits 2 minutes for the test run, then shows that it did not see the click (15.21). The timer counts from the stored I ran it time, so it survives a restart.

Service log events for Iconik Job starts. The service log (15.10) records the Iconik user for every Iconik Job start. Lines hold no email.

Event code When Fields
iconik_job_started A run of an Iconik Job starts, or a Workflow Test with an Iconik click starts. run_id, route_id, iconik_job_name, iconik_user_id, iconik_user_name
iconik_job_not_started A click becomes a job that did not start, or a skipped job. as above, plus reason
iconik_job_attribution_updated The name arrived after the run started. run_id, route_id, iconik_user_id, iconik_user_name
iconik_user_lookup_failed Relay could not read the name. run_id, iconik_user_id, reason_class: timeout, not_found, forbidden, or unavailable

When the name is not available, iconik_user_name is empty and the line adds iconik_user_name_unavailable=true. Each job's Download log file (11.4) also has an Attribution block: Started by <name> (<user ID>), or Started by <user ID> (name not available), and a second line when the name arrived after the run started.

Iconik Jobs and their click records live in store/relay.db, which this release upgrades to run journal version 23 (13.6). Backups and workflow exports do not include them. The payload is treated like other trigger data. It holds Iconik IDs and any values the Iconik user typed into the custom action's metadata form. It can appear in support bundles, downloaded logs, and backups. Relay removes credentials and secrets as it does for every job. Support bundles also keep the Iconik user ID and the name of the person who started each job. Backups and workflow exports carry no names. They can keep Iconik user IDs.