S3 Transfer

Upload a file to S3, or download one.

Read Shared action rules for disabling this action, outputs, error routing, and recovery. Also read File sources and destinations.

Configuration and behavior

Direction picks the job: Upload (the default) or Download. The step shows only the fields that job uses.

Shared parameters: Credential * (type S3), Direction, Endpoint * (default s3.amazonaws.com; host:port for S3-compatible stores; an http:// or https:// prefix also sets TLS), Region * (lists the AWS regions by code and place, filtered as you type; any other value typed, such as a MinIO or Wasabi region, is saved as typed), Use TLS (on), Bucket * (picker lists buckets the credential can see).

Upload - Plus Source file (${item.path}), Object key (blank uses the file's name; a key ending in /, such as incoming/, keeps the file's name in that folder; any other key is the full object name, e.g. incoming/${item.basename}${item.ext}), Chunk size (MiB) (default 64). The editor warns when a typed key starts with /, or has no / at the end and no extension, since S3 then makes a folder named / or one object with that name. Outputs: bucket, key, etag; Available data lists only these on an upload. Uploads always overwrite an existing object.

Files below 100 mebibytes (MiB) use one upload request. Files of 100 MiB or more use multipart upload, including files larger than 5 gibibytes (GiB). Multipart sends the file in chunks. The completed object becomes available after all chunks have arrived and S3 confirms completion.

Both directions show file transfer progress and recent speed in job detail. Multipart upload progress combines the concurrent parts without counting resent bytes twice toward completion. Finishing on upload waits for S3 to confirm completion. On download, it covers the final disk save and publication. See File transfer progress.

Chunk size (MiB) accepts whole numbers from 5 to 5120, with no variables. One MiB is 1,048,576 bytes. Start with 64. Smaller chunks reduce the amount resent after a failed chunk. Larger chunks reduce the number of requests. Relay increases the chunk size when necessary to stay within 10,000 parts. The last chunk can be smaller. Relay sends up to four chunks at once and reads them directly from the source file. Keep the source unchanged until the step finishes. Your S3-compatible service can impose a lower object-size limit.

The etag output is the object's entity tag (ETag). A multipart ETag is not a whole-file checksum. Use a separate file check when you need to verify the complete contents.

After failure or cancellation, Relay attempts to remove this attempt's unfinished upload. Cleanup has a separate 30-second budget. If cleanup fails, Relay keeps the upload ID on this node. The next multipart attempt to the same destination first removes its recorded abandoned uploads. It uses the current secrets from each attempt's saved credential reference. If that credential was deleted, Relay uses the step's current credential instead. It does not resume uploaded chunks. Another active multipart attempt to that same destination on this node fails with a busy error.

Unfinished parts can incur storage charges. Configure a seven-day bucket cleanup rule, with a longer period for uploads that can take longer. Relay does not change bucket policies. A lost start response can leave an upload that Relay cannot identify. A lost completion response can mean the object already exists even though the step failed. Inspect the destination before retrying an uncertain result. Retrying sends the file again and can create another version in a versioned bucket. See Set up S3 multipart uploads for permissions, cleanup, and verification.

Download - Plus Object key *, Destination folder * (must exist), File name, Overwrite (off). Output: path; the job now carries the downloaded file.

Sizes and storage class. Both directions also output source_bytes, transferred_bytes, and storage_class. Compare ${steps.<id>.source_bytes} with ${steps.<id>.transferred_bytes} in a Switch to confirm that the whole file arrived.

Output	Upload	Download
`source_bytes`	Size of the source file on disk when the upload starts.	Size of the S3 object.
`transferred_bytes`	Size of the stored object, as S3 reports it after the upload.	Bytes written to the downloaded file.
`storage_class`	Storage class of the stored object, as S3 reports it after the upload, for example `STANDARD`.	Storage class of the S3 object, for example `GLACIER`.

After an upload, Relay reads the stored object back from S3. This needs the s3:GetObject permission, and also s3:GetObjectVersion on a versioned bucket. If S3 refuses the read, the upload still succeeds, transferred_bytes and storage_class are empty, and the service log records the reason. A bucket rule that moves objects to another class acts later, so an upload normally reports STANDARD.

Failures. A failed transfer takes the step's Error output, so the workflow can send a notification. ${error.kind} names the failure:

Kind	Meaning
`configuration`	A step setting is not valid, or the credential is missing, of the wrong type, or incomplete.
`local_file`	The upload source is missing or cannot be read, or the download destination exists or cannot be written.
`permission`	S3 refused the credential or the request, for example `AccessDenied`.
`not_found`	The bucket or the object does not exist.
`busy`	Another multipart upload to the same destination is in progress on this node.
`transport`	Relay could not reach S3. No object was stored by this attempt.
`service`	S3 replied with another error.
`outcome_unknown`	An upload whose result Relay cannot confirm, such as a lost reply or a failed multipart completion. The object can exist. Check the bucket before you retry.
`expression`	A variable in a step setting has no value in this job, so the transfer did not start.

Error values, for every kind: ${error.s3.direction} (upload or download), ${error.s3.bucket}, ${error.s3.key}, ${error.s3.source} and ${error.s3.destination} (a local path or s3://bucket/key), and ${error.s3.endpoint}. On an upload, ${error.s3.key} is the full object key, with the file name added when Object key is blank or ends in /. On an expression failure, a setting that has no value keeps its ${…} text, and an upload from the job's file reports an empty source and the key as typed. The step's own outputs, such as ${steps.<id>.bucket}, exist only after a successful transfer. Use the error.s3 values on the Error branch. A node fault, such as an unavailable credential store, still stops the job without the Error output.