Migrate from AWS S3 to chili

This guide moves an existing bucket from AWS S3 (or any other S3-compatible source) to chili with rclone, in under 30 minutes. You configure a source and a target remote, copy with verification, run a checksum check, and optionally lock the target bucket. The closing read test is free, because chili charges no egress.

Prerequisites

• rclone installed (rclone version); use a recent release
• credentials for the source: AWS S3 access key and secret, plus the source region
• chili access keys (access key and secret) and a target bucket, or permission to create one
• on AWS, read permission on the source bucket and its objects; on chili, write permission on the target bucket

1. Configure the source remote (AWS S3)

Run rclone config and add the source as an s3 remote with provider = AWS. rclone uses provider = AWS to designate the storage provider as Amazon Web Services, as described in the rclone S3 documentation. The config lands in ~/.config/rclone/rclone.conf:

[aws-s3]
type = s3
provider = AWS
access_key_id = AKIA...
secret_access_key = ...
region = <source-region>

2. Configure the target remote (chili)

Add chili as a separate s3 remote with the generic provider and the chili endpoint:

[chili]
type = s3
provider = Other
access_key_id = AKIA...
secret_access_key = ...
endpoint = https://<ENDPOINT>

A region or location constraint, if your setup needs one, comes from the chili portal, not from rclone. Leave it out unless the portal tells you otherwise.

Test both connections:

rclone lsd aws-s3:
rclone lsd chili:

3. Dry run

Always preview the transfer before moving data. --dry-run lists the planned transfers without copying anything:

rclone copy aws-s3:SOURCE-BUCKET chili:TARGET-BUCKET --dry-run

4. Copy the data

rclone copy transfers only new and changed files and never deletes at the target, as described in the rclone copy documentation. That makes it the safe choice for a migration:

rclone copy aws-s3:SOURCE-BUCKET chili:TARGET-BUCKET \
  --checksum --transfers 8 --fast-list --progress --s3-no-check-bucket

A note on what happens under the hood: AWS S3 and chili are different providers behind different endpoints, so this is a client-side transfer (download from AWS, upload to chili), not a server-side copy. Per the rclone S3 documentation, server-side copy requires the same remote for source and target and the same region, so it does not apply here.

The flags:

• --checksum compares files by hash and size instead of modification time and size. When both ends are S3, the rclone S3 documentation recommends this flag for maximum efficiency.
• --transfers sets how many transfers run in parallel; the default is 4. The 8 above is only an example; tune it to your bandwidth and any rate limits.
• --fast-list uses a recursive listing, which means fewer transactions at the cost of more memory. Useful on buckets with many objects.
• --progress (-P) shows progress during the transfer.
• --s3-no-check-bucket tells rclone not to check that the bucket exists or to create it. Use it when the target bucket already exists or the credentials cannot create or list buckets.

If you instead want the target to match the source exactly, including deleting objects that no longer exist at the source, use rclone sync. Only do this when deleting at the target is intended, and run it with --dry-run first.

5. Verify the migration

rclone check compares both sides by size and hash (MD5 or SHA1) and reports any files that do not match, as described in the rclone check documentation:

rclone check aws-s3:SOURCE-BUCKET chili:TARGET-BUCKET --checksum

If no common hash is available, or you want to verify the full content, add --download. This downloads both remotes and compares them on the fly:

rclone check aws-s3:SOURCE-BUCKET chili:TARGET-BUCKET --download

Use rclone cryptcheck only if the target is a crypt (encrypted) remote; it is the equivalent of rclone check for encrypted remotes. For a plain S3-to-S3 migration without a crypt layer, stay with rclone check.

6. Optional: lock the target bucket

Object Lock (WORM) on chili is set server-side on the bucket with a default retention, not by the client. Create the target bucket with Object Lock enabled before the copy, and every object you migrate inherits the retention automatically. That keeps the migrated data immutable.

Verify

Read one object back from chili. Because chili charges no egress, this restore test costs nothing:

rclone cat chili:TARGET-BUCKET/path/to/object | head -c 64
rclone ls chili:TARGET-BUCKET | head

If the object content prints and the listing matches what you migrated, the data is in chili and readable. A clean rclone check run from step 5 with no reported differences confirms every object transferred intact.

Common pitfalls

• The endpoint from the chili portal must include the scheme (https://), not a bare host.
• Keep the source and target as two separate remotes. Mixing them into one remote does not enable server-side copy across providers.
• A region or location constraint on chili comes from the portal; do not invent one. If in doubt, check the chili portal or documentation.
• --s3-no-check-bucket is only needed when the bucket already exists or the credentials cannot create or list buckets; without it, rclone tries to verify or create the bucket.
• For very large objects, the multipart upload cutoff and chunk size matter; check the current defaults under Advanced options in the rclone S3 documentation before tuning them.

The result: your bucket lives in Switzerland, in Zurich or Geneva and geo-redundant on request, with no egress fees and 11 nines of durability, and the read test that proves it cost nothing.