Moving from another cloud, with no downtime, step by step

1How it works

1. You give the bucket a source: your old storage and a read-only key for it. plycdn never writes to it and never deletes anything there.
2. Check the source. plycdn tries the key, lists the first files and tells you which names cannot move (step 2).
3. Start serving. From now on a file that is asked for and is not in plycdn yet is read from your old storage, sent to the viewer and kept in the bucket at the same path. The next viewer gets it from plycdn.
4. Add your address to the bucket. Its certificate is made before you change anything, so there is no gap (step 4).
5. Switch your DNS record. Your address now reaches plycdn. Nobody notices.
6. The rest of the files are copied in the background, starting as soon as you choose Start serving. This runs before you switch your address too, so most files are already in plycdn on the day you switch.
7. You move your uploads to plycdn, mark them as moved, and later switch off old links and close the move.

You can go back at any point before you close: point your DNS record back at your old service.

2Step by step

1. Give plycdn read-only access to your old storage

Create a key that can read and list and nothing else. plycdn refuses anything broader.

• Azure Blob Storage: a container token (shared access signature) for the one container, with only the read and list permissions, and an expiry date after the day you plan to close the move. A token that can also write, delete or create, or that covers the whole storage account, is refused with migration_credential_too_broad.
• Amazon S3 or another S3-compatible store: an access key and secret for a user that can only read and list that bucket, and, if your store needs it, the bucket's region.
• A plain web address: no key. A web address has no list of files, so files move as they are opened, and the ones you upload to plycdn from now on. Use this when you cannot give access to a listing. Such a move never completes by itself, because plycdn cannot know every file was asked for: files nobody opened are not moved. Upload those yourself, then close the move with confirmation (step 9).

Keep the key to yourself: plycdn stores it sealed, never shows it again and deletes it when you close the move.

2. Set up the move and check it

In the dashboard: Storage → Storage buckets → open the bucket → Move from another cloud.

1. Choose Azure Blob Storage, S3-compatible or Web address, and fill in the address of the storage (https://youraccount.blob.core.windows.net), the container or bucket, and the folder in it that your addresses start from (leave it empty if the whole container maps to the bucket).
2. Paste the key. For an Azure source, turn on Accept old links if your application hands out Azure-signed links today (step 6). If the container name appears in those links' paths, turn on Container name is part of the link. Old links need a private bucket with plycdn links (otherwise old_links_need_private_bucket) and the storage address in its usual form, https://youraccount.blob.core.windows.net (otherwise migration_source_unreachable).
3. Choose Check. The check tries the key, lists the first page of files and shows a sample. It also lists the names that cannot move: names with characters plycdn's file names do not allow (spaces, for instance) or that start with plycdn's reserved prefix. Those files keep being served from your old storage until you rename them there; they are not lost.
4. When the check is good, choose Start serving. The background copy starts at once. Your site keeps using your old storage until you switch your address (step 5), so you can start serving days ahead and let the copy run.

From your code (admin key; organizationId and bucketId are shown with Copy in Storage → Storage buckets):

ASP.NET Core

var move = await api.StartMigrationAsync(orgId, bucketId, new MigrationCreate(
    Kind: "azure_blob",
    Endpoint: "https://youraccount.blob.core.windows.net",
    Container: "contentprod",
    Credential: Environment.GetEnvironmentVariable("OLD_CLOUD_TOKEN"),
    LegacyLinks: "azure_sas"));

MigrationCheckResult check = await api.CheckMigrationAsync(orgId, bucketId);
foreach (var name in check.UnmovableSample)
    Console.WriteLine(quot;{name.Key}: {name.Reason}");          // rename these in your old storage

await api.CheckMigrationAsync(orgId, bucketId, start: true);  // a good check also starts serving

Node.js

await storage.startMigration(orgId, bucketId, {
  kind: 'azure_blob',
  endpoint: 'https://youraccount.blob.core.windows.net',
  container: 'contentprod',
  credential: process.env.OLD_CLOUD_TOKEN!,
  legacyLinks: 'azure_sas',
});

const check = await storage.checkMigration(orgId, bucketId);
for (const name of check.unmovableSample) console.log(name.key, name.reason);   // rename these in your old storage

await storage.checkMigration(orgId, bucketId, { start: true });                 // a good check also starts serving

Python

storage.start_migration(org_id, bucket_id, {
    "kind": "azure_blob",
    "endpoint": "https://youraccount.blob.core.windows.net",
    "container": "contentprod",
    "credential": os.environ["OLD_CLOUD_TOKEN"],
    "legacyLinks": "azure_sas",
})

check = storage.check_migration(org_id, bucket_id)
for name in check["unmovableSample"]:
    print(name["key"], name["reason"])          # rename these in your old storage

storage.check_migration(org_id, bucket_id, start=True)   # a good check also starts serving

For an S3-compatible source the credential is a JSON text: {"accessKeyId": "...", "secretAccessKey": "...", "region": "eu-west-1"} (region is optional), and container is the bucket's name. For a web address leave credential and container out.

Before you go on, open one of your files at its plycdn address (https://<bucket>.plycdn.net/<path>): it must load. The first request reads it from your old storage; the second comes from plycdn.

3. Lower the TTL a day ahead

At your DNS host, lower the time-to-live (TTL) of the record you will change (your address's CNAME) to a few minutes, a day before you switch. That way the switch, and a switch back if you need one, take minutes instead of hours.

4. Add your address and create the records

In the dashboard: Delivery → Custom domains → Add domain, choose the bucket and type the address. When the bucket is moving and is a private bucket with plycdn links, the domain's page shows three records instead of two:

Create the TXT and the _acme-challenge records now. The certificate is issued while your address still points at your old service. Without the _acme-challenge record plycdn gives up on the certificate after 24 hours (certificate_failed). With it, the domain may sit in issuing certificate until you switch, and gives up after 72 hours, so switch within three days of adding it (or add the domain on the day you switch). The records are in each answer's records, with a purpose in words you can show people.

Any other bucket shows the usual two records: its certificate is issued in the minutes after you switch, so for those minutes your address answers with a certificate warning. To avoid that gap, move into a private bucket with plycdn links.

From your code: CreateDomainAsync, createDomain, create_domain: see Your own domain.

5. Switch your address

Change the CNAME at cdn.example.com to the value the domain's page shows. If your DNS host can proxy a name, make the record DNS-only. Within minutes (your TTL) your address reaches plycdn, with a certificate already in place. The domain turns active.

Keep your old service configured until the move is closed. To switch back, point the record at your old service again. Files plycdn has already kept stay in the bucket.

6. Old links

If your application hands out links signed by Azure (the addresses have ?sv=...&sig=... at the end), turn on Accept old links (legacyLinks: "azure_sas"). While it is on, such a link keeps working through plycdn: each one is checked with Azure, so a link that has expired or been revoked there is refused here too. Notes:

• A link bound to a viewer's address cannot be checked and is refused.
• Old links are accepted even on a bucket that binds plycdn's own links to addresses.
• Only a private bucket with plycdn links can accept them: on any other bucket turning them on is refused with old_links_need_private_bucket.
• Links you sign with plycdn's server packages work from day one, next to the old ones.
• To have old links stop by themselves on a day, set legacyLinksUntil (a date, yyyy-MM-dd; optional): LegacyLinksUntil in MigrationUpdate for ASP.NET Core, legacyLinksUntil in Node.js and Python. After that day they are refused as if switched off.

Move your application over to plycdn links when you can: Delivering files shows how to sign them.

See which old links still arrive: Storage → Storage buckets → the bucket → Move from another cloud → Old links. It shows, per day, the links that arrived by kind, which sites still send them and when the last one was seen. When none has arrived for seven full days, a quiet notice says so: that is the evidence that it is safe to switch them off.

What the Old links panel records: for each request to the bucket, the kind of link (an old link or a plycdn link), whether it was accepted, the answer's status and the name of the site the viewer came from (only the site's name, such as learning.example.com, never the page, the link itself or anything in it). It never records the viewer's address or the file asked for.

7. Move your uploads

Until your application writes new files to plycdn, files uploaded to your old storage are picked up by the background copy. When you are ready, change your application's uploads to plycdn's Azure-compatible or S3-compatible address (see S3 and Azure compatibility) or to the upload packages (Storage), then choose My uploads have moved (writesMoved: true).

• Until you mark the uploads as moved, a file that changes in your old storage is copied again.
• After you mark them, plycdn stops looking for new and changed files in the old storage.
• A file you delete in plycdn stays deleted; it is not copied back (its item shows deleted_by_customer).
• A file deleted in your old storage stays in plycdn if it was already moved.

From your code:

ASP.NET Core

await api.UpdateMigrationAsync(orgId, bucketId, new MigrationUpdate { WritesMoved = true });

Node.js

await storage.updateMigration(orgId, bucketId, { writesMoved: true });

Python

storage.update_migration(org_id, bucket_id, {"writesMoved": True})

8. Watch the copy

Files move in two ways: when they are first opened, and by a background pass that starts as soon as you choose Start serving, before or after you switch your address, and copies at the speed you allow with Files copied at once (1 to 64). Small files go in one step, so a move of many small files is as quick as one of a few large ones. The page shows files copied, bytes copied, files waiting, failed and files that cannot move.

ASP.NET Core

var move = await api.GetMigrationAsync(orgId, bucketId);
Console.WriteLine(quot;{move.State}: {move.Progress.CopiedObjects} copied, {move.Progress.Pending} waiting");

var stuck = await api.ListMigrationItemsAsync(orgId, bucketId, state: "unmovable");
var usage = await api.GetMigrationLinkUsageAsync(orgId, bucketId);
Console.WriteLine(quot;last old link: {usage.LastOldLinkAt}");

Node.js

const move = await storage.getMigration(orgId, bucketId);
console.log(move.state, move.progress.copiedObjects, move.progress.pending);

const stuck = await storage.listMigrationItems(orgId, bucketId, { state: 'unmovable' });
const usage = await storage.getMigrationLinkUsage(orgId, bucketId);
console.log('last old link', usage.lastOldLinkAt);

Python

move = storage.get_migration(org_id, bucket_id)
print(move["state"], move["progress"]["copiedObjects"], move["progress"]["pending"])

stuck = storage.list_migration_items(org_id, bucket_id, state="unmovable")
usage = storage.get_migration_link_usage(org_id, bucket_id)
print("last old link", usage["lastOldLinkAt"])

The state goes draft → serving → complete → closed, and paused if you pause (checking shows for a moment while a check runs). The background copy runs while the move is serving. A move becomes complete only after you have marked My uploads have moved and a full pass of your old storage has run after that; until then it stays serving, because files can still change there. When it reaches complete every file that can move has moved. Files listed as cannot move or failed need your attention (see the errors below): until you close the move they are still served from your old storage. A move from a plain web address stays serving until you close it. You can pause and resume the move at any time (PauseMigrationAsync / pauseMigration / pause_migration, and the resume calls). Pausing stops the background copy only: a file asked for that is not in plycdn yet is still read from your old storage, so your address keeps working.

9. Switch off old links, then close

1. When the Old links panel shows none for a week, choose Switch off old links (SwitchOffOldLinksAsync, switchOffOldLinks, switch_off_old_links). Within a minute, a link signed by Azure is refused everywhere.
2. When the move is complete, choose Close the move (CloseMigrationAsync, closeMigration, close_migration). Old links stop, plycdn stops reading your old storage, and the stored key is deleted. If files that cannot move or that failed still exist, closing asks you to confirm (confirm: true). Files you deleted in plycdn (deleted_by_customer) count here too: confirming is only to make sure you meant it, and they are already gone. Any other file that cannot move or failed stops being served from your address once the move is closed. You can also close a move that is still serving or paused (to give up, or to start again with another source), always with confirmation: files not copied yet then stop being served from your address.
3. Remove the old service and your old DNS configuration. Your old storage is yours to delete: plycdn never deletes there.

ASP.NET Core

await api.SwitchOffOldLinksAsync(orgId, bucketId);
await api.CloseMigrationAsync(orgId, bucketId);              // confirm: true when files that cannot move or failed remain

Node.js

await storage.switchOffOldLinks(orgId, bucketId);
await storage.closeMigration(orgId, bucketId);               // { confirm: true } when files that cannot move or failed remain

Python

storage.switch_off_old_links(org_id, bucket_id)
storage.close_migration(org_id, bucket_id)                   # confirm=True when files that cannot move or failed remain

3See every move

When you move more than one container, one screen shows them all.

In the dashboard: Storage → Moves from other clouds. Each row is one bucket that has a move (or had one: a closed move stays listed until you start another). You see the bucket (choose it to open its move), the source container and folder, the state, files copied, waiting, failed and cannot move, when an old link last arrived, and when the read-only key expires.

• Narrow the list with State, Needs a look and the bucket search. Each active filter shows as a chip you can remove, and the filters stay in the page address, so you can send the exact view to a colleague.
• The chips above the table count every move in your account, whatever the filters: choose Files failed, Files that cannot move, Old links still arriving or Key expiring soon to see just those.
• Use it as your daily check while a move runs: nothing needs doing when no chip is lit.

From your code (a standard or admin key):

ASP.NET Core

var page = await api.ListMigrationsAsync(orgId, attention: "failed");
Console.WriteLine(quot;{page.Totals.Buckets} moves, {page.Totals.Failed} with failed files");
foreach (var move in page.Migrations)
    Console.WriteLine(quot;{move.BucketName}: {move.State}, {move.Progress.Failed} failed");
// page.NextCursor continues: pass it back as cursor:

Node.js

const page = await storage.listMigrations(orgId, { attention: 'failed' });
console.log(page.totals.buckets, 'moves,', page.totals.failed, 'with failed files');
for (const move of page.migrations) console.log(move.bucketName, move.state, move.progress.failed);
// page.nextCursor continues: pass it back as { cursor }

Python

page = storage.list_migrations(org_id, attention="failed")
print(page["totals"]["buckets"], "moves,", page["totals"]["failed"], "with failed files")
for move in page["migrations"]:
    print(move["bucketName"], move["state"], move["progress"]["failed"])
# page["nextCursor"] continues: pass it back as cursor=

The call is GET /api/content/v1/migrations; the filters are state, attention, bucket, limit and cursor (see the API reference). attention is one of:

4Moving many containers

One bucket takes one move, so moving ten containers means ten buckets and ten moves. The steps are the same for each; this script does them in a loop. Give each container its own read-only key (step 1) and keep the list in your own configuration. It creates the bucket, waits until it is ready, sets up the move, checks it and, if the check is good, starts serving. It needs an admin API key.

Start with two or three containers, open Storage → Moves from other clouds to see them, and add the rest when you are happy. Your addresses (step 4 onward) are switched one container at a time, when you are ready for each.

ASP.NET Core

var containers = new[]
{
    new { Container = "contentprod", Bucket = "media-prod", Token = Environment.GetEnvironmentVariable("TOKEN_CONTENTPROD")! },
    new { Container = "contentqa",   Bucket = "media-qa",   Token = Environment.GetEnvironmentVariable("TOKEN_CONTENTQA")! },
};

foreach (var c in containers)
{
    var bucket = await api.CreateBucketAsync(orgId, new BucketCreate(c.Bucket, "mumbai"));
    while (bucket.State != "active")                      // creating takes a short while
    {
        await Task.Delay(TimeSpan.FromSeconds(5));
        bucket = await api.GetBucketAsync(orgId, bucket.Id);
    }

    await api.StartMigrationAsync(orgId, bucket.Id, new MigrationCreate(
        Kind: "azure_blob",
        Endpoint: "https://youraccount.blob.core.windows.net",
        Container: c.Container,
        Credential: c.Token));

    var check = await api.CheckMigrationAsync(orgId, bucket.Id);
    if (!check.Ok || check.UnmovableSample.Count > 0)
    {
        Console.WriteLine(quot;{c.Container}: look at the check before serving");
        continue;                                          // fix names in the old storage, then run it again for this one
    }
    await api.CheckMigrationAsync(orgId, bucket.Id, start: true);   // start serving
    Console.WriteLine(quot;{c.Container}: serving from {bucket.Name}");
}

// Later, and every day while the copies run: who needs a look?
var failed = await api.ListMigrationsAsync(orgId, attention: "failed");
foreach (var move in failed.Migrations)
    Console.WriteLine(quot;{move.BucketName}: {move.Progress.Failed} failed");

Node.js

const containers = [
  { container: 'contentprod', bucket: 'media-prod', token: process.env.TOKEN_CONTENTPROD! },
  { container: 'contentqa',   bucket: 'media-qa',   token: process.env.TOKEN_CONTENTQA! },
];

for (const c of containers) {
  let bucket = await storage.createBucket(orgId, { name: c.bucket, region: 'mumbai' });
  while (bucket.state !== 'active') {                      // creating takes a short while
    await new Promise((r) => setTimeout(r, 5000));
    bucket = await storage.getBucket(orgId, bucket.id);
  }

  await storage.startMigration(orgId, bucket.id, {
    kind: 'azure_blob',
    endpoint: 'https://youraccount.blob.core.windows.net',
    container: c.container,
    credential: c.token,
  });

  const check = await storage.checkMigration(orgId, bucket.id);
  if (!check.ok || check.unmovableSample.length > 0) {
    console.log(`${c.container}: look at the check before serving`);
    continue;                                              // fix names in the old storage, then run it again for this one
  }
  await storage.checkMigration(orgId, bucket.id, { start: true });   // start serving
  console.log(`${c.container}: serving from ${bucket.name}`);
}

// Later, and every day while the copies run: who needs a look?
const failed = await storage.listMigrations(orgId, { attention: 'failed' });
for (const move of failed.migrations) console.log(move.bucketName, move.progress.failed, 'failed');

Python

containers = [
    {"container": "contentprod", "bucket": "media-prod", "token": os.environ["TOKEN_CONTENTPROD"]},
    {"container": "contentqa", "bucket": "media-qa", "token": os.environ["TOKEN_CONTENTQA"]},
]

for c in containers:
    bucket = storage.create_bucket(org_id, {"name": c["bucket"], "region": "mumbai"})
    while bucket["state"] != "active":                     # creating takes a short while
        time.sleep(5)
        bucket = storage.get_bucket(org_id, bucket["id"])

    storage.start_migration(org_id, bucket["id"], {
        "kind": "azure_blob",
        "endpoint": "https://youraccount.blob.core.windows.net",
        "container": c["container"],
        "credential": c["token"],
    })

    check = storage.check_migration(org_id, bucket["id"])
    if not check["ok"] or check["unmovableSample"]:
        print(c["container"], "look at the check before serving")
        continue                                           # fix names in the old storage, then run it again for this one
    storage.check_migration(org_id, bucket["id"], start=True)   # start serving
    print(c["container"], "serving from", bucket["name"])

# Later, and every day while the copies run: who needs a look?
failed = storage.list_migrations(org_id, attention="failed")
for move in failed["migrations"]:
    print(move["bucketName"], move["progress"]["failed"], "failed")

Use the region you chose for your account instead of mumbai. Keep your old-link choices per container: add LegacyLinks: "azure_sas" / legacyLinks: 'azure_sas' / "legacyLinks": "azure_sas" to the move for a container whose links are signed (step 6), and make that bucket private with plycdn links.

Watching many moves is the same as watching one (step 8), done once for all of them: Storage → Moves from other clouds in the dashboard, or ListMigrationsAsync / listMigrations / list_migrations with attention set to failed, unmovable, old_links or credential_expiring. A move that shows up under failed has files to look at: open it and list its keys with state set to failed, as in step 8.

5Containers without a custom domain

Some containers are reached through your own address (cdn.example.com, for instance behind Azure Front Door). plycdn can take that address over, as in steps 4 and 5, and every link you have handed out keeps working.

Other containers are reached at the storage account's own address, youraccount.blob.core.windows.net. That address belongs to Azure, so it cannot move to plycdn, and links that point at it cannot be kept working through plycdn. For these containers:

1. Switch your application to the bucket's plycdn address (https://<bucket>.plycdn.net/<path>), or to plycdn signed links, for everything it hands out from now on. Files copied into plycdn at the same paths are served from there.
2. Keep Azure serving the container until your application has switched, and for as long as old pages, emails or downloads may still hold the old addresses. Do not close the move or delete anything in the old storage yet.
3. Let the background copy finish: the move reaches complete (step 8), and the counts match your old storage.
4. Close the move (step 9) when you no longer need the old container to answer.

Until step 4, plycdn only reads from the old container; it never writes or deletes there. The Old links panel and the old_links filter only report links signed by Azure that reach a plycdn address; links sent straight to youraccount.blob.core.windows.net never reach plycdn, so they do not show there. Check your own site and its logs for them before you delete the old container.

6What it costs

Storage and delivery are billed as usual, from the day a file is kept. Copying files in is not charged as upload. There is no separate price for moving.

7Check it works

1. After the check: a file you know exists loads at https://<bucket>.plycdn.net/<path>, and shows in Files the next time you look.
2. After the switch: https://cdn.example.com/<path> loads over HTTPS with no certificate warning, and the domain is active.
3. An old signed link still loads (while Accept old links is on); a changed or expired one is refused.
4. The page's counts rise: Files copied goes up, waiting goes down to zero, and the state reaches complete (a move from a plain web address stays serving until you close it).
5. Compare the counts with your old storage's own before you close.

8Questions and problems

bucket_not_supported. The bucket is a video library. Only file buckets can move.

bucket_not_ready. The bucket is still being created. Try again in a minute.

migration_already_running. The bucket already has a move. Change its key, old-link or copy settings with PATCH, or close it first (with confirmation if it is still running) and start a new one: the source itself (address, container, folder) cannot be changed.

migration_not_found. The bucket has no move, or it has been closed.

migration_state_conflict. The action does not fit the move's state: you can only start a move that is a draft, and resume one that is paused.

migration_source_unreachable. plycdn cannot reach the address from the internet: it is private, misspelt or not answering. Use the public address of the storage and check again. With Accept old links on, the address must also be your Azure storage's own, https://youraccount.blob.core.windows.net: old links are checked there.

old_links_need_private_bucket. Accept old links was turned on for a bucket that is public or does not use plycdn links. Old links are checked only on a private bucket with plycdn links: move into such a bucket, or leave old links off.

migration_credential_refused. Your old cloud refused the key (wrong, expired or without permission to list), or the key is not in the form expected (an S3 key is the JSON text shown above; a web address takes no key). Create a new key and update the move with it.

migration_credential_too_broad. An Azure token that can write, delete or create, or that covers the whole account. Create a container token with only read and list.

migration_not_closable. Files that cannot move or that failed remain, or the move has not completed yet. See the files with ListMigrationItemsAsync(..., state: "unmovable") (and state: "failed"), rename or fix them in your old storage, or close with confirmation.

A file shows as "cannot move". Its reason says why:

The domain stays in "issuing certificate". Normal until you switch: create the _acme-challenge record, then the routing record. After 72 hours it fails with dns_not_configured; check the records and start again. Without the _acme-challenge record the certificate request gives up after 24 hours (certificate_failed); with it, the domain may wait up to 72 hours for your switch.

Old links return 403. The link is bound to a viewer's address, has expired or been revoked in Azure, or Accept old links is off or was switched off.

The copy stopped. The key expired (Azure shows its expiry on the page) or the move is paused. Update the key with UpdateMigrationAsync / updateMigration / update_migration, or resume.

admin_key_required. Setting up, changing, checking, pausing, resuming, switching off and closing need an admin key.

More: Storage and delivery, the API reference and Delivering files and video.