How-To: Migrating from Secondary to Bucket Storage

Secondary storage was deprecated in LogScale version 1.217.0 and removed in version 1.231.0.

This How-To shows you how to migrate your data from secondary storage to bucket storage.

There are two methods for migrating data from secondary storage to bucket storage, depending on whether or not you can temporarily block ingest.

If you are able to temporarily block ingest:

  1. Check that there are no "absent" bytes (data that cannot be found on any of the nodes in the cluster) in the cluster UI (Cluster administration > Cluster nodes). If there are, delete those first using the API.

  2. Set up the new bucket and point LogScale to it, and restart the nodes.

  3. Wait until the metric bucket-storage-pending-upload stabilizes at a reasonably small number (it should start out very large and drop over time as segments are uploaded). This is to clear the large backlog of old segments that need to be uploaded.

  4. Block ingest and wait for bucket-storage-pending-upload to drop to zero.

  5. Remove the secondary storage configuration variables and restart the nodes.

  6. Check that there are no "absent" bytes. If there are, add the secondary storage configuration variables back, restart, and try again, waiting longer this time.

If you are not able to tolerate blocking ingest for any amount of time:

  1. Check that there are no "absent" bytes (data that cannot be found on any of the nodes in the cluster) in the cluster UI (Cluster administration > Cluster nodes). If there are, delete those first using the API.

  2. Set up the new bucket and point LogScale to it, and restart the nodes.

  3. Wait until the metric bucket-storage-pending-upload stabilizes at a reasonably small number (it should start out very large and drop over time as segments are uploaded). This is to clear the large backlog of old segments that need to be uploaded.

  4. Set USING_EPHEMERAL_DISKS to true and restart. This forces all new segments to be written to bucket storage before that data can be deleted from Kafka.

  5. Wait for at least 15 minutes.

  6. Remove the secondary storage configuration variables and restart the nodes.

  7. Check that there are no "absent" bytes. If there are, add the secondary storage configuration variables back, restart, and try again, waiting longer this time.