|Description||Sets the S3 storage size for improved performance|
For improved performance, you can set the S3/GCP storage sizes. The first option here sets the chunks for upload. The minimum is 5 MB, and the maximum is 8 MB — the default is 8 MB.
S3_STORAGE_CHUNK_SIZE=8388608 S3_STORAGE_CHUNK_COUNT=4 S3_STORAGE_UPLOAD_CONCURRENCY=vcores/2 S3_STORAGE_DOWNLOAD_CONCURRENCY=vcores/2
The second option here is for the number of parallel chunks at a time for each file. This is only for S3. The third option is for the number of concurrent uploading files. The last is the number of concurrent downloading files.