Performance Tuning for Long Retention Settings

The defaults for Humio are targeting retention times of data in the range of 1-6 months. If you plan to keep data for much longer you can reduce the number of files stored on disk by telling Humio to create larger files. As retention “chops off” old data in chunks consisting of one file at a time, this will make those chunks larger.

If your Humio repositories expect to have a retention of more than 6 months on average, you can increase the amount of data in each file, thus reducing the total number of files in the system. Changing these settings on a Humio cluster has effect for files created after the change and making such a change is okay at any point in time.

ini
# The default value in Humio for installs where the average data retention is 0-6 months.
# MAX_HOURS_SEGMENT_OPEN=24

# Suggest using the default for Humio installs where the average data retention is 6-18 months.
# MAX_HOURS_SEGMENT_OPEN=48

# Suggest using the default for Humio installs where the average data retention is 2+ years.
# MAX_HOURS_SEGMENT_OPEN=96