Primary Storage
When data is ingested it is stored in primary storage (also called local storage). Queries of data will be most efficient if the data is already available on primary storage. To maximize query performance, primary drives should be as fast as possible. The tradeoff is that fast storage is expensive, so primary storage is normally extended by adding bucket storage. Typically, primary storage will be based on fast technology such as NVMe drives, and bucket storage will typically be cloud-based storage such as AWS S3.
The main configuration parameters for primary
storage are DIRECTORY, and
PRIMARY_STORAGE_MAX_FILL_PERCENTAGE. These enable you to
configure the mount point for primary storage, and its maximum storage
capacity.
When LogScale is deployed in a cloud environment, primary storage is typically based on NVMe instance stores. These provide maximum performance, but due to the way they are deployed in the cloud environment, should be considered ephemeral storage. This is explained further in the introduction.
Upgrading Primary Storage
If you want to upgrade primary storage, there are two main ways:
Add more nodes - this not only increases storage capacity, but also increases redundancy and query performance, and is the preferred method of upgrading your LogScale cluster.
Upgrade individual nodes - in this scenario you remove a node, upgrade it (upgrade the instance to one with larger NVMe, replace the hard drive etc.), add the upgraded node back into the cluster.
For more information on adding, removing, or replacing nodes see Adding and Removing Nodes.