Log data is the lowest common denominator for getting both real-time and historical insights into running systems.

Most systems and applications append log data to log files on disk. However, some systems also support sending logs directly to a log management systems like LogScale.

In most situations, it's necessary to add a 'log shipping' layer to your system. A 'log shipper' is an application that can take logs from a file on disk and send them directly to a log management system.

An important aspect of shipping is how faults are handled. When evaluating a log shipper, examine what situations will cause the loss of log lines, or duplication. Also check which kinds of failures it can tolerate.

Because most log formats are unique, at some point you'll need to parse your logs. Parsing lets you take logs that are essentially lines of text, and extract their structure in order to incorporate a more in-depth analysis.

For example, you can extract useful items of data such as a timestamp or key-value pairs.

LogScale can parse logs for you. If you are familiar with Fluentd or LogStash, they can also parse logs.

Querying is where you will feel the real benefit of deploying a log management tool like LogScale. The system creates value by letting you ask a variety of questions about the data in ways it might not be possible to with a raw data log.

The list below shows some of the use cases where a log management system like LogScale could be invaluable: