Log Shippers

A log shipper is a system tool that looks at files and system properties on a server and sends them to LogScale. Log shippers take care of buffering, re-transmitting lost messages, log file rolling, network disconnects, and other log management functions to ensure that you send data to LogScale in a reliable, and regular, fashion.

graph LR; YA["Your App"] --Logging--> LF["Log File"] LF --> DS PP["Pre-Processing"] <--> DS["Log Shipper"] DS --Ingest API--> LS{LogScale} LS --Parser 1--> R1["Repository 1"] LS --Parser 2--> R2["Repository 2"] style R1 fill:#f5ad42 style R2 fill:#f5ad42 style DS fill:#40bd28 style LS fill:#fc0000 style PP stroke-dasharray: 5 5
Diagram illustrating the LogScale log shipping architecture and data flow. The flowchart shows how data moves through the system: an application writes logs to a log file, which a log shipper reads and optionally pre-processes before sending to LogScale via the Ingest API. LogScale then applies different parsers to route the data to appropriate repositories. The diagram uses color coding to distinguish components: the log shipper in green, LogScale in red, and repositories in orange, with pre-processing shown as a dashed element.

Figure 81. Log Shippers


In Log Shippers, your application is writing logs to a log file. The log shipper reads the data and pre-processes it (for example, this could be converting a multi-line stack-trace into a single event). It then ships the data to LogScale on one of LogScale's Ingest API.

Data shipping can be done in a few ways: