This parser can handle the accesslog format, which is the default log format used by Apache and Nginx. The parser also supports putting the response time at the end of the log line. If you have modified the logging of your web server, copy the built-in accesslog parser and modify it to suit your customizations.

localhost - - [25/Feb/2017:21:05:16 +0100] "POST /api/v1/ingest/elastic-bulk HTTP/1.1" 200 50 "-" "Go-http-client/1.1" 0.000 848`
192.168.1.102 - - [25/Feb/2017:21:06:15 +0100] "GET /api/v1/repositories/gotoconf/queryjobs/855620e9-1d1f-4b0e-91fe-c348795e68c9 HTTP/1.1" 200 591 "referrer" "Mozilla/5.0" 0.008 995

This parser can process audit logs in JSON format from LogScale itself.

This is a built-in parser that supports Corelight's Zeek sensors. Corelight sensors have default support for streaming out Zeek logs in either JSON or Elasticsearch format. LogScale can receive the streaming data in Elasticsearch format using this parser. For the JSON format, see corelight-json.

The name of the Zeek log file will become a #path tag in LogScale.

This is a built-in parser that supports Corelight's Zeek sensors. Corelight sensors have default support for streaming out Zeek logs in either JSON for Elasticsearch format. LogScale can receive the streaming data in JSON format using this parser and Ingest Listeners. For the Elasticsearch format, see corelight-es above.

The name of the Zeek log file will become a #path tag in LogScale.

This parser can process JSON data in log lines. It expects to find a JSON property called @timestamp containing an ISO 8601-formatted time string.

If you don't have control over the JSON format, you can create a Example Input.

{
 "@timestamp": "2017-02-25T20:18:43.598+00:00",
 "loglevel": "INFO",
 "service": {
 "name": "user service",
 "time": 123
}
}

This parser is the default parser for the Action Type: Falcon LogScale Repository. It processes JSON data in the @rawstring field. It expects to find a JSON property called @timestamp containing a time in Unix Time in milliseconds.

This parser is the key-value parser. It is the default parser, that LogScale uses if no other parser is specified. It can process standard key-value patterns in log lines. It expects the log line to contain a timestamp with a time zone within the first 128 characters (configurable, see findTimestamp()). The parser processes the rest of the line for key-value pairs.

2017-02-25T20:18:43.598+0000 created a new user user="John Doe" service=user-service as a freemium user

user=John Doe

service=user-service

This parser, like the kv, is a key-value parser. It works in the same way, except that it will also parse timestamps without a timezone. Such timestamps will be assumed to be in UTC.

This parser, like the kv, is a key-value parser. However, it expects the timestamp to be at the start of the log line and to be in UTC time in milliseconds.

1488054417000 created a new user user="John Doe" service=user-service as a freemium user

user=John Doe

service=user-service

This parser can process log lines written by Serilog's JsonFormatter.

Example serilog configuration:

Log.Logger = new LoggerConfiguration()
             .WriteTo.File(formatter: new JsonFormatter(renderMessage: true), path:logPath, rollingInterval: RollingInterval.Day)

{"Timestamp":"2019-01-21T13:26:25.1354930+01:00","Level":"Information","MessageTemplate":"Processed {@Position} in {Elapsed:000} ms.","RenderedMessage":"Processed { Latitude: 25, Longitude: 134 } in 034 ms.","Properties":{"Position":{"Latitude":25,"Longitude":134},"Elapsed":34,"ProcessId":"15133"},"Renderings":{"Elapsed":[{"Format":"000","Rendering":"034"}]}}

This parser aims to be compatible with a variety of syslog formats. This includes RFC 3164 and RFC 5424. The parser does not implement every aspect of the syslog RFCs, but is instead liberal in what it accepts.

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for foo on /dev/pts/8

<34>1 2003-10-11T22:14:15.003Z server1.com sshd - - pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.0.2.

<34>Oct 11 22:14:15 mymachine su: 'su root' failed for foo on /dev/pts/8

Oct 11 22:14:15 su: 'su root' failed for foo on /dev/pts/8

timezone="UTC"

This parser can process standard lines generated by the syslog system.

The parser expects lines to starts with a timestamp, followed by the optional fields host, app, and pid. It also expects the timestamp to be in the UTC time zone. If your timestamps are in your local timezone, and that is not UTC, you will want to create a new parser by copying this parser and modifying

timezone="UTC"

to your desired timezone.

This parser also leverages LogScale's built-in key value parser kv.

Feb 25 19:17:01 Myhost CRON[24886]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Feb 25 06:35:01 Myhost CRON[24272]: (root) CMD (command -v deb-sa1 > /dev/null && deb-sa1 1 1)

This parser can process JSON data generated from Zeek. It is tailored to handle the output generated from the Zeek script, and you can read about how to send Zeek data to LogScale on the Zeek (Bro) Network Security Monitor documentation page.

The name of the Zeek log file will become a #path tag in LogScale.