Parses a string into a timestamp.
This function is important for creating parsers, as it is used to parse the timestamp for an incoming event.
Before parsing the timestamp, the part of the log containing the timestamp
should be captured into a field. This is done using functions like
regex()
and parseJson()
before
parseTimestamp
.
The format string is specified using Java's DateTimeFormatter. LogScale also supports specifying the following in the format string:
unixtimeMillis
UTC time since 1970 in millisecondsunixtime
UTC time since 1970 in seconds
If the timestamp is parsed it will create a field @timestamp containing the parsed timestamp in UTC milliseconds and a @timezone field containing the original timezone.
It is possible to parse time formats leaving out the year designator as is
sometime seen in time formats from Syslog. For example
Mar 15 07:48:13
can be parsed using the
format MM d HH:mm:ss
. In this case
LogScale will guess the year.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
addErrors | boolean | false | true | Whether to add an error field to the event, if it was not possible to find a timestamp. |
as | string | false | @timestamp | Name of output field that will contain the parsed timestamp. The timestamp is represented as milliseconds since 1970 in UTC. LogScale expects to find the timestamp in the field @timestamp, so do not change this when creating parsers. |
caseSensitive | boolean | false | true | Whether the timestamp format pattern is case sensitive. For example, the format LLL will accept Feb but not feb in case sensitive mode, while both will be accepted in case insensitive mode. |
Valid Values | false | Pattern is not case sensitive | ||
true | Pattern is case sensitive | |||
field | string | true | The field holding the timestamp to be parsed. | |
format | string | false | yyyy-MM-dd'T'HH:mm:ss[.SSS]XXX | Pattern used to parse the timestamp. The format string is specified using Java's DateTimeFormatter. |
Valid Values | millis | Epoch time in milliseconds (UTC) | ||
seconds | Epoch time in seconds (UTC) | |||
unixTimeSeconds | Epoch time in seconds (UTC) | |||
unixTimemillis | Epoch time in milliseconds (UTC) | |||
timezone | string | false | If the timestamp does not contain a timezone, it can be specified using this parameter. Example are Europe/London, America/New_York and UTC. See the full list of timezones supported by LogScale at Supported Timezones. | |
timezoneAs | string | false | @timezone | Name of output field that will contain the parsed timezone. LogScale expects to find the timezone in the field @timezone , so do not change when creating parsers. |
parseTimestamp()
Examples
Events having a timestamp in ISO8601 format can be parsed using the default format.
An example is a timestamp like
2017-12-18T20:39:35Z
:
/(?<timestamp>\S+)/
| parseTimestamp(field=timestamp)
Parse timestamps in an accesslog where the timestamp includes an
explicit timezone offset like 192.168.1.19
[02/Apr/2014:16:29:32 +0200] GET /hello/test/123 ...
/(?<client>\S+) \[(?<@timestamp>.+)\] (?<method>\S+) (?<url>\S+)/
| parseTimestamp("dd/MMM/yyyy:HH:mm:ss Z", field=timestamp)
When parsing a timestamp without a timezone like
2015-12-18T20:39:35
you should declare
to LogScale what timezone a given time should be located:
parseTimestamp("yyyy-MM-dd'T'HH:mm:ss", field=timestamp, timezone="America/New_York")
Parse an event with a timestamp not containing year like
Feb 9 12:22:44 hello world
/(?<timestamp>\S+\s+\S+\s+\S+)/
| parseTimestamp("MM [ ]d HH:mm:ss", field=timestamp, timezone="Europe/London")