Parser Errors

When a parser fails to parse incoming input, LogScale automatically adds the following fields to the event:

  • @error

    Indicates whether an error occured during processing. Will be set true if there was an error.

  • #error

    Indicates whether an error occured during processing. Will be set true if there was an error.

  • @event_parsed

    Indicates whether the event has been successfully parsed during ingest. Will be set false if there was an error.

  • @error_msg

    Contains the error message generated when the parsing of the event failed.

You can search for these fields to determine what happened and update the parser accordingly. When you find an error, you can add the @rawstring of any events that fail to parse as a test case for your parser.

Finding Errors

You can search for all events that were not parsed correctly by searching the repository:

logscale
@error=*

You can extend the query in different ways. For example, you can display a timechart:

logscale
@error=* 
| timechart()

or group results by error message:

logscale
@error=* 
| groupBy(@error_msg)

The original log input is available in @rawstring.

Note

There is no way for LogScale to re-parse data once it's stored, even if there is an error on the event. You'll have to resend the event through the ingest API.

Parser Timeout

Parsers can timeout due to issues with parsers, for example due to catastrophic back tracing in regexes or due to excessively expensive computation. We have set timeouts to ensure availability and resilience to Denial-of-Service attacks.