HAProxy

The HAProxy package allows you to analyze HAProxy logs in LogScale to improve your network monitoring for both error rates and potentially malicious IPs.

Note that the HAProxy package currently only contains a parser for HAProxy, which should help customers being able to ship data to LogScale from HAProxy and extract fields according ECS.

Breaking Changes

This update includes parser changes, which means that data ingested after upgrade will not be backwards compatible with logs ingested with the previous version.

Updating to version 1.0.0 or newer will therefore result in issues with existing queries in for example dashboards or alerts created prior to this version.

See CrowdStrike Parsing Standard (CPS) for more details on the new parser schema.

Follow the CPS Migration to update your queries to use the fields and tags that are available in data parsed with version 1.0.0.

Configurations and Sending The Logs to LogScale

See HAProxy Syslog Specification manual for information on how to send HAProxy logs to Falcon LogScale Collector.

Installing the HAProxy Package in LogScale

Find the repository where you want to send the HAProxy logs, or create a new one.

  1. Navigate to your repository in the LogScale interface, click Settings and then Packages on the left.

  2. Click Marketplace and install the LogScale package for HAProxy.

  3. When the package has finished installing, click Ingest tokens on the left (still under the Settings, see Ingest Tokens).

  4. In the right panel, click + Add Token to create a new token. Give the token an appropriate name (e.g.the name of the server and the name of the server the token is ingesting logs for), and leave the parser unassigned. You can also assign the parser to the LogScale Collector Configuration as described in the documentation Sources & Examples using haproxy-syslog.

    Before leaving this page, view the ingest token and copy it to your clipboard — to save it temporarily elsewhere.

    Now that you have a repository set up in LogScale along with an ingest token you're ready to send logs to LogScale.

  5. Next, configure the Falcon LogScale Collector to ship the logs from your syslog server into LogScale. Follow LogScale Collector Installing the LogScale Collector and Configuring LogScale Collector. LogScale Collector documentation also provides an example of how you can configure your syslog datasource, see Sources & Examples.

Since HAProxy ships data through syslog customers also need to set up an appropriately configured Log Collector, containing a configuration similar to: 

sources:
  syslog_udp_514:
    type: syslog
    mode: udp
    port: 514
    sink: logscale
sinks:
  logscale:
    type: humio
    token: your generated ingest token
    url: https://cloud.humio.com LogScale Cloud and LogScale Self-Hosted users will have different URLs

Verify Data is Arriving in LogScale

Once you have completed the above steps the HAProxy data should be arriving in your LogScale repository.

You can verify this by doing a simple search for #Vendor = "haproxy" | Product = "haproxy" to see the events.

Package Contents Explained

This package parses incoming data, and normalizing the data as part of that parsing. The parser normalizes the data to CrowdStrike Parsing Standard (CPS) schema based on OpenTelemetry standards, while still preserving the original data.

If you want to search using the original field names and values, you can access those in the fields whose names are prefixed with the word "Vendor". Fields which are not prefixed with "Vendor" are standard fields which are either based on the schema (e.g. source.ip) or on LogScale conventions (e.g. @rawstring).

The fields which the parser currently maps the data to, are chosen based on what seems the most relevant, and will potentially be expanded in the future. But the parser won't necessarily normalize every field that has potential to be normalized.

Event Categorisation

As part of the schema, events are categorized by different fields.

event.category is an array, so needs to be searched like so: 

array:contains("event.category[]", value="info")

This will find events where some event.category[n] field contains the value "info", regardless of what `n` is. Note that not all events will be categorized to this level of detail.

Normalized Fields

Here are some of the normalized fields which are being set by this parser:

  • source.ip

  • source.message