Proofpoint TAP SIEM API

This package follows CrowdStrike Parsing Standard (CPS) and it provides parsers for Proofpoint's Targeted Attack Protection (TAP) SIEM API which collects data for security teams on messages and clicks that have been permitted and blocked. Documentation for the API can be found here

The parser normalizes data to a common schema based on CrowdStrike Parsing Standard (CPS). This schema allows you to search the data without knowing the data specifically, and just knowing the common schema instead. It also allows you to combine the data more easily with other data sources which conform to the same schema.

Breaking Changes

This update includes parser changes, which means that data ingested after upgrade will not be backwards compatible with logs ingested with the previous version.

Updating to version 1.0.0 or newer will therefore result in issues with existing queries in for example dashboards or alerts created prior to this version.

See CrowdStrike Parsing Standard (CPS) for more details on the new parser schema.

Follow the CPS Migration to update your queries to use the fields and tags that are available in data parsed with version 1.0.0.

Installing the Package in LogScale

Find the repository where you want to send the Proofpoint TAP SIEM API events, or Creating a Repository or View.

  1. Navigate to your repository in the LogScale interface, click Settings and then Packages on the left.

  2. Click Marketplace and install the LogScale package for Proofpoint TAP SIEM API (i.e. proofpoint/tap).

  3. When the package has finished installing, click Ingest tokens on the left still under the Settings.

  4. In the right panel, click + Add Token to create a new token. Give the token an appropriate name (e.g. the name of the Event Hub it will collect logs from), and assigned the required parser.

    Before leaving this page, view the ingest token and copy it to your clipboard — to save it temporarily elsewhere.

    Now that you have a repository set up in LogScale along with an ingest token you're ready to send logs to LogScale.

Configurations and Sending the Logs to LogScale

Next configure your preferred log shipper(s) to send the Proofpoint TAP SIEM data to LogScale in JSON format. It is preferred to use the HEC interface of LogScale.

This package was tested using a python based log collection and processing package which pulls the logs from Proofpoint and ingests them to LogScale via the HEC interface. This may help you configure your own log shipper or you could re-use this package which can be found here.

Verify Data is Arriving in LogScale

Once you have completed the above steps the Proofpoint TAP SIEM API data should be arriving in your LogScale repository.

You can verify this by doing a simple search to see the events:

#Vendor ="proofpoint"
| #event.module="tap"

Package Contents Explained

This package contains the following parsers:

  • clicksBlocked-json

  • clicksPermitted-json

  • messagesBlocked-json

  • messagesDelivered-json

This package parses incoming data, and normalizing the data as part of that parsing. The parser normalizes the data to CrowdStrike Parsing Standard (CPS) schema based on OpenTelemetry standards, while still preserving the original data.

If you want to search using the original field names and values, you can access those in the fields whose names are prefixed with the word Vendor. Fields which are not prefixed with Vendor are standard fields which are either based on the schema (e.g. source.ip) or on LogScale conventions (e.g. @rawstring).

The fields which the parser currently maps the data to, are chosen based on what seems the most relevant, and will potentially be expanded in the future. But the parser won't necessarily normalize every field that has potential to be normalized.

Event Categorisation

As part of the schema, events are categorized by fields:

  • event.kind

  • event.type

  • event.outcome

#event.outcome is a tag.

event.categoryis an arrays, so need to be searched using the following syntax:

array:contains("event.category[]", value="network")

For example, the following will find events where some event.type[n] field contains the value network, regardless of what n is.

Note that not all events will be categorized to this level of detail.

Normalized Fields

Here are some of the normalized fields which are being set by this parser:

  • event.* (e.g. event.kind, event.module, event.dataset, event.outcome )

  • ecs* (e.g. ecs.version )

  • Cps* (e.g. Cps.version )