Measuring Data Ingest
For monitoring your current usage in
Usage Page. The following
diagram illustrates the flow of data into LogScale and the various
measurement points. For most customers, Daily Ingest Capacity will
correspond to MP below:
Figure 21. Ingest Flow
When calculating the amount of data ingested by LogScale we use the following formula:
processed_events = @rawstring + fields + tags - removed fields and tags
This is the length of the message from the request
Are the keys and values of any additional fields, as fields that were extracted from the rawstring are not counted.
Are the keys and values of any additional tag, as the tags that are extracted from the rawstrings are not counted.
Removed fields and tags (MR)
In cases where it's not possible to remove data that is not needed before ingesting it, you can filter it out by adjusting a parser setting. See Optimizing Ingestion for more details.
Your data needs to have a @timestamp field in order to be searchable in LogScale. This field counts as ingest, unless it's included in the rawstring.