How is Data Impacted?

When data enters LogScale through ingestion, it undergoes a journey that fundamentally changes its nature - think of it like water flowing through a sophisticated filtration system. This guide explores the transformation process and the implications to data.

Data Transformation

Your raw data starts as a stream of information - which may be unstructured, and inconsistent. During ingestion, LogScale acts as a refinery, processing this raw material into something more valuable.

  • Standardization of timestamps

  • Transformation of unstructured log lines

  • Organization of fields

Positive Transformations

  • Structured and searchable data

  • Aligned timestamps

  • Consistent field normalization

  • Enhanced context through enrichment

Potential Challenges

Warning

Be aware of these common issues:

  • Parser misinterpretation

  • Unexpected field extraction

  • Time zone misalignment

  • Data duplication

Real-World Impact

The quality of your ingested data directly affects your ability to find answers during troubleshooting. Well-processed data enables:

  • Faster searches

  • More accurate alerts

  • Better insights

Making It Work For You

Success with data ingestion requires regular monitoring and maintenance:

  • Monitor ingestion processes

  • Watch for parsing errors

  • Validate data regularly

  • Adjust configurations as needed

Data Immutability

Once ingested, data in Falcon LogScale is immutable. Data can only be deleted under certain conditions and with specific elevated privileges:

  • By time โ€” Data is automatically purged at the end of the designated retention period. See Data Retention.

  • By manual deletion of the repository โ€” A user with sufficient permissions can delete an entire repository. See Delete a Repository or View.

  • By API โ€” A user with specific privileges and administrative power over a repository can leverage the Redact API to remove specific data. See Redact Events API.

All of the above actions can only be performed by authorized users with the specific mentioned permissions tied to specific repositories.