LogScale Overview

What is log management?

Log management involves continuously collecting, storing, processing, and analyzing data from various programs to achieve system optimization, troubleshooting, resource management, enhanced security, and improved compliance. This centralized and searchable data enables informed decision-making regarding network health, resource allocation, and security. The volume of operational data generated by modern IT systems requires a log management tool that is able to handle the content at scale.

Log management is typically categorized into six main functions:

  • Collection: Aggregating log data from all relevant sources (OS, applications, servers, endpoints, etc.) across the organization.

  • Monitoring: Tracking all logged events and activities, along with their timestamps.

  • Analysis: Reviewing collected logs to proactively identify issues like bugs, security threats, or operational problems, either through dashboards or regular searches.

  • Retention: Designating and managing the duration for which log data must be stored, for auditing or legal as well as operational reasons.

  • Indexing or Search: Allow the data to be searched, filtered, sorted and analyzed to identify trends and outliers.

  • Reporting: Automating the generation of reports from audit logs regarding operational performance, resource usage, security, and regulatory compliance.

While complex, these processes can be efficiently handled using LogScale.

Since log formats vary widely, parsing the incoming data is crucial for extracting information in a standardized form and allows for easy deep analysis of complex and correlated information.

The true value of log management emerges during querying, where LogScale enables various analytical capabilities, including:

  • Incident investigation

  • Real-time operation monitoring

  • Communication pattern analysis

  • Tracking system interdependencies

  • Performance and metrics monitoring

  • Status monitoring

  • Ad-hoc analysis

  • Visual system monitoring through dashboards

Log Management in LogScale

Log management processes in LogScale are handled with a variety of different components.

The collection phase is the initial first step in log management, where all future efforts stem from. The first part of this process is data ingestion, which is achieved using LogScale tools like Falcon LogScale Collector, integrations, and third-party packages. These tools are geared towards the many types of data an average organization might have, given that data sources are often complex. Falcon LogScale Collector is the primary and recommended ingest tool, with third party packages and integrations acting as supplemental ingest tools that handle more complex and/or specific ingest scenarios. Using parsers is the next part of the process, because they are the integral part of extracting fields and preparing the data for storage in a repository.

Next is the monitoring and analysis of the data, the primary method is to use the CrowdStrike Query Language (CQL) to extract data that can then be visualized using dashboards, widgets, and used in alerts and automations. CQL provides a rich method of querying, summarizing, and formatting data, including correlation and join data to provide data insights. The dashboards and widgets use the query language and can provide overviews and detailed information, making those insights accessible to others within an organization.

To scale and provide support for ingesting data across many thousand or millions of hosts, LogScale uses bucket storage to store data in a time-efficient fashion within both cloud-based and self-hosted environments. CQL provides a variety of filters and analytical tools that allow users to filter and analyze the data either across time or to pick out specific events. When users have determined queries that should happen on a regular basis, users can implement automations that complete scheduled searches to look for incidents and deliver reports on demand.

LogScale also includes other systems for management role-based access control, security controls, and a user interface for managing your LogScale deployment.