Data Management & Analysis
This manual provides a guide to parsing, searching and analysing in Falcon LogScale 1.89.0-1.100.0. Since these are essentials of LogScale, these pages of the documentation apply to both Cloud and Self-Hosted deployments.
Below are links to the major sections, with descriptions of each. They're listed somewhat from the most basic, key aspects to the more advanced and useful tools. They're grouped, though, by related topics.
Data Storage & SiftingYou may have an administrator who installed LogScale, as well as set up log shippers on your servers to send data to LogScale — making it easier for you to focus on the data itself.
As a data analyst, to start you'll need to know your tools (i.e., the UI), as well as understand the repositories where the data is stored and how to parse that data, properly. These topics are covered in sections below.
Most people utilize LogScale with the web-based user interface. Through the UI, you can access your repositories containing server logs and metrics. It's where you can view and search data, create and see charts of server activities. Here you'll learn about the LogScale UI. In a way, for most, it all starts with the UI.
The main storage entity within LogScale is the repository. It's where log shippers on your servers send server log entries and other server metrics, known as events. Events are converged and stored in repositories. It's where you can manage and query and monitor the data accumulated. This section of the documentation is where you can learn about repositories, the heart of LogScale.
LogScale without parsers is raw data; it's chaos. Parsers bring order and sanity to data. They take events and break them into useful and manageable components. It discerns dates from IP addresses, user names from file names, and much more. This section on parsers shows you how to assemble and organize data so that you can query that data.
With the data parsed and stored in your repositories, you'll want to be able to search it, to be able to get answers to questions you may have about activities on your servers. How to do this is covered in the sections listed here, some basic and some to more indepth levels.
This is where you can learn how to get information from LogScale that you can actually use — in making decisions about your servers, about your security, and about your business. This section explains the basics of how to search your repositories, how to query the data through the user interface.
The previous section explains how to use the UI to search data. This section provides much more details on how to write queries using the LogScale query language to search data. These queries may be used within the query field of the UI, or with API interfaces. That may seem more advanced: while it can be, it's primarily more of an intermediate level.
The previous sections cover the more common ways to query a repository. This section provides much more details; it's a full language syntax guide for writing queries. It includes using operators, conditional statements, regular expressions.
LogScale has an elaborate list of powerful functions for querying repositories. Functions have specific syntax requirements, parameters allowed, and how data is returned using various data types. Here you'll find all of the details for all LogScale functions used for parsing and executing queries.
To streamline queries you often use, and to make data available to others without your skills or permissions, such as business managers and clients, the sections below explain how to save favorite and useful queries, as well as how to display them in meaningful ways.
Dashboards and widgets provide graphical views of your event data. They can be configured using different graphs and display formats, such as pie charts, line graphs, as well as listing relevant text. These can be useful for you and others to monitor easily server usage and activity. This is all explained here.
Being able to monitor data with dashboards is great, but you can't review them constantly: you have other things to do and can't work twenty-four hours a day, every day. LogScale has a variety of ways to check automatically your repositories, using your queries. If certain events happen, if criteria you set are met or exceeded, an alert can be triggered and you or others in your organization can be notified — by email or telephone text message.