Now you have your logs being sent to LogScale the next step is to create the assets which will be included in the package.
Parsers Best Pratices
The parser needs to extract the timestamp correctly and apply any field names that will help people navigate and search the data.
Take care to get the right balance between completeness of functionality and speed of the parser. Follow the guidance given in our documentation on Creating a Parser.
Tags are labels that can be applied to identify a class/type of log message but they need to be used carefully and it's probably best to check in with us if you are considering use Tags.
Testing the parser with a representative sample of logs is critical. This is especially important for systems that generate different logs that can have different numbers of fields, different value types etc.
The parser should contain multiple test events that are incorporated into the parser yaml file when the package is created or the parser is exported. This demonstrates basic parser functionality by using only the parser yaml file uploaded to LogScale.
Note that these test cases are publicly available, and should be scrubbed of any sensitive information. If you still want the data to look real, we recommend using data that is explicitly made for testing or documentation purposes. For example, when using IPv4 addresses, section 3 of RFC5737 declares that:
TEST-NET-3) are provided for use in documentation.
Similarly for URLs, section 3 of RFC2606 says:
The Internet Assigned Numbers Authority (IANA) also currently has the following second level domain names reserved which can be used as examples.
example.com example.net example.org
Figure 183, “Parser” shows a parser that includes 8 test events.
Figure 183. Parser
LogScale Query Language Best Practices
When using the LogScale query language (in parsers, searches or dashboards ) it is important to follow best practices in order to maintain optimum performance and avoid unnecessary system load. Follow the general guidelines provided in Best Practice for Query Writing.
When running a search in LogScale an indicator of the system resources used to run the search is provided by LogScale and shown as Work at the bottom of the search window. (as highlighted in red in below screenshot) This is a useful relative measure of the amount of compute needed to run the search so you can try out different search queries for efficiency.
Figure 184. Query Language
Dashboard Best Practices
Dashboards are a great way to summarise key information from the logs and to engage users. Dashboards can contain many different widgets (e.g. charts, graphs, tables).
Think about whether it is better to have a single dashboard with many widgets or whether there is a logical grouping of content which could mean that multiple different dashboards , each with widgets relevant to a particular use are more appropriate.
Be clear about the expected use of the dashboards and think how the user's next steps could be anticipated and catered for in the dashboards .
To make dashboards more useful you can use parameters to take input from the user to re-draw the dashboard based on their inputs.
These can be very useful when dealing with large sets of data as they allow the user to narrow the scope of the dashboard widget to a subset of data or a particular single value.
All LogScale customers have access to the built in CrowdStrike IOC (Indicators of Compromise) feed except LogScale Community Edition users.
If the package is relevant for security users consider whether it makes sense to include using the IOC feed in the dashboard. Highlighting any IP, domain or URLs that are present in the customer logs and matching the IOC feed could be very useful for the user.
New Dashboard Widgets
LogScale is constantly improving the available dashboard widgets so check Dashboards for updates. An example of this is the single value widget, which replaced the old gauge widget. The single value widget added the ability to have e.g. spark lines and trends attached to the shown value, which was a very useful addition for users.
Figure 185. New Widgets
Alerts and Saved Searches Best Practices
Make sure to follow the general best practices for the LogScale Query Language described here Best Practice for Query Writing. This will minimise system load and ensure optimal performance of the service. Careless or inefficient queries can cause performance issues.
Alerts can have labels attached to them which are displayed next to the Alert name in the list of Alerts and are searchable. This can be a useful way to tag the alert with meaningful data and to help when trying to locate Alerts with a certain tag.
For example the corelight/threathuntingguide package uses Labels to show the ATT&CK tactics and techniques that each Alert covers.
Naming and Informational Notes
Try giving clear and unambiguous names to all LogScale Dashboards and the titles of each widget on a dashboard . Some assets have the ability to add a description and this can be useful to aid understanding of what the asset does.