Falcon LogScale Documentation

This is the third page of the LogScale with Your Data tutorial. After reading through and doing everything suggested in the previous pages, you should have data in your repository on LogScale Cloud. You're now ready to search the data to see how LogScale performs with your data.

Figure 456. Page Not Found

Before working on creating a dashboard, let's verify the data is coming through with some simple searches. These will also be the basis of the dashboard we'll put together. First, let's check if web server log entries are being received in the repository. From a web browser, try to access a web page you know doesn't exist on your server, for your domain. For the example here, enter in the browser's address field the domain name of the server, with humio-test.html at the end — use the server's IP address if it doesn't have a domain name.

When you try to go to that web page, it will return status code 404, with a message of "Page Not Found" — which is what would we expect. While you're at it, refresh that page a few times, so you'll have several entries.

Now let's see if the repository in LogScale Cloud shows those events. Click on Search in the top menu of LogScale. Enter humio-test in the search field, the input-box near the top left. After you hit Enter or click on the purple button labeled Run, LogScale will retrieve all entries that match. If you click on one of those entries, your screen will look something like the screenshot in Figure 5.

Notice the details it's parsed. There's the name of repo or repository name (i.e., Testeroo in our example), followed by the type. In the screenshot here it says, accesslog. That basically tells us that it's coming from vector on our server. Further down you can see the file field says it's from the /var/log/httpd/access_log file. That's from Apache. Even further down it shows the status code of 404 and the URL that was entered after the domain name by the user: /humio-text.html.

Knowing that which web pages were not found by users might be something you might want to monitor. They could be caused by broken internal links or something else you can fix easily. We'll put this query in a dashboard for monitoring: a dashboard is where you create widgets to store and view searches, which we'll cover on the next page.

Instead of searching for this particular URL, let's tell it to search for the status code, 404. Change the search to statuscode = "404" and see what you get. It may contain more entries if anyone else has had the same problem finding a web page on your server.

Now let's make two widgets based on status code. The first one will be displayed as a time based graph, but for all HTTP status codes. Enter the following in the search field:

timechart(statuscode)

This will show a chart, an area graph with a different color for each status code. Let's change the look a bit. Click on the gray button labeled, Style near the top left. That will open a box for choosing different settings for how the data will be displayed. Play with some of the choices.

Figure 457. Page Not Found Chart

For the screenshot here, we chose to leave it as an area style, but set it to Stack the values instead of overlaying them. We also changed the interpolation type to Step After. The problem with this graph is that there are mostly found pages, status code 200 — in purple. It's hard to notice the not-found pages, which is important. We'll do something about that in a bit.

Let's save this query as it is to a widget. Click on the gray button at the top right labeled, Save As. From the small pull-down menu that appears, select Dashboard Widget. A box will then appear that asks as to which dashboard to save it. We need to create one, so select New Dashboard. Then you'll have to give the new dashboard a name, maybe something like Web Server Dashboard. You'll also have to enter a name for the widget. Put HTTP Status Codes or something similar. You can enter a description of the widget if you want. Before finishing, uncheck the box where it says, Open Dashboard After Save. We have a little more to do on this screen. When you're ready, click on the purple button labeled, Save.

To be able to see easily the not-found pages in your dashboard, let's change the search query. This won't affect the widget you just created. For the new query, let's get events with a status code of 404 and put those results in a table, rather than a graph. To do this, we'll have to enter a little more complicated query. Enter the following in the search field and click Run when you're ready:

statuscode = "404"
| top(url, limit=30)
| table([url, _count], sortby=_count)

The first line here will give you all of the entries in the repository with a statuscode of 404. The second line says to group the results based on URL and to return the top thirty URLs. The third line says to put the results in a table with two columns — the URL and the number of occurrences for each URL — and to sort them based on the count of each URL.

When this query is run, you should see an ordered list of all not-found pages, with the ones with the highest count at the top. Let's turn this into a widget for the dashboard. Click the Save As button and select Dashboard Widget. This time you'll see that it offers to save it to the dashboard you already created. Use it; don't create a new dashboard. Enter an appropriate widget title and description. Then press Save — without unchecking the Open Dashboard... box.

On the next page of this tutorial, we'll look at the two widgets in the dashboard and see how we might improve them and add more widgets.