Search a Repository
Part of our Humio with Your Data series:
This is the third page of the Humio with Your Data tutorial. After reading through and doing everything suggested in the previous pages, you should have data in your repository on Humio Cloud. You're now ready to search the data to see how Humio performs with your data.
Figure 302. Page Not Found
Before working on creating a dashboard, let's verify the data is coming
through with some simple searches. These will also be the basis of the
dashboard we'll put together. First, let's check if web server log entries
are being received in the repository. From a web browser, try to access a
web page you know doesn't exist on your server, for your domain. For the
example here, enter in the browser's address field the domain name of the
humio-test.html at the end — use the
server's IP address if it doesn't have a domain name.
When you try to go to that web page, it will return status code 404, with a message of "Page Not Found" — which is what would we expect. While you're at it, refresh that page a few times, so you'll have several entries.
Now let's see if the repository in Humio Cloud shows those events. Click
Search in the top menu of Humio. Enter
humio-test in the search field, the input-box near the
top left. After you hit Enter or click on the purple button labeled
Run, Humio will retrieve all entries that match. If you
click on one of those entries, your screen will look something like the
screenshot in Figure 5.
Notice the details it's parsed. There's the name of repo or repository
Testeroo in our example), followed by the
type. In the screenshot here it says,
accesslog. That basically tells us that it's coming
vector on our server. Further down you can see the
file field says it's from the
/var/log/httpd/access_log file. That's from Apache.
Even further down it shows the status code of 404 and the URL that was
entered after the domain name by the user:
Knowing that which web pages were not found by users might be something you might want to monitor. They could be caused by broken internal links or something else you can fix easily. We'll put this query in a dashboard for monitoring: a dashboard is where you create widgets to store and view searches, which we'll cover on the next page.
Instead of searching for this particular URL, let's tell it to search for
the status code, 404. Change the search to
"404" and see what you get. It may contain more entries if
anyone else has had the same problem finding a web page on your server.
Now let's make two widgets based on status code. The first one will be displayed as a time based graph, but for all HTTP status codes. Enter the following in the search field:
This will show a chart, an area graph with a different color for each status code. Let's change the look a bit. Click on the gray button labeled, Style near the top left. That will open a box for choosing different settings for how the data will be displayed. Play with some of the choices.
Figure 303. Page Not Found Chart
For the screenshot here, we chose to leave it as an area style, but set it
Stack the values instead of overlaying them. We also
changed the interpolation type to
Step After. The
problem with this graph is that there are mostly found pages, status code
200 — in purple. It's hard to notice the not-found pages, which is
important. We'll do something about that in a bit.
Let's save this query as it is to a widget. Click on the gray button at
the top right labeled,
Save As. From the small
pull-down menu that appears, select
Dashboard Widget. A
box will then appear that asks as to which dashboard to save it. We need
to create one, so select
New Dashboard. Then you'll
have to give the new dashboard a name, maybe something like Web Server
Dashboard. You'll also have to enter a name for the widget. Put HTTP
Status Codes or something similar. You can enter a description of the
widget if you want. Before finishing, uncheck the box where it says, Open
Dashboard After Save. We have a little more to do on this screen. When
you're ready, click on the purple button labeled,
To be able to see easily the not-found pages in your dashboard, let's
change the search query. This won't affect the widget you just created.
For the new query, let's get events with a status code of 404 and put
those results in a table, rather than a graph. To do this, we'll have to
enter a little more complicated query. Enter the following in the search
field and click
Run when you're ready:
statuscode = "404" | top(url, limit=30) | table([url, _count], sortby=_count)
The first line here will give you all of the entries in the repository
statuscode of 404. The second line says to group
the results based on URL and to return the top thirty URLs. The third line
says to put the results in a table with two columns — the URL and
the number of occurrences for each URL — and to sort them based on
the count of each URL.
When this query is run, you should see an ordered list of all not-found
pages, with the ones with the highest count at the top. Let's turn this
into a widget for the dashboard. Click the
button and select
Dashboard Widget. This time you'll
see that it offers to save it to the dashboard you already created. Use
it; don't create a new dashboard. Enter an appropriate widget title and
description. Then press
Save — without unchecking
the Open Dashboard... box.
On the next page of this tutorial, we'll look at the two widgets in the dashboard and see how we might improve them and add more widgets.