Events
Part of our Foundational Concepts series:
Previous Concept Ingest Flow
Next Concept: Queries
The data stored in Humio are called events. An event is a piece of data and an associated timestamp.
Examples of events include:
Log Lines
Metric Data
Analytics Data
However, any piece of data with an associated timestamp can be thought of as an event.
When data is sent to Humio the associated parser converts the data into an event. For example, suppose Humio received this log line:
[2018-10-11 22:00:10] INFO - User Logged In. userId=97110
The result might be turned into an event data containing the following fields:
Field | Value |
---|---|
@rawstring
| [2018-10-11 22:00:10] INFO - User Logged In. userId=97110 |
@id
| 3gqidgqi_uwgdwqu121duqgdw2iqwud_721gqwdugqdwu1 |
@timestamp
| 2018-10-11 22:00:10 |
@timezone
| Z |
#repo
| server-logs |
#type
| my-parser |
loglevel
| INFO |
message
| User Logged In |
userId
| 97110 |
Field Types
There are three types of fields:
Metadata Fields
Each event has some metadata attached to it on ingestion; all events
will have an @id
, @timestamp
,
@timezone
, and @rawstring
field.
From Humio version 1.15 there is also a
@ingesttimestamp
that holds the time when the event
ingested. From Humio version 1.17 there is also a
@timestamp.nanos
for extended precision of
timestamp below millisecond.
Notice that all metadata fields start with @
to
make them easy to identify.
The two most important are @timestamp
and
@rawstring
and will be described in detail below.
Tag Fields
Tag fields define how events are physically stored and indexed. They are also used for speeding up queries.
Users can associate custom tags as part of the parsing and ingestion
process but their use is usually very limited. The only built-in tags
are #repo
and #type
and both are
described in detail below.
Usually the client sending data to Humio will be configured to include
#host
and #source
tags that
contain the hostname and file that the event was read from.
User Fields
Any field that is not a tag or metadata is a user field. They are extracted at ingest by a parser or at query time by a regular expression. User fields are usually the interesting part of an event, containing application specific information.
Field @rawstring
Humio represents the original text of the event in the
@rawstring
attribute.
One of the greatest strengths of Humio is that it keeps the original data and nothing is thrown away at ingest. This allows you to do free-text searching across all logs and to extract virtual fields at query time for parts of the data you did not even know would be important.
You can read more about free-text search and extracting fields in the search documentation.
Field @timestamp
The timestamp of an event is represented in the
@timestamp
field. This field defines where the
event is stored in Humio's database and is what defines whether an
event is included in search results when searching a time range.
The timestamp needs special treatment when parsing incoming data during ingestion.
Field @ingesttimestamp
The timestamp of when an event was ingested is represented in the
@ingesttimestamp
field. The value is
milliseconds-since-epoch. Searches can restrict the data they search
using this timestamp. Currently this can be done by including
@ingesttimestamp > X AND @ingesttimestamp < Y
early in the search pipeline, where X and Y
are the
bounds as milliseconds since 1/1/1970. Future versions of Humio will
allow selecting these numbers using the time selector in the UI as
well.
Field #repo
All events have a special #repo
tag that denotes
the repository that
the event is stored in. This is useful in cross-repository searches
when using views.
Field #type
The type field is the name of the parser used to ingest the data.
Part of our Foundational Concepts series:
Previous Concept Ingest Flow
Next Concept: Queries