Using Lookup Files
Lookup files allow LogScale to use a previously uploaded file as the basis of the data to be used in the join. These lookups files can be used when there is a defined set of data.
The primary use case is to upload a CSV (of JSON) file that contains the lookup information. The file could come from another source including static lookup files, reference files, system data (for example the system services or network files), or data exported or generated from other applications and systems.
For example, using a file that has been uploaded that contains a defined set of protocols, lookups can be used to match protocols found in a query and showing the protocol name or port number.
To use lookup files in queries, the file must be uploaded to LogScale. This can be achieved through the UI using Lookup Files or GraphQL. Alternatively, the process can use information from a query and then automatically upload the file to be used in a lookup. See Using Lookups with Auto-updated Files for more information.
LogScale supports the following lookup methods:
Using
match()
to lookup data in query and extend the fields in a result set with the lookup information.Using
readFile()
as the file input to a query. The advantage of using thereadFile()
function instead of thematch()
function, is that the lookup will not be matched against data.
Using Lookups with Auto-updated Files
A typical use case with readFile()
is to use a
scheduled
search or alert with the
Upload
File action to automatically update the contents of a scheduled
search to a file to use as the basis for a join.
In the following diagram, a scheduled search is used to generate results that are uploaded as a file as part of the action:
For example, this could be used in the following situation:
A scheduled search periodically queries a repository to identify a list of users to be investigated.
When the scheduled search creates an output list, the data is uploaded as a file using the Upload File action.
A dashboard query executes a Scheduled Search that uses the uploaded file to search for suspicious usernames in a repository of logins.
Using the Scheduled Search method enables a join between a regularly executed query and a dataset. The two datasets could be across different repositories or the same.