Available:readFile() v1.130.0
The readFile() function is available from
v1.130.0
The readFile() function outputs the content
of CSV lookup files or ad-hoc tables as events. This allows you
to use a CSV Lookup
File and ad-hoc table as data input. For more information
about ad-hoc tables, see
Using Ad-hoc Tables.
Note
It is recommended to use the readFile()
function at the beginning of the query. Using the function
later in the query will always discard anything before it, and
only return the content of the files or tables.
| Parameter | Type | Required | Default Value | Description |
|---|---|---|---|---|
file[a] | array of file/table names | required | The names of the input files or input tables. You can use table as the parameter name instead of file. | |
include | array of strings | optional[b] | Specifies the column names to read in the lookup file. If no argument is given, all columns are included. | |
limit | number | optional[b] | Limits the number of rows returned. Use limit=N to preview the first N rows of the files and tables. The files or tables will be outputted in the specified order, until the limit has been reached. | |
| Minimum | 1 | |||
[b] Optional parameters use their default value unless explicitly set. | ||||
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
filecan be omitted; the following forms of this function are equivalent:logscale SyntaxreadFile("value")and:
logscale SyntaxreadFile(file="value")These examples show basic structure only.
The readFile() function requires one of
these file or table sources to be available:
An uploaded file (see Upload Files).
An ad-hoc table defined in the query.
An installed package file.
When using the readFile() function, it will
output each file or table as an event per row. The order of the
output is as follows:
The files or tables will be outputted in the order specified in the file or table parameter.
For each file or table, the rows will be outputted as events in the order they are in the file or table.
When reading a file from a package, the package name should be specified in addition to the filename. For example:
readFile("falcon/investigate/logoninfo.csv")For more information on referring to package resources, see Referencing Package Assets.
If you are aiming to preview the content of large files,
LogScale recommends always including the
limit parameter to
ensure optimal UI performance. However, when the file is
utilized as data input for further manipulation, the
limit parameter
can be omitted.
readFile()Examples
Click next to an example below to get the full details.
Perform a Right Join Query to Combine Two Datasets
Query
defineTable(name="users",query={orgId=1},include=[username, name])
| defineTable(name="operations",query={*},include=[username, operation])
| readFile(users)
| match(operations, field=username, strict=false)
| select([username, operation])Introduction
In this example, the defineTable() function is used
as a right join query to extract and combine information from two
different datasets.
The event set for the query is in one repository, but the event set for each query is shown separately to identify the two sets of information. The first event set is:
| username | name | orgId |
|---|---|---|
| user1 | John Doe | 1 |
| user2 | Jane Doe | 1 |
| user3 | Bob Smith | 2 |
and the other event set:
| username | operation |
|---|---|
| user1 | createdFile |
| user3 | createdFile |
Step-by-Step
Starting with the source repository events.
- logscale
defineTable(name="users",query={orgId=1},include=[username, name])Generates an ad-hoc table named
usersthat has the fields username and name and includes users where orgId field equals1. - logscale
| defineTable(name="operations",query={*},include=[username, operation])Defines a new ad-hoc table that uses all the fields (username and operation) in a table named
operations. - logscale
| readFile(users)Reads the
usersad-hoc table as events usingreadFile(). - logscale
| match(operations, field=username, strict=false)Matches the events that have a matching operation from the
operationstable with theuserstable using the username as the common field. Events are not filtered if the events do not match, (implying a right join), by usingstrict=false - logscale
| select([username, operation])Selects the username and operation fields to be displayed from the event set.
Event Result set.
Summary and Results
The result will output two events:
| username | operation |
|---|---|
| user1 | createdFile |
| user2 | no value |
Note that in the event set all operations have been included even when
there is no match between the
username field, resulting in the
no value for
user2. If
strict=true had been used to
the match() function, then the event for
user2 would not have been outputted.
Preview Content in a Lookup File With readFile()
Preview content in a lookup file in the search portion of a repo without having to match the lookup against data
Query
readFile("host_names.csv")Introduction
In this example, the readFile() function is used to
look up a host_names.csv file just to preview the content in it.
Example incoming data might look like this:
|--------------------|
| host_name, host_id |
| DESKTOP-VSKPBK8, 1 |
| FINANCE, 2 |
| homer-xubuntu, 3 |
| logger, 4 |
| DESKTOP-1, 5 |
| DESKTOP-2, 6 |
| DESKTOP-3, 7 |
|--------------------|Step-by-Step
Starting with the source repository events.
- logscale
readFile("host_names.csv")Displays the content of the .csv file.
If you aim to preview the content of large files, we recommend always including the
limitparameter to ensure optimal UI performance. For example:readFile("host_names.csv", limit=5). However, if the file is utilized as data input for further manipulation, thelimitparameter can be omitted.Notice that if reading a file from a package, then the package name should be specified in addition to the filename. For example:
readFile("falcon/investigate/logoninfo.csv"). Event Result set.
Summary and Results
The query is used to preview content in CSV Lookup Files. After
previewing the content with the readFile()
function, it is possible to use the data for further manipulation, for
example combine it with count() to count the rows,
select() to filter data,
join() to match data, etc.
The readFile() function can also be used to read
tables defined with the defineTable() function. See
Perform a Right Join Query to Combine Two Datasets
Sample output from the incoming example data:
| host_id | host_name |
|---|---|
| 1 | DESKTOP-VSKPBK8 |
| 2 | FINANCE |
| 3 | homer-xubuntu |
| 4 | logger |
| 5 | DESKTOP-1 |
| 6 | DESKTOP-2 |
| 7 | DESKTOP-3 |
Sample output from the incoming example data with
limit parameter:
| host_id | host_name |
|---|---|
| 1 | DESKTOP-VSKPBK8 |
| 2 | FINANCE |
| 3 | homer-xubuntu |
| 4 | logger |
| 5 | DESKTOP-1 |
Preview Content in a Lookup File With readFile() and Filter With !join()
Preview content in a lookup file in the search portion of a repo
and filter for specific data with the !join()
function
Query
readFile("host_names.csv")
| !join(query={groupBy(host_name)}, field=host_name, key=host_name, include=[host_name, id])Introduction
In this example, the readFile() function is used to
look up a host_names.csv file, and then filter for host names that do
not send any logs.
Example incoming data might look like this:
|--------------------|
| host_name, host_id |
| DESKTOP-VSKPBK8, 1 |
| FINANCE, 2 |
| homer-xubuntu, 3 |
| logger, 4 |
| DESKTOP-1, 5 |
| DESKTOP-2, 6 |
| DESKTOP-3, 7 |
|--------------------|Step-by-Step
Starting with the source repository events.
- logscale
readFile("host_names.csv")Displays the content of the .csv file.
- logscale
| !join(query={groupBy(host_name)}, field=host_name, key=host_name, include=[host_name, id])Filters for host names that do not send any logs.
Event Result set.
Summary and Results
The query is used to preview content in CSV Lookup Files, and then filter for host names that do not send any logs.
Sample output from the incoming example data:
| host_id | host_name |
|---|---|
| 5 | DESKTOP-1 |
| 6 | DESKTOP-2 |
| 7 | DESKTOP-3 |