Available:readFile() v1.130.0
The readFile()
function is available from
v1.130.0
The readFile()
function outputs the content
of CSV lookup files or ad-hoc tables as events. This allows you
to use a CSV Lookup
File and ad-hoc table as data input.
readFile()
can also be used to combine
multiple CSV files and tables, regardless if the structure is
identical.
For more information about ad-hoc tables, see Using Ad-hoc Tables.
Note
It is recommended to use the readFile()
function at the beginning of the query. Using the function
later in the query will always discard anything before it, and
only return the content of the files or tables.
Parameter | Type | Required | Default Value | Description |
---|---|---|---|---|
file [a] | array of file/table names | required | The name of the input files or input tables. In case of ad-hoc tables, you can alternatively use table as an alias of the file parameter. | |
include | array of strings | optional[b] | Specifies the column names to read in the lookup file. If no argument is given, all columns are included. | |
limit | number | optional[b] | Limits the number of rows returned. Use limit=N to preview the first N rows of the files and tables. The files or tables will be outputted in the specified order, until the limit has been reached. | |
Minimum | 1 | |||
[b] Optional parameters use their default value unless explicitly set. |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
file
can be omitted; the following forms of this function are equivalent:logscale SyntaxreadFile("value")
and:
logscale SyntaxreadFile(file="value")
These examples show basic structure only.
readFile()
Function Operation
The readFile()
function requires one of
these file or table sources to be available:
An uploaded file (see Upload Files).
An ad-hoc table defined in the query.
An installed package file.
When using the readFile()
function, it
will output each file or table as an event per row. The order
of the output is as follows:
The files or tables will be outputted in the order specified in the file or table parameter.
For each file or table, the rows will be outputted as events in the order they are in the file or table.
When reading a file from a package, the package name should be specified in addition to the filename. For example:
readFile("falcon/investigate/logoninfo.csv")
For more information on referring to package resources, see Referencing Package Assets.
If you are aiming to preview the content of large files,
LogScale recommends always including the
limit
parameter
to ensure optimal UI performance. However, when the file is
utilized as data input for further manipulation, the
limit
parameter
can be omitted.
readFile()
Examples
Click
next to an example below to get the full details.Concatenate Multiple CSV Files
Combine data from multiple CSV files into a single result set
using the readFile()
function
Query
readFile("users_2024.csv", "users_2025.csv", "temp_users.csv")
Introduction
In this example, the readFile()
function is used to
concatenate data from multiple CSV files named
users_2024.csv
, users_2025.csv
,
and temp_users.csv
, combining all events into a
single result set.
Example data in the CSV files might look like this:
File users_2024.csv
:
user_id | name | department | join_date |
---|---|---|---|
001 | Alice Johnson | Engineering | 2024-01-15 |
002 | Bob Smith | Marketing | 2024-03-22 |
003 | Carol Davis | Sales | 2024-06-10 |
File users_2025.csv
:
user_id | name | department | join_date |
---|---|---|---|
004 | David Wilson | Engineering | 2025-01-08 |
005 | Emma Brown | HR | 2025-02-14 |
006 | Frank Miller | Sales | 2025-03-01 |
File temp_users.csv
:
user_id | name | department | join_date |
---|---|---|---|
007 | Grace Lee | Marketing | 2025-07-20 |
008 | Henry Taylor | Engineering | 2025-08-15 |
Step-by-Step
Starting with the source repository events.
- logscale
readFile("users_2024.csv", "users_2025.csv", "temp_users.csv")
Reads and concatenates data from three CSV files:
users_2024.csv
,users_2025.csv
, andtemp_users.csv
. ThereadFile()
function processes each CSV file in the order specified and combines all events into a single result set, with events from the first file appearing first, followed by events from subsequent files. Event Result set.
Summary and Results
The query is used to combine data from multiple CSV files into a single unified event set by concatenating their contents.
This query is useful, for example, to merge data exports from different time periods, combine CSV files from multiple departments or regions, or consolidate data from various external sources for comprehensive analysis.
Sample output from the incoming example data:
user_id | name | department | join_date |
---|---|---|---|
001 | Alice Johnson | Engineering | 2024-01-15 |
002 | Bob Smith | Marketing | 2024-03-22 |
003 | Carol Davis | Sales | 2024-06-10 |
004 | David Wilson | Engineering | 2025-01-08 |
005 | Emma Brown | HR | 2025-02-14 |
006 | Frank Miller | Sales | 2025-03-01 |
007 | Grace Lee | Marketing | 2025-07-20 |
008 | Henry Taylor | Engineering | 2025-08-15 |
Note that the CSV files could be different formats (field names and content), but typically the schemas would match which will make it easier to query and display in a widget.
The events appear in the output in the same order as the files are specified in the function call, with all events from the first file appearing before any events from the second file, and so on.
Concatenate Multiple Tables
Combine data from multiple tables into a single result set using
the readFile()
function
Query
readFile("users_2024", "users_2025", "temp_users")
Introduction
In this example, the readFile()
function is used to
concatenate data from multiple lookup tables named
users_2024
, users_2025
, and
temp_users
, combining all events into a single result
set.
Example data in the lookup tables might look like this:
Table users_2024
:
user_id | name | department | join_date |
---|---|---|---|
001 | Alice Johnson | Engineering | 2024-01-15 |
002 | Bob Smith | Marketing | 2024-03-22 |
003 | Carol Davis | Sales | 2024-06-10 |
Table users_2025
:
user_id | name | department | join_date |
---|---|---|---|
004 | David Wilson | Engineering | 2025-01-08 |
005 | Emma Brown | HR | 2025-02-14 |
006 | Frank Miller | Sales | 2025-03-01 |
Table temp_users
:
user_id | name | department | join_date |
---|---|---|---|
007 | Grace Lee | Marketing | 2025-07-20 |
008 | Henry Taylor | Engineering | 2025-08-15 |
Step-by-Step
Starting with the source repository events.
- logscale
readFile("users_2024", "users_2025", "temp_users")
Reads and concatenates data from three lookup tables:
users_2024
,users_2025
, andtemp_users
. ThereadFile()
function processes each table in the order specified and combines all events into a single result set, with events from the first table appearing first, followed by events from subsequent tables. Event Result set.
Summary and Results
The query is used to combine data from multiple lookup tables into a single unified event set by concatenating their contents.
This query is useful, for example, to merge historical data stored in separate yearly tables, combine data from different regional databases, or consolidate temporary and permanent datasets for comprehensive analysis.
Sample output from the incoming example data:
user_id | name | department | join_date |
---|---|---|---|
001 | Alice Johnson | Engineering | 2024-01-15 |
002 | Bob Smith | Marketing | 2024-03-22 |
003 | Carol Davis | Sales | 2024-06-10 |
004 | David Wilson | Engineering | 2025-01-08 |
005 | Emma Brown | HR | 2025-02-14 |
006 | Frank Miller | Sales | 2025-03-01 |
007 | Grace Lee | Marketing | 2025-07-20 |
008 | Henry Taylor | Engineering | 2025-08-15 |
Note that the tables could be different formats (field names and content), but typically the schemas would match which will make it easier to query and display in a widget.
The events appear in the output in the same order as the files are specified in the function call, with all events from the first table appearing before any events from the second table, and so on.
Perform a Right Join Query to Combine Two Datasets
Query
defineTable(name="users",query={orgId=1},include=[username, name])
| defineTable(name="operations",query={*},include=[username, operation])
| readFile(users)
| match(operations, field=username, strict=false)
| select([username, operation])
Introduction
In this example, the defineTable()
function is used
as a right join query to extract and combine information from two
different datasets.
The event set for the query is in one repository, but the event set for each query is shown separately to identify the two sets of information. The first event set is:
username | name | orgId |
---|---|---|
user1 | John Doe | 1 |
user2 | Jane Doe | 1 |
user3 | Bob Smith | 2 |
and the other event set:
username | operation |
---|---|
user1 | createdFile |
user3 | createdFile |
Step-by-Step
Starting with the source repository events.
- logscale
defineTable(name="users",query={orgId=1},include=[username, name])
Generates an ad-hoc table named
users
that has the fields username and name and includes users where orgId field equals1
. - logscale
| defineTable(name="operations",query={*},include=[username, operation])
Defines a new ad-hoc table that uses all the fields (username and operation) in a table named
operations
. - logscale
| readFile(users)
Reads the
users
ad-hoc table as events usingreadFile()
. - logscale
| match(operations, field=username, strict=false)
Matches the events that have a matching operation from the
operations
table with theusers
table using the username as the common field. Events are not filtered if the events do not match, (implying a right join), by usingstrict=false
- logscale
| select([username, operation])
Selects the username and operation fields to be displayed from the event set.
Event Result set.
Summary and Results
The result will output two events:
username | operation |
---|---|
user1 | createdFile |
user2 | no value |
Note that in the event set all operations have been included even when
there is no match between the
username field, resulting in the
no value
for
user2
. If
strict=true
had been used to
the match()
function, then the event for
user2
would not have been outputted.
Preview Content in a Lookup File With readFile()
Preview content in a lookup file in the search portion of a repo without having to match the lookup against data
Query
readFile("host_names.csv")
Introduction
In this example, the readFile()
function is used to
look up a host_names.csv file just to preview the content in it.
Example incoming data might look like this:
|--------------------|
| host_name, host_id |
| DESKTOP-VSKPBK8, 1 |
| FINANCE, 2 |
| homer-xubuntu, 3 |
| logger, 4 |
| DESKTOP-1, 5 |
| DESKTOP-2, 6 |
| DESKTOP-3, 7 |
|--------------------|
Step-by-Step
Starting with the source repository events.
- logscale
readFile("host_names.csv")
Displays the content of the .csv file.
If you aim to preview the content of large files, we recommend always including the
limit
parameter to ensure optimal UI performance. For example:readFile("host_names.csv", limit=5)
. However, if the file is utilized as data input for further manipulation, thelimit
parameter can be omitted.Notice that if reading a file from a package, then the package name should be specified in addition to the filename. For example:
readFile("falcon/investigate/logoninfo.csv")
. Event Result set.
Summary and Results
The query is used to preview content in CSV Lookup Files. After
previewing the content with the readFile()
function, it is possible to use the data for further manipulation, for
example combine it with count()
to count the rows,
select()
to filter data,
join()
to match data, etc.
The readFile()
function can also be used to read
tables defined with the defineTable()
function. See
Perform a Right Join Query to Combine Two Datasets
Sample output from the incoming example data:
host_id | host_name |
---|---|
1 | DESKTOP-VSKPBK8 |
2 | FINANCE |
3 | homer-xubuntu |
4 | logger |
5 | DESKTOP-1 |
6 | DESKTOP-2 |
7 | DESKTOP-3 |
Sample output from the incoming example data with
limit
parameter:
host_id | host_name |
---|---|
1 | DESKTOP-VSKPBK8 |
2 | FINANCE |
3 | homer-xubuntu |
4 | logger |
5 | DESKTOP-1 |
Preview Content in a Lookup File With readFile()
and Filter With !join()
Preview content in a lookup file in the search portion of a repo
and filter for specific data with the !join()
function
Query
readFile("host_names.csv")
| !join(query={groupBy(host_name)}, field=host_name, key=host_name, include=[host_name, id])
Introduction
In this example, the readFile()
function is used to
look up a host_names.csv file, and then filter for host names that do
not send any logs.
Example incoming data might look like this:
|--------------------|
| host_name, host_id |
| DESKTOP-VSKPBK8, 1 |
| FINANCE, 2 |
| homer-xubuntu, 3 |
| logger, 4 |
| DESKTOP-1, 5 |
| DESKTOP-2, 6 |
| DESKTOP-3, 7 |
|--------------------|
Step-by-Step
Starting with the source repository events.
- logscale
readFile("host_names.csv")
Displays the content of the .csv file.
- logscale
| !join(query={groupBy(host_name)}, field=host_name, key=host_name, include=[host_name, id])
Filters for host names that do not send any logs.
Event Result set.
Summary and Results
The query is used to preview content in CSV Lookup Files, and then filter for host names that do not send any logs.
Sample output from the incoming example data:
host_id | host_name |
---|---|
5 | DESKTOP-1 |
6 | DESKTOP-2 |
7 | DESKTOP-3 |