Action Type: Lookup File

Security Requirements and Controls

The Lookup File action creates a CSV lookup file from the events and uploads the file to LogScale in the repository/view of the action. The CSV lookup file can then be used in the match() function, among others.

It is recommended to use this action with a search that only returns the fields that are needed as columns in the CSV file. This can, for instance, be achieved by using the select() query function.

For more information about lookup files in LogScale, see Lookup Files.

Screenshot of the Lookup File Action configuration form displaying options to select existing file or create new file, file name input, update behavior dropdown with choices for overwrite, append, or update changed, and key column selection for matching rows

Figure 300. Configuring Lookup File Action


Parameter Description
Action Name The name provided for the action.
Select existing file If you want to use an existing lookup file for this action, choose Select existing file.
File Name When creating a new lookup file, the file name for the CSV file. Type the name of the new lookup file to create. The file name must include the .csv extension.
File The file name for the CSV file, if you choose Select existing file. Select an existing file from the drop-down list.
Update behavior Choose between the following options:

  • Overwrite contents replaces the contents of the file with the query results each time the action runs. Because the file is fully replaced each time, the query must run over the full search interval to produce a complete snapshot of the data.

  • Append results adds the new query results to the existing contents; changed results are not updated. This might result in duplicate rows. Because only new results are added, the query can run over a smaller, recent search interval. Rows that become obsolete are never removed. For information on how to remove obsolete rows, see Removing obsolete rows.

  • Update changed updates the lookup file with changed content and new content each time the action runs based on the key columns selected. When a row's key columns match the query results, LogScale replaces the matching rows. So fields that are not on the updated row will be removed, and new fields may also be added. The rows that do not match will be appended to the existing contents. Because only changed and new results are processed, the query can run over a smaller, recent search interval. Rows that become obsolete are never removed. For information on how to remove obsolete rows, see Removing obsolete rows.

Tip

Use Append results or Update changed when possible. These behaviors allow the query to run over a smaller search interval rather than the full interval required by Overwrite contents.

Key column selection If Update behavior is Update changed, you must select one or more columns to use as key values on which to match. This functionality works in the same way as the match(), in that if multiple key columns are selected, then all of them must match for the row to be updated. If no rows match the key values, then the content from the query results is appended to the lookup file. Deselect Match case sensitive if you do not require that the match is case sensitive when matching.

Go to Resources and select Files to find the lookup file.

Whenever the action triggers, the file is either appended, updated, or overwritten, depending on your configuration. If your action is configured to overwrite, then the action must receive all events necessary to populate the file, and not just the changes since the action was last triggered. If your action is configured to append, then the action receives all changes since the action was last triggered.

Note that the default file size allowed for lookup files is 200 MB. If the action will update or append the file in such a way that the file size is exceeded then it will fail. You can update the file manually to reduce the file size, create a new lookup file for the action to write to, or change the query to reduce the data populated to the file.

Removing obsolete rows

When using Append results or Update changed, rows that become obsolete are never automatically removed from the lookup file. To remove obsolete rows, set up a separate scheduled search that periodically replaces the file with a full, current snapshot:

  1. Create a new scheduled search configured to run infrequently, for example once a day or once a week.

  2. Configure the query to run over the full search interval so it returns all current data.

  3. Create a new Lookup File action configured with Overwrite contents, pointing to the same lookup file.

  4. Add the new action to the scheduled search.

Each time the scheduled search runs, it replaces the entire file with a complete snapshot of the data, removing any rows that are no longer in the results.