Remove Trailing Whitespace From Log Messages

Clean up log messages using the text:trim() function

Query

flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1["Expression"] result{{Result Set}} repo --> 1 1 --> result
logscale
text:trim(message, mode=end, as="cleaned_message")

Introduction

The text:trim() function can be used to remove whitespace from strings. When using the mode parameter with value end, it removes only trailing whitespace characters. The as parameter specifies the name of the field where the trimmed result is stored.

In this example, the text:trim() function is used to clean up log messages that contain unnecessary trailing whitespace characters, which often occur when log messages are padded to a fixed width or when concatenating strings.

Example incoming data might look like this:

csv
"@timestamp","level","message"
"2025-11-05T10:00:00Z","INFO","User login successful "
"2025-11-05T10:00:01Z","WARN","Failed password attempt "
"2025-11-05T10:00:02Z","INFO","Session timeout "
"2025-11-05T10:00:03Z","ERROR","Database connection lost "
"2025-11-05T10:00:04Z","INFO","Cache cleared "
"2025-11-05T10:00:05Z","DEBUG","Memory usage: 85% "

Step-by-Step

  1. Starting with the source repository events.

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1["Expression"] result{{Result Set}} repo --> 1 1 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
    logscale
    text:trim(message, mode=end, as="cleaned_message")

    Creates a field named cleaned_message that contains the message text with trailing whitespace removed. The text:trim() function with mode=end only removes whitespace characters from the end of the string, preserving any leading spaces if they exist. The as parameter specifies the output field name.

  3. Event Result set.

Summary and Results

The query is used to standardize log messages by removing unnecessary trailing whitespace.

This query is useful, for example, to clean up fixed-width formatted logs, improve data consistency for downstream processing, or prepare messages for exact string matching operations where trailing spaces might cause false negatives.

Sample output from the incoming example data:

csv
"cleaned_message","level","message"
"User login successful","INFO","User login successful "
"Failed password attempt","WARN","Failed password attempt "
"Session timeout","INFO","Session timeout "
"Database connection lost","ERROR","Database connection lost "
"Cache cleared","INFO","Cache cleared "
"Memory usage: 85%","DEBUG","Memory usage: 85% "

Note that internal spaces between words are preserved, only the trailing whitespace is removed.

This is particularly useful when working with fixed-width formatted logs where trailing spaces are used for padding but are not meaningful for analysis.