Hash Field Values Using hashRewrite()

Hash sensitive data using the hashRewrite() function with a salt value

Query

flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[\Add Field/] result{{Result Set}} repo --> 1 1 --> result
logscale
hashRewrite(ssn, salt="salt1")

Introduction

The hashRewrite() function can be used to hash values in a specified field using a salt value. A salt is a random string added to the data before hashing to make the hash more secure. Without a salt, identical values would create identical hashes, making them vulnerable to dictionary attacks. Adding a salt ensures that even identical values produce different hashes when using different salts.

In this example, the hashRewrite() function is used to hash Social Security Numbers (SSNs) using a specified salt value (salt equal to salt1).

Example incoming data might look like this:

@timestampuser_idssnaction
2025-09-01T10:00:00Zuser1123-45-6789profile_update
2025-09-01T10:00:05Zuser2123-45-6789new_account
2025-09-01T10:00:10Zuser3456-78-9012profile_view

Step-by-Step

  1. Starting with the source repository events.

  2. flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[\Add Field/] result{{Result Set}} repo --> 1 1 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
    logscale
    hashRewrite(ssn, salt="salt1")

    Hashes the values in the ssn field through a multi-step process. First the hashRewrite() function takes each value in the ssn field and adds the salt value salt1 to it before hashing. The function then creates a hash of the combined value (original + salt) and replaces the original value with the resulting hash.

    Note that the salt value requires specific handling for security purposes. It must be kept consistent when searching for these values later, stored securely outside the log data, and should be different for different types of sensitive data.

  3. Event Result set.

Summary and Results

The query is used to securely hash sensitive SSN values while maintaining the ability to search for specific SSNs later using the same salt.

The salt makes the hashing process more secure by:

  • Making the hash unique even if SSNs are identical.

  • Preventing the use of pre-computed hash tables (rainbow tables) to reverse the hash.

  • Adding an extra layer of security beyond the basic hash.

This query is useful, for example, when you need to protect sensitive data while still being able to analyze it.

Sample output from the incoming example data:

@timestampactionssnuser_id
2025-09-01T10:00:00Zprofile_updateC4ZkaokbIUltItryWgtdRmIdCCnsWVhhrOg3GDrTkx8user1
2025-09-01T10:00:05Znew_accountC4ZkaokbIUltItryWgtdRmIdCCnsWVhhrOg3GDrTkx8user2
2025-09-01T10:00:10Zprofile_viewnaHQMPbzY6pLiFG8aiJzfxw5Gj4mLQ+bf2b0AJv8OPQuser3

Notice that identical SSNs (first two rows) produce identical hashes because they use the same salt. If you used a different salt, then the same SSNs would produce different hashes.

The example demonstrates how to use hashRewrite() in LogScale to protect sensitive data while maintaining searchability. To search for specific SSNs in the hashed data, use hashMatch() with the same salt value.

After hashing sensitive data using this method, you can search for specific values using either Match Hashed Values in Specific Fields for field-specific searches or Match Events Containing Specific Hash Values for searching anywhere in the event data.