How-To: Redacting Data from a Repository

There may be certain data that you don't want stored in a LogScale repository, maybe not whole events, but specific text contained in events. For example, someone's password might have been inadvertently logged and stored in plain text in a repository. Another example could be that someone under the European GDPR has requested all information on them not be saved.

The best practice regarding these situations is either not to send the data to LogScale, or to have LogScale not store the data. For the first preventive measure, you might configure your log shipper to filter out passwords and other sensitive data.

For the second measure, you could configure the parser you assign to a datasource so as not to record specific data. You might configure a parser like so:

logscale Syntax
parseJson()
|
case {
   data=sensitive
| dropEvent();
   password=*
| replace(field=password,with="XXXXXX");
}

These measures should help greatly to reduce the amount of sensitive data that is recorded. However, there may still be data that makes it through and is stored in a repository. For those, you'll have to redact the specific text.

Solution

You can't use the LogScale web interface to delete text contained in an event entry in a repository. Instead, you'll have to do this from the command-line, using the Redact Events API.

To eliminate specific events you can use the redactEvents() GraphQL mutation, which supports removal of a small number of events from LogScale.