Parse a CSV-encoded field into known columns. It can parse values of the form:
value 1, value 2, value 3
"value 1", "value 2", value 3
(Quoted values. Quotes are optional.)"value 1"; "value 2"; value 3
(Using ; as delimiter. Delimiter is configurable.)
Parameter | Type | Required | Default Value | Description |
---|---|---|---|---|
columns | string or array | required | Names of columns to extract from field. | |
delimiter | string | optional[a] | , | Delimiter character to split records by. |
excludeEmpty | boolean | optional[a] | false | If the value of a column is empty, exclude the field. |
field [b] | string | required | @rawstring | Field that holds the input in CSV form. |
trim | boolean | optional[a] | false | Allows to ignore whitespace before and after values. If the value is quoted, the quotes can start after the spaces. Useful for parsing data created by sources that do not adhere to the CSV standard. |
[a] Optional parameters use their default value unless explicitly set. |
Hide omitted argument names for this function
Omitted Argument NamesThe argument name for
field
can be omitted; the following forms of this function are equivalent:logscale SyntaxparseCsv("value",columns="value")
and:
logscale SyntaxparseCsv(field="value",columns="value")
These examples show basic structure only.
For a log line like this:
2017-02-22T13:14:01.917+0000 [main thread] INFO statsModule got result="117,success,27%,3.14"
Using parseCsv(result, columns=[count, status,
completion, precision, sourcetask])
will add these fields:
count | 117 |
status | success |
completion | 27% |
precision | 3.14 |
sourcetask will not get assigned a value, as there were too few columns in the input for that.
Use the (unnamed) field
parameter to specify which
field should be CSV parsed. Specify @rawstring to
parse the rawstring.
parseCsv()
Examples
Click
next to an example below to get the full details.Parse String as CSV
Parse a CSV-encoded field into known columns using parseCsv()
function
Query
kvparse()
parseCsv(result, columns=[count, status,
completion, precision, sourcetask])
Introduction
The parseCsv()
function can be used to Parse
a CSV-encoded field into known columns.
Example incoming data might look like this:
2017-02-22T13:14:01.917+0000 [main thread] INFO statsModule got result="117 ,success ,27%% ,3.14"
Step-by-Step
Starting with the source repository events.
- logscale
kvparse()
Parses the string into key value pairs.
- logscale
parseCsv(result, columns=[count, status, completion, precision, sourcetask])
CSV parses the result field from a log line (extracted by the
kvParse()
function) and adds the following fields to the event: count with the value117
, status with the valuesuccess
, completion with the value27%%
, and precision with the value3.14
. Event Result set.
Summary and Results
The query is used to parse a string as CSV.
Sample output from the incoming example data:
"completion","count","precision","result","status"
"27% ","117 ","3.14 ","117 ,success ,27% ,3.14 ","success "
Parse String as CSV - Example 2
Parse a CSV-encoded field into known columns using parseCsv()
function and trim parameter defined
Query
parseCsv(fields=[status, hosts, rest], trim=true)
Introduction
The parseCsv()
function can be used to Parse
a CSV-encoded field into known columns. In this example, the
parseCsv()
function is used to parse a log
line with spaces and quotes and trim the output. Trimming the
output is done by setting the trim
parameter to true
. When true
and using
quotes with trim, the spaces inside the quotes are not removed,
but the quotes may come after spaces.
Example incoming data might look like this:
117, " crowdstrike.com, logscale.com ", 3.14
Step-by-Step
Starting with the source repository events.
- logscale
parseCsv(fields=[status, hosts, rest], trim=true)
CSV parses the columns field from a log line and adds the following fields to the event: status with the value
117,
, hosts with the value" crowdstrike.com, logscale.com \"
, rest with the value3.14"
. Event Result set.
Summary and Results
The query is used to parse a string as CSV.
Note that if you use quotes with trim
the
behavior is as follows:
When
trim
set totrue
, spaces around the separation character (for example a comma) are ignored, but retained within quoted columns. For example:117 , " crowdstrike.com, humio.com " , 3.14
Would identify three columns:
csv117," crowdstrike.com, humio.com ",3.14
Retaining the spaces at the beginning and end of a quoted column.
Without trim (
trim=false
), the spaces around the character separated would be included in the values. For example:117 , " crowdstrike.com, humio.com " , 3.14
Would identify three columns:
csv"117 "," crowdstrike.com, humio.com "," 3.14"
In the preceding example, there are spaces after and before columns due to the spaces around the comma separator.