Limits & Standards

This page lists the various limits and standard operating parameters of LogScale. See Best Practice for the best practices relative to ingest via the Ingest API.

General Limits & Parameters

Below is a list of general limits and parameters:

Note

Unless otherwise specified, all multiple-byte data sizes are expressed in SI units using decimal (Base 10). For example:

  • 1KB = 1,000 Bytes

  • 1MB = 1,000,000 Bytes

  • 1GB = 1,000,000,000 Bytes

  • 1TB = 1,000,000,000,000 Bytes

Data Structure

LimitDefault ValueDynamic VariableConfig VariableAvailability

Max number of fields in an event. During ingest, fields are sorted alphabetically by name and the first fields are parsed, the remainder of the named fields are dropped. The @rawstring field is not modified and will contain all data.

  

MAX_EVENT_FIELD_COUNT

Variable default: 1,000 fields (from 1.0)

 
Max event size. When the configured event size max is reached, either in @rawstring and/or in other fields, the overall data will be truncated. Fields will be removed entirely, and @rawstring will be truncated down to the allowed max size with added ... at the end, such that the of all other fields + size of @rawstring is less than the configured max event size. Only @rawstring, @timestamp and @timezone are added when truncation occurs.

1 MB

 

MAX_EVENT_SIZE

Variable default: 1 MiB

 
Max file upload size (see Lookup Files)

2,048 MB

   

Query Related Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability

Max memory a live query can consume during its execution.

LiveQueryMemoryLimit determines how much memory in bytes a live query can consume during its execution. For non-live queries, their memory limit is determined by QueryMemoryLimit, which is 100MB by default. By default LiveQueryMemoryLimit has the same value as QueryMemoryLimit.

100,000,000 bytes

LiveQueryMemoryLimit

Variable default: 100,000,000 bytes

  
Max query length in characters

66,000 characters

   

Number of pipes that can be included in a query.

99 pipes

   

Max number of results that any query can give.

Increasing this value beyond the default limit may cause browser performance issues, including memory exhaustion or UI hanging when rendering large result sets.

StateRowLimit

  

Function Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability

Default value for the limit parameter in groupBy(), selfJoin() and some other functions, when not specified.

GroupDefaultLimit has a default value of either 20,000 or the value of the GroupMaxLimit configuration, whichever is smallest. GroupDefaultLimit cannot be larger than GroupMaxLimit.

20,000 rows

GroupDefaultLimit

Variable default: 20,000 group elements

  

Max value for the limit parameter in the groupBy() function, meaning that the function can collect up to one million groups.

Due to stability concerns this variable will not allow groupBy() to return the full million rows as a result when this function is the last aggregator: this is governed by the QueryResultRowCountLimit dynamic configuration.

200,000(from 1.0)

GroupMaxLimit

Variable default: 1,000,000 group elements

  
Max number of events in tail(), head(), and sort() functions.

20,000 events

StateRowLimit

  
Max number of events in an RDNS request. See rdns() function.

20,000

RdnsMaxLimit

  

API Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability
Max body size in POST requests after decompression.

32M bytes

   
Max body size in POST requests before decompression.

32M bytes

   

Standards

Below are some standards in LogScale.

Character sets

LogScale works with the JVM standard charsets. The character sets below are guaranteed to be supported in future JVM versions.

Note

These are the character sets LogScale accepts in the HTTP request and this may not necessarily be the same character sets accepted by the log shipper, such as Log Collector or another third-party log shipper, when ingesting files. When ingesting files, they are read by a log shipper that then makes HTTP requests to LogScale, and the character set supported by the log shipper is unlikely to have anything to do with what LogScale supports in the HTTP layer.

  • US-ASCII

  • ISO-8859-1

  • UTF-8

  • UTF-16

  • UTF-16BE

  • UTF-16LE

  • UTF-32

  • UTF-32BE

  • UTF-32LE

If you have any questions about whether a character set is accepted or supported, contact LogScale Support Team.