Limits & Standards

This page lists the various limits and standard operating parameters of LogScale. See Best Practice for the best practices relative to ingest via the Ingest API.

General Limits & Parameters

Below is a list of general limits and parameters of LogScale:

Note

Unless otherwise specified, all multiple-byte data sizes in LogScale are expressed in SI units using decimal (Base 10). For example:

  • 1KB = 1,000 Bytes

  • 1MB = 1,000,000 Bytes

  • 1GB = 1,000,000,000 Bytes

  • 1TB = 1,000,000,000,000 Bytes

Data Structure

LimitDefault ValueDynamic VariableConfig VariableAvailability

Max number of fields in an event. During ingest, fields are sorted alphabetically by name and the first fields are parsed, the remainder of the named fields are dropped. The @rawstring field is not modified and will contain all data.

  

MAX_EVENT_FIELD_COUNT

Variable default: 1,000 fields

 
Max event size. When the configured event size max is reached, either in @rawstring and/or in other fields, the overall data will be truncated. Fields will be removed entirely, and @rawstring will be truncated down to the allowed max size with added ... at the end, such that the of all other fields + size of @rawstring is less than the configured max event size. Only @rawstring, @timestamp and @timezone are added when truncation occurs.

1 MB

 

MAX_EVENT_SIZE

Variable default: 1 MiB

 
Max CSV lookup file size (see Lookup Files).

200 MB

MaxCsvFileUploadSizeBytes

Variable default: 209,715,200 bytes (200 MB)

MAX_FILEUPLOAD_SIZE

Variable default: 104,857,600 bytes (100 MB)

 
Max JSON lookup file upload size (see Lookup Files).

100 MB

MaxJsonFileUploadSizeBytes

Variable default: 104,857,600 bytes (100 MB)

MAX_FILEUPLOAD_SIZE

Variable default: 104,857,600 bytes (100 MB)

 
Max file upload size (see Lookup Files)

2,048 MB

   
Max tag-key size (not configurable). See Tag Fields.

65,535

   
Max tag-value size (not configurable). See Tag Fields.

65,535

   

Query Related Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability

Max memory a live query can consume during its execution.

LiveQueryMemoryLimit determines how much memory in bytes a live query can consume during its execution. For non-live queries, their memory limit is determined by QueryMemoryLimit, which is 100MB by default. By default LiveQueryMemoryLimit has the same value as QueryMemoryLimit.

1 GB

LiveQueryMemoryLimit

Variable default: 100,000,000 bytes

  
Max query length in characters

66,000 characters

   

Number of pipes that can be included in a query.

99 pipes

   

Max number of results that any query can give.

Increasing this value beyond the default limit may cause browser performance issues, including memory exhaustion or UI hanging when rendering large result sets.

StateRowLimit

  

Function Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability
Default number of events set in an RDNS request. See rdns() function.

-

RdnsDefaultLimit

Variable default: 5,000 events

  

Default value for the limit parameter in groupBy(), selfJoin() and some other functions, when not specified.

GroupDefaultLimit has a default value of either 20,000 or the value of the GroupMaxLimit configuration, whichever is smallest. GroupDefaultLimit cannot be larger than GroupMaxLimit.

20,000 rows

GroupDefaultLimit

Variable default: 20,000 group elements

  

Max value for the limit parameter in the groupBy() function, meaning that the function can collect up to one million groups.

Due to stability concerns this variable will not allow groupBy() to return the full million rows as a result when this function is the last aggregator: this is governed by the QueryResultRowCountLimit dynamic configuration.

1,000,000

GroupMaxLimit

Variable default: 1,000,000 group elements

  
Max number of events in tail(), head(), and sort() functions.

20,000 events

StateRowLimit

  
Max number of events in an RDNS request. See rdns() function.

-

RdnsMaxLimit

Variable default: 20,000 events

  
Memory limit for the mapper phase of a collect() function running as a top-level function, that is, how much data such a function can store.

10 MiB

   
Memory limit for the mapper phase of a collect() function running in a subquery, or as a subaggregator to another function, that is, how much data such a function can store.

1 MiB

   

API Limits

LimitDefault ValueDynamic VariableConfig VariableAvailability
Limits for GraphQL queries on the total number of selected fields and fragments for authenticated users.

1,000

AuthenticatedGraphQLSelectionSizeLimit

  
Limits for GraphQL queries on the total number of selected fields and fragments for unauthenticated users.

150

UnauthenticatedGraphQLSelectionSizeLimit

Variable default: 150 queries

  
Max body size in POST requests after decompression.

32M bytes

   
Max body size in POST requests before decompression.

32M bytes

   

Standards

Below are some standards in LogScale.

Character sets

LogScale works with the JVM standard charsets. The character sets below are guaranteed to be supported in future JVM versions.

Note

These are the character sets LogScale accepts in the HTTP request and this may not necessarily be the same character sets accepted by the log shipper, such as Log Collector or another third-party log shipper, when ingesting files. When ingesting files, they are read by a log shipper that then makes HTTP requests to LogScale, and the character set supported by the log shipper is unlikely to have anything to do with what LogScale supports in the HTTP layer.

  • US-ASCII

  • ISO-8859-1

  • UTF-8

  • UTF-16

  • UTF-16BE

  • UTF-16LE

  • UTF-32

  • UTF-32BE

  • UTF-32LE

If you have any questions about whether a character set is accepted or supported, contact LogScale Support Team.