Skip to content
LogScale Self-Hosted
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

## `MAX_EVENT_SIZE`

Option | `MAX_EVENT_SIZE`
Description | Specifies the maximum allowed event size
Default | `1 MiB`
Maximum | `10,000,000 bytes` (10 MB)

Controls the maximum allowed size for an event. If you increase this value, you will also need to increase the following variables:

- `TOPIC_MAX_MESSAGE_BYTES` - this enables the producer to send larger messages, and for LogScale to configure the topic to enable it to store larger messages. If the topic already exists, you need to edit the Kafka topic configuration - the relevant setting to change is `max.message.bytes`.

  Refer to the Kafka documentation for more details.

- `KAFKA_INGEST_QUEUE_PRODUCER_SEND_BUFFER_BYTES` - This enables you to increase the TCP buffer size for the producer. See also `KAFKA_INGEST_QUEUE_PRODUCER_`, especially on how to map between LogScale variables and Kafka variables.

  Refer to the Kafka documentation for more details.

- `KAFKA_INGEST_QUEUE_PRODUCER_MAX_REQUEST_SIZE` - This sets the maximum size (in bytes) of a request that the producer can send. If a message batch exceeds this size, the producer will split it into smaller requests.

  Refer to the Kafka documentation for more details.

- `KAFKA_INGEST_QUEUE_CONSUMER_FETCH_MAX_BYTES` - This defines the maximum number of bytes the server should return for a fetch request. If a message batch is larger than this value, the server will still return it, but only fetch one message. This setting must be larger than the largest message a broker will accept, `max.message.bytes`.

  This setting should be coordinated with other related configurations such as `max.partition.fetch.bytes` and `message.max.bytes` for optimal performance.

  Refer to the Kafka documentation for more details.

- `KAFKA_INGEST_QUEUE_CONSUMER_MAX_PARTITION_FETCH_BYTES` - This controls the maximum number of bytes that will be fetched from each partition at a time. This setting applies at the per-partition level, unlike `fetch.max.bytes` which is for the entire fetch request. Must be larger than the largest message in the partition.

  This should be set lower than the consumer's total memory allocation. Multiple partitions can fetch in parallel, so total memory use could be `max.partition.fetch.bytes * number_of_partitions`.

  Default value is typically 1MB (1048576 bytes).

  Refer to the Kafka documentation for more details.

- `KAFKA_INGEST_QUEUE_PRODUCER_BUFFER_MEMORY` - This determines the total amount of memory the producer can use for buffering. When this buffer memory is exhausted, additional `send()` calls will block or throw exceptions depending on configuration. This parameter acts as a crucial flow control mechanism.

  Should be sized based on:

  Expected production rate
- Network latency
- Broker response time
- Available system memory

This parameter works in conjunction with `batch.size`, `linger.ms` and `max.block.ms`.

Default is typically 32MB (33554432 bytes).

Refer to the Kafka documentation for more details.

In addition, you need to configure the topics on the Kafka side, see the Kafka documentation for Broker-level parameters.

### Warning

Increasing this variable may have adverse affects for overall system performance, specifically that of LogScale and Kafka. If you increase this value, make sure you also configure LogScale and Kafka accordingly. It is important to determine that your use case warrants adjusting this variable - if in doubt please Contact Support.

#### Enter search term