Dynamic Configuration and System Tuning
Adjust configuration settings and optimize performance for your self-hosted LogScale deployment
New to LogScale Configuration?
Learn how LogScale configuration works before you start changing settings. Understand the difference between environment variables and dynamic configuration, what the platform limits mean, and how storage and JVM settings affect overall performance.
- Environment Variables vs Dynamic Configuration
- Platform Limits and Defaults
- Storage and Retention Settings
- JVM and System-Level Configuration
Ready to Tune Your Deployment?
You understand how LogScale configuration works and are ready to optimize for your specific workloads. Identify bottlenecks, adjust configuration parameters, tune query resource limits, and measure the effect of your changes.
- Measuring Current Performance
- Tuning Dynamic Configuration
- Optimizing Ingest and Storage
- Setting Query and Resource Limits
Workflow 1: New to LogScale Configuration?
Build a solid understanding of how LogScale is configured before making changes. Knowing the configuration model helps you make informed tuning decisions and avoid unintended consequences in production.
The Configuration Model
Understand the two mechanisms for configuring LogScale
Configuration Variables
guideintermediateLearn how LogScale uses environment variables for configuration. Understand which settings require a restart and how to apply environment variables in your deployment environment.
Dynamic Configuration Parameters
guideintermediateUnderstand dynamic configuration: settings that can be changed at runtime without restarting LogScale. Learn which parameters are available dynamically and how to update them using the UI or API.
Platform Limits and Defaults
Understand the default limits that govern data, queries, and API usage
Limits and Standards
guideintermediateReview the default limits LogScale applies to data structures, queries, API calls, triggers, and scheduled reports. Understanding defaults helps you identify where limits may need adjustment for your workloads.
Query-Related Limits
guideintermediateUnderstand the specific limits that apply to queries: maximum query duration, result size limits, and concurrent query limits. These defaults are tuned for general use and may need adjustment for heavy analytical workloads.
Storage and Retention Configuration
Understand how storage and retention settings affect performance and capacity
Data Retention
guideintermediateLearn how retention settings control the balance between data availability and storage consumption. Understand compressed versus uncompressed retention, age-based retention, and how retention interacts with query performance.
Performance Tuning for Long Retention
guideadvancedUnderstand the specific configuration adjustments needed when running LogScale with long retention periods. Long retention changes query behavior and requires different tuning than short-retention deployments.
Kafka Retention Settings
guideadvancedLearn how Kafka retention settings interact with LogScale's ingest pipeline. Kafka retention must be aligned with LogScale's processing capacity to avoid data loss during high-volume ingest.
JVM and System Configuration
Understand the system-level settings that affect LogScale's resource usage
JVM Configuration
guideintermediateLearn how the JVM configuration affects LogScale's memory usage and garbage collection behavior. JVM settings are among the most impactful tuning parameters for stable, high-performance deployments.
Java Memory Options
guideintermediateUnderstand heap size configuration and how to set memory limits appropriate for your hardware. Undersized heap causes excessive garbage collection; oversized heap wastes resources and increases GC pause times.
Garbage Collection
guideintermediateUnderstand garbage collection tuning for LogScale workloads. Learn which GC configuration options are recommended and why GC pause behavior matters for query latency.
Workflow 2: Ready to Tune Your Deployment?
Follow this path to identify performance bottlenecks, adjust configuration for your workloads, and verify that changes have the intended effect.
Measure Current Performance
Establish a baseline before making any configuration changes
Measure and Monitor
guideintermediateUse LogScale's built-in monitoring to capture current ingest throughput, query load, and resource utilization. Establish a performance baseline so you can measure the effect of configuration changes.
Cluster Statistics
guideintermediateReview cluster statistics for a real-time view of ingest rates, data volumes, and node resource usage. Use this to identify which nodes or repositories are contributing most to resource consumption.
Tune Dynamic Configuration
Adjust runtime parameters without restarting LogScale
Optimize Ingest
Tune ingest throughput and reduce ingestion overhead
Best Practices to Optimize Data Ingestion
guideintermediateApply recommended practices for maximizing ingest throughput: parser selection, field extraction strategies, tag design, and Kafka configuration. These optimizations reduce the per-event processing cost and increase sustainable ingest rates.
Kafka Configuration
guideadvancedTune Kafka configuration to match your ingest volume and LogScale's processing capacity. Kafka is the ingest buffer between data sources and LogScale's indexing pipeline - its configuration directly affects ingest reliability and throughput.
Kafka Topic Configuration
guideadvancedAdjust topic-level settings such as partition count and replication factor. Partition count affects the parallelism of ingest processing and is one of the most impactful Kafka tuning parameters.
Tune Storage and Retention
Balance data availability, query performance, and storage costs
Data Retention
guideintermediateAdjust retention settings per repository to match your availability requirements and storage budget. Fine-tune compressed and uncompressed retention limits to optimize the balance between hot and cold data.
Performance Tuning for Long Retention
guideadvancedApply specific configuration adjustments for deployments with long retention periods. Long retention requires different segment sizes, merge strategies, and query timeout settings than short-retention deployments.
Long-Retention Performance Tuning Configuration
guideadvancedReference for the specific environment variable settings recommended for long-retention deployments. Apply these after reviewing the performance tuning guidance.
Set Query and Resource Limits
Prevent resource-intensive queries from affecting overall cluster stability
Query Quotas
guideintermediateConfigure per-user query quotas to prevent individual users from consuming excessive CPU, memory, or I/O. Quotas are the primary mechanism for ensuring fair resource sharing in multi-user deployments.
Limits and Standards
guideintermediateReview and adjust platform-wide limits for query duration, result sizes, API call frequency, and trigger execution. Increase limits only where your workload genuinely requires it - defaults exist for good reasons.
JVM Tuning
Adjust JVM settings to match your hardware and workload profile
Java Memory Options
guideintermediateSet heap size based on your available RAM and workload mix. Adjust memory settings when you observe excessive garbage collection frequency or out-of-memory errors under load.
Garbage Collection
guideintermediateTune GC behavior to minimize pause times for your query latency requirements. Different workload profiles (high-throughput ingest vs. interactive search) benefit from different GC configurations.
Transparent Huge Pages
guideintermediateConfigure Transparent Huge Pages (THP) on Linux for improved memory throughput. THP settings have a measurable effect on LogScale performance under high-memory workloads.
