Skip to content

Getting Data Into LogScale

Send logs, metrics, and traces to LogScale from any source using multiple ingestion methods

Getting Started

Learn data ingestion fundamentals and set up your first repository

What is Data Ingestion?

Understand how LogScale collects and processes logs from multiple sources for centralized analysis.

Prerequisites and System Requirements

Hardware, software, and network requirements for successful data ingestion.

Create Your First Repository

Set up a repository to store your log data with proper retention and access control.

📥 Core Ingestion Methods
Primary methods for sending data to LogScale - APIs, agents, and protocols

Ingest API

Send data directly via HTTP endpoints. Flexible REST API for custom integrations and scripting.

LogScale Collector

Lightweight agent for log collection from files, containers, and system logs. Easy to deploy and configure.

Log Shippers

Integrate with Filebeat, Fluentd, Logstash, and other popular log forwarding tools.

Syslog

Standard syslog protocol support for network devices, firewalls, and legacy systems.

💻 Parse Your Data
Extract structured fields from logs using built-in or custom parsers

What are Parsers?

Parsers extract structured fields from unstructured logs. Learn how parsing works and why it's essential for effective log analysis.

Built-in Parsers

LogScale includes parsers for common formats like JSON, syslog, CEF, LEEF, and more. Use them out of the box.

Create Custom Parsers

Build parsers for your proprietary log formats. Use regex, JSON paths, and LogScale's parsing language.

Test Parsers

Validate parsing logic before deploying to production. Test with sample logs and verify field extraction.

Parser Best Practices

Performance tips, efficiency guidelines, and patterns for writing maintainable parsers.

PASTA - Parsing Standard

LogScale Parsing Standard (PASTA) provides consistent, efficient parsing patterns for common log types.

☁️ Cloud and Platform Integrations
Native integrations for AWS, Azure, Google Cloud, and Kubernetes

AWS Integration

Ingest from CloudTrail, CloudWatch Logs, S3, EventBridge, VPC Flow Logs, and other AWS services using native integrations.

Azure Integration

Collect logs from Azure Event Hubs, Blob Storage, Activity Logs, and Azure Monitor.

Google Cloud Platform

Ingest from Cloud Logging, Pub/Sub, and GCS buckets. Monitor GCP resources and services.

Kubernetes Deployment

Deploy LogScale Collector as a DaemonSet using Helm charts. Collect logs from all pods and nodes automatically.

📦 Pre-Built Integration Packages
60+ ready-to-use packages with parsers, dashboards, and queries for popular platforms

Browse All Packages

Explore 60+ pre-built integration packages for cloud platforms, security tools, networking equipment, containers, and applications. Each package includes parsers, dashboards, saved searches, and queries tailored for that specific technology.

Install a Package

Learn how to install and configure integration packages. Packages automatically set up parsers, create dashboards, and provide pre-built queries so you can start analyzing your data immediately.

⚙️ Configuration and Management
Manage repositories, tokens, retention, views, and enrichment

Repository Settings

Configure retention policies, access control, ingest tokens, and storage settings. Control who can access and modify your data.

Ingest Tokens

Manage authentication tokens for data sources. Assign parsers, control routing, and secure access.

Data Retention

Configure how long data is stored. Set retention policies based on compliance requirements and storage capacity.

Views

Create combined or filtered perspectives across multiple repositories. Perfect for cross-team visibility and aggregated analysis.

Lookup Files

Enrich data with additional context using lookup files. Map IPs to locations, user IDs to names, or asset IDs to details.

📈 Monitoring and Troubleshooting
Track ingestion statistics, troubleshoot issues, and manage ingest tokens

Ingest Statistics

Monitor data volume, ingestion rate, latency, and throughput. Track metrics per repository and source.

Parser Monitoring

Track parser performance, rejection rates, and errors. Identify parsing issues before they impact search.

Troubleshoot Ingest Issues

Common ingestion problems and solutions. Debug missing data, parsing errors, and connection issues.

🎯 Top Ingest Methods by Source
Find the right ingestion method for your specific data source type

Security Tools

Ingest from SIEM platforms, EDR solutions, firewalls, IDS/IPS, and security appliances.

Cloud Platforms

Ingest from AWS, Azure, and Google Cloud services.

Containers and Orchestration

Collect logs from Docker containers, Kubernetes clusters, and container platforms.

Applications

Collect logs from web servers, databases, and custom applications.

Infrastructure

Monitor traditional IT infrastructure including servers, network devices, and legacy systems.


🔔 New Pages
🔔 Recently Updated Pages