What are Data Sources?

A data source is a logical bucket within a repository where LogScale physically groups and stores related events together. LogScale automatically decides which bucket an event belongs to based on its tag values. Events that share the same tag values are stored together in the same data source.

The tagging of events in LogScale is fundamentally about query efficiency. When you run a query, LogScale does not need to scan everything โ€” it can identify which data sources are relevant and skip the rest entirely. The more precisely your tags describe your data, the more LogScale can eliminate irrelevant data before it even starts searching. Organizing related events together enables faster query performance, similar to how organized filing systems enable faster document retrieval than unsorted collections.

Tags and Data Sources

Tags are the organizing principle behind data sources. Every unique combination of tag values creates its own data source. This means your tagging strategy directly shapes your storage structure. Tags are not just metadata โ€” they are the foundation of how your data is physically organized and how efficiently it can be retrieved.

Choosing the Right Tags

The organizing power of data sources depends on balance. Too few tags and your buckets are too broad to be useful. Too many unique tag values and the system fractures into an unmanageable sprawl.

This balance hinges on cardinality. If you tag on a field with a high number of unique values โ€” such as a user ID or session ID โ€” LogScale creates a separate data source for every single value. Instead of a small number of well-populated buckets that queries can efficiently navigate, you end up with thousands or millions of nearly empty ones.

At that point, the organizational system works against you. More data sources means more overhead, more memory pressure, and ultimately slower queries โ€” the opposite of what tags are meant to achieve.

Data Source Strategy

Data sources are largely invisible during normal operation โ€” they are an internal mechanism, not something you interact with directly. However, they are the reason why decisions made early in your implementation, such as parser design and tag selection, have a lasting impact on how well LogScale performs at scale.

Configure your tagging strategy before ingesting data. Changing your tagging strategy after data ingestion requires reingesting data.