Skip to content
LogScale Self-Hosted
This is an archived version of the documentation downloaded from LogScale Documentation Archives. Up to date documentation, and documentation for later versions, is available on LogScale Documentation

# Ingesting Data

After installing LogScale on a server, you will want to put in place a system to feed data automatically into LogScale. This loading of information into LogScale is known as ingesting data. Configuring Data ingestion is an essential configuration step when you are setting up LogScale.

The following diagram provides an overview of the configuration flow to ingest data using LogScale.

graph LR; A["Install and Configure LogScale"]--> B B["Create a Repository"]--> C C["Configure Data Ingest"]--> D D["Parse and Filter Data"]--> E E["Enrich Data"]--> F F["Query Data"] style C fill:#A6A0D2
Flowchart illustrating the complete LogScale setup and usage process.

Figure 120. Process graph

- Ingesting Data

  You can use different methods to ingest data depending on your requirements; OS, Log format and so on:

  - Ingesting FDR Data

    Falcon LogScale can ingest Falcon Data Replicator (FDR) data into LogScale without having to configure log shippers. See Ingesting FDR Data.

  - Ingesting AWS S3 Bucket Data

    Falcon LogScale can ingest and parse AWS S3 bucket data. See Ingest Data from AWS S3.

    S3 Ingest was introduced in v1.120.0

  - Log Shippers

    Falcon LogScale is able to ingest data from a wide range of log shippers. Log shippers use the Ingest API to send one or more logs to LogScale. A log shipper can handle multiple logs, multiple log types, manage the log storage on disk, and pre-process the logs before sending them to LogScale. Log shippers are covered in more details in Log Shippers.

  - Listeners

    Ingest listeners are a great way of shipping data to LogScale through raw sockets, using either UDP or TCP. See Ingest Listeners.

  - LogScale Ingest Tokens

    A repository can have one or more ingest tokens associated with it. Ingest tokens are used with the Ingest API to enable data to be routed to the right repository, and to associate a parser. See Ingest Tokens.

  - LogScale API

    The Ingest API can be used directly or through one of LogScale's APIs or software libraries. See the Ingest API reference page for more information. For a list of supported software, see the Software Libraries in the Appendix.

- Parsing data

  Parsing the data that is ingested enables the information to be tagged, specific fields and elements of the log data to be extracted, and enables an additional level of detail. The use of a parser also enables the type of the data and fields extracted to be configured, supporting metrics, graphing and dashboards. See Parse Data.

In most cases you will want to use a log shipper or one of the LogScale platform integrations. If you are interested in getting some data into LogScale quickly, see Use Case: Ingesting Application Logs.

LogScale is optimized for live streaming of events in real time. If you ship data that are not live, you must observe some basic rules so that the resulting events are stored in LogScale as efficiently as if they had been received live. See Backfilling Data.

#### Enter search term