Ingest Feeds

Ingest feeds allow Falcon LogScale to ingest data from AWS S3 buckets and Azure.

The ingest feeds page lists all of the ingest feeds you have configured. From this page you can monitor and manage AWS and Azure ingest feeds and perform activities likes, adding new feeds, enabling or disabling existing feeds, and editing feeds.

To get a list of ingest feeds for all repositories through you own custom application, use the GraphQL API, in particular the repositories query with the ingestFeeds parameter. For ingest feeds for a specific repository, use repository.

Monitor Ingest Feeds

Errors during ingest will be shown within the Ingest feeds page.

To display the Ingest feeds page and monitor ingest feeds:

  1. Go to Repositories and views page and select a relevant repository in which you want to ingest the data.

  2. Click Settings, under Ingest on the side menu click Ingest feeds.

The Ingest feeds page shows the individual feeds, their configuration and current status:

  • Name

    Name of the ingest feed given during configuration.

  • Preprocessing

    A description of any preprocessing performed on the data before it is provided to the parser.

  • Parser

    The configured parser for this ingest feed.

  • Last activity

    The time of the last ingest activity for this feed.

  • Status

    The current status of the feed, including whether there is an error or if the feed has been disabled.

    Transient errors during ingest should automatically be cleared. You can click on the status message to get more information.

To get the status on an ingest feed with your own custom application, use the repository query in GraphQL API with the ingestFeed parameter and the return sub-parameter executionInfo, and its sub-parameter IngestFeedStatus — whichh will return any problems, the cause, and the timestamp.

To edit, delete, enable and disable an existing feed, click the ⋮ icon next to each feed configuration. To add a new feed, click the + New Ingest feed button.