Set up a database as an ingest method
Step 1 - Configure the destination sink
Why? The Database Source collects rows locally; a Collector sink defines where those events are sent.
For a LogScale destination, configure a Log Collector sink with the destination URL and ingest token. For example:
sinks:
logscale:
type: humio
token: "${INGEST_TOKEN}"
url: "https://cloud.humio.com"Use the URL for your Log Collector deployment and protect the ingest token using the credential-management approach appropriate for your Collector deployment.
Step 2 - Prepare the database source
Why? The Collector needs a reliable way to identify new rows and a value that becomes each event's raw message.
Choose a database account with the minimum read permissions required for the target table, view, or query.
Identify the database hostname, port, database name, and credentials.
Choose the key/offset column used to track collection progress. Prefer a unique, monotonically increasing integer.
Choose the message column. Its value is ingested as the event raw string (
@rawstring).Optionally identify additional database columns to add to each event as fields.
Important
Rows that share the same offset value can be skipped. CrowdStrike recommends integer identifiers and advises against timestamps/time values for starting offsets.
Step 3 - Configure the database connection
Add a source under sources in the Collector configuration. Set the type to postgres, mysql, mssql, or oracle as appropriate.
Example PostgreSQL source:
sources:
application_logs:
type: postgres
hostname: postgres.example.com
port: 5432
database: application
username: logscale_reader
password: "${POSTGRES_PASSWORD}"
sink: logscale
query:
type: builder
table: app_logs
keyColumn: log_id
messageColumn: log_message
additionalColumns:
level: log_level
timestamp: created_atStep 4 - Configure the query
Why? The query defines which rows become events and how the Collector tracks its progress.
Option A - Builder mode:
table - Source table.
keyColumn - Column used as the collection offset/checkpoint.
messageColumn - Column whose value becomes
@rawstring.additionalColumns - Optional mappings from database columns to event fields.
Option B - Custom SQL:
Use a custom query for joins, filtering, calculated values, or constructed messages. Include the Collector checkpoint placeholder and order by the key column
query:
type: custom
selectQuery: |
SELECT audit_id, username, action, message
FROM audit_log
WHERE audit_id >= {{CHECKPOINT}}
ORDER BY audit_id
keyColumn: audit_id
messageColumn: message
additionalColumns:
user: username
action: actionStep 5 - Configure the starting position
By default, a new Database Source collects historical rows from the
earliest available offset. Optionally set startFrom
to an integer offset; it applies to keyColumn.
startFrom: 1000
Use a sequential integer identifier where possible rather than a timestamp.
Step 6 - Configure database connection security
The Database Source security setting defaults to full TLS validation.
Use a trusted CA certificate where necessary.
tls-no-verify and disabled are
insecure alternatives and require insecure database sources to be
explicitly allowed; avoid them for normal production use.
Step 7 - Validate and run the Collector configuration
Save or deploy the configuration using your normal local configuration or Fleet Management workflow.
Validate the Collector configuration before relying on it in production.
Start or restart the Collector as required so the source is active.
Check Collector logs for database connection, query, or TLS errors.
The Database Source polls every five seconds. This interval is fixed and cannot be configured.
Step 8 - Verify ingestion and checkpointing
Ensure the configured table or query returns a representative row at or after the starting offset.
In Log Collector or Next-Gen SIEM, confirm that the resulting event appears.
Verify that the message value appears as the raw string and additional columns appear as expected fields.
Insert a new test row with a higher offset value and confirm that it is collected on a subsequent poll.
Check Collector logs and checkpoint advancement if new rows are not collected.
If events are missing, check duplicate/non-monotonic key values, query filters, starting offset, database permissions, network connectivity, and sink configuration.