LogScale Multi-Cluster Search

Important

Multi-Cluster search is limited to Self-hosted (On-Prem) customers only at this time.

Multi-Cluster search enables you to create a view that searches across multiple views from local and remote clusters. Using multi-cluster search provides a number of benefits:

  • Aggregation of results from multiple clusters and views, for example aggregating information from one or more clusters

  • Increased distribution of workload across clusters when performing searches

  • Ability to combine and aggregate query data across clusters in multiple regional locations

Multi-Cluster search works by creating a special type of view, the Multi-Cluster View. The multi-cluster view creates a connection between local and remote clusters and allows queries to be executed across each of the configured downstream clusters in the view configuration. The query is executed using the standard query API and uses a Repository Token which provides secure access to the remote repository or view.

Two types of connections within the upstream, or parent, cluster that creates the Multi-Cluster view are supported; local and remote:

  • A local connection refers to a view on the same cluster as the multi-cluster view. Only one local connection can be part of a multi-cluster view.

  • A remote connection refers to a view on a remote cluster. Multiple remote connections are supported, but only one connection to each remote cluster.

Connectivity and authentication for accessing data on a remote cluster is provided at the remote cluster view or repository level. Connectivity must be granted for each individual repository, and the secret token created through Repository Token system limits the access to those users with rights to create a suitable API token with Data read access. Remote cluster search does not enable unlimited access to all views on the remote cluster. For more information, see Multi-Cluster Security.

An overview of the connectivity between local and remote connections and the parent multi-cluster view are shown in the diagram below:

%%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% graph LR C1["Cluster"] V1["Local View"] V2["Local View"] F3["Multi-Cluster View"] C1---V1 C1---V2 C1---F3 RC1["Remote Cluster"] RC1V1["Remote View"] RC1V2["Remote View"] RC1V1---RC1 RC1V2---RC1 RC2["Remote Cluster"] RC2V1["Remote View"] RC2V2["Remote View"] RC2V1---RC2 RC2V2---RC2 F3---V2 F3---RC1V1 F3---RC2V2 linkStyle 8 stroke:#ff0000 linkStyle 9 stroke:#ff0000 linkStyle 7 stroke:#ff0000

When executing the search, the process that is followed for the execution is as follows:

  • Send the query to each configured connection

  • Read the event data from each connection

  • Combine the result set

This process is outlined in the diagram below:

%%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% graph LR C1["Multi-Cluster View Search"] V1["Local View"] V2["Remote View"] V3["Remote View"] CM["Combine Results"] CCE["Combine Event Data"] C1 -- Send Query --- V1 -- Return Events---CM---CCE C1 -- Send Query --- V2 -- Return Events---CM C1 -- Send Query --- V3 -- Return Events---CM

Because the queries are distributed to each connection, the time taken for the query to be processed and returned may be longer than for a purely local query. LogScale must query, receive, and merge the query result information from each cluster connection within the Multi-Cluster configuration before returning the results.