LogScale Log collector can be used for this specific use case. Vector and Filebeat do not have access to the Windows APIs that are required to read the Windows Event Log.

If LogScale Log collector is not suitable for your use case you can use Winlogbeat. Organizations can choose to put Winlogbeat or LogScale Log Collector on each server individually however at scale this becomes difficult to manage in terms of deploying log shippers to every Windows machine. The solution is to use the Windows Event Forwarding to forward events from each Windows machine to a central aggregation server that will then forward the events on to LogScale. While Winlogbeat is often used in this role, there are other log shippers like LogScale Log Collector and Fluentd that forward the events on to LogScale.

For organizations using Kafka as part of their data pipeline there are multiple solutions for moving data from Kafka to LogScale. Vector and Fluentd for example can both read from Kafka and then ship that data to LogScale. Alternatively LogScale has a Kafka Connect Log Format that is also capable of sending data from Kafka topics to LogScale.

One of the primary advantages to using Kafka in front of LogScale as part of your log shipping pipeline is that Kafka can be used as a queue to help buffer events under various failure conditions (network outages, log shipper failures, etc.)

One way to achieve high availability with log shipping is to place a pool of log shipping agents behind a load balancer. In this model source systems sending logs to LogScale would send their logs to a single address (URL or IP) and the load balancer would manage the process of balancing the traffic across a series of log shippers. This model of load balancing log shippers is commonly used under for the following reasons:

For most organizations the primary reason to deploy log shippers in a highly available configuration is when those log shippers are acting as a centralized point for forwarding logs on to LogScale. Most common load balancers (HAProxy, Nginx, etc.) would support this use case. The following diagram presents a high level architectural overview of a load balanced log shipping solution:

The following sample Nginx configuration demonstrates how to load balance Syslog connections over TCP and UDP to two log shipping nodes (10.100.0.5:514, 10.100.0.6:514) from the NGINX proxy (10.100.0.26:514):

stream
{
   upstream syslog_udp {
      Server 10.100.0.5:514;
      Server 10.100.0.6:514;
   }

   upstream syslog_tcp {
      Server 10.100.0.5:514;
      Server 10.100.0.6:514;
   }

   server {
      Listen 10.100.0.26:514 udp;
      Proxy_pass syslog_udp;
      Proxy_responses 0;
   }

   server {
      Listen 10.100.0.26:514;
      Proxy_pass syslog_tcp;
   }
}

worker_rlimit_nofile 1000000;

events
{
   worker_connections 20000;
}

An alternative solution is to use a queue. In this high availability model all of your log producers write events to the target queue. Multiple agents (log shippers) are deployed to read from the queue. There are several possible queue platforms that you can use including Kafka, Amazon SQS, and Amazon S3. The decision of which platform to use for your queue will depend on your organization's existing infrastructure. It should be noted however that one of the important aspects of a queuing system is that there is a mechanism in place to prevent events from being read and forwarded by multiple log shippers.

Generally speaking organizations should not attempt to build ad hoc queuing systems when it is possible to use a platform like Amazon SQS or Kafka. These platforms have existing integrations with LogScale that minimize the level of effort required to implement a queuing solution.