How LogScale interacts with the Kafka Admin API
When LogScale uses Kafka for persistent queueing, it needs to manage its own Kafka resources rather than relying on a human administrator to manually configure everything. The Kafka Admin API is how it does that programmatically.
The admin client is used primarily for:
Topic creation during startup (when
KAFKA_MANAGED_BY_HUMIO=true)Monitoring cluster health and partition states
Tracking consumer group lag
Managing partition reassignments
Providing cluster metadata via GraphQL API
Kafka Admin API Operations Used
LogScale has internal classes that wrap Kafka's AdminClient class and uses the Admin API methods shown in the following tables.
Cluster Management:
| Method / Property | Purpose |
|---|---|
describeCluster()
| Get cluster ID, nodes, controller info |
numLiveKafkaBrokers
| Count of active brokers |
Topic Management:
| Method / Property | Purpose |
|---|---|
createTopics()
| Create topics with replication/partition config |
describeTopics()
| Get partition assignments and metadata |
fetchTopicPartitionAssignments()
| Get replica assignments per partition |
Configuration:
| Method / Property | Purpose |
|---|---|
describeConfigs()
| Get topic and broker configurations |
describeLogDirs()
| Get log directory information |
Offset Management:
| Method / Property | Purpose |
|---|---|
listOffsets() /
getKafkaOffsets()
| Query partition offsets |
listConsumerGroups()
| List all consumer groups |
listConsumerGroupOffsets()
| Get consumer group lag information |
Data Management:
| Method / Property | Purpose |
|---|---|
deleteRecords()
| Delete records up to a specific offset |
Configuration
Admin client properties are set by:
Environment variables prefixed with
KAFKA_ADMIN_*(converted to lowercase with dots)Common Kafka config from
KAFKA_COMMON_*Bootstrap servers from main Kafka config
Default metadata max age: 10 seconds