The reference architecture diagram for a GCP deployment looks like the following diagram:

This includes a number of distinct components described below.

Private GKE cluster

Private clusters offer increased isolation by utilizing private IP addresses for nodes and providing private or public endpoints for control plane nodes, which can be further isolated. With private clusters, we can still access Google APIs through Private Google Access. Within a private cluster, Pods are segregated from both inbound and outbound communication, establishing a cluster perimeter. The directional flows of these communications can be managed by exposing services through load balancing and Cloud NAT. To enable a private endpoint when you create a cluster, use the --enable-private-endpoint flag.

Bastion Host

A jump host plays a crucial role in enhancing the security and manageability of the GKE private cluster. The cluster nodes are not directly accessible from the public internet, which provides an inherent layer of security. The host serves as an intermediary server that allows authorised users to access the private cluster securely. It acts as a single entry point for SSH access.

The Bastion host acts as a barrier between the public internet and the cluster's internal network, mitigating the risk of unauthorised access or attacks. Access to the bastion host can be tightly controlled using standard authentication mechanisms, in this setup we use SSH key pairs to access the Bastion host and the cluster private nodes, user can generate ssh key pair.

The Bastion host can also be utilised for tasks like maintenance, troubleshooting, and updates. Once deployed, the host can be used as a proxy to run changes on the private cluster as described in Google Documentation.

By default, ssh sessions to GCP VMs timeout after 10 minutes, the session can be extended by adding --ssh-flag="-ServerAliveInterval=60" to gcloud compute ssh command.

The host has a dedicated ssh firewall rule to restrict access except from IP addresses that gcloud IAP uses for TCP forwarding as per Google Documentation

Cert Manager and Let's Encrypt issuer

Cert Manager is a popular Kubernetes tool used for managing the TLS certificates. It is combined with Let's Encrypt issuer to automate the process of obtaining and renewing SSL/TLS certificates for Logscale application. Cert Manager is responsible for certificate management within the GKE cluster, it can generate renew and keep track of TLS certificates. Let's encrypt is a certificate authority that provides TLS/SSL certificates.

Cloud NAT:

To enable outbound internet connectivity for the pods within the private GKE cluster, you can set up Cloud NAT in your VPC network. Cloud NAT acts as a gateway that translates the private IP addresses of the pods to public IP addresses, allowing them to access the internet. It's configured for logscale egress traffic.