Quay Enterprise is designed to be run as a single global high availability service with minimal setup. This guide explains the best practices for running Quay Enterprise in an HA setup.
The following services are required in order to run Quay Enterprise as HA:
Perform the basic Quay Enterprise Setup process, using the above database, storage, redis and using the load balancer hostname as the server hostnane. Once complete, save the contents of the conf/stack
directory.
The ensure high availability, it is recommended that machines running in the cluster be self-initializing.
Each machine will need to be able to:
conf/stack
directory saved in the previous stepdocker pull
. It is highly recommended that you lock to a specific tagged version of Quay EnterpriseOnce a cluster has been setup, the next step is to setup health checking to ensure that if a machine fails, it is automatically removed and replaced in the cluster.
Quay Enterprise exports a health checking endpoint at https://{yourloadbalancerhostname}/health/instance
that will return a 200
HTTP status code if the machine is healthy.
If the backing database is RDS, an additional healthcheck config can be specified to ensure that machines report as healthy even if RDS is current in failover mode.
HEALTH_CHECKER:
- RDSAwareHealthCheck
- access_key: access_key_here
secret_key: secret_key_here
region: us-east-1
db_instance: quay-database
To ensure the cluster as a whole is healthy, Quay Enterprise also exports a cluster-wide services status endpoint at https://{yourloadbalancerhostname}/health/endtoend
that will return a 200
HTTP status code only if all services (database, redis, etc) are healthy.
The final step in ensuring high availability for a Quay Enterprise cluster is ensuring it can scale to meet incoming demand. Scaling is typically accomplished by monitoring metrics such as CPU and memory, and adding (or removing) machines based on thresholds. On Amazon, CloudWatch alarms and Autoscaling Groups can be used to accomplish this task.