We are bringing the best of Tectonic to Red Hat OpenShift to build the most secure, hybrid Kubernetes application platform.
Chargeback supports configuration options which may be set in the spec
section of the Chargeback
resource.
An example configuration file can be found in custom-values.yaml. A minimal configuration example that doesn't override anything can be found in default.yaml. For details on customizing these files, read the common-configuration-options section below.
This document follows the convention of describing nested fields in configuration settings using dots as separators. For example,
spec.chargeback-operator.config.awsAccessKeyID
refers to the following YAML structure and value:
spec:
chargeback-operator:
config:
awsAccessKeyID: "REPLACEME"
To install the custom configuration file, run the following command:
kubectl -n $CHARGEBACK_NAMESPACE apply -f manifests/chargeback-config/custom-values.yaml
The example manifest custom-values.yaml contains the most common user configuration values, including resource limits. The values listed are the defaults, which may be uncommented and adjusted as needed.
Chargeback requires at least 1 Persistent Volume to operate. (The example manifest includes 3 by default.) The Persistent Volume Claims (PVCs) are listed below:
hive-metastore-db-data
is the only required volume. It is used by
hive metastore to retain information about the location of Presto data.hdfs-namenode-data-hdfs-namenode-0
and hdfs-datanode-data-hdfs-datanode-$i
are used by the single node HDFS cluster which is deployed by default for
storing data within the cluster. These two PVCs are not required to store data in AWS S3.Each of these Persistent Volume Claims is created dynamically by a Stateful Set. Enabling this requires that dynamic volume provisioning be enabled via a Storage Class, or persistent volumes of the correct size must be manually pre-created.
Storage Classes may be used when dynamically provisioning Persistent Volume Claims using a Stateful Set. Use kubectl get
to determine if Storage Classes have been created in your cluster. (By default, Tectonic does not install cloud provider specific
Storage Classes.)
$ kubectl get storageclasses
If the output includes (default)
next to the name
of any StorageClass
, then that StorageClass
is the default for the cluster. The default is used when StorageClass
is unspecified or set to null
in a PersistentVolumeClaim
spec.
If no StorageClass
is listed, or if you wish to use a non-default StorageClass
, see Configuring the StorageClass for Chargeback below.
For more information, see Storage Classes in the Kubernetes documentation.
To configure and specify a StorageClass
for use in Chargeback, specify the StorageClass
in custom-values.yaml
. A example StorageClass
section is included in custom-storageclass-values.yaml.
Uncomment the following sections and replace the null
in class: null
value with the name of the StorageClass
to use. Leaving the value null
will cause Chargeback to use the default StorageClass for the cluster.
spec.presto.hive.metastore.storage.class
spec.hdfs.datanode.storage.class
spec.hdfs.namenode.storage.class
If a Storage Class that supports dynamic volume provisioning does not exist in the cluster, it is possible to manually create a Persistent Volume with the correct capacity. By default, the PVCs listed above each request 5Gi of storage. This can be adjusted in the same section as adjusting the Storage Class.
Use custom-storageclass-values.yaml as a template and adjust the size: "5Gi"
value to the desired capacity for the following sections:
presto.hive.metastore.storage.size
hdfs.datanode.storage.size
hdfs.namenode.storage.size
By default, the data that Chargeback collects and generates is stored in a single node HDFS cluster which is backed by a Persistent Volume. To store the data in a location outside of the cluster, configure Chargeback to store data in S3.
To use S3 for storage, uncomment the defaultStorage:
section in the example
custom-values.yaml configuration.
Once uncommented, set awsAccessKeyID
and awsSecretAccessKey
in the chargeback-operator.config
and presto.config
sections.
Please note that this must be done before installation. Changing these settings after installation may result in unexpected behavior.
Because the deployed HDFS cluster will not be used to store data, it may also be disabled. Uncomment the hdfs.enabled: true
setting in custom-values.yaml
, and set the
value to false
.
spec:
hdfs:
enabled: false
Chargeback is able to correlate cluster usage information with AWS detailed billing information, attaching a dollar amount to resource usage. For clusters running in EC2, this can be enabled by modifying the example custom-values.yaml configuration.
To enable AWS billing correlation, first ensure the AWS Cost and Usage Reports are enabled. For more information, see Turning on the AWS Cost and Usage report in the AWS documentation.
Next, update the awsBillingDataSource
section in the custom-values.yaml example configuration manifest.
Change the enabled
value to true
, and update the bucket
and prefix
to the location of your AWS Detailed billing report.
Then, set the awsAccessKeyID
and awsSecretAccessKey
in the spec.chargeback-operator.config
and spec.presto.config
sections.
This can be done either pre-install or post-install. Note that disabling it post-install can cause errors in the chargeback-operator.