Chargeback supports configuration options which may be set in the
spec section of the
An example configuration file can be found in custom-values.yaml. A minimal configuration example that doesn't override anything can be found in default.yaml. For details on customizing these files, read the common-configuration-options section below.
This document follows the convention of describing nested fields in configuration settings using dots as separators. For example,
refers to the following YAML structure and value:
spec: chargeback-operator: config: awsAccessKeyID: "REPLACEME"
To install the custom configuration file, run the following command:
kubectl -n $CHARGEBACK_NAMESPACE apply -f manifests/chargeback-config/custom-values.yaml
The example manifest custom-values.yaml contains the most common user configuration values, including resource limits. The values listed are the defaults, which may be uncommented and adjusted as needed.
Chargeback requires at least 1 Persistent Volume to operate. (The example manifest includes 3 by default.) The Persistent Volume Claims (PVCs) are listed below:
hive-metastore-db-datais the only required volume. It is used by hive metastore to retain information about the location of Presto data.
hdfs-datanode-data-hdfs-datanode-$iare used by the single node HDFS cluster which is deployed by default for storing data within the cluster. These two PVCs are not required to store data in AWS S3.
Each of these Persistent Volume Claims is created dynamically by a Stateful Set. Enabling this requires that dynamic volume provisioning be enabled via a Storage Class, or persistent volumes of the correct size must be manually pre-created.
Storage Classes may be used when dynamically provisioning Persistent Volume Claims using a Stateful Set. Use
kubectl get to determine if Storage Classes have been created in your cluster. (By default, Tectonic does not install cloud provider specific
$ kubectl get storageclasses
If the output includes
(default) next to the
name of any
StorageClass, then that
StorageClass is the default for the cluster. The default is used when
StorageClass is unspecified or set to
null in a
StorageClass is listed, or if you wish to use a non-default
StorageClass, see Configuring the StorageClass for Chargeback below.
For more information, see Storage Classes in the Kubernetes documentation.
To configure and specify a
StorageClass for use in Chargeback, specify the
custom-values.yaml. A example
StorageClass section is included in custom-storageclass-values.yaml.
Uncomment the following sections and replace the
class: null value with the name of the
StorageClass to use. Leaving the value
null will cause Chargeback to use the default StorageClass for the cluster.
If a Storage Class that supports dynamic volume provisioning does not exist in the cluster, it is possible to manually create a Persistent Volume with the correct capacity. By default, the PVCs listed above each request 5Gi of storage. This can be adjusted in the same section as adjusting the Storage Class.
Use custom-storageclass-values.yaml as a template and adjust the
size: "5Gi" value to the desired capacity for the following sections:
By default, the data that Chargeback collects and generates is stored in a single node HDFS cluster which is backed by a Persistent Volume. To store the data in a location outside of the cluster, configure Chargeback to store data in S3.
To use S3 for storage, uncomment the
defaultStorage: section in the example
Once uncommented, set
awsSecretAccessKey in the
Please note that this must be done before installation. Changing these settings after installation may result in unexpected behavior.
Because the deployed HDFS cluster will not be used to store data, it may also be disabled. Uncomment the
hdfs.enabled: true setting in
custom-values.yaml, and set the
spec: hdfs: enabled: false
Chargeback is able to correlate cluster usage information with AWS detailed billing information, attaching a dollar amount to resource usage. For clusters running in EC2, this can be enabled by modifying the example custom-values.yaml configuration.
To enable AWS billing correlation, first ensure the AWS Cost and Usage Reports are enabled. For more information, see Turning on the AWS Cost and Usage report in the AWS documentation.
Next, update the
awsBillingDataSource section in the custom-values.yaml example configuration manifest.
enabled value to
true, and update the
prefix to the location of your AWS Detailed billing report.
Then, set the
awsSecretAccessKey in the
This can be done either pre-install or post-install. Note that disabling it post-install can cause errors in the chargeback-operator.