4.2.4

Changes

• Add the --client flag to pgo version to output the client version of pgo

Fixes

• Ensure WAL archives are pushed to all repositories when pgBackRest is set to use both a local and a S3-based repository

4.4.1

Crunchy Data announces the release of the PostgreSQL Operator 4.4.1 on August 18, 2020.

The PostgreSQL Operator is released in conjunction with the Crunchy Container Suite.

The PostgreSQL Operator 4.4.1 release includes the following software versions upgrades:

• The PostgreSQL containers now use versions 12.4, 11.9, 10.14, 9.6.19, and 9.5.23

PostgreSQL Operator is tested with Kubernetes 1.13 - 1.18, OpenShift 3.11+, OpenShift 4.3+, Google Kubernetes Engine (GKE), and VMware Enterprise PKS 1.3+.

Fixes

• The pgBackRest URI style defaults to host if it is not set.
• Fix pgo label when applying multiple labels at once.
• pgBadger now has a default memory limit of 64Mi, which should help avoid a visit from the OOM killer.
• Fix pgo create pgorole so that the expression --permissions=* works.

4.4.1 Release Candidate 1

v4.4.1-rc.1

Add 4.4.1 release notes

4.3.3

Crunchy Data announces the release of the PostgreSQL Operator 4.3.3 on August 18, 2020.

The PostgreSQL Operator is released in conjunction with the Crunchy Container Suite.

The PostgreSQL Operator 4.3.3 release includes the following software versions upgrades:

• The PostgreSQL containers now use versions 12.4, 11.9, 10.14, 9.6.19, and 9.5.23
• pgBouncer is now at version 1.14.

PostgreSQL Operator is tested with Kubernetes 1.13 - 1.18, OpenShift 3.11+, OpenShift 4.3+, Google Kubernetes Engine (GKE), and VMware Enterprise PKS 1.3+.

Changes

• Perform a pg_dump from a specific database using the --database flag when using pgo backup with --backup-type=pgdump.
• Restore a pg_dump to a specific database using the --pgdump-database flag using pgo restore when --backup-type=pgdump is specified.
• Add the --client flag to pgo version to output the client version of pgo.
• The PostgreSQL cluster scope is now utilized to identify and sync the ConfigMap responsible for the DCS for a PostgreSQL cluster.
• The PGMONITOR_PASSWORD is now populated by an environmental variable secret. This environmental variable is only set on a primary instance as it is only needed at the time a PostgreSQL cluster is initialized.
• Remove "Operator Start Time" from pgo status as it is more convenient and accurate to get this information from kubectl and the like, and it was not working due to RBAC privileges. (Reported by @mw-0).
• pgo-rmdata container no longer runs as the root user, but as daemon (UID 2)
• Remove dependency on the expenv binary that was included in the PostgreSQL Operator release. All expenv calls were either replaced with the native envsubst program or removed.

Fixes

• Add validation to ensure that limits for CPU/memory are greater-than-or-equal-to the requests. This applies to any command that can set a limit/request.
• Ensure WAL archives are pushed to all repositories when pgBackRest is set to use both a local and a S3-based repository
• Silence expected error conditions when a pgBackRest repository is being initialized.
• Add the watch permissions to the pgo-deployer ServiceAccount.
• Ensure client-setup.sh works with when there is an existing pgo client in the install path
• Ensure the PostgreSQL Operator can be uninstalled by adding list verb ClusterRole privileges to several Kubernetes objects.
• Bring up the correct number of pgBouncer replicas when pgo update cluster --startup is issued.
• Fixed issue where pgo scale would not work after pgo update cluster --shutdown and pgo update cluster --startup were run.
• Ensure pgo scaledown deletes external WAL volumes from the replica that is removed.
• Fix for PostgreSQL cluster startup logic when performing a restore.
• Do not consider non-running Pods as primary Pods when checking for multiple primaries (Reported by @djcooklup).
• Fix race condition that could occur while pgo upgrade was running while a HA configuration map attempted to sync. (Reported by Paul Heinen @v3nturetheworld).
• Silence "ConfigMap not found" error messages that occurred during PostgreSQL cluster initialization, as these were not real errors.
• Fix an issue with controller processing, which could manifest in PostgreSQL clusters not being deleted.
• Eliminate gcc from the postgres-ha and pgadmin4 containers.
• Fix pgo label when applying multiple labels at once.

4.3.3 Release Candidate 1

v4.3.3-rc.1

Add 4.3.3 release notes

4.4.0

Crunchy Data announces the release of the PostgreSQL Operator 4.4.0 on July 17, 2020. Instructions for installing the Postgres Operator are here:

https://www.crunchydata.com/developers/download-postgres/containers/postgres-operator

The PostgreSQL Operator is released in conjunction with the Crunchy Container Suite.

The PostgreSQL Operator 4.4.0 release includes the following software versions upgrades:

• PostGIS 3.0 is now supported. There is now a manual upgrade path between PostGIS containers.
• pgRouting is now included in the PostGIS containers.
• pgBackRest is now at version 2.27.
• pgBouncer is now at version 1.14.

PostgreSQL Operator is tested with Kubernetes 1.15 - 1.18, OpenShift 3.11+, OpenShift 4.4+, Google Kubernetes Engine (GKE), and VMware Enterprise PKS 1.3+.

Major Features

• Create New PostgreSQL Clusters from pgBackRest Repositories
• Improvements to RBAC Reconciliation.
• TLS Authentication for PostgreSQL Instances.
• A Helm Chart is now available and support for deploying the PostgreSQL Operator.

Create New PostgreSQL Clusters from pgBackRest Repositories

A technique frequently used in PostgreSQL data management is to have a pgBackRest repository that can be used to create new PostgreSQL clusters. This can be helpful for a variety of purposes:

• Creating a development or test database from a production data set
• Performing a point-in-time-restore on a database that is separate from the primary database

and more.

This can be accomplished with the following new flags on pgo create cluster:

• --restore-from: used to specify the name of the pgBackRest repository to restore from via the name of the PostgreSQL cluster (whether the PostgreSQL cluster is active or not).
• --restore-opts: used to specify additional options like the ones specified to pgbackrest restore (e.g. --type and --target if performing a point-in-time-recovery).

Only one restore can be performed against a pgBackRest repository at a given time.

RBAC Reconciliation

PostgreSQL Operator 4.3 introduced a change that allows for the Operator to manage the role-based access controls (RBAC) based upon the Namespace Operating mode that is selected. This ensures that the PostgreSQL Operator is able to function correctly within the Namespace or Namespaces that it is permitted to access. This includes Service Accounts, Roles, and Role Bindings within a Namespace.

PostgreSQL Operator 4.4 removes the requirements of granting the PostgreSQL Operator bind and escalate privileges for being able to reconcile its own RBAC, and further defines which RBAC is specifically required to use the PostgreSQL Operator (i.e. the removal of wildcard * privileges). The permissions that the PostgreSQL Operator requires to perform the reconciliation are assigned when it is deployed and is a function of which NAMESPACE_MODE is selected (dynamic, readonly, or disabled).

This change renames the DYNAMIC_RBAC parameter in the installer to RECONCILE_RBAC and is set to true by default.

For more information on how RBAC reconciliation works, please visit the RBAC reconciliation documentation.

TLS Authentication for PostgreSQL Instances

Certificate-based authentication is a powerful PostgreSQL feature that allows for a PostgreSQL client to authenticate using a TLS certificate. While there are a variety of permutations for this can be set up, we can at least create a standardized way for enabling the replication connection to authenticate with a certificate, as we do have a known certificate authority.

PostgreSQL Operator 4.4 introduces the --replication-tls-secret flag on the pgo create cluster command, which, if specified and if the prerequisites are specified (--server-tls-secret and --server-ca-secret), then the replication account ("primaryuser") is configured to use certificate-based authentication. Combine with --tls-only for powerful results.

Note that the common name (CN) on the certificate MUST be "primaryuser", otherwise one must specify a mapping in a pg_ident configuration block to map to "primary" user.

When mounted to the container, the connection sslmode that the replication user uses is set to verify-ca by default. We can make that guarantee based on the certificate authority that is being mounted. Using verify-full would cause the Operator to make assumptions about the cluster that we cannot make, and as such a custom pg_ident configuration block is needed for that. However, using verify-full allows for mutual authentication between primary and replica.

Breaking Changes

• The parameter to set the RBAC reconciliation settings is renamed to RECONCILE_RBAC (from DYNAMIC_RBAC).

Features

• Added support for using the URI path style feature of pgBackRest. This includes:
  • Adding the BackrestS3URIStyle configuration parameter to the PostgreSQL Operator ConfigMap (pgo.yaml), which accepts the values of host or path.
  • Adding the --pgbackrest-s3-uri-style flag to pgo create cluster, which accepts values of host or path.
• Added support to disable TLS verification when connecting to a pgBackRest repository. This includes:
  • Adding the BackrestS3VerifyTLS configuration parameter to the PostgreSQL Operator ConfigMap (pgo.yaml). Defaults to true.
  • Adding the --pgbackrest-s3-verify-tls flag to pgo create cluster, which accepts values of true or false.
• Perform a pg_dump from a specific database using the --database flag when using pgo backup with --backup-type=pgdump.
• Restore a pg_dump to a specific database using the --pgdump-database flag using pgo restore when --backup-type=pgdump is specified.
• Allow for support of authentication parameters in the pgha-config (e.g. sslmode). See the documentation for words of caution on using these.
• Add the --client flag to pgo version to output the client version of pgo.
• A Helm Chart using Helm v3 is now available.

Changes

• pgo clone is now deprecated. For a better cloning experience, please use pgo create cluster --restore-from
• The PostgreSQL cluster scope is now utilized to identify and sync the ConfigMap responsible for the DCS for a PostgreSQL cluster.
• The PGMONITOR_PASSWORD is now populated by an environmental variable secret. This environmental variable is only set on a primary instance as it is only needed at the time a PostgreSQL cluster is initialized.
• Remove "Operator Start Time" from pgo status as it is more convenient and accurate to get this information from kubectl and the like, and it was not working due to RBAC privileges. (Reported by @mw-0).
• Removed unused pgcluster attributes PrimaryHost and SecretFrom.
• pgo-rmdata container no longer runs as the root user, but as daemon (UID 2)
• Remove dependency on the expenv binary that was included in the PostgreSQL Operator release. All expenv calls were either replaced with the native envsubst program or removed.

Fixes

• Add validation to ensure that limits for CPU/memory are greater-than-or-equal-to the requests. This applies to any command that can set a limit/request.
• Ensure PVC capacities are being accurately reported when using pgo show cluster
• Ensure WAL archives are pushed to all repositories when pgBackRest is set to use both a local and a S3-based repository
• Silence expected error conditions when a pgBackRest repository is being initialized.
• Deployments with pgo-deployer using the default file with hostpathstorage will now successfully deploy PostgreSQL clusters without any adjustments.
• Add the watch permissions to the pgo-deployer ServiceAccount.
• Ensure the PostgreSQL Operator can be uninstalled by adding list verb ClusterRole privileges to several Kubernetes objects.
• Ensure client-setup.sh executes to completion if existing PostgreSQL Operator credentials exist that were created by a different installation method.
• Ensure client-setup.sh works with when there is an existing pgo client in the install path.
• Update the documentation to properly name CCP_IMAGE_PULL_SECRET_MANIFEST and PGO_IMAGE_PULL_SECRET_MANIFEST in the pgo-deployer configuration.
• Bring up the correct number of pgBouncer replicas when pgo update cluster --startup is issued.
• Fixed issue where pgo scale would not work after pgo update cluster --shutdown and pgo update cluster --startup were run.
• Ensure pgo scaledown deletes external WAL volumes from the replica that is removed.
• Fix for PostgreSQL cluster startup logic when performing a restore.
• Several fixes for selecting default storage configurations and sizes when using the pgo-deployer container. These include #1, #4, and #8.
• Do not consider non-running Pods as primary Pods when checking for multiple primaries (Reported by @djcooklup).
• Fix race condition that could occur while pgo upgrade was running while a HA configuration map attempted to sync. (Reported by Paul Heinen @v3nturetheworld).
• The custom setup example was updated to reflect the current state of bootstrapping the PostgreSQL container.
• Silence "ConfigMap not found" error messages that occurred during PostgreSQL cluster initialization, as these were not real ...

4.4.0 Release Candidate 1

v4.4.0-rc.1

Add v4.4.0 release notes

4.4.0 Beta 2

Crunchy Data is pleased to announce the release of PostgreSQL Operator 4.4.0 Beta 2. We encourage you to download it and try it out.

Changes Since Beta 2:

Breaking Changes

• pgBackRest is updated to 2.27

Changes

• Added a Helm Chart for installing the metrics stack.
• Certain artifacts that would show up in pgo show cluster (e.g. a repo sync pod from pgo clone or creating a cluster from a pgBackRest repository) no longer show up.
• Removed unused pgcluster attributes PrimaryHost and SecretFrom.

Fixes

• Ensure the new pgBackRest S3 URI path style and the disable TLS verification work when using both local and S3 storage.
• Reset the primary instance after a manual restart (i.e. pgo restart)
• Update version label for the pgo client when using the Ansible installer to ensure that it can actually download the installer.

4.4.0 Beta 1

Crunchy Data announces the release of the PostgreSQL Operator 4.4.0-beta.1 on July 2, 2020.

The PostgreSQL Operator is released in conjunction with the Crunchy Container Suite.

The PostgreSQL Operator 4.4.0 release includes the following software versions upgrades:

• PostGIS 3.0 is now supported. There is now a manual upgrade path between PostGIS containers.
• pgBackRest is now at version 2.27

PostgreSQL Operator is tested with Kubernetes 1.15 - 1.18, OpenShift 3.11+, OpenShift 4.4+, Google Kubernetes Engine (GKE), and VMware Enterprise PKS 1.3+.

Major Features

• Create New PostgreSQL Clusters from pgBackRest Repositories
• Improvements to RBAC Reconciliation.
• TLS Authentication for PostgreSQL Instances.
• A Helm Chart is now available and support for deploying the PostgreSQL Operator.

Create New PostgreSQL Clusters from pgBackRest Repositories

A technique frequently used in PostgreSQL data management is to have a pgBackRest repository that can be used to create new PostgreSQL clusters. This can be helpful for a variety of purposes:

• Creating a development or test database from a production data set
• Performing a point-in-time-restore on a database that is separate from the primary database

and more.

This can be accomplished with the following new flags on pgo create cluster:

• --restore-from: used to specify the name of the pgBackRest repository to store from via the name of the PostgreSQL cluster (whether the PostgreSQL cluster is active or not).
• --restore-opts: used to specify additional options like the ones specified to pgbackrest restore (e.g. --type and --target if performing a point-in-time-recovery).

Only one restore can be performed against a pgBackRest repository at a given time.

RBAC Reconciliation

PostgreSQL Operator 4.3 introduced a change that allows for the Operator to manage the role-based access controls (RBAC) based upon the Namespace Operating mode that is selected. This ensures that the PostgreSQL Operator is able to function correctly within the Namespace or Namespaces that it is permitted to access. This includes Service Accounts, Roles, and Role Bindings within a Namespace.

PostgreSQL Operator 4.4 removes the requirements of granting the PostgreSQL Operator bind and escalate privileges for being able to reconcile its own RBAC, and further defines which RBAC is specifically required to use the PostgreSQL Operator (i.e. the removal of wildcard * privileges). The permissions that the PostgreSQL Operator requires to perform the reconciliation are assigned when it is deployed and is a function of which NAMESPACE_MODE is selected (dynamic, readonly, or disabled).

This change renames the DYNAMIC_RBAC parameter in the installer to RECONCILE_RBAC and is set to true by default.

For more information on how RBAC reconciliation works, please visit the RBAC reconciliation documentation.

TLS Authentication for PostgreSQL Instances

Certificate-based authentication is a powerful PostgreSQL feature that allows for a PostgreSQL client to authenticate using a TLS certificate. While there are a variety of permutations for this can be set up, we can at least create a standardized way for enabling the replication connection to authenticate with a certificate, as we do have a known certificate authority.

PostgreSQL Operator 4.4 introduces the --replication-tls-secret flag on the pgo create cluster command, which, if specified and if the prerequisites are specified (--server-tls-secret and --server-ca-secret), then the replication account ("primaryuser") is configured to use certificate-based authentication. Combine with --tls-only for powerful results.

Note that the common name (CN) on the certificate MUST be "primaryuser", otherwise one must specify a mapping in a pg_ident configuration block to map to "primary" user.

When mounted to the container, the connection sslmode that the replication user uses is set to verify-ca by default. We can make that guarantee based on the certificate authority that is being mounted. Using verify-full would cause the Operator to make assumptions about the cluster that we cannot make, and as such a custom pg_ident configuration block is needed for that. However, using verify-full allows for mutual authentication between primary and replica.

Breaking Changes

• The parameter to set the RBAC reconciliation settings is renamed to RECONCILE_RBAC (from DYNAMIC_RBAC).

Features

• Added support for using the URI path style feature of pgBackRest. This includes:
  • Adding the BackrestS3URIStyle configuration parameter to the PostgreSQL Operator ConfigMap (pgo.yaml), which accepts the values of host or path.
  • Adding the --pgbackrest-s3-uri-style flag to pgo create cluster, which accepts values of host or path.
• Added support to disable TLS verification when connecting to a pgBackRest repository. This includes:
  • Adding the BackrestS3VerifyTLS configuration parameter to the PostgreSQL Operator ConfigMap (pgo.yaml). Defaults to true.
  • Adding the --pgbackrest-s3-verify-tls flag to pgo create cluster, which accepts values of true or false.
• Perform a pg_dump from a specific database using the --database flag when using pgo backup with --backup-type=pgdump.
• Restore a pg_dump to a specific database using the --pgdump-database flag using pgo restore when --backup-type=pgdump is specified.
• Allow for support of authentication parameters in the pgha-config (e.g. sslmode). See the documentation for words of caution on using these.
• Add the --client flag to pgo version to output the client version of pgo.
• A Helm Chart using Helm v3 is now available.

Changes

• The PostgreSQL cluster scope is now utilized to identify and sync the ConfigMap responsible for the DCS for a PostgreSQL cluster.
• The PGMONITOR_PASSWORD is now populated by an environmental variable secret. This environmental variable is only set on a primary instance as it is only needed at the time a PostgreSQL cluster is initialized.
• Remove "Operator Start Time" from pgo status as it is more convenient and accurate to get this information from kubectl and the like, and it was not working due to RBAC privileges. (Reported by @mw-0).
• Remove dependency on the expenv binary that was included in the PostgreSQL Operator release. All expenv calls were either replaced with the native envsubst program or removed.

Fixes

• Add validation to ensure that limits for CPU/memory are greater-than-or-equal-to the requests. This applies to any command that can set a limit/request.
• Ensure PVC capacities are being accurately reported when using pgo show cluster
• Ensure WAL archives are pushed to all repositories when pgBackRest is set to use both a local and a S3-based repository
• Silence expected error conditions when a pgBackRest repository is being initialized.
• Deployments with pgo-deployer using the default file with hostpathstorage will now successfully deploy PostgreSQL clusters without any adjustments.
• Ensure the PostgreSQL Operator can be uninstalled by adding list verb ClusterRole privileges to several Kubernetes objects.
• Ensure client-setup.sh executes to completion if existing PostgreSQL Operator credentials exist that were created by a different installation method.
• Update the documentation to properly name CCP_IMAGE_PULL_SECRET_MANIFEST and PGO_IMAGE_PULL_SECRET_MANIFEST in the pgo-deployer configuration.
• Several fixes for selecting default storage configurations and sizes when using the pgo-deployer container. These include #1, #4, and #8.
• Do not consider non-running Pods as primary Pods when checking for multiple primaries (Reported by @djcooklup).
• The custom setup example was updated to reflect the current state of bootstrapping the PostgreSQL container.
• Fix an issue with controller processing, which could manifest in PostgreSQL clusters not being deleted.
• Eliminate gcc from the postgres-ha and pgadmin4 containers.

4.3.2

Crunchy Data announces the release of the PostgreSQL Operator 4.3.2 on June 3, 2020.

The PostgreSQL Operator is released in conjunction with the Crunchy Container Suite.

Version 4.3.2 of the PostgreSQL Operator contains bug fixes to the installer container and changes to how CPU/memory requests and limits can be specified.

PostgreSQL Operator is tested with Kubernetes 1.13 - 1.18, OpenShift 3.11+, OpenShift 4.3+, Google Kubernetes Engine (GKE), and VMware Enterprise PKS 1.3+.

Changes

Resource Limit Flags

PostgreSQL Operator 4.3.0 introduced some new options to tune the resource requests for PostgreSQL instances under management and other associated deployments, including pgBackRest and pgBouncer. From some of our learnings of running PostgreSQL in Kubernetes, we heavily restricted how the limits on the Pods could be set, and tied them to be the same as the requests.

Due to feedback from a variety of sources, this caused more issues than it helped, and as such, we decided to introduce a breaking change into a patch release and remove the --enable-*-limit and --disable-*-limit series of flags and replace them with flags that allow you to specifically choose CPU and memory limits.

This release introduces several new flags to various commands, including:

• pgo create cluster --cpu-limit
• pgo create cluster --memory-limit
• pgo create cluster --pgbackrest-cpu-limit
• pgo create cluster --pgbackrest-memory-limit
• pgo create cluster --pgbouncer-cpu-limit
• pgo create cluster --pgbouncer-memory-limit
• pgo update cluster --cpu-limit
• pgo update cluster --memory-limit
• pgo update cluster --pgbackrest-cpu-limit
• pgo update cluster --pgbackrest-memory-limit
• pgo create pgbouncer --cpu-limit
• pgo create pgbouncer --memory-limit
• pgo update pgbouncer --cpu-limit
• pgo update pgbouncer --memory-limit

Additionally, these values can be modified directly in a pgcluster Custom Resource and the PostgreSQL Operator will react and make the modifications.

Other Changes

• The pgo-deployer container can now run using an aribtrary UID.
• For deployments of the PostgreSQL Operator using the pgo-deployer container to OpenShift 3.11 environments, a new template YAML file, postgresql-operator-ocp311.yml is provided. This YAML file requires that the pgo-deployer is run with cluster-admin role for OpenShift 3.11 environments due to the lack of support of the escalate RBAC verb. Other environments (e.g. Kubernetes, OpenShift 4+) still do not require cluster-admin.
• Allow for the resumption of download the pgo client if the client-setup.sh script gets interrupted. Contributed by Itay Grudev (@itay-grudev).

Fixes

• The pgo-deployer container now assigns the required Service Account all the appropriate get RBAC privileges via the postgres-operator.yml file that it needs to properly install. This allows the install functionality to properly work across multiple runs.
• For OpenShift deploymments, the pgo-deployer leverages version 4.4 of the oc client.
• Use numeric UIDs for users in the PostgreSQL Operator management containers to support MustRunAsNonRoot Pod Security Policies and the like. Reported by Olivier Beyler (@obeyler).