1.14.5 recreated all my certs as the same secret

[russell]

Describe the bug:

I have a namespace with 4 certs in it. All of them were recreated as the same cert, downgrading to 1.14.4 fixed the problem

Once i noticed the bug, basically a can see that all certificate requests are created for the same set of DNS entries. I extracted and printed the x509 certifcate request from a domain to see this.

all secrets in the namespace have the exact same creation time, so on upgrade it must have re-issued them all, or maybe one was close to expiry and it triggered updating them all?

This is the cert with some of the dns entries redacted in a consistent way, but the point is that this is then used as the template for all cert requests, so every cert secret became this cert's entry. I'm not sure why it was chosen as the template, i use some jsonnet to generate the certs so they are all the same format, just with different DNS settings.

$ kubectl -n certificates describe certs wild-redacted1
Name:         wild-redacted1
Namespace:    certificates
Labels:       app.kubernetes.io/instance=halifax-cert-manager
              qbec.io/application=cert-manager
              qbec.io/environment=halifax
Annotations:  qbec.io/component: certificates
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2021-07-03T05:56:50Z
  Generation:          3
  Resource Version:    201503680
  UID:                 13b756c1-e835-4a2d-8ec5-7c8a8963606e
Spec:
  Common Name:  *.<redacted1>
  Dns Names:
    *.<redacted1>
    *.<redacted2>
  Issuer Ref:
    Kind:       ClusterIssuer
    Name:       letsencrypt
  Secret Name:  wild-redacted1
Status:
  Conditions:
    Last Transition Time:  2024-05-09T12:48:09Z
    Message:               Certificate is up to date and has not expired
    Observed Generation:   3
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2024-08-07T11:48:07Z
  Not Before:              2024-05-09T11:48:08Z
  Renewal Time:            2024-07-08T11:48:07Z
  Revision:                19
Events:                    <none>

Expected behaviour:

I have 4 certs in i get 4 secrets that match them

Steps to reproduce the bug:

I'm going to have to assume that upgrading to 1.14.5 and putting multiple certs in the same namespace will trigger it.

I tried to delete all certificate requests and delete all the controller deployments, and then delete one cert and secret.

On recreation of the deployments and the cert resource, it recreate the same wrong cert request and secret as described above.

So I couldn't find any stored state that would indicate that this was a transient issue

Anything else we need to know?:

Environment details::

• Kubernetes version: v1.27.10+k3s2
• Cloud-provider/provisioner:vps
• cert-manager version: 1.14.5
• Install method: release yaml from github release page

/kind bug