WIP: volumebinding: scheduler queueing hints - phase1

[bells17]

What type of PR is this?

/kind feature

What this PR does / why we need it:

kube-scheduler implements scheduling hints for the VolumeBinding plugin.
The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated.

Which issue(s) this PR fixes:

Part of #118893
KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md

Special notes for your reviewer:

Fields Impacting QueueingHintFn

PersistentVolume (PV) is not included in this table because it can undergo extensive changes when a conversion is performed by csi-translation-lib.

resource	field	Referenced in PreFilter+Filter?	Admission Overwrite Prevention Config	Need to Check Changes in QHint?
StorageClass	.metadata.labels	x	x	x
StorageClass	.metadata.annotations	x	x	x
StorageClass	.provisioner	o	o	x
StorageClass	.parameters	x	o	x
StorageClass	.reclaimPolicy	x	o	x
StorageClass	.mountOptions	x	o	x
StorageClass	.allowVolumeExpansion	x	x	x
StorageClass	.volumeBindingMode	o	x	o
StorageClass	.allowedTopologies	o	x	o
PersistentVolumeClaim(PVC)	.metadata.labels	x	x	x
PersistentVolumeClaim(PVC)	.metadata.annotations	o	o	o
PersistentVolumeClaim(PVC)	.spec.accessModes	o	x	o
PersistentVolumeClaim(PVC)	.spec.selector	o	o	x
PersistentVolumeClaim(PVC)	.spec.resources	o	o	x
PersistentVolumeClaim(PVC)	.spec.volumeName	o	o	x
PersistentVolumeClaim(PVC)	.spec.storageClassName	o	x	o
PersistentVolumeClaim(PVC)	.spec.volumeMode	x	o	x
PersistentVolumeClaim(PVC)	.spec.dataSource	x	o	x
PersistentVolumeClaim(PVC)	.spec.dataSourceRef	x	o	x
PersistentVolumeClaim(PVC)	.spec.volumeAttributesClassName	x	o	x
PersistentVolumeClaim(PVC)	.status.phase	o	x	o
PersistentVolumeClaim(PVC)	.status.accessModes	x	x	x
PersistentVolumeClaim(PVC)	.status.capacity	x	x	x
PersistentVolumeClaim(PVC)	.status.conditions	x	x	x
PersistentVolumeClaim(PVC)	.status.allocatedResources	x	x	x
PersistentVolumeClaim(PVC)	.status.allocatedResourceStatuses	x	x	x
PersistentVolumeClaim(PVC)	.status.currentVolumeAttributesClassName	x	x	x
PersistentVolumeClaim(PVC)	.status.modifyVolumeStatus	x	x	x
Node	.metadata.labels	o	x	o
Node	.metadata.annotations	x	o	x
Node	.spec.podCIDR	x	x	x
Node	.spec.podCIDRs	x	o	x
Node	.spec.providerID	x	o	x
Node	.spec.unschedulable	x	x	x
Node	.spec.taints	x	x	x
Node	.spec.configSource	x	x	x
Node	.spec.externalID	x	o	x
Node	.status.capacity	x	x	x
Node	.status.allocatable	x	x	x
Node	.status.phase	x	x	x
Node	.status.conditions	x	x	x
Node	.status.addresses	x	o	x
Node	.status.daemonEndpoints	x	x	x
Node	.status.nodeInfo	x	x	x
Node	.status.images	x	x	x
Node	.status.volumesInUse	x	x	x
Node	.status.volumesAttached	x	x	x
Node	.status.config	x	x	x
Node	.status.runtimeHandlers	x	x	x
CSINode	.metadata.labels	x	x	x
CSINode	.metadata.annotations	o	x	o
CSINode	.spec.drivers.name	x	o	x
CSINode	.spec.drivers.nodeID	x	o	x
CSINode	.spec.drivers.topologyKeys	x	o	x
CSINode	.spec.drivers.allocatable	x	o	x
CSIDriver	.metadata.labels	x	o	x
CSIDriver	.metadata.annotations	o	o	x
CSIDriver	.spec.attachRequired	x	o	x
CSIDriver	.spec.podInfoOnMount	x	x	x
CSIDriver	.spec.VolumeLifecycleMode	x	o	x
CSIDriver	.spec.storageCapacity	o	x	o
CSIDriver	.spec.fsGroupPolicy	x	x	x
CSIDriver	.spec.tokenRequests	x	x	x
CSIDriver	.spec.requiresRepublish	x	x	x
CSIDriver	.spec.seLinuxMount	x	x	x
CSIStorageCapacity	.metadata.labels	x	x	x
CSIStorageCapacity	.metadata.annotations	x	x	x
CSIStorageCapacity	.nodeTopology	o	o	x
CSIStorageCapacity	.storageClassName	o	o	x
CSIStorageCapacity	.capacity	o	x	o
CSIStorageCapacity	.maximumVolumeSize	o	x	o

ref(ja): https://zenn.dev/bells17/scraps/65bd6891012bdc

Does this PR introduce a user-facing change?

kube-scheduler implements scheduling hints for the VolumeBinding plugin.
The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bells17
Once this PR has been reviewed and has the lgtm label, please assign gnufied for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/scheduler/framework/plugins/volumebinding/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bells17]

/cc @sanposhiho @utam0k @YamasouA

[k8s-ci-robot]

@bells17: GitHub didn't allow me to request PR reviews from the following users: YamasouA.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @sanposhiho @utam0k @YamasouA

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bells17]

/assign @YamasouA

[k8s-ci-robot]

@bells17: GitHub didn't allow me to assign the following users: YamasouA.

Note that only kubernetes members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @YamasouA

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bells17]

/retest

[bells17]

If a PV was subject to processing by csi-translation-lib, it may undergo significant changes. Therefore, even if only a part of the PV changes, I believe it should be treated as a target for queueing. I would appreciate your thoughts on this.

[bells17]

The fields of PVC referenced within Volumebinding are wide-ranging.
Therefore, if we implement a logic that strictly checks for changes in the fields being used, it would be difficult to correctly reflect those changes in the QueueingHintFn when there are modifications to PreFilter or Filter, and I think it would result in high maintenance costs.
So, for the Phase, Annotations, and Spec fields, I'm using DeepEqual to check for differences.
I would appreciate your thoughts on about this approach.

[bells17]

/retest

[sanposhiho]

can we close this PR in favor of other smaller PRs you opened?

[bells17]

/close

[k8s-ci-robot]

@bells17: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.