-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
calico-kube-controllers fix concurrent map writes issue #8706
calico-kube-controllers fix concurrent map writes issue #8706
Conversation
✅ Deploy Preview for calico-v3-25 canceled.
|
/sem-approve |
Thanks for the PR. Could you show where the conflicting Write is happening on the resource? If I understand the issue & panic correctly, the panic is generated by If the concurrent write is also occurring in the same Do we know what component is writing to that On a side-note, can you submit a PR of the fix to master branch instead, rather than directly to the release branch, to ensure we don't regress in future releases. We can then backport the master patch to 3.26. |
@@ -126,7 +126,9 @@ func (c *calicoCache) Set(key string, newObj interface{}) { | |||
if reflect.TypeOf(newObj) != c.ObjectType { | |||
c.log.Fatalf("Wrong object type received to store in cache. Expected: %s, Found: %s", c.ObjectType, reflect.TypeOf(newObj)) | |||
} | |||
|
|||
// lock the cache |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something isn't quite adding up to me here - the underlying map is within the threadSafeCache
instance, which IIUC should already handle this.
Do you know why we need this second mutex?
Description
Add mutex lock to the Set function to prevent a race condition that causes the process to panic with fatal error: concurrent map read and map write
Related issues/PRs
fixes 8705
Todos
Release Note
Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one
docs-*
label.docs-pr-required
: This change requires a change to the documentation that has not been completed yet.docs-completed
: This change has all necessary documentation completed.docs-not-required
: This change has no user-facing impact and requires no docs.Every PR needs one
release-note-*
label.release-note-required
: This PR has user-facing changes. Most PRs should have this label.release-note-not-required
: This PR has no user-facing changes.Other optional labels:
cherry-pick-candidate
: This PR should be cherry-picked to an earlier release. For bug fixes only.needs-operator-pr
: This PR is related to install and requires a corresponding change to the operator.