Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv/kvserver: TestReplicateAfterRemoveAndSplit failed #124396

Open
cockroach-teamcity opened this issue May 19, 2024 · 6 comments
Open

kv/kvserver: TestReplicateAfterRemoveAndSplit failed #124396

cockroach-teamcity opened this issue May 19, 2024 · 6 comments
Labels
branch-master Failures on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv-replication KV Replication Team

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented May 19, 2024 •
edited by cockroach-jira-scripts

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ 93ad913106b6f0f6ec98bc2cfa788ff6d8085bd4:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit4213416496
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit4213416496
--- FAIL: TestReplicateAfterRemoveAndSplit (10.60s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/replication

This test on roachdash | Improve this report!

Jira issue: CRDB-38870
@cockroach-teamcity cockroach-teamcity added branch-master Failures on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv-replication KV Replication Team labels May 19, 2024
@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ e4d744226f47e37b9179632711067277b2b0ab80:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit2944513682
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit2944513682
--- FAIL: TestReplicateAfterRemoveAndSplit (7.42s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@kvoli
Copy link
Collaborator

kvoli commented May 20, 2024 

/Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range

Could be related to recent changes to speed up descriptor iteration on startup, hitting:

cockroach/pkg/keys/keys.go

Lines 428 to 431 in 3a3c1d8

if !bytes.HasPrefix(key, LocalRangePrefix) {
return nil, nil, nil, errors.Errorf("key %q does not have %q prefix",
key, LocalRangePrefix)
}

@kvoli kvoli added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. P-1 Issues/test failures with a fix SLA of 1 month labels May 20, 2024
@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ d146ecff6f687e438706cf63591cafca60cc116d:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit502155402
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit502155402
--- FAIL: TestReplicateAfterRemoveAndSplit (7.62s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

This was referenced May 21, 2024
Open
Open
@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ 8156d095961d7271df61fe2e2f716ef072bc5f96:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit3463261215
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit3463261215
--- FAIL: TestReplicateAfterRemoveAndSplit (7.58s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ c7f16c96d4db6ef97085f972f094c7e4e7c1518f:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit1451704657
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit1451704657
--- FAIL: TestReplicateAfterRemoveAndSplit (7.32s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

This was referenced May 30, 2024
Open
Open
@cockroach-teamcity
Copy link
Member Author

kv/kvserver.TestReplicateAfterRemoveAndSplit failed on master @ c84a89569fab4cc4896a0234bec1e1822dd420d8:

=== RUN   TestReplicateAfterRemoveAndSplit
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestReplicateAfterRemoveAndSplit3196754340
    test_log_scope.go:81: use -show-logs to present logs inline
    client_raft_test.go:1851: failed to start store: key /Local/RangeID/5/r/RangeGCThreshold does not have /Local/Range prefix
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestReplicateAfterRemoveAndSplit3196754340
--- FAIL: TestReplicateAfterRemoveAndSplit (7.98s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=2
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-master Failures on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv-replication KV Replication Team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cockroach-teamcity @kvoli