sql/stats: TestCreateStatsProgress failed

[cockroach-teamcity]

sql/stats.TestCreateStatsProgress failed on release-24.1.0-rc @ 5e4ca9e26f1a25681de9c944298cfa139c344466:

Fatal error:

panic: test timed out after 41m37s
running tests:
	TestCreateStatsProgress (41m31s)

Stack:

goroutine 18407 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2366 +0x265
created by time.goFunc
	GOROOT/src/time/sleep.go:177 +0x45

Log preceding fatal error

* github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3({0xb19a418, 0xc008c210e0}, {0x82a27e0, 0xc0068df200}, 0xc001216b80, 0xc007e14640)
* 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:169 +0xf1
* google.golang.org/grpc.getChainUnaryHandler.func1({0xb19a418, 0xc008c210e0}, {0x82a27e0, 0xc0068df200})
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1163 +0x12d
* github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor({0xc0047ee000?, {{0x0?}, {0xb1dde20?, 0xc0045301c0?}}}, {0xb19a418, 0xc008c21020}, {0x82a27e0, 0xc0068df200}, 0xc001216b80, 0xc007e14600)
* 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/auth.go:95 +0x2b7
* google.golang.org/grpc.getChainUnaryHandler.func1({0xb19a418, 0xc008c21020}, {0x82a27e0, 0xc0068df200})
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1163 +0x12d
* github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1({0xb19a418, 0xc008c21020})
* 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:136 +0x6b
* github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc0056bf360, {0xb19a418, 0xc008c21020}, {0xc000d68808?, 0xc007e145c0?}, 0xc004b43700)
* 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:337 +0x186
* github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1({0xb19a418, 0xc008c21020}, {0x82a27e0, 0xc0068df200}, 0xc001216b80, 0xc007e145c0)
* 	github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:134 +0x151
* google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0xb19a418, 0xc008c21020}, {0x82a27e0, 0xc0068df200}, 0xc001216b80, 0xc008d2d020)
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1154 +0xe8
* github.com/cockroachdb/cockroach/pkg/kv/kvpb._Internal_Batch_Handler({0x8284140, 0xc002785408}, {0xb19a418, 0xc008c21020}, 0xc00366cb60, 0xc0014ecb80)
* 	github.com/cockroachdb/cockroach/pkg/kv/kvpb/bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:10634 +0x1f4
* google.golang.org/grpc.(*Server).processUnaryRPC(0xc005255860, {0xb1e75a0, 0xc0052571e0}, 0xc006887c20, 0xc0084158f0, 0xe6c0540, 0x0)
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1336 +0x1814
* google.golang.org/grpc.(*Server).handleStream(0xc005255860, {0xb1e75a0, 0xc0052571e0}, 0xc006887c20, 0x0)
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:1704 +0x1013
* google.golang.org/grpc.(*Server).serveStreams.func1.2()
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:965 +0xe5
* created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 4689
* 	google.golang.org/grpc/external/org_golang_google_grpc/server.go:963 +0x4a8
* 
* goroutine 15272 [sync.Cond.Wait, 2 minutes]:
* sync.runtime_notifyListWait(0xc003c080a8, 0x1bb)
* 	GOROOT/src/runtime/sema.go:569 +0x159
* sync.(*Cond).Wait(0xc003c08098)
* 	GOROOT/src/sync/cond.go:70 +0x75
* github.com/cockroachdb/pebble/record.(*flusherCond).Wait(...)
* 	github.com/cockroachdb/pebble/record/external/com_github_cockroachdb_pebble/record/log_writer.go:369
* github.com/cockroachdb/pebble/record.(*LogWriter).flushLoop(0xc003c08000, {0xc005d8a030?, 0x7b5b200?})
* 	github.com/cockroachdb/pebble/record/external/com_github_cockroachdb_pebble/record/log_writer.go:643 +0xb79
* runtime/pprof.Do({0xb19a248, 0xfc52080}, {{0xc000abb6c0?, 0x1ffc760?, 0xc001c6c7e0?}}, 0xc00464dfc0)
* 	GOROOT/src/runtime/pprof/runtime.go:51 +0x118
* github.com/cockroachdb/pebble/record.NewLogWriter.func2()
* 	github.com/cockroachdb/pebble/record/external/com_github_cockroachdb_pebble/record/log_writer.go:562 +0x92
* created by github.com/cockroachdb/pebble/record.NewLogWriter in goroutine 2923
* 	github.com/cockroachdb/pebble/record/external/com_github_cockroachdb_pebble/record/log_writer.go:561 +0x8e5
* 
* goroutine 15271 [select, 2 minutes]:
* github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker.func1()
* 	github.com/cockroachdb/pebble/vfs/external/com_github_cockroachdb_pebble/vfs/disk_health.go:243 +0x19b
* created by github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker in goroutine 2923
* 	github.com/cockroachdb/pebble/vfs/external/com_github_cockroachdb_pebble/vfs/disk_health.go:238 +0xad
* 
*

Parameters:

• attempt=1
• race=true
• run=3
• shard=7

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• sql/stats: TestCreateStatsProgress failed #121770 sql/stats: TestCreateStatsProgress failed [C-test-failure O-robot P-3 T-sql-queries branch-master]

This test on roachdash | Improve this report!

Jira issue: CRDB-38871

[yuzefovich]

dup of #121770