Update to polkadot v0.9.39 #4705

mnaamani · 2023-03-21T05:05:18Z

Implements #4704

Changes:

Testing:

A new genesis config for Multiplier appears in tx_payment pallet, if not set during upgrade what value will it have and is it problematic. -> no the pallet has NextFeeMultiplierOnEmpty() -> 1 so it is not a problem and no migration is required.
Ensure integration tests work and pass
Ensure runtime upgrade integration tests pass - on runtime upgrade processor seem to stop processing, indexer appears to not correctly handle the block with upgrade. Currently working around it. Hydra Indexer unstable during runtime upgrade #4741 - on more recent run everything seems to have worked correctly https://github.com/Joystream/joystream/actions/runs/4872691408/jobs/8692880377 in this setup we kept the old node running for the upgrade, then started up new node after upgrade..
Test running new node against existing chain
Test running old node and upgrade to new runtime (covered by integration test)
- any database migrations run - no
Test all expected substrate pallet runtime migrations run successfully.

Package Versioning:

Bump package versions

Misc Fixes:

Fix for Runtime returns incorrect values for constants: DefaultGlobalWeeklyNftLimit, DefaultChannelDailyNftLimit & DefaultChannelWeeklyNftLimit #4726

TBD:

From version v0.9.25 transaction version was bumped bump transaction version (polkadot & kusama) paritytech/polkadot#5690 but only requirement seems to be due to use of treasury pallet which we do not use.
Optional:
Should we change benchmarking code to use new syntax - migrate new benchmarking syntax from frame_support::benchmarking to frame_benchmarking::v2 paritytech/substrate#13235

┆Issue is synchronized with this Asana task by Unito

vercel · 2023-03-21T05:05:22Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated (UTC)
pioneer-testnet	⬜️ Ignored (Inspect)		May 23, 2023 6:23am

mnaamani · 2023-04-11T19:46:01Z

Integration test failing:

2023-04-06T11:50:00.385Z integration-tests:job:direct channel payment by members flow 0 failed:
Error: Could not find matching query-node event (expected 2966:3)!
    at DirectChannelPaymentsHappyCaseFixture.findMatchingQueryNodeEvent (/home/runner/work/joystream/joystream/tests/network-tests/src/Fixture.ts:100:13)
    at /home/runner/work/joystream/joystream/tests/network-tests/src/Fixture.ts:121:27
    at Array.forEach (<anonymous>)
    at DirectChannelPaymentsHappyCaseFixture.assertQueryNodeEventsAreValid (/home/runner/work/joystream/joystream/tests/network-tests/src/Fixture.ts:119:17)
    at /home/runner/work/joystream/joystream/tests/network-tests/src/fixtures/content/channelPayouts/DirectChannelPaymentHappyCaseFixture.ts:75:25
    at QueryNodeApi.tryQueryWithTimeout (/home/runner/work/joystream/joystream/tests/network-tests/src/QueryNodeApi.ts:521:9)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async DirectChannelPaymentsHappyCaseFixture.runQueryNodeChecks (/home/runner/work/joystream/joystream/tests/network-tests/src/fixtures/content/channelPayouts/DirectChannelPaymentHappyCaseFixture.ts:73:5)
    at async FixtureRunner.runQueryNodeChecks (/home/runner/work/joystream/joystream/tests/network-tests/src/Fixture.ts:190:5)
    at async FixtureRunner.runWithQueryNodeChecks (/home/runner/work/joystream/joystream/tests/network-tests/src/Fixture.ts:195:5)
2023-04-06T11:50:00.412Z integration-tests:job:direct channel payment by members [Failed]

edit:
Fixed in 5156465

compiling now requires at least rustc 1.66.0

runtime weights need to be adjusted for fee/weight failing tests

…docker.sh

…e test scenario

ignazio-bovo · 2023-05-16T13:15:55Z

I should mention that the new version should be 2002 and not 3000

ignazio-bovo

I left a few questions, but overall seems very good

ignazio-bovo · 2023-05-12T08:35:18Z

.github/workflows/joystream-node-docker-dev.yml

        with:
          context: .
          file: joystream-node.Dockerfile
+          # arm64 cross compiling takes longer than 6h, so we only build for amd64
+          # platforms: linux/amd64,linux/arm64


Suggested change

# platforms: linux/amd64,linux/arm64

# platforms: linux/amd64

Since you said "only amd64" above.

The line is already commented out, but left it for reference if we want to add it back.
It might be better to drop this whole workflow and improve joystream-node-docker.yml which deploys two worker machines on AWS (amd64 and arm64) to build natively for each platform arch have the ansible playbook build for all the runtime profiles.

ignazio-bovo · 2023-05-12T08:36:50Z

.github/workflows/joystream-node.yml

@@ -36,16 +36,15 @@ jobs:

      - name: Build
        env:
-          WASM_BUILD_TOOLCHAIN: nightly-2022-05-11 
+          WASM_BUILD_TOOLCHAIN: nightly-2022-11-15


Was this toolchain ver. recommended by substrate?

Yes the nightly version at time the version was released
https://github.com/paritytech/polkadot/releases/tag/v0.9.39-rc6

ignazio-bovo · 2023-05-12T09:13:30Z

.github/workflows/joystream-node.yml

@@ -78,6 +77,19 @@ jobs:
          yarn cargo-checks && yarn cargo-build
          ./target/release/call-sizes
        if: env.GIT_DIFF
+      - name: OnRuntimeUpgrade (try-runtime)


This make the runtime-upgrade in the network tests obsolete, if I correctly understood the try runtime feature.
Should we drop the checks using the fork-off and move them to the pre/post hooks in the OnRuntimeUpgrade impl.?

That's a good point.
There might still be some use for the runtime upgrade integration tests to see how it affects the query-node, storage and distributor nodes during an upgrade?

ignazio-bovo · 2023-05-12T09:20:41Z

Cargo.toml

+# https://doc.rust-lang.org/rustc/linker-plugin-lto.html
+lto = "fat"
+# https://doc.rust-lang.org/rustc/codegen-options/index.html#codegen-units
+codegen-units = 1


Nice, where did you get the values from?

From substrate/Cargo.toml https://github.com/paritytech/substrate/blob/31491fa24164b1cc9fffd5f07f06f41c8fef437d/Cargo.toml#L376

ignazio-bovo · 2023-05-12T12:54:48Z

runtime/src/utils.rs

-    let len_fee =
-        <Runtime as pallet_transaction_payment::Config>::LengthToFee::weight_to_fee(&length);
+    let len_fee = <Runtime as pallet_transaction_payment::Config>::LengthToFee::weight_to_fee(
+        &Weight::from_ref_time(length),


Suggested change

&Weight::from_ref_time(length),

&Weight::from_parts(length, 0u64),

I did some research and this function will be deprecated: https://paritytech.github.io/substrate/master/sp_weights/struct.Weight.html#method.from_ref_time
The suggestion is what they recommend

Updated in 51fb763

…apt package

mnaamani · 2023-07-30T22:57:11Z

I updated the PR from latest master branch.
I also updated the nara branch to match master.

mnaamani · 2023-07-30T22:58:11Z

I should mention that the new version should be 2002 and not 3000

yes it has already been modified to 2002.

mnaamani added the runtime label Mar 21, 2023

mnaamani self-assigned this Mar 21, 2023

mnaamani added the right-after-ephesus label Mar 21, 2023

mnaamani force-pushed the update-to-polkadot-v0.9.39 branch from 6f56afc to 9712c6d Compare April 6, 2023 07:56

mnaamani added 24 commits April 12, 2023 13:32

v0.9.39 - set dependency

cccc77e

v0.9.39 - pallet-randomness-collective-flip renamed (insecure)

f74dc18

v0.9.39 - drop uncles code

bb28c30

v0.9.39 - adjust for renamed packages and update crates versions

b770d81

compiling now requires at least rustc 1.66.0

v0.9.39 - Origin rename to RuntimeOrigin

ab2d5c9

v0.9.39 - rename Event to RuntimeEvent

eac1549

v0.9.39 - workaround weights v2

547cbf8

v0.9.39 - more weights fixes

cc64367

v0.9.39 - fix pallet builds

e35ed91

v0.9.39 - fix mock and tests use of Origin and Event types

3a0ab6c

v0.9.39 - Fix runtime build

bb22687

v0.9.39 - fix runtime tests build

9b069a8

runtime weights need to be adjusted for fee/weight failing tests

v0.9.39 - fix cli dependency on clap

1033c1c

v0.9.39 - tweak Cargo.toml build settings

a2475af

v0.9.39 - set MaxActiveValidators to not exceed genesis limit

7ab781e

v0.9.39 - fix call-sizes build

b78efb4

v0.9.39 - fix node rpc,service

c400c98

v0.9.39 - no mmr

b51dd88

v0.9.39 - fix node build

40cb698

v0.9.39 - Fix max block weight

8872a09

v0.9.39 - fix runtime-benchmarks feature build

9697a84

v0.9.39 - re-enable node-inspect command

caca06c

v0.9.39 - fix joystream-node unit tests

b5c2462

v0.9.39 - update wastime to latest

7f4adef

mnaamani added 6 commits May 2, 2023 17:38

docker build workflows: different cache names to avoid clash

8de82d4

docker build, remove arm64 cross compile takes too long

e1fd7e7

warn if host arch and docker image arch differ when using build-node-…

b3fb6d6

…docker.sh

Merge branch 'master' into update-to-polkadot-v0.9.39

d930492

update runtime changelog

914c4d8

run runtime upgrade: use old node before upgrade, and run post upgrad…

354cf7d

…e test scenario

ignazio-bovo mentioned this pull request May 3, 2023

Tracker: Runtime CRT work for crt-1 release #4748

Closed

9 tasks

mnaamani added 2 commits May 3, 2023 17:10

fix post runtime upgrade check

5d5c9c1

update types to v2.2.0 - metadata update (joystream-utility-pallet)

85fd568

mnaamani requested review from ignazio-bovo and Lezek123 May 4, 2023 04:36

mnaamani marked this pull request as ready for review May 4, 2023 04:36

ignazio-bovo suggested changes May 17, 2023

View reviewed changes

mnaamani added 8 commits May 19, 2023 11:24

Merge branch 'master' into update-to-polkadot-v0.9.39

1efdd5e

setup.sh drop getsubstrate.io - tries to install deprecated protobuf …

b3eebca

…apt package

rustup install script - non-interactive shell fix

7fc95a4

setup.sh - rustup update after install

779c774

Merge branch 'master' into update-to-polkadot-v0.9.39

f936481

Merge branch 'nara' into update-to-polkadot-v0.9.39

0825bd1

update cargo-chef to work with latest nightly

09fac54

fix cargo-chef install no --locked arg

2f82527

mnaamani added 2 commits July 31, 2023 02:33

use Weight:from_parts in place of from_ref_time which is deprecated

d53ddd4

generate weights using Weight:from_parts in template

51fb763

mnaamani requested review from ignazio-bovo and removed request for Lezek123 July 31, 2023 07:02

mnaamani merged commit 19de3f9 into nara Jul 31, 2023
52 checks passed

mnaamani deleted the update-to-polkadot-v0.9.39 branch September 23, 2023 20:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to polkadot v0.9.39 #4705

Update to polkadot v0.9.39 #4705

mnaamani commented Mar 21, 2023 •

edited

vercel bot commented Mar 21, 2023 •

edited

mnaamani commented Apr 11, 2023 •

edited

ignazio-bovo commented May 16, 2023

ignazio-bovo left a comment

ignazio-bovo May 12, 2023

mnaamani Jul 30, 2023

ignazio-bovo May 12, 2023

mnaamani Jul 30, 2023

ignazio-bovo May 12, 2023

mnaamani Jul 30, 2023

ignazio-bovo May 12, 2023

mnaamani Jul 30, 2023

ignazio-bovo May 12, 2023

mnaamani Jul 31, 2023

mnaamani commented Jul 30, 2023

mnaamani commented Jul 30, 2023

	# platforms: linux/amd64,linux/arm64
	# platforms: linux/amd64

	&Weight::from_ref_time(length),
	&Weight::from_parts(length, 0u64),

Update to polkadot v0.9.39 #4705

Update to polkadot v0.9.39 #4705

Conversation

mnaamani commented Mar 21, 2023 • edited

vercel bot commented Mar 21, 2023 • edited

mnaamani commented Apr 11, 2023 • edited

ignazio-bovo commented May 16, 2023

ignazio-bovo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnaamani commented Jul 30, 2023

mnaamani commented Jul 30, 2023

mnaamani commented Mar 21, 2023 •

edited

vercel bot commented Mar 21, 2023 •

edited

mnaamani commented Apr 11, 2023 •

edited