guix: Use LTO to build releases #25391

fanquake · 2022-06-16T13:21:34Z

Changes required to use LTO in Guix (release) builds.

DrahtBot · 2022-06-16T15:02:15Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj, TheCharlatan
Approach ACK	hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#30201 (depends: remove FORCE_USE_SYSTEM_CLANG by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

laanwj · 2022-06-22T11:54:14Z

Concept ACK. This would be great.

hebasto · 2022-07-05T10:05:56Z

... build all binaries except for bitcoin-qt

See #25542.

hebasto · 2022-07-10T18:48:55Z

Tested 076aa5e on Ubuntu 22.04:

$ make -C depends HOST=riscv64-linux-gnu LTO=1 NO_QT=1
$ ./autogen.sh
$ ./configure CONFIG_SITE=depends/riscv64-linux-gnu/share/config.site
$ make clean
$ make
...
  CXXLD    libbitcoinconsensus.la
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:6729: libbitcoinconsensus.la] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/home/hebasto/git/bitcoin/src'
make[1]: *** [Makefile:18615: all-recursive] Error 1
make[1]: Leaving directory '/home/hebasto/git/bitcoin/src'
make: *** [Makefile:818: all-recursive] Error 1

UPDATE: CONFIG_SITE needs a absolute path. Sorry for the noise.

fanquake · 2022-07-13T14:44:41Z

Tested 076aa5e on Ubuntu 22.04:

I'm not sure what's causing that (maybe a not-completely-clean build dir, and mix of compiled objects), but I don't have any issues with this branch, following the same steps, using Ubuntu 22.04 and riscv64-linux-gnu-g++-11.

depends/packages/expat.mk

… code on Linux 3442865 build: Use Link Time Optimization for Qt code on Linux (Hennadii Stepanov) ebce66e build: pass -fno-lto when building expat (fanquake) Pull request description: See: https://www.qt.io/blog/2019/01/02/qt-applications-lto `bitcon-qt` unstripped size: | host | master (31c6309) | this PR, depends built with `LTO=1` | |---|:-:|:-:| | x86_64-pc-linux-gnu | 42 MB | 35 MB | | arm-linux-gnueabihf | 31 MB | 26 MB | | aarch64-linux-gnu | 41 MB | 32 MB | | powerpc64-linux-gnu | 51 MB | 41 MB | | powerpc64le-linux-gnu | 48 MB | 39 MB | | riscv64-linux-gnu | 35 MB | 29 MB | Based on the first commit from bitcoin/bitcoin#25391. Using LTO for macOS and Windows hosts has some issues which could be addressed in follow ups. x86_64 build: ![image](https://user-images.githubusercontent.com/32963518/179326902-f91853ca-23c1-4c04-9a6d-161b695f27b5.png) ACKs for top commit: fanquake: ACK 3442865 Tree-SHA512: 03eef2568358df9336e24d6c4e12f28b89d649076fb74e7e5303d61e52865c2360c5345a4fb2b1e4bdfdae194f273fc27a5f67e6cf797ed01a154f3da9117247

fanquake · 2022-07-18T09:43:02Z

Rebased for #25542. All binaries for our Linux HOSTS now build.

…th GCC) 658685a depends: default to using GCC tool wrappers (with GCC) (fanquake) 6fdc13c build: Fix autoconf variable names for tools found by `AC_PATH_TOOL` (Hennadii Stepanov) Pull request description: This improves support for LTO by using gcc wrappers for `ar`, `nm`, `ranlib`, that correctly setup plugin arguments for LTO, when using GCC. Other HOSTS are using clang. Portion of #25391. Guix Build (x86_64): ```bash ``` Guix Build (arm64): ```bash ``` ACKs for top commit: dongcarl: Code Review ACK 658685a hebasto: ACK 658685a jarolrod: ACK 658685a Tree-SHA512: 28d6127c118f74336c97e2523117f8a0d11b32cd565124cd4052baeb7cc53e71909d3037cb080d996ae4e3ce600326fced37ee36adcc53d839ba7dd7974ebcd2

hebasto · 2024-01-06T12:37:07Z

build: remove --enable-lto #29185

4133c81 guix: use gcc tool wrappers (fanquake) Pull request description: This way, correct `--plugin` arguments are passed through. This is a prerequisite for LTO (see bitcoin#25391). Split out, to try move things along, as this change is isolated, and should be straight-forward. ACKs for top commit: TheCharlatan: ACK [4133c81](bitcoin@4133c81) hebasto: ACK 4133c81 Tree-SHA512: 4311a72a613cf027bd4490caa29604c985ed455589acd972285f13cbdf4806d2184a4dc6f20cb6f47c3fa751d58bfd0bacc257b87d4a804bf5ecf5b240e4a757

Summary: This way, correct `--plugin` argument are passed through. This is a prerequisite for LTO (see [[bitcoin/bitcoin#25391 | core#25391]]). This is a backport of [[bitcoin/bitcoin#27345 | core#27345]] Depends on D15328 Test Plan: `contrib/guix.guix-build` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D15329

4133c81 guix: use gcc tool wrappers (fanquake) Pull request description: This way, correct `--plugin` arguments are passed through. This is a prerequisite for LTO (see bitcoin#25391). Split out, to try move things along, as this change is isolated, and should be straight-forward. ACKs for top commit: TheCharlatan: ACK [4133c81](bitcoin@4133c81) hebasto: ACK 4133c81 Tree-SHA512: 4311a72a613cf027bd4490caa29604c985ed455589acd972285f13cbdf4806d2184a4dc6f20cb6f47c3fa751d58bfd0bacc257b87d4a804bf5ecf5b240e4a757

```bash bitcoind: export of symbol in6addr_any not allowed! bitcoind: failed EXPORTED_SYMBOLS test/test_bitcoin: export of symbol in6addr_any not allowed! test/test_bitcoin: failed EXPORTED_SYMBOLS ``` Also new exports for bitcoin-qt.

maflcko · 2024-06-04T10:02:01Z

Is this safe to do in light of https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105469 ?

Also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113359 (in combination with #29881)

Would be nice to extend the motivation (pull request description) a bit about the benefits and risks.

DrahtBot · 2024-06-11T16:54:00Z

🐙 This pull request conflicts with the target branch and needs rebase.

fanquake added Brainstorming Build system labels Jun 16, 2022

This was referenced Jun 16, 2022

build: Improve depends build system robustness #22552

Closed

scripts: remove no-longer-needed ignored exports #25392

Merged

DrahtBot added the Needs rebase label Jun 17, 2022

fanquake force-pushed the lto_in_guix branch from 15cbee0 to 135000d Compare June 20, 2022 20:58

DrahtBot removed the Needs rebase label Jun 20, 2022

fanquake force-pushed the lto_in_guix branch 3 times, most recently from 32d3049 to c6b8b4d Compare June 22, 2022 10:46

fanquake force-pushed the lto_in_guix branch 2 times, most recently from 545abfa to 076aa5e Compare July 5, 2022 08:46

hebasto mentioned this pull request Jul 5, 2022

build: Use Link Time Optimization for Qt code on Linux #25542

Merged

fanquake mentioned this pull request Jul 8, 2022

[POC] guix: produce a fully -static-pie bitcoind #25573

Draft

fanquake mentioned this pull request Jul 14, 2022

depends: default to using GCC tool wrappers for LTO (with GCC) #25612

Merged

hebasto reviewed Jul 14, 2022

View reviewed changes

depends/packages/expat.mk Outdated Show resolved Hide resolved

fanquake force-pushed the lto_in_guix branch from 076aa5e to 463cf59 Compare July 18, 2022 09:40

fanquake changed the title ~~[POC] guix: Use LTO to build releases~~ guix: Use LTO to build releases Jul 18, 2022

maflcko added DrahtBot Guix build requested and removed DrahtBot Guix build requested labels Jul 18, 2022

fanquake force-pushed the lto_in_guix branch from 463cf59 to bbd663e Compare July 18, 2022 11:15

fanquake force-pushed the lto_in_guix branch from bbd663e to e10eee9 Compare July 19, 2022 07:45

DrahtBot mentioned this pull request Jan 11, 2024

depends: remove dependency on Darwin libtool #29232

Closed

DrahtBot added the CI failed label Jan 14, 2024

fanquake force-pushed the lto_in_guix branch from 9367f6e to 8cafa3d Compare January 16, 2024 10:14

DrahtBot removed the CI failed label Jan 16, 2024

DrahtBot mentioned this pull request Jan 16, 2024

Remove the legacy wallet and BDB dependency #28710

Draft

DrahtBot added the Needs rebase label Mar 14, 2024

fanquake force-pushed the lto_in_guix branch from 8cafa3d to 229dc35 Compare March 14, 2024 14:54

DrahtBot removed the Needs rebase label Mar 14, 2024

fanquake force-pushed the lto_in_guix branch from 229dc35 to 6cedf8a Compare April 17, 2024 12:57

fanquake force-pushed the lto_in_guix branch from 6cedf8a to 92cdbc7 Compare May 1, 2024 02:39

DrahtBot mentioned this pull request May 2, 2024

releases: use LLVM 18 for macOS #30022

Closed

fanquake force-pushed the lto_in_guix branch 2 times, most recently from 9907c82 to 739826f Compare May 22, 2024 09:32

fanquake marked this pull request as ready for review May 22, 2024 09:58

fanquake force-pushed the lto_in_guix branch from 739826f to 0441b71 Compare May 29, 2024 10:37

DrahtBot mentioned this pull request May 31, 2024

depends: remove FORCE_USE_SYSTEM_CLANG #30201

Merged

fanquake added 2 commits June 4, 2024 10:05

scripts: update exports for LTO

c81fcd0

```bash bitcoind: export of symbol in6addr_any not allowed! bitcoind: failed EXPORTED_SYMBOLS test/test_bitcoin: export of symbol in6addr_any not allowed! test/test_bitcoin: failed EXPORTED_SYMBOLS ``` Also new exports for bitcoin-qt.

guix: build depends with LTO=1 and -flto

0367ee6

fanquake force-pushed the lto_in_guix branch from 0441b71 to 0367ee6 Compare June 4, 2024 09:05

DrahtBot added the Needs rebase label Jun 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

guix: Use LTO to build releases #25391

guix: Use LTO to build releases #25391

fanquake commented Jun 16, 2022 •

edited

DrahtBot commented Jun 16, 2022 •

edited

laanwj commented Jun 22, 2022

hebasto commented Jul 5, 2022

hebasto commented Jul 10, 2022 •

edited

fanquake commented Jul 13, 2022

fanquake commented Jul 18, 2022

hebasto commented Jan 6, 2024

maflcko commented Jun 4, 2024

DrahtBot commented Jun 11, 2024

guix: Use LTO to build releases #25391

Are you sure you want to change the base?

guix: Use LTO to build releases #25391

Conversation

fanquake commented Jun 16, 2022 • edited

DrahtBot commented Jun 16, 2022 • edited

Code Coverage

Reviews

Conflicts

laanwj commented Jun 22, 2022

hebasto commented Jul 5, 2022

hebasto commented Jul 10, 2022 • edited

fanquake commented Jul 13, 2022

fanquake commented Jul 18, 2022

hebasto commented Jan 6, 2024

maflcko commented Jun 4, 2024

DrahtBot commented Jun 11, 2024

fanquake commented Jun 16, 2022 •

edited

DrahtBot commented Jun 16, 2022 •

edited

hebasto commented Jul 10, 2022 •

edited