Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) #52876

FireBurn · 2024-05-07T14:43:39Z

Version

20.12.1 - 22.1.0

Platform

Gentoo with GCC 14 or clang 18.1.

Subsystem

simdjson

What steps will reproduce the bug?

~~This didn't happen with node 20.12.1 which used the older simd~~ (Comments indicate that the build issue is present with earlier versions of Node)

FAILED: obj/deps/simdutf/simdutf.simdutf.o 
clang++ -MMD -MF obj/deps/simdutf/simdutf.simdutf.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DNODE_OPENSSL_CERT_STORE -DICU_NO_USER_DATA_OVERRIDE -D__STDC_FORMAT_MACROS -I../../deps/simdutf -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-omit-frame-pointer -O3 -pipe -march=native -fno-rtti -fno-exceptions -std=gnu++17  -c ../../deps/simdutf/simdutf.cpp -o obj/deps/simdutf/simdutf.simdutf.o
../../deps/simdutf/simdutf.cpp:23203:26: error: always_inline function '_mm512_setzero_si512' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23203 |     __m512i currentmax = _mm512_setzero_si512();
       |                          ^
../../deps/simdutf/simdutf.cpp:23203:26: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23205:20: error: always_inline function '_mm512_loadu_si512' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23205 |       __m512i in = _mm512_loadu_si512((__m512i *)buf);
       |                    ^
../../deps/simdutf/simdutf.cpp:23205:20: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23206:43: error: always_inline function '_mm512_set1_epi16' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23206 |       __m512i diff = _mm512_sub_epi16(in, _mm512_set1_epi16(uint16_t(0xD800)));
       |                                           ^
../../deps/simdutf/simdutf.cpp:23206:43: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23206:22: error: always_inline function '_mm512_sub_epi16' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23206 |       __m512i diff = _mm512_sub_epi16(in, _mm512_set1_epi16(uint16_t(0xD800)));
       |                      ^
../../deps/simdutf/simdutf.cpp:23206:22: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23208:11: error: '__builtin_ia32_ucmpw512_mask' needs target feature avx512bw,evex512
 23208 |           _mm512_cmplt_epu16_mask(diff, _mm512_set1_epi16(uint16_t(0x0800)));
       |           ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:357:5: note: expanded from macro '_mm512_cmplt_epu16_mask'
  357 |     _mm512_cmp_epu16_mask((A), (B), _MM_CMPINT_LT)
      |     ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:256:15: note: expanded from macro '_mm512_cmp_epu16_mask'
  256 |   ((__mmask32)__builtin_ia32_ucmpw512_mask((__v32hi)(__m512i)(a), \
      |               ^
../../deps/simdutf/simdutf.cpp:23208:41: error: always_inline function '_mm512_set1_epi16' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23208 |           _mm512_cmplt_epu16_mask(diff, _mm512_set1_epi16(uint16_t(0x0800)));
       |                                         ^
../../deps/simdutf/simdutf.cpp:23208:41: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23222:38: error: '__builtin_ia32_ucmpw512_mask' needs target feature avx512bw,evex512
 23222 |           __mmask32 highsurrogates = _mm512_cmplt_epu16_mask(
       |                                      ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:357:5: note: expanded from macro '_mm512_cmplt_epu16_mask'
  357 |     _mm512_cmp_epu16_mask((A), (B), _MM_CMPINT_LT)
      |     ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:256:15: note: expanded from macro '_mm512_cmp_epu16_mask'
  256 |   ((__mmask32)__builtin_ia32_ucmpw512_mask((__v32hi)(__m512i)(a), \
      |               ^
../../deps/simdutf/simdutf.cpp:23223:21: error: always_inline function '_mm512_set1_epi16' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23223 |               diff, _mm512_set1_epi16(uint16_t(0x0400)));
       |                     ^
../../deps/simdutf/simdutf.cpp:23223:21: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23265:20: error: always_inline function '_mm512_max_epu32' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23265 |       currentmax = _mm512_max_epu32(in, currentmax);
       |                    ^
../../deps/simdutf/simdutf.cpp:23265:20: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdutf/simdutf.cpp:23268:15: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
 23268 |       checker.check_next_input(in);
       |               ^
../../deps/simdutf/simdutf.cpp:23278:30: error: always_inline function '_mm512_maskz_loadu_epi8' requires target feature 'evex512', but would be inlined into function 'detect_encodings' that is compiled without support for 'evex512'
 23278 |         const __m512i utf8 = _mm512_maskz_loadu_epi8(
       |                              ^
../../deps/simdutf/simdutf.cpp:23278:30: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

FAILED: obj/deps/simdjson/simdjson.simdjson.o 
clang++ -MMD -MF obj/deps/simdjson/simdjson.simdjson.o.d -D_GLIBCXX_USE_CXX11_ABI=1 -DNODE_OPENSSL_CONF_NAME=nodejs_conf -DNODE_OPENSSL_CERT_STORE -DICU_NO_USER_DATA_OVERRIDE -D__STDC_FORMAT_MACROS -I../../deps/simdjson -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-omit-frame-pointer -O3 -pipe -march=native -fno-rtti -fno-exceptions -std=gnu++17  -c ../../deps/simdjson/simdjson.cpp -o obj/deps/simdjson/simdjson.simdjson.o
../../deps/simdjson/simdjson.cpp:20344:14: error: always_inline function '_mm512_loadu_si512' requires target feature 'evex512', but would be inlined into function 'load' that is compiled without support for 'evex512'
 20344 |       return _mm512_loadu_si512(reinterpret_cast<const __m512i *>(values));
       |              ^
../../deps/simdjson/simdjson.cpp:20344:14: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20344:14: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20470:51: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
 20470 |     simdjson_inline simd8(const __m512i _value) : base8_numeric<uint8_t>(_value) {}
       |                                                   ^
../../deps/simdjson/simdjson.cpp:20364:59: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
 20364 |     simdjson_inline base8_numeric(const __m512i _value) : base8<T>(_value) {}
       |                                                           ^
../../deps/simdjson/simdjson.cpp:20310:51: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
 20310 |     simdjson_inline base8(const __m512i _value) : base<simd8<T>>(_value) {}
       |                                                   ^
../../deps/simdjson/simdjson.cpp:20535:52: error: always_inline function '_mm512_movepi8_mask' requires target feature 'evex512', but would be inlined into function 'is_ascii' that is compiled without support for 'evex512'
 20535 |     simdjson_inline bool is_ascii() const { return _mm512_movepi8_mask(*this) == 0; }
       |                                                    ^
../../deps/simdjson/simdjson.cpp:20535:52: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20291:71: error: always_inline function '_mm512_or_si512' requires target feature 'evex512', but would be inlined into function 'operator|' that is compiled without support for 'evex512'
 20291 |     simdjson_inline Child operator|(const Child other) const { return _mm512_or_si512(*this, other); }
       |                                                                       ^
../../deps/simdjson/simdjson.cpp:20291:71: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20291:71: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20322:14: error: '__builtin_ia32_palignr512' needs target feature avx512bw,evex512
 20322 |       return _mm512_alignr_epi8(*this, _mm512_permutex2var_epi64(prev_chunk, _mm512_set_epi64(13, 12, 11, 10, 9, 8, 7, 6), *this), shift);
       |              ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:1977:13: note: expanded from macro '_mm512_alignr_epi8'
 1977 |   ((__m512i)__builtin_ia32_palignr512((__v64qi)(__m512i)(A), \
      |             ^
../../deps/simdjson/simdjson.cpp:20322:78: error: always_inline function '_mm512_set_epi64' requires target feature 'evex512', but would be inlined into function 'prev' that is compiled without support for 'evex512'
 20322 |       return _mm512_alignr_epi8(*this, _mm512_permutex2var_epi64(prev_chunk, _mm512_set_epi64(13, 12, 11, 10, 9, 8, 7, 6), *this), shift);
       |                                                                              ^
../../deps/simdjson/simdjson.cpp:20322:78: error: AVX vector return of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20322:40: error: always_inline function '_mm512_permutex2var_epi64' requires target feature 'evex512', but would be inlined into function 'prev' that is compiled without support for 'evex512'
 20322 |       return _mm512_alignr_epi8(*this, _mm512_permutex2var_epi64(prev_chunk, _mm512_set_epi64(13, 12, 11, 10, 9, 8, 7, 6), *this), shift);
       |                                        ^
../../deps/simdjson/simdjson.cpp:20322:40: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
../../deps/simdjson/simdjson.cpp:20322:14: error: AVX vector argument of type 'const __m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
 20322 |       return _mm512_alignr_epi8(*this, _mm512_permutex2var_epi64(prev_chunk, _mm512_set_epi64(13, 12, 11, 10, 9, 8, 7, 6), *this), shift);
       |              ^
/usr/lib/llvm/18/bin/../../../../lib/clang/18/include/avx512bwintrin.h:1977:3: note: expanded from macro '_mm512_alignr_epi8'
 1977 |   ((__m512i)__builtin_ia32_palignr512((__v64qi)(__m512i)(A), \
      |   ^
../../deps/simdjson/simdjson.cpp:20543:72: error: always_inline function '_mm512_srli_epi16' requires target feature 'evex512', but would be inlined into function 'shr' that is compiled without support for 'evex512'
 20543 |     simdjson_inline simd8<uint8_t> shr() const { return simd8<uint8_t>(_mm512_srli_epi16(*this, N)) & uint8_t(0xFFu >> N); }
       |                                                                        ^
../../deps/simdjson/simdjson.cpp:20543:72: error: AVX vector argument of type '__m512i' (vector of 8 'long long' values) without 'evex512' enabled changes the ABI
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior? Why is that the expected behavior?

No response

What do you see instead?

Compile failure

Additional information

No response

The text was updated successfully, but these errors were encountered:

Uzlopak · 2024-05-07T14:51:27Z

ping
@lemire
@anonrig

FireBurn · 2024-05-07T14:53:32Z

Gentoo bug https://bugs.gentoo.org/931267

lemire · 2024-05-07T14:57:46Z

This didn't happen with node 20.12.1 which used the older simd

Can you elaborate?

thesamesam · 2024-05-07T14:58:53Z

We've had a few people hit this. I can hit something similar with -march=native on znver2 (3950x) with GCC 14 (https://bugs.gentoo.org/931150).

EDIT: @lemire, would you prefer it if I file a separate bug for the GCC 14 issue? It's probably related but it might well be distinct.

FireBurn · 2024-05-07T15:01:09Z

Hmm seems it is happening with nodejs 20.12.1 too, I wonder if something has happened in a clang-18.1.x update

FireBurn · 2024-05-07T15:02:17Z

I bet it's ninja 1.12.0

lemire · 2024-05-07T15:30:17Z

it is happening with nodejs 20.12.1 too

We had an issue that would be triggered by GCC 14... (e.g., see simdutf fix), but this was fixed (for simdutf) in 5.2.3. Node 22.1.0 has recent versions of simdutf and simdjson that are not affected by such issues.

There is currently no known build issues reported directly against simdjson and simdutf (which share a similar runtime dispatching).

I find it suspicious that there would be an issue affecting both GCC and LLVM at the same time... and going back to previous Node versions.

thesamesam · 2024-05-07T15:32:25Z

I'll file my own bug (on the simdjson repo, or here?) with reproduction instructions once I have them for GCC 14, as I can still hit it with the fixes for that bug you linked.

lemire · 2024-05-07T15:34:11Z

I'll file my own bug (on the simdjson repo, or here?) with reproduction instructions once I have them for GCC 14, as I can still hit it with the fixes for that bug you linked.

Yeah. If you have reproduction instructions, then file the bug with the libraries, we will fix it there. That would be best. It is easy afterward to bump automagically the fix to Node.

Cheers!!!!

FireBurn · 2024-05-07T15:49:15Z

gcc version 13.2.1 20240503 (Gentoo 13.2.1_p20240503 p15) works with Nodejs 20.12.1 & 21.1.0
Clang 18.1.5 doesn't build 20.12.1 or 21.1.0
Clang 18.1.3 successfully built 20.12.1 back on 16/04/24

from my logs I can see:

1712284321: ::: completed emerge (6 of 12) sys-devel/clang-18.1.3 to /
1713263296: ::: completed emerge (2 of 5) net-libs/nodejs-20.12.1 to /
1713485952: ::: completed emerge (6 of 15) sys-devel/clang-18.1.4 to /
1715029866: ::: completed emerge (21 of 39) sys-devel/clang-18.1.5 to /

lemire · 2024-05-07T16:46:16Z

@thesamesam @FireBurn Can you verify that you are building the correct dependencies in gentoo? That is, Node 22.1 should use simdjson 3.8.0 (or better) and simdutf 5.2.4 (or better). That is what the current Node releases are built with.

lemire · 2024-05-07T16:50:35Z

If you want to build with GCC 14 (currently not used in Node.js releases), you need simdjson 3.9.1.

Version 3.9.1 is not yet available in Node but should be shortly.

lemire · 2024-05-07T16:51:55Z

I am currently installing gentoo (latest) and it seems to be using GCC 13.2.1.

thesamesam · 2024-05-07T16:54:53Z

@thesamesam @FireBurn Can you verify that you are building the correct dependencies in gentoo? That is, Node 22.1 should use simdjson 3.8.0 (or better) and simdutf 5.2.4 (or better). That is what the current Node releases are built with.

Yeah, we're not unbundling simdjson or simdutf, so we're using the versions in the nodejs-22.1 tarball.

If you want to build with GCC 14 (currently not used in Node.js releases), you need simdjson 3.9.1.
Version 3.9.1 is not yet available in Node but should be shortly.

I'll see if we can update within nodejs downstream for now. It's not critical, as users who are using GCC 14 are opting-in (see below), but it inhibits further testing with the new compiler.

I'm going to play with this now. If it's too invasive, I'll just tell people to wait.

I am currently installing gentoo (latest) and it seems to be using GCC 13.2.1.

Yeah, I committed GCC 14 earlier today, but it's "unkeyworded" (not exposed to users by default).

If you put <sys-devel/gcc-14.1.9999:14 ** in /etc/portage/package.accept_keywords, you should get GCC 14.

Thank you for your assistance!

lemire · 2024-05-07T23:31:14Z

For people reading this. GCC 14 introduced a breaking change (and possibly LLVM 18 followed suit though I am not sure) whereas the value of some macros can change within the same execution unit (something that did not happen before). This has caused breakage in some libraries.

Gentoo users caught the issue because they are using a bleeding edge compiler (GCC 14) to compile Node from source.

Bug https://bugs.gentoo.org/931150 was marked fixed : the patch is to update simdjson to 3.9.1 or better.

Bug https://bugs.gentoo.org/931267 remains unconfirmed, but it is reported as not happening with GCC 13.2.1. At a glance, it looks like a duplicate of https://bugs.gentoo.org/931150

It is likely that all that is needed to fix this is for Node.js to adopt the latest versions of simdjson which should happen automagically.

RedYetiDev added the build Issues and PRs related to build files or the CI. label May 7, 2024

thesamesam mentioned this issue May 7, 2024

Build failure with GCC 14 in nodejs with -O2 -march=znver2 simdjson/simdjson#2175

Closed

lemire changed the title ~~Compile failure in simdutf and simdjson~~ Compile failure in simdutf and simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) May 7, 2024

lemire changed the title ~~Compile failure in simdutf and simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0)~~ Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) May 7, 2024

fahrradflucht mentioned this issue May 13, 2024

deps: update simdjson to 3.9.2 #52947

Merged

lemire linked a pull request May 13, 2024 that will close this issue

deps: update simdjson to 3.9.2 #52947

Merged

nodejs-github-bot closed this as completed in #52947 May 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) #52876

Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) #52876

FireBurn commented May 7, 2024 •

edited by lemire

Uzlopak commented May 7, 2024

FireBurn commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024 •

edited

FireBurn commented May 7, 2024

FireBurn commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024

lemire commented May 7, 2024

FireBurn commented May 7, 2024 •

edited

lemire commented May 7, 2024

lemire commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024 •

edited

lemire commented May 7, 2024

Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) #52876

Compile failure in simdjson using GCC 14 or clang 18.1 (Node 20.12.1 - 22.1.0) #52876

Comments

FireBurn commented May 7, 2024 • edited by lemire

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Uzlopak commented May 7, 2024

FireBurn commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024 • edited

FireBurn commented May 7, 2024

FireBurn commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024

lemire commented May 7, 2024

FireBurn commented May 7, 2024 • edited

lemire commented May 7, 2024

lemire commented May 7, 2024

lemire commented May 7, 2024

thesamesam commented May 7, 2024 • edited

lemire commented May 7, 2024

FireBurn commented May 7, 2024 •

edited by lemire

thesamesam commented May 7, 2024 •

edited

FireBurn commented May 7, 2024 •

edited

thesamesam commented May 7, 2024 •

edited