Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

high memory requirement during monomorphization #125212

Open
matthiaskrgr opened this issue May 17, 2024 · 1 comment
Open

high memory requirement during monomorphization #125212

matthiaskrgr opened this issue May 17, 2024 · 1 comment
Labels
A-monomorphization Area: Monomorphization I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@matthiaskrgr
Copy link
Member

matthiaskrgr commented May 17, 2024

I noticed this while fuzzing because rustc would crash after hitting the memory limit inside the sandbox
We need around 1.5 gb of memory during monomorphization here it seems.
Next solver does not help.

I tried this code:
rustc file.rs -Ztime-passes -Clink-dead-code

struct Foo {
    bar: dyn for<'r> Fn(usize, &'r dyn Fn(
            &dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(&dyn Fn(u32))))))))),
        ))
}

fn main() {
}

Meta

rustc --version --verbose:

rustc 1.80.0-nightly (8c127df75 2024-05-16)
binary: rustc
commit-hash: 8c127df75fde3d5ad8ef9af664962a7676288b52
commit-date: 2024-05-16
host: x86_64-unknown-linux-gnu
release: 1.80.0-nightly
LLVM version: 18.1.4
Backtrace

 rustc ice.rs -Ztime-passes -Clink-dead-code
time:   0.000; rss:   46MB ->   47MB (   +2MB)	parse_crate
time:   0.000; rss:   48MB ->   49MB (   +1MB)	setup_global_ctxt
time:   0.000; rss:   50MB ->   50MB (   +1MB)	crate_injection
time:   0.002; rss:   50MB ->   63MB (  +12MB)	expand_crate
time:   0.002; rss:   50MB ->   63MB (  +12MB)	macro_expand_crate
time:   0.000; rss:   63MB ->   63MB (   +0MB)	AST_validation
time:   0.000; rss:   63MB ->   63MB (   +0MB)	finalize_imports
time:   0.000; rss:   63MB ->   67MB (   +4MB)	finalize_macro_resolutions
time:   0.000; rss:   67MB ->   67MB (   +0MB)	late_resolve_crate
time:   0.000; rss:   67MB ->   67MB (   +0MB)	resolve_check_unused
time:   0.000; rss:   67MB ->   67MB (   +0MB)	resolve_postprocess
time:   0.001; rss:   63MB ->   67MB (   +5MB)	resolve_crate
time:   0.000; rss:   68MB ->   68MB (   +0MB)	complete_gated_feature_checking
time:   0.000; rss:   68MB ->   69MB (   +2MB)	looking_for_entry_point
time:   0.000; rss:   70MB ->   71MB (   +0MB)	unused_lib_feature_checking
time:   0.000; rss:   68MB ->   71MB (   +3MB)	misc_checking_1
time:   0.001; rss:   71MB ->   78MB (   +7MB)	coherence_checking
time:   0.001; rss:   71MB ->   79MB (   +8MB)	type_check_crate
time:   0.000; rss:   79MB ->   81MB (   +2MB)	MIR_borrow_checking
time:   0.000; rss:   81MB ->   82MB (   +1MB)	MIR_effect_checking
warning: struct `Foo` is never constructed
 --> ice.rs:1:8
  |
1 | struct Foo {
  |        ^^^
  |
  = note: `#[warn(dead_code)]` on by default

time:   0.000; rss:   86MB ->   86MB (   +0MB)	module_lints
time:   0.000; rss:   86MB ->   86MB (   +0MB)	lint_checking
time:   0.002; rss:   82MB ->   86MB (   +4MB)	misc_checking_3
time:   0.000; rss:   86MB ->   87MB (   +1MB)	monomorphization_collector_root_collections
time:   5.850; rss:   87MB -> 1537MB (+1450MB)	monomorphization_collector_graph_walk
time:   0.000; rss: 1537MB -> 1538MB (   +1MB)	partition_and_assert_distinct_symbols
time:   0.000; rss: 1538MB -> 1541MB (   +3MB)	write_allocator_module
time:   0.001; rss: 1547MB -> 1557MB (   +9MB)	codegen_to_LLVM_IR
time:   5.853; rss:   86MB -> 1557MB (+1471MB)	codegen_crate
time:   0.000; rss: 1557MB -> 1557MB (   +0MB)	serialize_dep_graph
time:   0.023; rss: 1557MB -> 1279MB ( -277MB)	LLVM_passes
time:   0.078; rss: 1557MB ->  209MB (-1349MB)	free_global_ctxt
time:   0.051; rss:  209MB ->  209MB (   +0MB)	run_linker
time:   0.052; rss:  209MB ->  209MB (   +0MB)	link_binary
time:   0.052; rss:  209MB ->  209MB (   +0MB)	link_crate
time:   0.052; rss:  209MB ->  209MB (   +0MB)	link
warning: 1 warning emitted

time:   5.999; rss:   32MB ->  102MB (  +70MB)	total

@matthiaskrgr matthiaskrgr added the I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. label May 17, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label May 17, 2024
@fmease fmease added A-monomorphization Area: Monomorphization T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels May 17, 2024
@Nilstrieb
Copy link
Member

Nilstrieb commented May 18, 2024

I profiled this with massif: 300MB appear to be coming from interned types, over 400MB of region constraints
The file is attached here, check it out with massif-visualizer: msout.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-monomorphization Area: Monomorphization I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

4 participants