perf2bolt: huge RAM consumption

[zamazan4ik]

Hi!

I am trying to optimize YDB database (https://github.com/ydb-platform/ydb) with Bolt. I have run YDB binary (ydbd) with perf record, then I want to convert it to Bolt format with perf2bolt. Unfortunately, perf2bolt uses more than 30 Gib RAM and is OOM killed since I have only 32 Gib RAM.

Is there a way to reduce memory usage?

My setup:

• OS: Fedora 37
• Linux kernel: 6.2.7
• Compiler: clang-15 from Fedora packages (clang 15.0.7 (I've patched a few sources to support this compiler)
• Hardware: Ryzen 9 5900X, 32 Gib RAM, SSD
• Bolt version: built manually from the main branch with Clang 15.0.7 in Release mode

A related issue in YDB upstream: ydb-platform/ydb#140

[llvmbot]

@llvm/issue-subscribers-bolt

[maksfb]

Could you try —strict=0 option?

[zamazan4ik]

@maksfb yes, it reduces memory usage significantly. Didn't hear about this option before... Is it somehow related to the "lightweight" BOLT approach (I have read about it recently in the latest Propeller paper)?

[maksfb]

In a nutshell, yes. In some, relatively rare cases, --strict=1 allows to capture/optimize more functions, but the drawback is that it disables the lite mode.

[zamazan4ik]

@maksfb could you please mention this information somewhere in the BOLT' README file? Probably something like "if you want to reduce memory consumption during BOLT optimization - you could try to use -strict=0 option". Also, could be useful to put this info into llvm-bolt -help too.

It would users in the future quicker resolve possible issues. Thanks!

[ptr1337]

@zamazan4ik

How big is your perf.data?

I already had a discussion in the llvm discord about that issue and according them is not that much profile data (e.G 2GB or more) not required. This would also fix your issue without disabling strict and lite.

[zamazan4ik]

How big is your perf.data?

236 Mib. If you need, I can share this file - if it could help for debugging purposes.

[dustdfg]

I don't know hum much memory you will need but zram (on linux) is pretty good. With zstd and default settings it allows to compress ram in ratio 1:4. So 20GB of compressed memory will be stored in 5GB of real memory. It requires a bit of cpu when used though it is still faster than usual drive swap so it can be used even when you need additional 10GB of memory.