[llvm-dev] Tool to help hunt down binary size regressions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Tool to help hunt down binary size regressions

Tingyuan LIANG via llvm-dev
Hi folks,

Sometimes I make a change to the compiler that causes a binary size regression and I need to track down where that regression came from. One relatively naive way of doing this is to take two build trees, one compiled with the old compiler and another compiled with the new compiler, and then compare the symbol sizes (st_size) in the final output files in each build tree. The problem with this is that the results can be very noisy, especially if your change is early in the compiler pipeline, since it can tend to affect heuristics used by the inliner to make inlining decisions.

What we really want to see is a size delta that ignores any changes that resulted from different inlining decisions having been made. In the past I've done this by passing -fno-inline when compiling both build trees, but this won't be entirely representative given how important inlining is to optimization. What I later realised was that we can use the relocation list of each section to help identify functions that ended up having the same inlining decisions apply to them -- if a symbol's relocations point to different symbols in the two build trees, it is likely that different inlining decisions were made.

I wrote a quick and dirty tool in Go that takes two build trees and compares the sizes of the symbols in any object files found recursively in the build trees, while ignoring any symbols whose relocations point to different symbols. It can be found here:

And it can be used like this:
$ benchcomp sym_reloc_sizes builddir1 builddir2 | sort -n -k 4

If this seems useful I can see if I can find some time to contribute a tool based on this idea to LLVM in some form.

Thanks,
--
-- 
Peter

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Tool to help hunt down binary size regressions

Tingyuan LIANG via llvm-dev
On Tue, 16 Apr 2019 at 22:11, Peter Collingbourne via llvm-dev
<[hidden email]> wrote:
>
> Hi folks,
>
> Sometimes I make a change to the compiler that causes a binary size regression and I need to track down where that regression came from. One relatively naive way of doing this is to take two build trees, one compiled with the old compiler and another compiled with the new compiler, and then compare the symbol sizes (st_size) in the final output files in each build tree. The problem with this is that the results can be very noisy, especially if your change is early in the compiler pipeline, since it can tend to affect heuristics used by the inliner to make inlining decisions.
>
> What we really want to see is a size delta that ignores any changes that resulted from different inlining decisions having been made. In the past I've done this by passing -fno-inline when compiling both build trees, but this won't be entirely representative given how important inlining is to optimization. What I later realised was that we can use the relocation list of each section to help identify functions that ended up having the same inlining decisions apply to them -- if a symbol's relocations point to different symbols in the two build trees, it is likely that different inlining decisions were made.
>
> I wrote a quick and dirty tool in Go that takes two build trees and compares the sizes of the symbols in any object files found recursively in the build trees, while ignoring any symbols whose relocations point to different symbols. It can be found here:
> https://github.com/pcc/llvm-project/blob/sym-reloc-sizes/llgo/utils/benchcomp/main.go

You might want to also take a look at the size-info tool from Vedant
Kumar (CCed) <http://lists.llvm.org/pipermail/llvm-dev/2018-September/126501.html>
<https://github.com/vedantk/llvm-project/tree/sizeinfo>

Vedant: do you plan to push size-info upstream?

Best,

Alex
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Tool to help hunt down binary size regressions

Tingyuan LIANG via llvm-dev
Hi,

On Apr 17, 2019, at 7:09 AM, Alex Bradbury <[hidden email]> wrote:

On Tue, 16 Apr 2019 at 22:11, Peter Collingbourne via llvm-dev
<[hidden email]> wrote:

Hi folks,

Sometimes I make a change to the compiler that causes a binary size regression and I need to track down where that regression came from. One relatively naive way of doing this is to take two build trees, one compiled with the old compiler and another compiled with the new compiler, and then compare the symbol sizes (st_size) in the final output files in each build tree. The problem with this is that the results can be very noisy, especially if your change is early in the compiler pipeline, since it can tend to affect heuristics used by the inliner to make inlining decisions.

What we really want to see is a size delta that ignores any changes that resulted from different inlining decisions having been made. In the past I've done this by passing -fno-inline when compiling both build trees, but this won't be entirely representative given how important inlining is to optimization. What I later realised was that we can use the relocation list of each section to help identify functions that ended up having the same inlining decisions apply to them -- if a symbol's relocations point to different symbols in the two build trees, it is likely that different inlining decisions were made.

I wrote a quick and dirty tool in Go that takes two build trees and compares the sizes of the symbols in any object files found recursively in the build trees, while ignoring any symbols whose relocations point to different symbols. It can be found here:
https://github.com/pcc/llvm-project/blob/sym-reloc-sizes/llgo/utils/benchcomp/main.go

You might want to also take a look at the size-info tool from Vedant
Kumar (CCed) <http://lists.llvm.org/pipermail/llvm-dev/2018-September/126501.html>
<https://github.com/vedantk/llvm-project/tree/sizeinfo>

This tool does display inlining trees, but doesn't have support for ignoring symbols with different inlining histories while diffing.


Vedant: do you plan to push size-info upstream?

I'd like to upstream a size analysis library derived from the prototype, but don't know when I'll have time to do this. If anyone's interested in grabbing the code and running with it, I'd be happy to give reviews!

best,
vedant


Best,

Alex


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev