Questions about LLVM PGO and autoFDO

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about LLVM PGO and autoFDO

猫妖
Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 
  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:
  clang -O2 -fprofile-instr-generate test.c -o a.out
  ./a.out     (then default.profraw is generated)
  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out
  error: pgo data file has malformed function entry
  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 
  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:
  CurPtr = strchr(CurPtr, '\n');
  if (!CurPtr) {
    ReportBadPGOData(CGM, "pgo data file has malformed function entry");
    return;
  }
  
  2. Problems in autoFDO: 
  Actually the problem happened in using create_llvm_prof, transformation is failed.
  clang -O2 -g test.c -o a.out
  perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not recognized, why?)
  change to: perf record ./a.out   (so perf.data is generated)
  create_llvm_prof --binary=./a.out --out=code.prof  (it read a.out and perf.data)
  error: Malformed .note.gnu.build-id section
           no build id found in the binary
  it seems that a.out has no build id.

  Someone who can help me?

  Thanks,
  Kaidong Chen


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Questions about LLVM PGO and autoFDO

Diego Novillo-3
On Mon, May 12, 2014 at 12:59 PM, 猫妖 <[hidden email]> wrote:

>   2. Problems in autoFDO:
>   Actually the problem happened in using create_llvm_prof, transformation is
> failed.
>   clang -O2 -g test.c -o a.out
>   perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not
> recognized, why?)
>   change to: perf record ./a.out   (so perf.data is generated)
>   create_llvm_prof --binary=./a.out --out=code.prof  (it read a.out and
> perf.data)
>   error: Malformed .note.gnu.build-id section
>            no build id found in the binary
>   it seems that a.out has no build id.

This error message from create_llvm_prof is actually harmless. Dehao
will be removing it at some point. The file code.prof should have
content and clang will be able to read it. Please try:

$ clang -fprofile-sample-use=code.prof -g ...


Diego.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Questions about LLVM PGO and autoFDO

Justin Bogner-2
In reply to this post by 猫妖
On Monday, May 12, 2014, 猫妖 <[hidden email]> wrote:
Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 
  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:
  clang -O2 -fprofile-instr-generate test.c -o a.out
  ./a.out     (then default.profraw is generated)
  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out
  error: pgo data file has malformed function entry
  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 
  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:
  CurPtr = strchr(CurPtr, '\n');
  if (!CurPtr) {
    ReportBadPGOData(CGM, "pgo data file has malformed function entry");
    return;
  }

Sorry that this isn't documented very well. I'm working on some documentation for the instrumentation based profiling that should make this less of a guessing game.

You need to run `llvm-profdata merge` on the raw profile data to generate the format that clang understands. Something like:

  llvm-profdata merge default.profraw -o default.profdata

Then use default.profdata as the input to -fprofile-instr-use.
 
  2. Problems in autoFDO: 
  Actually the problem happened in using create_llvm_prof, transformation is failed.
  clang -O2 -g test.c -o a.out
  perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not recognized, why?)
  change to: perf record ./a.out   (so perf.data is generated)
  create_llvm_prof --binary=./a.out --out=code.prof  (it read a.out and perf.data)
  error: Malformed .note.gnu.build-id section
           no build id found in the binary
  it seems that a.out has no build id.

  Someone who can help me?

  Thanks,
  Kaidong Chen


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Questions about LLVM PGO and autoFDO

猫妖


On Monday, May 12, 2014, 猫妖 <[hidden email]> wrote:
Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 
  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:
  clang -O2 -fprofile-instr-generate test.c -o a.out
  ./a.out     (then default.profraw is generated)
  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out
  error: pgo data file has malformed function entry
  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 
  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:
  CurPtr = strchr(CurPtr, '\n');
  if (!CurPtr) {
    ReportBadPGOData(CGM, "pgo data file has malformed function entry");
    return;
  }

>>Sorry that this isn't documented very well. I'm working on some documentation for the instrumentation 
>>based profiling that should make this less of a guessing game.

>>You need to run `llvm-profdata merge` on the raw profile data to generate the format that clang 
>>understands. Something like:

>>  llvm-profdata merge default.profraw -o default.profdata

>>Then use default.profdata as the input to -fprofile-instr-use.

     It has worked well after I do as you say. Thank you!
     And I want to confirm what passes benefits from profdata-guided feedback currently.
     Is the profiledata feeded back to all of basicblock replacement, function inline, register allocation
     and loop-unroll, or only basicblock replacement ?
     
     I'll test performance improvement of PGO on some benchmarks.
     I think you already have some test results.

     Thanks,
     Kaidong Chen
 
  

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Questions about LLVM PGO and autoFDO

猫妖
In reply to this post by 猫妖


On Tue, May 13, 2014 at 1:15 PM, 猫妖 <[hidden email]> wrote:

   2. Problems in autoFDO:
  Actually the problem happened in using create_llvm_prof, transformation is
  failed.
  clang -O2 -g test.c -o a.out
  perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not recognized, why?)

>>Hm, now I noticed the perf version you are using. That's too old. You
>>will need perf and kernel newer than 3.4. I'm currently using
>>3.8.13.13


>>Diego.

  Yes, my perf is so old. It would bring errors like:Read wrong magic,(because
        "PERFFILE" has been changed to "PERFILE2") and others.

And I have updated my perf to about version 3.4, which already support "-b"
        option, but still convert failed.  (I didn't add "-b" option because my linux OS
        is version2.6, lower than 3.4. Meaning no Last Branch Records samples.UserGuide
        said LBR is not prerequisite. I don't know whether it is the reason for convert failure,
        or it is because of different perfdata format from different perf version. )

I will try the newest perf you recommended.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev