[llvm-dev] How data is laid out in default.profraw when doing profiling?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] How data is laid out in default.profraw when doing profiling?

Chris Lattner via llvm-dev
Hi all,

  I'm now working on llvm-cov for a new target and have a problem here.
  Because of some reasons, users do not stub in the main function and  after running elf file, they cannot get a default profraw. Now they want to construct a default profraw manually but don't know how data is laid out in `default profraw` file.
  We found a struct ProfDataIOVec in InstrProfilingWriter.c in compiler-rt and followed this layout.
```
  ProfDataIOVec IOVec[] = {
      {&Header, sizeof(__llvm_profile_header), 1},
      {DataBegin, sizeof(__llvm_profile_data), DataSize},
      {CountersBegin, sizeof(uint64_t), CountersSize},
      {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
      {Zeroes, sizeof(uint8_t), Padding}};
```
  This helped us successfully passed some small cases(not know for sure whether the coverage result is true, but we can see coverage report). But when we added more cases, `llvm-cov show -arch=xxx ...` gave us an assertion when entering fuction ` loadFunctionRecord ` in CoverageMapping.cpp. The OrigFuncName is empty when inserting into FunctionNames. Seems the llvm-cov cannot find a function name. I think it's because we didn't construct a correct default.profraw. So how data is laid out in default.profraw when doing profiling? Any help will be appreciated.
  A small tip which may help. We are using coverage tool for a new target so when building compiler-rt, we use InstrProfilingPlatformOther.c

Best,
Ruobin
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] How data is laid out in default.profraw when doing profiling?

Chris Lattner via llvm-dev
I would recommend reusing the existing code for writing those files, if possible. If atexit() doesn't work in your environment, you can still invoke the routines explicitly. https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#using-the-profiling-runtime-without-static-initializers describes this.

If llvm-cov isn't working, the most likely cause is that there's something wrong with your ELF file.  Are you sure your ELF file matches the profile data, and has all the relevant __llvm_* sections?

-Eli

> -----Original Message-----
> From: llvm-dev <[hidden email]> On Behalf Of xuruobin via
> llvm-dev
> Sent: Wednesday, May 15, 2019 8:27 PM
> To: [hidden email]
> Cc: Yuchao (Michael) <[hidden email]>
> Subject: [EXT] [llvm-dev] How data is laid out in default.profraw when doing
> profiling?
>
> Hi all,
>
>   I'm now working on llvm-cov for a new target and have a problem here.
>   Because of some reasons, users do not stub in the main function and  after
> running elf file, they cannot get a default profraw. Now they want to construct a
> default profraw manually but don't know how data is laid out in `default
> profraw` file.
>   We found a struct ProfDataIOVec in InstrProfilingWriter.c in compiler-rt and
> followed this layout.
> ```
>   ProfDataIOVec IOVec[] = {
>       {&Header, sizeof(__llvm_profile_header), 1},
>       {DataBegin, sizeof(__llvm_profile_data), DataSize},
>       {CountersBegin, sizeof(uint64_t), CountersSize},
>       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
>       {Zeroes, sizeof(uint8_t), Padding}};
> ```
>   This helped us successfully passed some small cases(not know for sure whether
> the coverage result is true, but we can see coverage report). But when we added
> more cases, `llvm-cov show -arch=xxx ...` gave us an assertion when entering
> fuction ` loadFunctionRecord ` in CoverageMapping.cpp. The OrigFuncName is
> empty when inserting into FunctionNames. Seems the llvm-cov cannot find a
> function name. I think it's because we didn't construct a correct default.profraw.
> So how data is laid out in default.profraw when doing profiling? Any help will be
> appreciated.
>   A small tip which may help. We are using coverage tool for a new target so
> when building compiler-rt, we use InstrProfilingPlatformOther.c
>
> Best,
> Ruobin
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] How data is laid out in default.profraw when doing profiling?

Chris Lattner via llvm-dev
Hi Eli,

    For the first problem, I will try reuse the existing code.
    For the second problem that my llvm-cov isn't working, I compared elf file with original one(no -fprofile-instr-generate  -fcoverage-mapping flag). 5 new sections were introduced. __llvm_prf_names, __llvm_prf_cnts, __llvm_prf_data, __llvm_prf_vnds and __llvm_covmap. Does the sequence of 5 sections matter or are there more sections I need to use?

Best,
Ruobin

> -----Original Message-----
> From: Eli Friedman [mailto:[hidden email]]
> Sent: Friday, May 17, 2019 2:44 AM
> To: xuruobin <[hidden email]>; llvm-dev <[hidden email]>
> Cc: Yuchao (Michael) <[hidden email]>
> Subject: RE: [llvm-dev] How data is laid out in default.profraw when doing
> profiling?
>
> I would recommend reusing the existing code for writing those files, if
> possible. If atexit() doesn't work in your environment, you can still invoke the
> routines explicitly.
> https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#using-the-
> profiling-runtime-without-static-initializers describes this.
>
> If llvm-cov isn't working, the most likely cause is that there's something
> wrong with your ELF file.  Are you sure your ELF file matches the profile data,
> and has all the relevant __llvm_* sections?
>
> -Eli
>
> > -----Original Message-----
> > From: llvm-dev <[hidden email]> On Behalf Of xuruobin
> > via llvm-dev
> > Sent: Wednesday, May 15, 2019 8:27 PM
> > To: [hidden email]
> > Cc: Yuchao (Michael) <[hidden email]>
> > Subject: [EXT] [llvm-dev] How data is laid out in default.profraw when
> > doing profiling?
> >
> > Hi all,
> >
> >   I'm now working on llvm-cov for a new target and have a problem here.
> >   Because of some reasons, users do not stub in the main function and
> > after running elf file, they cannot get a default profraw. Now they
> > want to construct a default profraw manually but don't know how data
> > is laid out in `default profraw` file.
> >   We found a struct ProfDataIOVec in InstrProfilingWriter.c in
> > compiler-rt and followed this layout.
> > ```
> >   ProfDataIOVec IOVec[] = {
> >       {&Header, sizeof(__llvm_profile_header), 1},
> >       {DataBegin, sizeof(__llvm_profile_data), DataSize},
> >       {CountersBegin, sizeof(uint64_t), CountersSize},
> >       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
> >       {Zeroes, sizeof(uint8_t), Padding}}; ```
> >   This helped us successfully passed some small cases(not know for
> > sure whether the coverage result is true, but we can see coverage
> > report). But when we added more cases, `llvm-cov show -arch=xxx ...`
> > gave us an assertion when entering fuction ` loadFunctionRecord ` in
> > CoverageMapping.cpp. The OrigFuncName is empty when inserting into
> > FunctionNames. Seems the llvm-cov cannot find a function name. I think
> it's because we didn't construct a correct default.profraw.
> > So how data is laid out in default.profraw when doing profiling? Any
> > help will be appreciated.
> >   A small tip which may help. We are using coverage tool for a new
> > target so when building compiler-rt, we use
> > InstrProfilingPlatformOther.c
> >
> > Best,
> > Ruobin
> > _______________________________________________
> > LLVM Developers mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] How data is laid out in default.profraw when doing profiling?

Chris Lattner via llvm-dev
That looks like all the sections.  If you're using --gc-sections or something like that, you might want to make sure the linker is including sections from all of the source .o files.  If there's some sort of mismatch, that can cause llvm-cov to fail.  Or if you're using a linker script, make sure you aren't accidentally inserting padding somewhere in one of those sections.

It might be simpler to compile only one .o file with "-fprofile-instr-generate  -fcoverage-mapping" to start with, to make it more obvious what data is supposed to be in the final executable.

-Eli

> -----Original Message-----
> From: xuruobin <[hidden email]>
> Sent: Friday, May 17, 2019 2:08 AM
> To: Eli Friedman <[hidden email]>; llvm-dev <[hidden email]>
> Cc: Yuchao (Michael) <[hidden email]>
> Subject: [EXT] RE: [llvm-dev] How data is laid out in default.profraw when doing
> profiling?
>
> Hi Eli,
>
>     For the first problem, I will try reuse the existing code.
>     For the second problem that my llvm-cov isn't working, I compared elf file
> with original one(no -fprofile-instr-generate  -fcoverage-mapping flag). 5 new
> sections were introduced. __llvm_prf_names, __llvm_prf_cnts, __llvm_prf_data,
> __llvm_prf_vnds and __llvm_covmap. Does the sequence of 5 sections matter or
> are there more sections I need to use?
>
> Best,
> Ruobin
>
> > -----Original Message-----
> > From: Eli Friedman [mailto:[hidden email]]
> > Sent: Friday, May 17, 2019 2:44 AM
> > To: xuruobin <[hidden email]>; llvm-dev <[hidden email]>
> > Cc: Yuchao (Michael) <[hidden email]>
> > Subject: RE: [llvm-dev] How data is laid out in default.profraw when doing
> > profiling?
> >
> > I would recommend reusing the existing code for writing those files, if
> > possible. If atexit() doesn't work in your environment, you can still invoke the
> > routines explicitly.
> > https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#using-the-
> > profiling-runtime-without-static-initializers describes this.
> >
> > If llvm-cov isn't working, the most likely cause is that there's something
> > wrong with your ELF file.  Are you sure your ELF file matches the profile data,
> > and has all the relevant __llvm_* sections?
> >
> > -Eli
> >
> > > -----Original Message-----
> > > From: llvm-dev <[hidden email]> On Behalf Of xuruobin
> > > via llvm-dev
> > > Sent: Wednesday, May 15, 2019 8:27 PM
> > > To: [hidden email]
> > > Cc: Yuchao (Michael) <[hidden email]>
> > > Subject: [EXT] [llvm-dev] How data is laid out in default.profraw when
> > > doing profiling?
> > >
> > > Hi all,
> > >
> > >   I'm now working on llvm-cov for a new target and have a problem here.
> > >   Because of some reasons, users do not stub in the main function and
> > > after running elf file, they cannot get a default profraw. Now they
> > > want to construct a default profraw manually but don't know how data
> > > is laid out in `default profraw` file.
> > >   We found a struct ProfDataIOVec in InstrProfilingWriter.c in
> > > compiler-rt and followed this layout.
> > > ```
> > >   ProfDataIOVec IOVec[] = {
> > >       {&Header, sizeof(__llvm_profile_header), 1},
> > >       {DataBegin, sizeof(__llvm_profile_data), DataSize},
> > >       {CountersBegin, sizeof(uint64_t), CountersSize},
> > >       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
> > >       {Zeroes, sizeof(uint8_t), Padding}}; ```
> > >   This helped us successfully passed some small cases(not know for
> > > sure whether the coverage result is true, but we can see coverage
> > > report). But when we added more cases, `llvm-cov show -arch=xxx ...`
> > > gave us an assertion when entering fuction ` loadFunctionRecord ` in
> > > CoverageMapping.cpp. The OrigFuncName is empty when inserting into
> > > FunctionNames. Seems the llvm-cov cannot find a function name. I think
> > it's because we didn't construct a correct default.profraw.
> > > So how data is laid out in default.profraw when doing profiling? Any
> > > help will be appreciated.
> > >   A small tip which may help. We are using coverage tool for a new
> > > target so when building compiler-rt, we use
> > > InstrProfilingPlatformOther.c
> > >
> > > Best,
> > > Ruobin
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > [hidden email]
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev