[llvm-dev] Debugify and Verify-each mode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline. Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable! Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev

On Mar 16, 2018, at 2:30 PM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

No, there's no reserved work. If you'd like to work on this I encourage you to do so. There's plenty of other work slated for the GSoC project. That said, let's make sure to sync up on the mailing lists to make sure work isn't being duplicated.


However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

The intermediate textual output is all irrelevant. And clang isn't in the picture here. Opt's regular mode of operation is to run pass1, then run pass2, etc. In the debugify-each mode, this instead looks like: debugify, pass1, check-debugify, strip debug info, debugify, pass2, etc. etc.

vedant



Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu




_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Mhm I see now, thanks for your explanation! 

Son Tuan Vu

On Fri, Mar 16, 2018 at 10:58 PM, Vedant Kumar <[hidden email]> wrote:

On Mar 16, 2018, at 2:30 PM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

No, there's no reserved work. If you'd like to work on this I encourage you to do so. There's plenty of other work slated for the GSoC project. That said, let's make sure to sync up on the mailing lists to make sure work isn't being duplicated.


However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

The intermediate textual output is all irrelevant. And clang isn't in the picture here. Opt's regular mode of operation is to run pass1, then run pass2, etc. In the debugify-each mode, this instead looks like: debugify, pass1, check-debugify, strip debug info, debugify, pass2, etc. etc.

vedant



Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu





_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hi Vedant,

I have tried to implement the fix you proposed, but it didn't work as expected. I created a new Module Pass Manager (not Function Pass Manager) and override the add() method like this:

class DebugifyEachPassManager : public legacy::PassManager {
 public:
   void add(Pass *P) override {
     PassManager::add(createDebugifyPass());
     PassManager::add(P);
     PassManager::add(createCheckDebugifyPass());
   }    
        
   bool run(Module &M) {
     return PassManager::run(M);
   }            
};     

However, we can't apply and check debugify of FunctionPasses since these passes are managed by Function Pass Managers created on the fly during schedulePass(). I guess we'd have to implement the FunctionPass version of Debugify and CheckDebugify. But what about Call Graph SCC Pass and Loop Pass?

Do we have a way to know whether a pass is an analysis or a transformation at the moment of adding the pass to the manager? I want to try to debugify only transformation passes because analysis passes do not modify the IR, hence do not modify the DI. But I'm not sure if there's a way to achieve this.

I'm currently trying to strip debug info after each CheckDebugify. I have been searching everywhere but can't seem to find an API to remove MDNodes. I have seen a MDNode::deleteAsSubclass() method, unfortunately it is private.I guess nobody has ever needed to remove MDNodes/Metadata before. Can you give me some advice on how to do this please?

Thank you for your help


Son Tuan Vu

On Fri, Mar 16, 2018 at 11:54 PM, Son Tuan VU <[hidden email]> wrote:
Mhm I see now, thanks for your explanation! 

Son Tuan Vu

On Fri, Mar 16, 2018 at 10:58 PM, Vedant Kumar <[hidden email]> wrote:

On Mar 16, 2018, at 2:30 PM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

No, there's no reserved work. If you'd like to work on this I encourage you to do so. There's plenty of other work slated for the GSoC project. That said, let's make sure to sync up on the mailing lists to make sure work isn't being duplicated.


However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

The intermediate textual output is all irrelevant. And clang isn't in the picture here. Opt's regular mode of operation is to run pass1, then run pass2, etc. In the debugify-each mode, this instead looks like: debugify, pass1, check-debugify, strip debug info, debugify, pass2, etc. etc.

vedant



Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu






_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hello,

On Apr 26, 2018, at 6:44 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

I have tried to implement the fix you proposed, but it didn't work as expected. I created a new Module Pass Manager (not Function Pass Manager) and override the add() method like this:

class DebugifyEachPassManager : public legacy::PassManager {
 public:
   void add(Pass *P) override {
     PassManager::add(createDebugifyPass());
     PassManager::add(P);
     PassManager::add(createCheckDebugifyPass());
   }    
        
   bool run(Module &M) {
     return PassManager::run(M);
   }            
};     

However, we can't apply and check debugify of FunctionPasses since these passes are managed by Function Pass Managers created on the fly during schedulePass(). I guess we'd have to implement the FunctionPass version of Debugify and CheckDebugify.

Yes, Anastasis (CC'd) has started exploring this idea: https://gramanas.github.io/posts/community-bonding/. Please sync up with him before you start hacking, so you can collaborate on the design and split up work :). Now that GSoC has officially started, Anastasis needs to plan work ahead of time, so we need to find a way to avoid duplicating work.

If you find that it's difficult to split up work on this, consider working on enhancing opt's -verify-each mode so that it runs after each pass in -O1/-O2/etc. This would help address use-before-def scenarios with debug info intrinsics. There's an ongoing discussion about this here: https://reviews.llvm.org/D46100.


But what about Call Graph SCC Pass and Loop Pass?

Great question! It might not be worthwhile to shoehorn what debugify does onto cgscc passes. The major cgscc pass I know of is the inliner, and its handling of debug info can be tested in a more targeted way. I think it would be worth adding a debugify wrapper for loop passes, however.


Do we have a way to know whether a pass is an analysis or a transformation at the moment of adding the pass to the manager? I want to try to debugify only transformation passes because analysis passes do not modify the IR, hence do not modify the DI. But I'm not sure if there's a way to achieve this.

I'm not sure if there's a way to do this, but I see it as an optimization that can be deferred for now.


I'm currently trying to strip debug info after each CheckDebugify.

There's a function called StripDebugInfo which does this.

vedant


I have been searching everywhere but can't seem to find an API to remove MDNodes. I have seen a MDNode::deleteAsSubclass() method, unfortunately it is private.I guess nobody has ever needed to remove MDNodes/Metadata before. Can you give me some advice on how to do this please?

Thank you for your help


Son Tuan Vu

On Fri, Mar 16, 2018 at 11:54 PM, Son Tuan VU <[hidden email]> wrote:
Mhm I see now, thanks for your explanation! 

Son Tuan Vu

On Fri, Mar 16, 2018 at 10:58 PM, Vedant Kumar <[hidden email]> wrote:

On Mar 16, 2018, at 2:30 PM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

No, there's no reserved work. If you'd like to work on this I encourage you to do so. There's plenty of other work slated for the GSoC project. That said, let's make sure to sync up on the mailing lists to make sure work isn't being duplicated.


However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

The intermediate textual output is all irrelevant. And clang isn't in the picture here. Opt's regular mode of operation is to run pass1, then run pass2, etc. In the debugify-each mode, this instead looks like: debugify, pass1, check-debugify, strip debug info, debugify, pass2, etc. etc.

vedant



Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu







_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
Hello,

@Anastasis: what do you think about this? Do you mind if I implement the debugify-each mode and refactor the code to have DebugifyFunctionPass and DebugifyLoopPass? I am sorry if it bothers you, I should have done this earlier, so you could focus on fixing the optimization passes. FYI, my debugify-each is quite ready, I am having Debugify and CheckDebugify before and after each ModulePass, then I just need to implement the FunctionPass and LoopPass.


On Thu, Apr 26, 2018 at 8:53 PM, Vedant Kumar <[hidden email]> wrote:
Hello,

On Apr 26, 2018, at 6:44 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

I have tried to implement the fix you proposed, but it didn't work as expected. I created a new Module Pass Manager (not Function Pass Manager) and override the add() method like this:

class DebugifyEachPassManager : public legacy::PassManager {
 public:
   void add(Pass *P) override {
     PassManager::add(createDebugifyPass());
     PassManager::add(P);
     PassManager::add(createCheckDebugifyPass());
   }    
        
   bool run(Module &M) {
     return PassManager::run(M);
   }            
};     

However, we can't apply and check debugify of FunctionPasses since these passes are managed by Function Pass Managers created on the fly during schedulePass(). I guess we'd have to implement the FunctionPass version of Debugify and CheckDebugify.

Yes, Anastasis (CC'd) has started exploring this idea: https://gramanas.github.io/posts/community-bonding/. Please sync up with him before you start hacking, so you can collaborate on the design and split up work :). Now that GSoC has officially started, Anastasis needs to plan work ahead of time, so we need to find a way to avoid duplicating work.

If you find that it's difficult to split up work on this, consider working on enhancing opt's -verify-each mode so that it runs after each pass in -O1/-O2/etc. This would help address use-before-def scenarios with debug info intrinsics. There's an ongoing discussion about this here: https://reviews.llvm.org/D46100.


Yes I am still waiting for Chandler's thoughts on verify-each mode of LegacyPassManager (because it works if we use the new PassManager) and on what would be the best way to tackle it.
But what about Call Graph SCC Pass and Loop Pass?

Great question! It might not be worthwhile to shoehorn what debugify does onto cgscc passes. The major cgscc pass I know of is the inliner, and its handling of debug info can be tested in a more targeted way. I think it would be worth adding a debugify wrapper for loop passes, however.


Do we have a way to know whether a pass is an analysis or a transformation at the moment of adding the pass to the manager? I want to try to debugify only transformation passes because analysis passes do not modify the IR, hence do not modify the DI. But I'm not sure if there's a way to achieve this.

I'm not sure if there's a way to do this, but I see it as an optimization that can be deferred for now.


I'm currently trying to strip debug info after each CheckDebugify.

There's a function called StripDebugInfo which does this.

Wow thanks, I was reimplementing that function.... 

vedant


I have been searching everywhere but can't seem to find an API to remove MDNodes. I have seen a MDNode::deleteAsSubclass() method, unfortunately it is private.I guess nobody has ever needed to remove MDNodes/Metadata before. Can you give me some advice on how to do this please?

Thank you for your help


Son Tuan Vu

On Fri, Mar 16, 2018 at 11:54 PM, Son Tuan VU <[hidden email]> wrote:
Mhm I see now, thanks for your explanation! 

Son Tuan Vu

On Fri, Mar 16, 2018 at 10:58 PM, Vedant Kumar <[hidden email]> wrote:

On Mar 16, 2018, at 2:30 PM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant,

Thank you for your reply. I think I can make this debugify-each mode, but I guess this is reserved for your GSoC project ?

No, there's no reserved work. If you'd like to work on this I encourage you to do so. There's plenty of other work slated for the GSoC project. That said, let's make sure to sync up on the mailing lists to make sure work isn't being duplicated.


However, if I understand correctly, we do not want to take the output of the first check-debugify (I mean the .ll file with potentially all the WARNINGs and ERRORs after the first pass) as input for the second debugify. What we need is to take the fresh output of clang -Xclang -emit-llvm -disable-O0-optnone -S and iteratively test each optimization. Am I right?

The intermediate textual output is all irrelevant. And clang isn't in the picture here. Opt's regular mode of operation is to run pass1, then run pass2, etc. In the debugify-each mode, this instead looks like: debugify, pass1, check-debugify, strip debug info, debugify, pass2, etc. etc.

vedant



Cheers,


Son Tuan Vu

On Thu, Mar 15, 2018 at 4:05 AM, Vedant Kumar <[hidden email]> wrote:
Hi Son Tuan,

Thanks for taking a look at this :). Responses inline --

On Mar 14, 2018, at 8:11 AM, Son Tuan VU <[hidden email]> wrote:

Hi Vedant, hi all,

My goal is to measure debug info loss of each optimization pass in LLVM. I am trying to create a debugify-each mode in opt, inspired by verify-each mode which is supposed to already work.

+ Anastasis, who's interested in working on this as well. There's definitely enough work to go around: once we can measure debug info loss after each pass, we'll need a testing harness.


However, if I understand correctly, the verify-each mode (triggered by -verify-each option in opt) only works when we provide a pass list or a pass pipeline.

Yes, you're correct.


Is this intended? I mean, why do not let people verify each pass in -O{1,2,3} pipeline?

That's a good question! Like you, I assumed -verify-each "does the right thing" when you pass -O1/-O2/etc. to opt.

I'm not sure if the current behavior is intended (hopefully others will chime in about this :). If no one does, please file a bug.

I imagine this is pretty simple to fix. You can just define and use custom pass managers within opt which inject debugify passes as needed:

// In opt.cpp:

class DebugifyEachFunctionPassManager : public legacy::FunctionPassManager {
public:
  explicit DebugifyEachFunctionPassManager(Module *M)
      : FunctionPassManager(M) {}

  void add(Pass *P) override {
    // FunctionPassManager::add(<debugify>)
    FunctionPassManager::add(P);
  }
};


My second question is more about debugify: what should be the best way to debugify each pass? Adding a debugify-each mode would make the output unreadable!

The intermediate output is all irrelevant. I think it'd be best to simply throw it away. What really matters are the debug info loss statistics: we should capture these stats after each pass and dump them as JSON, at the end of the pipeline.

vedant


Maybe writing a script that collects all optimization options (like -mem2reg or -constmerge), then pass each one of them to opt with -enable-debugify so that we have 1 output file for each debugified pass?

Thank you for your help,

Son Tuan Vu








_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Debugify and Verify-each mode

Adam Nemet via llvm-dev
On 180427 17:13:40, Son Tuan VU wrote:
> Hello,
>
> @Anastasis: what do you think about this? Do you mind if I implement the
> debugify-each mode and refactor the code to have DebugifyFunctionPass and
> DebugifyLoopPass? I am sorry if it bothers you, I should have done this
> earlier, so you could focus on fixing the optimization passes.

Sure go ahead, I don't mind at all.

> FYI, my debugify-each is quite ready, I am having Debugify and CheckDebugify before and
> after each ModulePass, then I just need to implement the FunctionPass and
> LoopPass.

Add me as a reviewer when you post the patch, I'd be glad to help.

Anastasis
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev