[llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
Hi,

I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.  

Specifically, below is what I did:

1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
7) Link against libraries and get the final binary: a.out.

The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?

Thanks!
Yuanfeng Peng
 

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
Is source level information present in generated IR files? 

On Wed, Jun 14, 2017 at 12:29 PM, Yuanfeng Peng via llvm-dev <[hidden email]> wrote:
Hi,

I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.  

Specifically, below is what I did:

1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
7) Link against libraries and get the final binary: a.out.

The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?

Thanks!
Yuanfeng Peng
 

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




--
Disclaimer: Views, concerns, thoughts, questions, ideas expressed in this mail are of my own and my employer has no take in it. 
Thank You.
Madhur D. Amilkanthwar


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
Yes, I've inspected all the intermediate IR files and source level information is present in them.

------------------ Original ------------------
From:  "Madhur Amilkanthwar";<[hidden email]>;
Send time: Wednesday, Jun 14, 2017 5:03 PM
To: "Yuanfeng Peng"<[hidden email]>;
Cc: "via llvm-dev"<[hidden email]>;
Subject:  Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Is source level information present in generated IR files? 

On Wed, Jun 14, 2017 at 12:29 PM, Yuanfeng Peng via llvm-dev <[hidden email]> wrote:
Hi,

I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.  

Specifically, below is what I did:

1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
7) Link against libraries and get the final binary: a.out.

The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?

Thanks!
Yuanfeng Peng
 

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




--
Disclaimer: Views, concerns, thoughts, questions, ideas expressed in this mail are of my own and my employer has no take in it. 
Thank You.
Madhur D. Amilkanthwar


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
Try dumping .out file using cuobjdump and see if it has .debug_info section.
If it does not have then PTX compilation skipped debug information generation.
If it does have then there is some other issue.


On Wed, Jun 14, 2017 at 4:35 PM, Yuanfeng Peng <[hidden email]> wrote:
Yes, I've inspected all the intermediate IR files and source level information is present in them.

------------------ Original ------------------
From:  "Madhur Amilkanthwar";<[hidden email]>;
Send time: Wednesday, Jun 14, 2017 5:03 PM
To: "Yuanfeng Peng"<[hidden email]>;
Cc: "via llvm-dev"<[hidden email]>;
Subject:  Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Is source level information present in generated IR files? 

On Wed, Jun 14, 2017 at 12:29 PM, Yuanfeng Peng via llvm-dev <[hidden email]> wrote:
Hi,

I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.  

Specifically, below is what I did:

1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
7) Link against libraries and get the final binary: a.out.

The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?

Thanks!
Yuanfeng Peng
 

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




--
Disclaimer: Views, concerns, thoughts, questions, ideas expressed in this mail are of my own and my employer has no take in it. 
Thank You.
Madhur D. Amilkanthwar




--
Disclaimer: Views, concerns, thoughts, questions, ideas expressed in this mail are of my own and my employer has no take in it. 
Thank You.
Madhur D. Amilkanthwar


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
In reply to this post by Alex Bradbury via llvm-dev
On 06/14/2017 02:59 AM, Yuanfeng Peng via llvm-dev wrote:
> Hi,
>
> I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.  
>

The NVPTX backend in LLVM does not support DWARF emission.

-Tom

> Specifically, below is what I did:
>
> 1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
> 2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
> 3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
> 4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
> 5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
> 6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
> 7) Link against libraries and get the final binary: a.out.
>
> The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?
>
> Thanks!
> Yuanfeng Peng
>  
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
Hi Tom,

Thanks for the response!  I wonder whether there's any other way that I can get the source level information when using cuda-gdb to debug a binary compiled by LLVM? Or is there any way to feed the IR files to NVCC and get a binary with the debug info I need?

Thanks!
Yuanfeng


------------------ Original ------------------
From:  "Tom Stellard";<[hidden email]>;
Send time: Wednesday, Jun 14, 2017 9:06 PM
To: "Yuanfeng Peng"<[hidden email]>; "via llvm-dev"<[hidden email]>;
Subject:  Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

On 06/14/2017 02:59 AM, Yuanfeng Peng via llvm-dev wrote:
> Hi,
>
> I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck. 
>

The NVPTX backend in LLVM does not support DWARF emission.

-Tom

> Specifically, below is what I did:
>
> 1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
> 2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
> 3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
> 4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
> 5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
> 6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
> 7) Link against libraries and get the final binary: a.out.
>
> The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?
>
> Thanks!
> Yuanfeng Peng

>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>





_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code

Alex Bradbury via llvm-dev
On 06/14/2017 11:33 AM, Yuanfeng Peng wrote:
> Hi Tom,
>
> Thanks for the response!  I wonder whether there's any other way that I can get the source level information when using cuda-gdb to debug a binary compiled by LLVM? Or is there any way to feed the IR files to NVCC and get a binary with the debug info I need?
>

I don't know, I think you will have to ask NVIDIA about this.

-Tom

> Thanks!
> Yuanfeng
>
>
> ------------------ Original ------------------
> *From: * "Tom Stellard";<[hidden email]>;
> *Send time:* Wednesday, Jun 14, 2017 9:06 PM
> *To:* "Yuanfeng Peng"<[hidden email]>; "via llvm-dev"<[hidden email]>;
> *Subject: * Re: [llvm-dev] [CUDA] Lost debug information when compiling CUDA code
>
> On 06/14/2017 02:59 AM, Yuanfeng Peng via llvm-dev wrote:
>> Hi,
>>
>> I needed to debug some CUDA code in my project; however, although I used -g when compiling the source code, no source-level information is available in cuda-gdb or cuda-memcheck.
>>
>
> The NVPTX backend in LLVM does not support DWARF emission.
>
> -Tom
>
>> Specifically, below is what I did:
>>
>> 1) For a CUDA file a.cu, generate IR files: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c a.cu;
>> 2) Instrument the device code a-cuda-nvptx64-nvidia-cuda-sm_35.bc (generated in the previous step), inserting a call to a hook function before each device memory access.  The hook function is defined in another file, b.cu.  Let's say we get a file named intrumented-a-device.bc after this step;
>> 3) Generate IR files for b.cu: clang++ -g -emit-llvm --cuda-gpu-arch=sm_35 -c b.cu;
>> 4) Link instrumented-a.device.bc with the device code generated for b.cu: llvm-link intrumented-a-device.bc b-cuda-nvptx64-nvidia-cuda-sm_35.bc -o ab-device.bc;
>> 5) Use llc, ptxas & fatbinary on ab-device.bc to get ab-device.ptx, ab-device.o & ab-device.fatbin;
>> 6) Call clang again the generate the host object file ab.o, with ab-device.o & ab-device.fatbin embedded;
>> 7) Link against libraries and get the final binary: a.out.
>>
>> The binary a.out fails with an exception I when run it; but when I try to debug it with cuda-gdb or cuda-memcheck, no source information was available.  Why?
>>
>> Thanks!
>> Yuanfeng Peng
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
>
>

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev