[llvm-dev] Broken relocation for generating offsets?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Broken relocation for generating offsets?

Robin Eklind via llvm-dev
Hello LLVM-Mailing-List,

I discovered a strange behavior when dealing with object files generated by the compiler of Visual Studio 2015.

When jitting bc files I also add object files to look up functions. These object files are coming from visual studio. When using a switch case instruction that compiler often generates code based of __ImageBase. I show you a short snippet of the assembly output.
mov         eax, DWORD PTR ?myInt@@3HA ; myInt
lea         rdi, OFFSET FLAT:__ImageBase
xor         ebx, ebx


Then these offset is used to jump to some labels like "$LL4@execute:".

When the object file gets added to the jitting process this offset generation seems to be broken. Executing the code coming from the object file will lead to a crash. The crash address will always be exactly the address I used to overload __ImageBase with. So it seems that the address relocation is wrong with generating offsets?

Kind regards
Björn

Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Broken relocation for generating offsets?

Robin Eklind via llvm-dev
Hello,

I append another clue I found out: The problem is definitely not caused by "__ImageBase" the problem comes with the "OFFSET". I generated another object file which crashed. The commonality:
mov         edx, DWORD PTR ?normalPlanschbecken@@3HA ; normalPlanschbecken
lea         rcx, OFFSET FLAT:??_C@_0CC@LCMJAIPO@Reading?5?$CCnormalPlanschbecken?$CC?5?$CFi@

jmp         printf





From:        
via llvm-dev <[hidden email]>
To:        
[hidden email]
Date:        
06.03.2018 10:45
Subject:        
[llvm-dev] Broken relocation for generating offsets?
Sent by:        
"llvm-dev" <[hidden email]>




Hello LLVM-Mailing-List,


I discovered a strange behavior when dealing with object files generated by the compiler of Visual Studio 2015.


When jitting bc files I also add object files to look up functions. These object files are coming from visual studio. When using a switch case instruction that compiler often generates code based of __ImageBase. I show you a short snippet of the assembly output.
mov         eax, DWORD PTR ?myInt@@3HA ; myInt
lea         rdi, OFFSET FLAT:__ImageBase

xor         ebx, ebx




Then these offset is used to jump to some labels like "$LL4@execute:".


When the object file gets added to the jitting process this offset generation seems to be broken. Executing the code coming from the object file will lead to a crash. The crash address will always be exactly the address I used to overload __ImageBase with. So it seems that the address relocation is wrong with generating offsets?


Kind regards

Björn


Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list
[hidden email]

http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Broken relocation for generating offsets?

Robin Eklind via llvm-dev
I wouldn't be surprised if JITing COFF files on Windows doesn't work so well, since the object file format assumes most symbols are dllimport or within the local 2GB module address range.

I'm not familiar with the current JIT state of the art, though.


On Thu, Mar 22, 2018 at 1:45 AM via llvm-dev <[hidden email]> wrote:
Hello,

I append another clue I found out: The problem is definitely not caused by "__ImageBase" the problem comes with the "OFFSET". I generated another object file which crashed. The commonality:
mov         edx, DWORD PTR ?normalPlanschbecken@@3HA ; normalPlanschbecken
lea         rcx, OFFSET FLAT:??_C@_0CC@LCMJAIPO@Reading?5?$CCnormalPlanschbecken?$CC?5?$CFi@

jmp         printf





From:        
via llvm-dev <[hidden email]>
To:        
[hidden email]
Date:        
06.03.2018 10:45
Subject:        
[llvm-dev] Broken relocation for generating offsets?
Sent by:        
"llvm-dev" <[hidden email]>




Hello LLVM-Mailing-List,


I discovered a strange behavior when dealing with object files generated by the compiler of Visual Studio 2015.


When jitting bc files I also add object files to look up functions. These object files are coming from visual studio. When using a switch case instruction that compiler often generates code based of __ImageBase. I show you a short snippet of the assembly output.
mov         eax, DWORD PTR ?myInt@@3HA ; myInt
lea         rdi, OFFSET FLAT:__ImageBase

xor         ebx, ebx




Then these offset is used to jump to some labels like "$LL4@execute:".


When the object file gets added to the jitting process this offset generation seems to be broken. Executing the code coming from the object file will lead to a crash. The crash address will always be exactly the address I used to overload __ImageBase with. So it seems that the address relocation is wrong with generating offsets?


Kind regards

Björn


Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list
[hidden email]

http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Broken relocation for generating offsets?

Robin Eklind via llvm-dev
In my case I wouldn't exceed the 2GB module address range...
As I understood - but I'm still a cub with the LLVM and everything - the "OFFSET" thingy should calculate the offset from the instruction to the address of the symbol. But LLVM creates a jump to that symbol.



From:        Reid Kleckner <[hidden email]>
To:        [hidden email], Lang Hames <[hidden email]>
Cc:        llvm-dev <[hidden email]>
Date:        22.03.2018 18:43
Subject:        Re: [llvm-dev] Broken relocation for generating offsets?




I wouldn't be surprised if JITing COFF files on Windows doesn't work so well, since the object file format assumes most symbols are dllimport or within the local 2GB module address range.

I'm not familiar with the current JIT state of the art, though.


On Thu, Mar 22, 2018 at 1:45 AM via llvm-dev <[hidden email]> wrote:
Hello,

I append another clue I found out: The problem is definitely not caused by "__ImageBase" the problem comes with the "OFFSET". I generated another object file which crashed. The commonality:
mov         edx, DWORD PTR ?normalPlanschbecken@@3HA ; normalPlanschbecken
lea         rcx, OFFSET FLAT:??_C@_0CC@LCMJAIPO@Reading?5?$CCnormalPlanschbecken?$CC?5?$CFi@

jmp         printf






From:        
via llvm-dev <[hidden email]>
To:        
[hidden email]
Date:        
06.03.2018 10:45
Subject:        
[llvm-dev] Broken relocation for generating offsets?
Sent by:        
"llvm-dev" <[hidden email]>




Hello LLVM-Mailing-List,


I discovered a strange behavior when dealing with object files generated by the compiler of Visual Studio 2015.


When jitting bc files I also add object files to look up functions. These object files are coming from visual studio. When using a switch case instruction that compiler often generates code based of __ImageBase. I show you a short snippet of the assembly output.
mov         eax, DWORD PTR ?myInt@@3HA ; myInt
lea         rdi, OFFSET FLAT:__ImageBase

xor         ebx, ebx





Then these offset is used to jump to some labels like "$LL4@execute:".


When the object file gets added to the jitting process this offset generation seems to be broken. Executing the code coming from the object file will lead to a crash. The crash address will always be exactly the address I used to overload __ImageBase with. So it seems that the address relocation is wrong with generating offsets?


Kind regards

Björn


Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list

[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.




Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list

[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Possibilities with LLVM

Robin Eklind via llvm-dev
In reply to this post by Robin Eklind via llvm-dev
Hello everyone,

I have some questions about the possibilities with the LLVM but I'm not sure where to gather the information.

1.) Can I teach the LLVM new platform depended intrinsics?
Like I provide assembly code and want to create a custom intrinsic for it.

2.) Does the IR language have some kind of template support?
I'm not sure if this even possible - but I thought about having a template function and when jitting the IR it could instantiate that template with the now known data - okay writing this already sounds weird and I'm not sure if I could explain what I wanted.

3.) Can I implement my own custom calling convention? Like my own "__planschiCall" or something?

When these things are possible - are there any documentation or tutorials for this? And how does Clang learn about all these custom things?

Kind regards
Björn

Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Possibilities with LLVM

Robin Eklind via llvm-dev
Also a newbie myself. However afaik: 1&3 should be possible , as for the 2nd, I have no idea what do you mean 

Zhang

On 9 Apr 2018, at 09:43, via llvm-dev <[hidden email]> wrote:

Hello everyone,

I have some questions about the possibilities with the LLVM but I'm not sure where to gather the information.

1.) Can I teach the LLVM new platform depended intrinsics?
Like I provide assembly code and want to create a custom intrinsic for it.

2.) Does the IR language have some kind of template support?
I'm not sure if this even possible - but I thought about having a template function and when jitting the IR it could instantiate that template with the now known data - okay writing this already sounds weird and I'm not sure if I could explain what I wanted.

3.) Can I implement my own custom calling convention? Like my own "__planschiCall" or something?

When these things are possible - are there any documentation or tutorials for this? And how does Clang learn about all these custom things?

Kind regards
Björn

Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Possibilities with LLVM

Robin Eklind via llvm-dev
In reply to this post by Robin Eklind via llvm-dev
Hi Björn,

On 9 April 2018 at 09:43, via llvm-dev <[hidden email]> wrote:
> 1.) Can I teach the LLVM new platform depended intrinsics?
> Like I provide assembly code and want to create a custom intrinsic for it.

Yes. You'd add a declaration to include/llvm/IR/IntrinsicsXYZ.td
(where XYZ is your target), and then you can select them with a normal
pattern in your target's .td files.

On the Clang side you'd add it to include/clang/Basic/BuiltinsXYZ.def
and lib/CodeGen/CGBuiltin.cpp. Possibly lib/Sema/SemaChecking.cpp too
if it has strange validity requirements.

> 2.) Does the IR language have some kind of template support?
> I'm not sure if this even possible - but I thought about having a template
> function and when jitting the IR it could instantiate that template with the
> now known data - okay writing this already sounds weird and I'm not sure if
> I could explain what I wanted.

No template support, I'm afraid. That would be handled by a language's
front-end.

> 3.) Can I implement my own custom calling convention? Like my own
> "__planschiCall" or something?

Yes. The easiest way would be to use a simple numbered calling
convention "cc N" in the LLVM IR. Then you just have to add support to
lib/Target/XYZ/XYZISelLowering.cpp and the corresponding calling
convention .td file (the name varies a bit). ISelLowering usually just
involves a switch based on the call or definition that selects the
right implementation from the .td file. Grep for CCAssignFnForCall to
see the kind of thing you'll be changing.

Clang side, you'll be changing lib/Targets/XYZ.cpp to either make it
the default or support it via __attribute__((pcs("whatever"))). See
ARM.cpp for an example, it already uses both methods.

Cheers.

Tim.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Possibilities with LLVM

Robin Eklind via llvm-dev
Hello Tim,

thank you for the response! Are there tutorials for the points 1 and 3 available? I'm still a cub with the LLVM...

Kind regards
Björn



From:        Tim Northover <[hidden email]>
To:        [hidden email]
Cc:        LLVM Developers Mailing List <[hidden email]>
Date:        09.04.2018 12:09
Subject:        Re: [llvm-dev] Possibilities with LLVM




Hi Björn,

On 9 April 2018 at 09:43, via llvm-dev <[hidden email]> wrote:
> 1.) Can I teach the LLVM new platform depended intrinsics?
> Like I provide assembly code and want to create a custom intrinsic for it.

Yes. You'd add a declaration to include/llvm/IR/IntrinsicsXYZ.td
(where XYZ is your target), and then you can select them with a normal
pattern in your target's .td files.

On the Clang side you'd add it to include/clang/Basic/BuiltinsXYZ.def
and lib/CodeGen/CGBuiltin.cpp. Possibly lib/Sema/SemaChecking.cpp too
if it has strange validity requirements.

> 2.) Does the IR language have some kind of template support?
> I'm not sure if this even possible - but I thought about having a template
> function and when jitting the IR it could instantiate that template with the
> now known data - okay writing this already sounds weird and I'm not sure if
> I could explain what I wanted.

No template support, I'm afraid. That would be handled by a language's
front-end.

> 3.) Can I implement my own custom calling convention? Like my own
> "__planschiCall" or something?

Yes. The easiest way would be to use a simple numbered calling
convention "cc N" in the LLVM IR. Then you just have to add support to
lib/Target/XYZ/XYZISelLowering.cpp and the corresponding calling
convention .td file (the name varies a bit). ISelLowering usually just
involves a switch based on the call or definition that selects the
right implementation from the .td file. Grep for CCAssignFnForCall to
see the kind of thing you'll be changing.

Clang side, you'll be changing lib/Targets/XYZ.cpp to either make it
the default or support it via __attribute__((pcs("whatever"))). See
ARM.cpp for an example, it already uses both methods.

Cheers.

Tim.



Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Hiroshi Kawamura, Takashi Nagano, Takeshi Fukushima.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] [JIT] Cross references

Robin Eklind via llvm-dev
In reply to this post by Robin Eklind via llvm-dev
Hello LLVM-people,

I have a question!
Lets take I have two bc-files: "Plansch.bc" and "Becken.bc"

"Plansch.bc" has an external variable which will be found in "Becken.bc"
"Becken.bc" has also an external variable which will be found in "Plansch.bc"
They cross reference each other.

As I understand - the ExecutionEngine will easily merge those files and resolve them as one big file.
So "Plansch.bc" and "Becken.bc" will be "Planschbecken.bc"

That's cool! But what if I don't want that to happen? I want to load "Plansch.bc" and "Becken.bc" to their own memory addresses but also want to cross-reference them.

I thought:
First I load "Plansch.bc". Then the ExecutionEngine will get an undefined reference to the extern variable coming from "Becken.bc". What now?

1.) Can I tell the ExecutionEngine something like "Please wait for the address and skip it for now"?
2.) Can I trigger in that moment another ExecutionEngine loading "Becken.bc" to resolve at least that address?
-> But then "Becken.bc" would run into its own undefined reference to "Plansch.bc" and I don't think I would get the value at that point.

Any ideas, solutions or opinions on that?

Kind regards
Björn

Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789
Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Takeshi Fukushima. Junichi Tajika

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev