Using a function from another module

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Using a function from another module

Michael Muller

Hi all,

I'm trying to use a function defined in one LLVM module from another module
(in the JIT) but for some reason it's not working out.  My sequence of
activity is roughly like this:

  1) Create moduleA
  2) Create moduleB with "func()"
  3) execEng = ExecutionEngine::create(
         new ExistingModuleProvider(moduleB));
  4) execute "func()" (this works fine)
  4) add "func()" to moduleA as a declaration (no code blocks) with External
     linkage.
  5) execEng->addModuleProvider(new ExistingModuleProvider(moduleA));
  6) run a function in moduleA that calls "func()"

I get:
  LLVM ERROR: Program used external function 'func' which could not be resolved!

I'm guessing I'm either going about this wrong or missing something.  Can
anyone offer me some insight?

=============================================================================
michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
-----------------------------------------------------------------------------
We are the music-makers, and we are the dreamers of dreams
 - Arthur O'Shaughnessy
=============================================================================
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Michael Muller

Michael Muller wrote:

>
> Hi all,
>
> I'm trying to use a function defined in one LLVM module from another module
> (in the JIT) but for some reason it's not working out.  My sequence of
> activity is roughly like this:
>
>   1) Create moduleA
>   2) Create moduleB with "func()"
>   3) execEng = ExecutionEngine::create(
>          new ExistingModuleProvider(moduleB));
>   4) execute "func()" (this works fine)
>   4) add "func()" to moduleA as a declaration (no code blocks) with External
>      linkage.
>   5) execEng->addModuleProvider(new ExistingModuleProvider(moduleA));
>   6) run a function in moduleA that calls "func()"
>
> I get:
>   LLVM ERROR: Program used external function 'func' which could not be resolved!
>
> I'm guessing I'm either going about this wrong or missing something.  Can
> anyone offer me some insight?

I've played around with this some more.

It looks like the only way that I can get this to work is to do an
ExecutionEngine::addGlobalMapping() on the function declaration in moduleA to
map it to the function pointer in moduleB.

This seems awkward, is there a better way to do this?

>
> =============================================================================
> michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
> -----------------------------------------------------------------------------
> We are the music-makers, and we are the dreamers of dreams
>  - Arthur O'Shaughnessy
> =============================================================================
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


=============================================================================
michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
-----------------------------------------------------------------------------
you and I are only different in our minds, the universe makes no such
distinction
=============================================================================
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Kenneth Uildriks
On Sun, Jan 10, 2010 at 8:58 AM, Michael Muller <[hidden email]> wrote:

>
> Michael Muller wrote:
>>
>> Hi all,
>>
>> I'm trying to use a function defined in one LLVM module from another module
>> (in the JIT) but for some reason it's not working out.  My sequence of
>> activity is roughly like this:
>>
>>   1) Create moduleA
>>   2) Create moduleB with "func()"
>>   3) execEng = ExecutionEngine::create(
>>          new ExistingModuleProvider(moduleB));
>>   4) execute "func()" (this works fine)
>>   4) add "func()" to moduleA as a declaration (no code blocks) with External
>>      linkage.
>>   5) execEng->addModuleProvider(new ExistingModuleProvider(moduleA));
>>   6) run a function in moduleA that calls "func()"
>>
>> I get:
>>   LLVM ERROR: Program used external function 'func' which could not be resolved!
>>
>> I'm guessing I'm either going about this wrong or missing something.  Can
>> anyone offer me some insight?
>
> I've played around with this some more.
>
> It looks like the only way that I can get this to work is to do an
> ExecutionEngine::addGlobalMapping() on the function declaration in moduleA to
> map it to the function pointer in moduleB.
>
> This seems awkward, is there a better way to do this?
>

I'm doing the same thing, and had to do it in the same way.

Just because the JIT loads two modules doesn't mean that they're
automatically linked together within the JIT... one module cannot call
functions in the other unless the external functions are declared and
explicitly mapped using addGlobalMapping.  I'm guessing it's meant to
be that way.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Garrison Venn
Won't passing llvm::Function* around vs strings (function names), also work, at code generation time,
without the need for a module A dec to module B impl. mapping?

Garrison

On Jan 10, 2010, at 10:31, Kenneth Uildriks wrote:

> On Sun, Jan 10, 2010 at 8:58 AM, Michael Muller <[hidden email]> wrote:
>>
>> Michael Muller wrote:
>>>
>>> Hi all,
>>>
>>> I'm trying to use a function defined in one LLVM module from another module
>>> (in the JIT) but for some reason it's not working out.  My sequence of
>>> activity is roughly like this:
>>>
>>>   1) Create moduleA
>>>   2) Create moduleB with "func()"
>>>   3) execEng = ExecutionEngine::create(
>>>          new ExistingModuleProvider(moduleB));
>>>   4) execute "func()" (this works fine)
>>>   4) add "func()" to moduleA as a declaration (no code blocks) with External
>>>      linkage.
>>>   5) execEng->addModuleProvider(new ExistingModuleProvider(moduleA));
>>>   6) run a function in moduleA that calls "func()"
>>>
>>> I get:
>>>   LLVM ERROR: Program used external function 'func' which could not be resolved!
>>>
>>> I'm guessing I'm either going about this wrong or missing something.  Can
>>> anyone offer me some insight?
>>
>> I've played around with this some more.
>>
>> It looks like the only way that I can get this to work is to do an
>> ExecutionEngine::addGlobalMapping() on the function declaration in moduleA to
>> map it to the function pointer in moduleB.
>>
>> This seems awkward, is there a better way to do this?
>>
>
> I'm doing the same thing, and had to do it in the same way.
>
> Just because the JIT loads two modules doesn't mean that they're
> automatically linked together within the JIT... one module cannot call
> functions in the other unless the external functions are declared and
> explicitly mapped using addGlobalMapping.  I'm guessing it's meant to
> be that way.
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Kenneth Uildriks
On Sun, Jan 10, 2010 at 12:38 PM, Garrison Venn <[hidden email]> wrote:
> Won't passing llvm::Function* around vs strings (function names), also work, at code generation time,
> without the need for a module A dec to module B impl. mapping?
>
> Garrison

Nope.  You cannot place a call instruction into one module whose
callee is a Function from another module.  You have to put a
declaration into the same module, and have your call instruction call
that.  And then they need to be linked together, either by llvm-link
or (if JITting) by addGlobalMapping.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Michael Muller

Kenneth Uildriks wrote:

> On Sun, Jan 10, 2010 at 12:38 PM, Garrison Venn <[hidden email]> wrote:
> > Won't passing llvm::Function* around vs strings (function names), also work, at code generation time,
> > without the need for a module A dec to module B impl. mapping?
> >
> > Garrison
>
> Nope.  You cannot place a call instruction into one module whose
> callee is a Function from another module.  You have to put a
> declaration into the same module, and have your call instruction call
> that.  And then they need to be linked together, either by llvm-link
> or (if JITting) by addGlobalMapping.
>

Actually, this is the first thing I tried, and the correct function does seem
to get called - but it looks like the Verifier complains about it, which leads
me to believe that there may be broader issues involved.


=============================================================================
michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
-----------------------------------------------------------------------------
Scnozwangers? Vermicious Knids? What kind of rubbish is that?
 - Mr. Salt, "Willy Wonka and the Chocolate Factory"
=============================================================================
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Garrison Venn
In reply to this post by Kenneth Uildriks
Cool! I wouldn't have believed it until I saw my test results.

Thanks for ed.

Garrison

On Jan 10, 2010, at 14:02, Kenneth Uildriks wrote:

> On Sun, Jan 10, 2010 at 12:38 PM, Garrison Venn <[hidden email]> wrote:
>> Won't passing llvm::Function* around vs strings (function names), also work, at code generation time,
>> without the need for a module A dec to module B impl. mapping?
>>
>> Garrison
>
> Nope.  You cannot place a call instruction into one module whose
> callee is a Function from another module.  You have to put a
> declaration into the same module, and have your call instruction call
> that.  And then they need to be linked together, either by llvm-link
> or (if JITting) by addGlobalMapping.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Garrison Venn
In reply to this post by Michael Muller
So, having given my last response, I'm still bothered by this. I think what I find unusual is
that one has to manually JIT and map (ExecutionEngine::addGlobalMapping(...)). Maybe
I'm out there, but I keep on wanting to have the linkage supplied in the Module A decl. take
care of this for me. So instead of an enum value of llvm::GlobalValue::ExternalLinkage,
I could conceptually give it (the decl.), a module (module B in this case), or maybe an enum
linkage, module pair. Of course representing this in IR might be a problem.

Anyway what do I know.  

Garrison

On Jan 10, 2010, at 14:23, Michael Muller wrote:

>
> Kenneth Uildriks wrote:
>> On Sun, Jan 10, 2010 at 12:38 PM, Garrison Venn <[hidden email]> wrote:
>>> Won't passing llvm::Function* around vs strings (function names), also work, at code generation time,
>>> without the need for a module A dec to module B impl. mapping?
>>>
>>> Garrison
>>
>> Nope.  You cannot place a call instruction into one module whose
>> callee is a Function from another module.  You have to put a
>> declaration into the same module, and have your call instruction call
>> that.  And then they need to be linked together, either by llvm-link
>> or (if JITting) by addGlobalMapping.
>>
>
> Actually, this is the first thing I tried, and the correct function does seem
> to get called - but it looks like the Verifier complains about it, which leads
> me to believe that there may be broader issues involved.
>
>
> =============================================================================
> michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
> -----------------------------------------------------------------------------
> Scnozwangers? Vermicious Knids? What kind of rubbish is that?
> - Mr. Salt, "Willy Wonka and the Chocolate Factory"
> =============================================================================


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Jeffrey Yasskin
In reply to this post by Michael Muller
The JIT tries to handle this in some cases
(http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/ExecutionEngine.cpp?annotate=92771#l942),
but doesn't handle it for functions. There aren't any tests, so I'm
not surprised it's broken.

The JIT would be simpler if we just dropped multiple-module support
and asked people to link their modules together before trying to JIT
them.  Is there a reason you can't do that?

If there is, could you write a test for
http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/ExecutionEngine/JIT/
that exercises the behavior you want, and file a bug with it attached?
I'm not likely to actually implement that any time soon, but having a
bug with a test will make it easier for someone else to pick it up.

Thanks,
Jeffrey

On Fri, Jan 8, 2010 at 4:49 PM, Michael Muller <[hidden email]> wrote:

>
> Hi all,
>
> I'm trying to use a function defined in one LLVM module from another module
> (in the JIT) but for some reason it's not working out.  My sequence of
> activity is roughly like this:
>
>  1) Create moduleA
>  2) Create moduleB with "func()"
>  3) execEng = ExecutionEngine::create(
>         new ExistingModuleProvider(moduleB));
>  4) execute "func()" (this works fine)
>  4) add "func()" to moduleA as a declaration (no code blocks) with External
>     linkage.
>  5) execEng->addModuleProvider(new ExistingModuleProvider(moduleA));
>  6) run a function in moduleA that calls "func()"
>
> I get:
>  LLVM ERROR: Program used external function 'func' which could not be resolved!
>
> I'm guessing I'm either going about this wrong or missing something.  Can
> anyone offer me some insight?
>
> =============================================================================
> michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
> -----------------------------------------------------------------------------
> We are the music-makers, and we are the dreamers of dreams
>  - Arthur O'Shaughnessy
> =============================================================================
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Kenneth Uildriks
On Mon, Jan 11, 2010 at 1:39 PM, Jeffrey Yasskin <[hidden email]> wrote:
> The JIT tries to handle this in some cases
> (http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/ExecutionEngine.cpp?annotate=92771#l942),
> but doesn't handle it for functions. There aren't any tests, so I'm
> not surprised it's broken.
>
> The JIT would be simpler if we just dropped multiple-module support
> and asked people to link their modules together before trying to JIT
> them.  Is there a reason you can't do that?

I'd like to be able to JIT a module, call imported functions from it,
and then write out that module without including the bodies of the
imported functions in the output .bc file.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Using a function from another module

Michael Muller

Kenneth Uildriks wrote:

> On Mon, Jan 11, 2010 at 1:39 PM, Jeffrey Yasskin <[hidden email]> wrote:
> > The JIT tries to handle this in some cases
> > (http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/ExecutionEngine.cpp?annotate=92771#l942),
> > but doesn't handle it for functions. There aren't any tests, so I'm
> > not surprised it's broken.
> >
> > The JIT would be simpler if we just dropped multiple-module support
> > and asked people to link their modules together before trying to JIT
> > them.  Is there a reason you can't do that?
>
> I'd like to be able to JIT a module, call imported functions from it,
> and then write out that module without including the bodies of the
> imported functions in the output .bc file.
>
Yeah, I have a similar use case - I want to be able to execute and dump the
.bc file to a filesystem cache to avoid subsequent compiles.

I've come up with a fairly elegant work-around, but I'll try to put together a
unit test that illustrates the failure.  If I feel ambitious, I'll also take
a stab at a fix.


=============================================================================
michaelMuller = [hidden email] | http://www.mindhog.net/~mmuller
-----------------------------------------------------------------------------
Society in every state is a blessing, but government even in its best state
is but a necessary evil; in its worst state an intolerable one...
 - Thomas Paine
=============================================================================

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev