[llvm-dev] Since MCJIT I can't get libm functions to work

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
I am upgrading our JIT application to use LLVM 6.0.0, and with this
transition I am making the move to use the new MCJIT facility.

A problem I am encountering is that the math functions from libm are not
resolved/found. I am using the lambda resolver from the KaleidoscopeJIT
class which first searches the present modules and, if that is
unsuccessful, continues the search in the whole process space.

The generated modules would declare external functions like

declare float @cosf(float)

and would call it like:

   %17 = call float @cosf(float %16)

The datalayout is set to

target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"

The code snippet that adds the module is pretty much from the JIT tutorial:

       auto Resolver = llvm::orc::createLambdaResolver(
                               [&](const std::string &Name) {
                             if (auto Sym =
CompileLayer.findSymbol(Name, false))
                               return Sym;
                             return llvm::JITSymbol(nullptr);
                               },
                               [](const std::string &Name) {
                             if (auto SymAddr =
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
                               return llvm::JITSymbol(SymAddr,
llvm::JITSymbolFlags::Exported);
                             return llvm::JITSymbol(nullptr);
                               });

       cantFail(CompileLayer.addModule(std::move(M),
                       std::move(Resolver)));

When running the program I receive the following error:

LLVM ERROR: Program used external function 'cosf' which could not be
resolved!

This is on an Intel i7 CPU with LLVM targets configured as "X86".

Adding the '-lm' and '-ldl' option to the linker command that links the
final program doesn't help. (I even called the 'cosf' function by hand
in the host code to make sure it is mapped - didn't change the behavior
of the MCJIT resolver.)

Any ideas?


Best wishes,

Frank



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
Hi Frank,

This is on Linux, right? If so, you will need to pass --export-dynamic to ld to export symbols from the main binary.

For example, given source file foo.c:

#include <stdio.h>
#include <dlfcn.h>

void bar() {}

int main(int argc, char *argv[]) {
  printf("bar is %p\n", dlsym(RTLD_DEFAULT, "bar")); 
  return 0;
}

You will see:

$ clang -O3 -D_GNU_SOURCE=1 -o foo foo.c -ldl
$ ./foo
bar is (nil)
$ clang -O3 -D_GNU_SOURCE=1 -Wl,--export-dynamic -o foo foo.c -ldl
$ ./foo
bar is 0x400820

Hope this helps!

Cheers,
Lang.


On Thu, Jun 28, 2018 at 12:40 PM Frank Winter via llvm-dev <[hidden email]> wrote:
I am upgrading our JIT application to use LLVM 6.0.0, and with this
transition I am making the move to use the new MCJIT facility.

A problem I am encountering is that the math functions from libm are not
resolved/found. I am using the lambda resolver from the KaleidoscopeJIT
class which first searches the present modules and, if that is
unsuccessful, continues the search in the whole process space.

The generated modules would declare external functions like

declare float @cosf(float)

and would call it like:

   %17 = call float @cosf(float %16)

The datalayout is set to

target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"

The code snippet that adds the module is pretty much from the JIT tutorial:

       auto Resolver = llvm::orc::createLambdaResolver(
                               [&](const std::string &Name) {
                             if (auto Sym =
CompileLayer.findSymbol(Name, false))
                               return Sym;
                             return llvm::JITSymbol(nullptr);
                               },
                               [](const std::string &Name) {
                             if (auto SymAddr =
llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
                               return llvm::JITSymbol(SymAddr,
llvm::JITSymbolFlags::Exported);
                             return llvm::JITSymbol(nullptr);
                               });

       cantFail(CompileLayer.addModule(std::move(M),
                       std::move(Resolver)));

When running the program I receive the following error:

LLVM ERROR: Program used external function 'cosf' which could not be
resolved!

This is on an Intel i7 CPU with LLVM targets configured as "X86".

Adding the '-lm' and '-ldl' option to the linker command that links the
final program doesn't help. (I even called the 'cosf' function by hand
in the host code to make sure it is mapped - didn't change the behavior
of the MCJIT resolver.)

Any ideas?


Best wishes,

Frank



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
In reply to this post by Muhui Jiang via llvm-dev
Hi Frank,

If I am not mistaken it doesn't look in the whole process space by default.
Please, try loading all the symbols explicitly:

        sys::DynamicLibrary::LoadLibraryPermanently(nullptr);

If it doesn't help, then you may try compiling your host program with -rdynamic, otherwise dlsym may not see all the symbols. At least it was the case for me on Linux.

I hope it helps.

Cheers,
Alex.

> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <[hidden email]> wrote:
>
> I am upgrading our JIT application to use LLVM 6.0.0, and with this transition I am making the move to use the new MCJIT facility.
>
> A problem I am encountering is that the math functions from libm are not resolved/found. I am using the lambda resolver from the KaleidoscopeJIT class which first searches the present modules and, if that is unsuccessful, continues the search in the whole process space.
>
> The generated modules would declare external functions like
>
> declare float @cosf(float)
>
> and would call it like:
>
>   %17 = call float @cosf(float %16)
>
> The datalayout is set to
>
> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>
> The code snippet that adds the module is pretty much from the JIT tutorial:
>
>       auto Resolver = llvm::orc::createLambdaResolver(
>                               [&](const std::string &Name) {
>                             if (auto Sym = CompileLayer.findSymbol(Name, false))
>                               return Sym;
>                             return llvm::JITSymbol(nullptr);
>                               },
>                               [](const std::string &Name) {
>                             if (auto SymAddr =
> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>                               return llvm::JITSymbol(SymAddr, llvm::JITSymbolFlags::Exported);
>                             return llvm::JITSymbol(nullptr);
>                               });
>
>       cantFail(CompileLayer.addModule(std::move(M),
>                       std::move(Resolver)));
>
> When running the program I receive the following error:
>
> LLVM ERROR: Program used external function 'cosf' which could not be resolved!
>
> This is on an Intel i7 CPU with LLVM targets configured as "X86".
>
> Adding the '-lm' and '-ldl' option to the linker command that links the final program doesn't help. (I even called the 'cosf' function by hand in the host code to make sure it is mapped - didn't change the behavior of the MCJIT resolver.)
>
> Any ideas?
>
>
> Best wishes,
>
> Frank
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
Hi Alex,

loading the symbols explicitly helped. With this the build process
doesn't need any additional linker flags. Very nice.

Thanks,
Frank


On 06/28/2018 04:38 PM, Alex Denisov wrote:

> Hi Frank,
>
> If I am not mistaken it doesn't look in the whole process space by default.
> Please, try loading all the symbols explicitly:
>
> sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
>
> If it doesn't help, then you may try compiling your host program with -rdynamic, otherwise dlsym may not see all the symbols. At least it was the case for me on Linux.
>
> I hope it helps.
>
> Cheers,
> Alex.
>
>> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <[hidden email]> wrote:
>>
>> I am upgrading our JIT application to use LLVM 6.0.0, and with this transition I am making the move to use the new MCJIT facility.
>>
>> A problem I am encountering is that the math functions from libm are not resolved/found. I am using the lambda resolver from the KaleidoscopeJIT class which first searches the present modules and, if that is unsuccessful, continues the search in the whole process space.
>>
>> The generated modules would declare external functions like
>>
>> declare float @cosf(float)
>>
>> and would call it like:
>>
>>    %17 = call float @cosf(float %16)
>>
>> The datalayout is set to
>>
>> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>
>> The code snippet that adds the module is pretty much from the JIT tutorial:
>>
>>        auto Resolver = llvm::orc::createLambdaResolver(
>>                                [&](const std::string &Name) {
>>                              if (auto Sym = CompileLayer.findSymbol(Name, false))
>>                                return Sym;
>>                              return llvm::JITSymbol(nullptr);
>>                                },
>>                                [](const std::string &Name) {
>>                              if (auto SymAddr =
>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>>                                return llvm::JITSymbol(SymAddr, llvm::JITSymbolFlags::Exported);
>>                              return llvm::JITSymbol(nullptr);
>>                                });
>>
>>        cantFail(CompileLayer.addModule(std::move(M),
>>                        std::move(Resolver)));
>>
>> When running the program I receive the following error:
>>
>> LLVM ERROR: Program used external function 'cosf' which could not be resolved!
>>
>> This is on an Intel i7 CPU with LLVM targets configured as "X86".
>>
>> Adding the '-lm' and '-ldl' option to the linker command that links the final program doesn't help. (I even called the 'cosf' function by hand in the host code to make sure it is mapped - didn't change the behavior of the MCJIT resolver.)
>>
>> Any ideas?
>>
>>
>> Best wishes,
>>
>> Frank
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
Ahh. It is the other in-process-symbol-lookup gotcha.

I will think about adding a convenience option for that.

Thanks Alex!

-- Lang.

On Thu, Jun 28, 2018 at 1:49 PM Frank Winter via llvm-dev <[hidden email]> wrote:
Hi Alex,

loading the symbols explicitly helped. With this the build process
doesn't need any additional linker flags. Very nice.

Thanks,
Frank


On 06/28/2018 04:38 PM, Alex Denisov wrote:
> Hi Frank,
>
> If I am not mistaken it doesn't look in the whole process space by default.
> Please, try loading all the symbols explicitly:
>
>       sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
>
> If it doesn't help, then you may try compiling your host program with -rdynamic, otherwise dlsym may not see all the symbols. At least it was the case for me on Linux.
>
> I hope it helps.
>
> Cheers,
> Alex.
>
>> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <[hidden email]> wrote:
>>
>> I am upgrading our JIT application to use LLVM 6.0.0, and with this transition I am making the move to use the new MCJIT facility.
>>
>> A problem I am encountering is that the math functions from libm are not resolved/found. I am using the lambda resolver from the KaleidoscopeJIT class which first searches the present modules and, if that is unsuccessful, continues the search in the whole process space.
>>
>> The generated modules would declare external functions like
>>
>> declare float @cosf(float)
>>
>> and would call it like:
>>
>>    %17 = call float @cosf(float %16)
>>
>> The datalayout is set to
>>
>> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>
>> The code snippet that adds the module is pretty much from the JIT tutorial:
>>
>>        auto Resolver = llvm::orc::createLambdaResolver(
>>                                [&](const std::string &Name) {
>>                              if (auto Sym = CompileLayer.findSymbol(Name, false))
>>                                return Sym;
>>                              return llvm::JITSymbol(nullptr);
>>                                },
>>                                [](const std::string &Name) {
>>                              if (auto SymAddr =
>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
>>                                return llvm::JITSymbol(SymAddr, llvm::JITSymbolFlags::Exported);
>>                              return llvm::JITSymbol(nullptr);
>>                                });
>>
>>        cantFail(CompileLayer.addModule(std::move(M),
>>                        std::move(Resolver)));
>>
>> When running the program I receive the following error:
>>
>> LLVM ERROR: Program used external function 'cosf' which could not be resolved!
>>
>> This is on an Intel i7 CPU with LLVM targets configured as "X86".
>>
>> Adding the '-lm' and '-ldl' option to the linker command that links the final program doesn't help. (I even called the 'cosf' function by hand in the host code to make sure it is mapped - didn't change the behavior of the MCJIT resolver.)
>>
>> Any ideas?
>>
>>
>> Best wishes,
>>
>> Frank
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Since MCJIT I can't get libm functions to work

Muhui Jiang via llvm-dev
Great! I'm glad that it worked :)

> On 28. Jun 2018, at 23:27, Lang Hames <[hidden email]> wrote:
>
> Ahh. It is the other in-process-symbol-lookup gotcha.
>
> I will think about adding a convenience option for that.
>
> Thanks Alex!
>
> -- Lang.
>
> On Thu, Jun 28, 2018 at 1:49 PM Frank Winter via llvm-dev <[hidden email]> wrote:
> Hi Alex,
>
> loading the symbols explicitly helped. With this the build process
> doesn't need any additional linker flags. Very nice.
>
> Thanks,
> Frank
>
>
> On 06/28/2018 04:38 PM, Alex Denisov wrote:
> > Hi Frank,
> >
> > If I am not mistaken it doesn't look in the whole process space by default.
> > Please, try loading all the symbols explicitly:
> >
> >       sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
> >
> > If it doesn't help, then you may try compiling your host program with -rdynamic, otherwise dlsym may not see all the symbols. At least it was the case for me on Linux.
> >
> > I hope it helps.
> >
> > Cheers,
> > Alex.
> >
> >> On 28. Jun 2018, at 21:40, Frank Winter via llvm-dev <[hidden email]> wrote:
> >>
> >> I am upgrading our JIT application to use LLVM 6.0.0, and with this transition I am making the move to use the new MCJIT facility.
> >>
> >> A problem I am encountering is that the math functions from libm are not resolved/found. I am using the lambda resolver from the KaleidoscopeJIT class which first searches the present modules and, if that is unsuccessful, continues the search in the whole process space.
> >>
> >> The generated modules would declare external functions like
> >>
> >> declare float @cosf(float)
> >>
> >> and would call it like:
> >>
> >>    %17 = call float @cosf(float %16)
> >>
> >> The datalayout is set to
> >>
> >> target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> >>
> >> The code snippet that adds the module is pretty much from the JIT tutorial:
> >>
> >>        auto Resolver = llvm::orc::createLambdaResolver(
> >>                                [&](const std::string &Name) {
> >>                              if (auto Sym = CompileLayer.findSymbol(Name, false))
> >>                                return Sym;
> >>                              return llvm::JITSymbol(nullptr);
> >>                                },
> >>                                [](const std::string &Name) {
> >>                              if (auto SymAddr =
> >> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(Name))
> >>                                return llvm::JITSymbol(SymAddr, llvm::JITSymbolFlags::Exported);
> >>                              return llvm::JITSymbol(nullptr);
> >>                                });
> >>
> >>        cantFail(CompileLayer.addModule(std::move(M),
> >>                        std::move(Resolver)));
> >>
> >> When running the program I receive the following error:
> >>
> >> LLVM ERROR: Program used external function 'cosf' which could not be resolved!
> >>
> >> This is on an Intel i7 CPU with LLVM targets configured as "X86".
> >>
> >> Adding the '-lm' and '-ldl' option to the linker command that links the final program doesn't help. (I even called the 'cosf' function by hand in the host code to make sure it is mapped - didn't change the behavior of the MCJIT resolver.)
> >>
> >> Any ideas?
> >>
> >>
> >> Best wishes,
> >>
> >> Frank
> >>
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> [hidden email]
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

signature.asc (499 bytes) Download Attachment