Crash using the JIT on x86 but work on x64

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Crash using the JIT on x86 but work on x64

Skykill Skyoverside
Hello everyone, i’m using LLVM (updated to 3.1 after seeing that bug, but it’s the same with 3.0) for running a bitcode on a C++ program, and Clang for compiling it. My code work perfectly, as expected on x64, but crash on x86. I’m on Windows 7 x64 and LLVM + Clang was compiled using Visual Studio 2010 (tested in both Release and Debug build). Project was make using CMake.
 
Here is my code:
 
main.cpp:
 
 
#include "llvm/LLVMContext.h"
#include "llvm/Module.h"
//#include "llvm/Type.h"
//#include "llvm/ADT/Triple.h"
//#include "llvm/Bitcode/ReaderWriter.h"
#include "llvm/CodeGen/LinkAllCodegenComponents.h"
#include "llvm/ExecutionEngine/GenericValue.h"
//#include "llvm/ExecutionEngine/Interpreter.h"
#include "llvm/ExecutionEngine/JIT.h"
#include "llvm/ExecutionEngine/JITEventListener.h"
#include "llvm/ExecutionEngine/JITMemoryManager.h"
//#include "llvm/ExecutionEngine/MCJIT.h"
//#include "llvm/Support/CommandLine.h"
#include "llvm/Support/IRReader.h"
//#include "llvm/Support/ManagedStatic.h"
#include "llvm/Support/MemoryBuffer.h"
//#include "llvm/Support/PluginLoader.h"
#include "llvm/Support/PrettyStackTrace.h"
#include "llvm/Support/raw_ostream.h"
//#include "llvm/Support/Process.h"
#include "llvm/Support/Signals.h"
#include "llvm/Support/TargetSelect.h"
//#include <cerrno>
 
#include "llvm/DerivedTypes.h"
//#include "llvm/Support/DynamicLibrary.h"
#include "llvm/Linker.h"
 
//using namespace llvm;
 
 
llvm::LLVMContext* LLVM_Context;
static llvm::ExecutionEngine* LLVM_ExecE = 0;
llvm::Module* LLVM_Module;
llvm::Type* LLVM_pointerType;
 
bool loadChunk(void* chunk)
{
    llvm::Module* m = (llvm::Module*)chunk;
    std::string msg;
 
    if (llvm::Linker::LinkModules(LLVM_Module, m, llvm::Function::ExternalLinkage, &msg))
    {
        printf("\nLINK ERROR - %s", &msg);
    }
    return true;
}
bool loadFile(const char* filename)
{
    // Load the bitcode...
    llvm::SMDiagnostic Err;
    llvm::Module* Mod = ParseIRFile(filename, Err, *LLVM_Context);
    if (!Mod) {
        Err.print("Error loading bitcode", llvm::errs());
        return 1;
    }
    printf("\nLink with main module.\n");
    loadChunk(Mod);
 
    printf("\nDone loading %s", filename);
    return true;
}
void callFunction(const char* name, unsigned int argc, ...)
{
    llvm::Function *f = LLVM_Module->getFunction(name);
    if (f)
    {
        std::vector<llvm::GenericValue> args;
        if (argc > 0)
        {
            va_list argv;
 
            va_start(argv, argc);
            for (unsigned int x = 0; x < argc; x++) {
                llvm::GenericValue v;
                v.DoubleVal = (va_arg(argv, double)); // this might not work for anything other than doubles!
                args.push_back(v);
            }
            va_end(argv);
        }
        LLVM_ExecE->runFunction(f, args);
    }
    else
    {
        printf("\nTried to execute non-existent function '%c'.\n", name);
    }
}
 
template<typename T>
const static void* void_cast(const T& object)
{
    union Retyper
    {
        const T object;
        void* pointer;
        Retyper(T obj) : object(obj) { }
    };
 
    return Retyper(object).pointer;
}
template<typename T, typename M>
const static void* getMethodPointer(const T* object, M method) // will work for virtual methods
{
    union MethodEntry
    {
        intptr_t offset;
        void* function;
    };
 
    const MethodEntry* entry = static_cast<const MethodEntry*>(void_cast(&method));
 
    if (entry->offset % sizeof(intptr_t) == 0) // looks like that's how the runtime guesses virtual from static
        return getMethodPointer(method);
 
    const void* const* const vtable = *reinterpret_cast<const void* const* const* const>(object);
    return vtable[(entry->offset - 1) / sizeof(void*)];
}
template<typename M>
const static void* getMethodPointer(M method) // will only work with non-virtual methods
{
    union MethodEntry
    {
        intptr_t offset;
        void* function;
    };
 
    return static_cast<const MethodEntry*>(void_cast(&method))->function;
}
 
static llvm::Type* voidptr_type = NULL;
 
void registerSymbolFunction(const char* mangled_name, void* func)
{
    // This is the JIT way for register a function
    // Commented "sig.push_back" before defined function because they don't seem to need args
    std::vector<llvm::Type*> sig;
    llvm::FunctionType* FT = NULL;
    llvm::Function* F = NULL;
 
    sig.clear();
    //sig.push_back(LLVM_pointerType);
    // Get the function type
    FT = llvm::FunctionType::get(voidptr_type, sig, false);
    // Create the function
    F = llvm::Function::Create(FT, llvm::Function::ExternalLinkage, mangled_name, LLVM_Module);
    // Register the function to the global module
    // This method register the function to the module and the ExecutionEngine
    LLVM_ExecE->addGlobalMapping(F, func);
 
    // This also work, i don't know what method is better, addGlobalMapping seem to be for the JIT
    // and AddSymbol for the interpreter
    // This method register the function directly to LLVM
    //llvm::sys::DynamicLibrary::AddSymbol(mangled_name, func);
}
 
std::string getPrintText()
{
    printf("\nRunning getPrintText()...\n");
    return std::string("Hello world from getPrintText()!");
}
class TestClass1
{
public:
    static void test1()
    {
        printf("\nHello World from TestClass1::test1()!\n");
    }
    void test2(std::string s)
    {
        printf("\nHello World from TestClass1::test2()! : %s\n",+s.data());
        //ConsoleM.newMsg(S+"Hello World from TestClass1::test2()! : "+s.data(), YELLOW);
    }
} testclass1;
 
int main(int argc, char** argv, char* const* envp)
{
    printf("argc: %i ", argc);
    printf("argv: %c ", argv);
    printf("envp: %c ", envp);
 
    llvm::sys::PrintStackTraceOnErrorSignal();
    llvm::PrettyStackTraceProgram X(argc, argv);
 
    LLVM_Context = new llvm::LLVMContext();
 
    // If we have a native target, initialize it to ensure it is linked in and
    // usable by the JIT.
    llvm::InitializeNativeTarget();
    llvm::InitializeNativeTargetAsmPrinter();
 
    llvm::SMDiagnostic Err;
    LLVM_Module = new llvm::Module("LLVM Test", *LLVM_Context);
    if (!LLVM_Module) {
        Err.print(argv[0], llvm::errs());
        return 1;
    }
 
    std::string ErrorMsg;
    llvm::EngineBuilder builder(LLVM_Module);
    builder.setErrorStr(&ErrorMsg);
    builder.setJITMemoryManager(llvm::JITMemoryManager::CreateDefaultMemManager());
    builder.setEngineKind(llvm::EngineKind::JIT);
 
    //CodeGenOpt::Level OLvl = CodeGenOpt::Default;
 
    //builder.setOptLevel(OLvl);
 
    llvm::TargetOptions Options;
    Options.JITExceptionHandling = false;
    Options.JITEmitDebugInfo = false;
    Options.JITEmitDebugInfoToDisk = false;
    builder.setTargetOptions(Options);
 
    LLVM_ExecE = builder.create();
    if (!LLVM_ExecE) {
        if (!ErrorMsg.empty())
            llvm::errs() << argv[0] << ": error creating ExecE: " << ErrorMsg << "\n";
        else
            llvm::errs() << argv[0] << ": unknown error creating ExecE!\n";
        exit(1);
    }
 
    // The following functions have no effect if their respective profiling
    // support wasn't enabled in the build configuration.
    LLVM_ExecE->RegisterJITEventListener(
        llvm::JITEventListener::createOProfileJITEventListener());
    LLVM_ExecE->RegisterJITEventListener(
        llvm::JITEventListener::createIntelJITEventListener());
 
    LLVM_ExecE->DisableLazyCompilation(false);
 
 
    voidptr_type = llvm::IntegerType::get(*LLVM_Context, 8)->getPointerTo();
 
    LLVM_pointerType = (llvm::Type*)llvm::Type::getInt32Ty(*LLVM_Context);
 
 
    registerSymbolFunction("_Z12getPrintTextv", (void*)&getPrintText);
 
    registerSymbolFunction("_ZdlPv", (void*)(void(*) (void*))operator delete);
 
    registerSymbolFunction("_ZN10TestClass15test2ESs", (void*)getMethodPointer(&TestClass1::test2));
 
    //registerSymbolFunction("_ZN10TestClass15test1Ev", (void *)TestClass1::test1);
 
 
    // Run static constructors.
    LLVM_ExecE->runStaticConstructorsDestructors(LLVM_Module, false);
 
    loadFile("engine_test.bc");
 
    // Run main.
    std::vector<llvm::GenericValue> args;
    LLVM_ExecE->runFunction(LLVM_Module->getFunction("_Z4Initv"), args);
 
    // Run static destructors.
    LLVM_ExecE->runStaticConstructorsDestructors(true);
 
    return 1;
}
 
 
engine_test.cpp:
 
 
#include <string>
 
std::string getPrintText();
 
class TestClass1
{
public:
    static void test1();
    void test2(std::string s);
} testclass1;
 
bool Init()
{
    printf("\n\nStarting bitcode...\n");
 
    //This work on x86 and x64 using LLVM in Release, but crash in x86 Debug
    printf("Result from getPrintText(): %s", getPrintText().data());
    printf("\ngetPrintText() finished!\n");
 
    //This work on x64 using LLVM in Release, but crash in x86 both Release and Debug
    //testclass1.test2(std::string("HI! testclass1"));
    TestClass1 _testclass1;
    _testclass1.test2(std::string("Hello world!"));
 
    printf("\nEnding bitcode...\n\n");
    return true;
}
 
 
And the command i’m using for compiling the bitcode:
 
clang++ -v -O3 -I "../CodeHeaders/VS2010/include" -std=c++11 -emit-llvm "../engine_test.cpp" -c -o engine_test.bc
 
From what i have see it could be a bug with the "getMethodPointer" function (i’m using it for getting the address of the function in a class). Or with the "delete" operator from what i could see on the Visual Studio call stack. Basicaly "_testclass1.test2" is executed, but crash just after, so probably when he get deleted.
 
After trying to test my code with the x64 Debug build i have found that it also crash, but the call stack is different then the x86 Debug build.
 
Call stack using LLVM in x86 Debug:
 
     ntdll.dll!778515de()    
     [Les frames ci-dessous sont peut-être incorrects et/ou manquants, aucun symbole chargé pour ntdll.dll]   
     ntdll.dll!778515de()    
     ntdll.dll!7784014e()    
>    msvcr100d.dll!_write(int fh, const void * buf, unsigned int cnt)  Ligne 83 + 0x9 octets    C
     msvcr100d.dll!_write(int fh, const void * buf, unsigned int cnt)  Ligne 82 + 0xc octets    C
     013df8f4()   
     msvcp100d.dll!std::_Container_base12::_Orphan_all()  Ligne 200    C++
     LLVM_Test_Console.exe!std::_String_val<char,std::allocator<char> >::~_String_val<char,std::allocator<char> >()  Ligne 478 + 0xb octets    C++
     LLVM_Test_Console.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::~basic_string<char,std::char_traits<char>,std::allocator<char> >()  Ligne 754 + 0xf octets    C++
     LLVM_Test_Console.exe!TestClass1::test2(std::basic_string<char,std::char_traits<char>,std::allocator<char> > s)  Ligne 172 + 0xf octets    C++
     001e0091()   
     LLVM_Test_Console.exe!llvm::JIT::runFunction(llvm::Function * F, const std::vector<llvm::GenericValue,std::allocator<llvm::GenericValue> > & ArgValues)  Ligne 455 + 0x7 octets    C++
     LLVM_Test_Console.exe!main(int argc, char * * argv, char * const * envp)  Ligne 254 + 0x43 octets    C++
     LLVM_Test_Console.exe!__tmainCRTStartup()  Ligne 555 + 0x19 octets    C
     LLVM_Test_Console.exe!mainCRTStartup()  Ligne 371    C
     kernel32.dll!76dd339a()    
     ntdll.dll!77869ef2()    
     ntdll.dll!77869ec5()    
 
Call stack using LLVM in x86 Release:
 
     ntdll.dll!778515de()    
     [Les frames ci-dessous sont peut-être incorrects et/ou manquants, aucun symbole chargé pour ntdll.dll]   
     ntdll.dll!778515de()    
     ntdll.dll!7784014e()    
     msvcr100.dll!56a7a5d0()    
     msvcr100.dll!56a70949()    
     msvcr100.dll!56a7f00d()    
     kernel32.dll!76dd14dd()    
     msvcr100.dll!56a7016a()    
     LLVM_Test_Console.exe!TestClass1::test2(std::basic_string<char,std::char_traits<char>,std::allocator<char> > s)  Ligne 172 + 0x10 octets    C++
     00160091()   
>    LLVM_Test_Console.exe!llvm::JIT::runFunction()  + 0x2ed octets    C++
     LLVM_Test_Console.exe!main(int argc, char * * argv, char * const * envp)  Ligne 254 + 0x41 octets    C++
     msvcr100.dll!56a7263d()    
     kernel32.dll!76dd339a()    
     ntdll.dll!77869ef2()    
     ntdll.dll!77869ec5()    
 
Call stack using LLVM in x64 Debug:
 
>    msvcr100d.dll!_output_l(_iobuf * stream, const char * format, localeinfo_struct * plocinfo, char * argptr)  Ligne 1644 + 0x23 octets    C++
     msvcr100d.dll!printf(const char * format, ...)  Ligne 62 + 0x1e octets    C
     LLVM_Test_Console.exe!TestClass1::test2(std::basic_string<char,std::char_traits<char>,std::allocator<char> > * s)  Ligne 170 + 0x1a octets    C++
     00000000003500c7()   
     00000000001eed78()   
     00000000001eed50()   
     00000000003e00d0()   
 
Realy hope you could help me, since i don’t know if it’s bug in my code or in LLVM. :).
 
Thanks you.
 

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Crash using the JIT on x86 but work on x64

Duncan Sands
Hi Skykill,

> Hello everyone, i’m using LLVM (updated to 3.1 after seeing that bug, but it’s
> the same with 3.0) for running a bitcode on a C++ program, and Clang for
> compiling it. My code work perfectly, as expected on x64, but crash on x86. I’m
> on Windows 7 x64 and LLVM + Clang was compiled using Visual Studio 2010 (tested
> in both Release and Debug build). Project was make using CMake.

did you build LLVM with assertions enabled?  -DLLVM_ENABLE_ASSERTIONS=true.  (I
don't recall if cmake Debug build enables assertions by default).

Ciao, Duncan.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Crash using the JIT on x86 but work on x64

Skykill Skyoverside
Hi, so yes assertions are enabled by default in Debug from what i could see
in the CMakeLists.txt

if( uppercase_CMAKE_BUILD_TYPE STREQUAL "RELEASE" )
  option(LLVM_ENABLE_ASSERTIONS "Enable assertions" OFF)
else()
  option(LLVM_ENABLE_ASSERTIONS "Enable assertions" ON)
endif()

Without assertions enabled, when i'm running the program without the
debugger, it simply freeze until i close it, but not crash at all.

Here is the call stack for the x86 Debug build without assertions enabled:

     ntdll.dll!778515de()
     [Les frames ci-dessous sont peut-être incorrects et/ou manquants, aucun
symbole chargé pour ntdll.dll]
     ntdll.dll!778515de()
     ntdll.dll!7784014e()
>    msvcp100d.dll!std::_Lockit::_Lockit(int kind)  Ligne 64 + 0x14 octets
> C++
     msvcp100d.dll!std::_Container_base12::_Orphan_all()  Ligne 200    C++
     LLVM_Test_Console.exe!std::_String_val<char,std::allocator<char>
 >::~_String_val<char,std::allocator<char> >()  Ligne 478 + 0xb octets    C++
     LLVM_Test_Console.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char>
 >::~basic_string<char,std::char_traits<char>,std::allocator<char> >()  Ligne
754 + 0xf octets    C++
     LLVM_Test_Console.exe!TestClass1::test2(std::basic_string<char,std::char_traits<char>,std::allocator<char>
 > s)  Ligne 172 + 0xf octets    C++
     01180091()
     LLVM_Test_Console.exe!llvm::JIT::runFunction(llvm::Function * F, const
std::vector<llvm::GenericValue,std::allocator<llvm::GenericValue> > &
ArgValues)  Ligne 455 + 0x7 octets    C++
     LLVM_Test_Console.exe!main(int argc, char * * argv, char * const *
envp)  Ligne 254 + 0x43 octets    C++
     LLVM_Test_Console.exe!__tmainCRTStartup()  Ligne 555 + 0x19 octets    C
     LLVM_Test_Console.exe!mainCRTStartup()  Ligne 371    C
     kernel32.dll!76dd339a()
     ntdll.dll!77869ef2()
     ntdll.dll!77869ec5()

Thank you.

-----Message d'origine-----
From: Duncan Sands
Sent: Saturday, July 07, 2012 9:22 AM
To: [hidden email]
Subject: Re: [LLVMdev] Crash using the JIT on x86 but work on x64

Hi Skykill,

> Hello everyone, i’m using LLVM (updated to 3.1 after seeing that bug, but
> it’s
> the same with 3.0) for running a bitcode on a C++ program, and Clang for
> compiling it. My code work perfectly, as expected on x64, but crash on
> x86. I’m
> on Windows 7 x64 and LLVM + Clang was compiled using Visual Studio 2010
> (tested
> in both Release and Debug build). Project was make using CMake.

did you build LLVM with assertions enabled?  -DLLVM_ENABLE_ASSERTIONS=true.
(I
don't recall if cmake Debug build enables assertions by default).

Ciao, Duncan.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev 


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Crash using the JIT on x86 but work on x64

Óscar Fuentes
Skykill Skyoverside <[hidden email]> writes:

[snip]

My guess is that you are hitting a VS C++ ABI missing feature in
Clang. Ask in the Clang mailing list for a confirmation, but IIRC there
are still missing pieces on either object construction/destrucion,
parameter passing, returning classes and C++ exception support.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Crash using the JIT on x86 but work on x64

Skykill Skyoverside
Thank, i have posted a mail about that. What i don't understand, is why it
only work on x64, it should be more logical that it work only on x86, since
most library focus first on working x86 before x64.

The function getMethodPointer come from here
http://stackoverflow.com/questions/3104389/can-i-bind-an-existing-method-to-a-llvm-function-and-use-it-from-jitted-code 
they said "that probably will only be compatible with the Itanium ABI", so
does it could be because of that?

-----Message d'origine-----
From: Óscar Fuentes
Sent: Saturday, July 07, 2012 4:57 PM
To: [hidden email]
Subject: Re: [LLVMdev] Crash using the JIT on x86 but work on x64

Skykill Skyoverside <[hidden email]> writes:

[snip]

My guess is that you are hitting a VS C++ ABI missing feature in
Clang. Ask in the Clang mailing list for a confirmation, but IIRC there
are still missing pieces on either object construction/destrucion,
parameter passing, returning classes and C++ exception support.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev 


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Crash using the JIT on x86 but work on x64

Thomson
Which exception caused the crash?  I could take a look if you send me the crash dump file privately.

Thanks,
-Thomson

On Sun, Jul 8, 2012 at 8:36 AM, Skykill Skyoverside <[hidden email]> wrote:
Thank, i have posted a mail about that. What i don't understand, is why it
only work on x64, it should be more logical that it work only on x86, since
most library focus first on working x86 before x64.

The function getMethodPointer come from here
http://stackoverflow.com/questions/3104389/can-i-bind-an-existing-method-to-a-llvm-function-and-use-it-from-jitted-code
they said "that probably will only be compatible with the Itanium ABI", so
does it could be because of that?

-----Message d'origine-----
From: Óscar Fuentes
Sent: Saturday, July 07, 2012 4:57 PM
To: [hidden email]
Subject: Re: [LLVMdev] Crash using the JIT on x86 but work on x64

Skykill Skyoverside <[hidden email]> writes:

[snip]

My guess is that you are hitting a VS C++ ABI missing feature in
Clang. Ask in the Clang mailing list for a confirmation, but IIRC there
are still missing pieces on either object construction/destrucion,
parameter passing, returning classes and C++ exception support.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev