JIT Leaks?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

JIT Leaks?

Paolo Invernizzi
Hi all,

I'm playing with the JIT trying to find out the best way to handle a  
genetic algorithm that we are developing: I need to:

1) generate about 50.000 functions a time
2) JIT them and execute them over a large dataset
3) Discard everything
4) ... stuff...
5) Loop to 1 again and again and again...

I've the code running, under LLVM 2.0 on OS X 10.4.10 but it leaks,  
so I started to tweak the standard "fibonacci" example to understand  
how to do in a proper way the step 3.

- Simply deleting the execution engine instance in the fibonacci code  
and looping all the main code give me memory leaks. Can anybody  
confirm this?

- Conserving the module, the execution engine and the module  
provider, but releasing the fibonacci function with:

EE->freeMachineCodeForFunction(FibF);
FibF->eraseFromParent();

and recreating the function from scratch leaks even more. Idem,  
someone can confirm this?

Thanks in advance.
Paolo Invernizzi

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Anton Korobeynikov
Hello, Paolo.

> and recreating the function from scratch leaks even more. Idem,  
> someone can confirm this?
Could you please provide some example? I can try to valgrind then.

--
With best regards, Anton Korobeynikov.

Faculty of Mathematics & Mechanics, Saint Petersburg State University.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Holger Schurig-2
In reply to this post by Paolo Invernizzi
> - Simply deleting the execution engine instance in the fibonacci code  
> and looping all the main code give me memory leaks. Can anybody  
> confirm this?
>
> - Conserving the module, the execution engine and the module  
> provider, but releasing the fibonacci function with:

You can find out what exactly leaks with the help of valgrind.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Anton Korobeynikov
In reply to this post by Paolo Invernizzi
Holger.

> You can find out what exactly leaks with the help of valgrind.
It seems, that Paolo is on Mac OS X. No valgrind there :(

--
With best regards, Anton Korobeynikov.

Faculty of Mathematics & Mechanics, Saint Petersburg State University.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Gordon Henriksen-3
On 2007-07-14, at 13:56, Anton Korobeynikov wrote:

>> You can find out what exactly leaks with the help of valgrind.
>
> It seems, that Paolo is on Mac OS X. No valgrind there :(

All is not lost…

http://developer.apple.com/documentation/Performance/Conceptual/ 
ManagingMemory/Articles/FindingLeaks.html

— Gordon


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Bill Wendling
In reply to this post by Anton Korobeynikov
On Jul 14, 2007, at 10:56 AM, Anton Korobeynikov wrote:

> Holger.
>
>> You can find out what exactly leaks with the help of valgrind.
> It seems, that Paolo is on Mac OS X. No valgrind there :(
>
On OS X, you can use the "leaks" command to find out what's leaked in  
a particular process. It's not exact, but may give a good indication  
of what's wrong.

-bw

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Paolo Invernizzi
In reply to this post by Gordon Henriksen-3
First, I'm not sure if deleting the ExecutionEngine is all I need to  
clean-up... so I started with a minimal test just to check

int main( int argc, char **argv ){
     while( true ){
         Module *M = new Module("M");
         Function *F =  cast<Function>(M->getOrInsertFunction("F",  
Type::Int32Ty, (Type*)0));
         BasicBlock *BB = new BasicBlock("BB",F);
         new ReturnInst( ConstantInt::get(Type::Int32Ty,1), BB );
         std::vector<GenericValue> Args(0);
         ExistingModuleProvider *MP = new ExistingModuleProvider(M);
         ExecutionEngine *EE = ExecutionEngine::create(MP, false);
         GenericValue GV = EE->runFunction(F, Args);
         delete EE;
     }
}

The memory goes up, and OS X leaks command returns pages of:

Leak: 0x011036b0  size=16
         0x00000000 0x00000000 0x00000000  
0x01102988     .............)..
Leak: 0x011036a0  size=16
         0x00000000 0x00000000 0x00000000  
0x01102988     .............)..
Leak: 0x01103680  size=16
         0x00000000 0x00000000 0x00000000  
0x01102c78     ............x,..
Leak: 0x01103670  size=16
         0x00000000 0x00000000 0x00000000 0x01102728     ............
('..
Leak: 0x01103640  size=16
         0x00000000 0x00000000 0x00000000  
0x01102978     ............x)..
Leak: 0x01103630  size=16
         0x00000000 0x00000000 0x00000000  
0x01103368     ............h3..
Leak: 0x01103620  size=16
         0x00000000 0x00000000 0x00000000  
0x01102a58     ............X*..
Leak: 0x01103610  size=16
         0x00000000 0x00000000 0x00000000  
0x01103368     ............h3..

So all I know is that It's a size 16 leak *grin*.

But MallocDebug reports as leaks

2.4M start
2.4M operator new(unsigned long)
2.4M MDmalloc
2.4M main
2.4M llvm::PMDataManager::add(llvm::Pass*, bool)
2.4M llvm::LoopPass::assignPassManager(llvm::PMStack&,  
llvm::PassManagerType)
2.4M llvm::LLVMTargetMachine::addPassesToEmitMachineCode
(llvm::FunctionPassManager&, llvm::MachineCodeEmitter&, bool)
2.4M llvm::JIT::JIT[in-charge](llvm::ModuleProvider*,  
llvm::TargetMachine&, llvm::TargetJITInfo&)
2.4M llvm::JIT::create(llvm::ModuleProvider*, std::basic_string<char,  
std::char_traits<char>, std::allocator<char> >*)
2.4M llvm::FunctionPass::assignPassManager(llvm::PMStack&,  
llvm::PassManagerType)
2.4M llvm::ExecutionEngine::create(llvm::ModuleProvider*, bool,  
std::basic_string<char, std::char_traits<char>, std::allocator<char> >*)
2.4M _start
2.4M 0x1
2.4M llvm::PMTopLevelManager::schedulePass(llvm::Pass*)
2.4M llvm::FunctionPassManagerImpl::addTopLevelPass(llvm::Pass*)

I hope this is usefull, I'm pretty new to OS X.

Cheers, Paolo Invernizzi

On 14/lug/07, at 22:29, Gordon Henriksen wrote:

> On 2007-07-14, at 13:56, Anton Korobeynikov wrote:
>
>>> You can find out what exactly leaks with the help of valgrind.
>>
>> It seems, that Paolo is on Mac OS X. No valgrind there :(
>
> All is not lost…
>
> http://developer.apple.com/documentation/Performance/Conceptual/
> ManagingMemory/Articles/FindingLeaks.html
>
> — Gordon
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Anton Korobeynikov
In reply to this post by Gordon Henriksen-3
Hello, Paolo.

> I hope this is usefull, I'm pretty new to OS X.
Well. From your example you have got the only valgrind results:

==28336== 58 bytes in 2 blocks are possibly lost in loss record 70 of
127
==28336==    at 0x4020A92: operator new(unsigned)
(in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==28336==    by 0x410010B: std::string::_Rep::_S_create(unsigned,
unsigned, std::allocator<char> const&)
(in /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/libstdc++.so.6.0.3)
==28336==    by 0x4102A04:
(within /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/libstdc++.so.6.0.3)
==28336==    by 0x4102B25: std::string::string(char const*,
std::allocator<char> const&)
(in /usr/lib/gcc/i686-pc-linux-gnu/3.4.6/libstdc++.so.6.0.3)
==28336==    by 0x85530D6:
__static_initialization_and_destruction_0(int, int)
(MachineFunction.cpp:40)
==28336==    by 0x855311D:
_GLOBAL__I__ZN4llvm19MachineFunctionPass6virtfnEv
(MachineFunction.cpp:484)
==28336==    by 0x86E1A37:
(within /home/asl/proj/llvm/build_debug/Debug/examples/JITLeak)
==28336==    by 0x8355D50:
(within /home/asl/proj/llvm/build_debug/Debug/examples/JITLeak)
==28336==    by 0x86E197A: __libc_csu_init
(in /home/asl/proj/llvm/build_debug/Debug/examples/JITLeak)
==28336==    by 0x418E7F3: (below main) (in /lib/libc-2.5.so)

--
With best regards, Anton Korobeynikov.

Faculty of Mathematics & Mechanics, Saint Petersburg State University.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Chris Lattner
In reply to this post by Paolo Invernizzi
On Sun, 15 Jul 2007, Paolo Invernizzi wrote:
> First, I'm not sure if deleting the ExecutionEngine is all I need to
> clean-up... so I started with a minimal test just to check

Is this llvm 2.0 or llvm svn head?  Several minor memory leaks have been
fixed since llvm 2.0.

-Chris

> int main( int argc, char **argv ){
>     while( true ){
>         Module *M = new Module("M");
>         Function *F =  cast<Function>(M->getOrInsertFunction("F",
> Type::Int32Ty, (Type*)0));
>         BasicBlock *BB = new BasicBlock("BB",F);
>         new ReturnInst( ConstantInt::get(Type::Int32Ty,1), BB );
>         std::vector<GenericValue> Args(0);
>         ExistingModuleProvider *MP = new ExistingModuleProvider(M);
>         ExecutionEngine *EE = ExecutionEngine::create(MP, false);
>         GenericValue GV = EE->runFunction(F, Args);
>         delete EE;
>     }
> }
>
> The memory goes up, and OS X leaks command returns pages of:
>
> Leak: 0x011036b0  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01102988     .............)..
> Leak: 0x011036a0  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01102988     .............)..
> Leak: 0x01103680  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01102c78     ............x,..
> Leak: 0x01103670  size=16
>         0x00000000 0x00000000 0x00000000 0x01102728     ............
> ('..
> Leak: 0x01103640  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01102978     ............x)..
> Leak: 0x01103630  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01103368     ............h3..
> Leak: 0x01103620  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01102a58     ............X*..
> Leak: 0x01103610  size=16
>         0x00000000 0x00000000 0x00000000
> 0x01103368     ............h3..
>
> So all I know is that It's a size 16 leak *grin*.
>
> But MallocDebug reports as leaks
>
> 2.4M start
> 2.4M operator new(unsigned long)
> 2.4M MDmalloc
> 2.4M main
> 2.4M llvm::PMDataManager::add(llvm::Pass*, bool)
> 2.4M llvm::LoopPass::assignPassManager(llvm::PMStack&,
> llvm::PassManagerType)
> 2.4M llvm::LLVMTargetMachine::addPassesToEmitMachineCode
> (llvm::FunctionPassManager&, llvm::MachineCodeEmitter&, bool)
> 2.4M llvm::JIT::JIT[in-charge](llvm::ModuleProvider*,
> llvm::TargetMachine&, llvm::TargetJITInfo&)
> 2.4M llvm::JIT::create(llvm::ModuleProvider*, std::basic_string<char,
> std::char_traits<char>, std::allocator<char> >*)
> 2.4M llvm::FunctionPass::assignPassManager(llvm::PMStack&,
> llvm::PassManagerType)
> 2.4M llvm::ExecutionEngine::create(llvm::ModuleProvider*, bool,
> std::basic_string<char, std::char_traits<char>, std::allocator<char> >*)
> 2.4M _start
> 2.4M 0x1
> 2.4M llvm::PMTopLevelManager::schedulePass(llvm::Pass*)
> 2.4M llvm::FunctionPassManagerImpl::addTopLevelPass(llvm::Pass*)
>
> I hope this is usefull, I'm pretty new to OS X.
>
> Cheers, Paolo Invernizzi
>
> On 14/lug/07, at 22:29, Gordon Henriksen wrote:
>
>> On 2007-07-14, at 13:56, Anton Korobeynikov wrote:
>>
>>>> You can find out what exactly leaks with the help of valgrind.
>>>
>>> It seems, that Paolo is on Mac OS X. No valgrind there :(
>>
>> All is not lost…
>>
>> http://developer.apple.com/documentation/Performance/Conceptual/
>> ManagingMemory/Articles/FindingLeaks.html
>>
>> — Gordon
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-Chris

--
http://nondot.org/sabre/
http://llvm.org/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: JIT Leaks?

Paolo Invernizzi
Chris,

It's the 2.0. I'll try the head as soon as I can (I installed the 2.0  
via macports).
All started because I was not sure if deleting the ExecutionEngine is  
all I have to do to cleanup the LLVM stuff.

Cheers, Paolo

On Jul 15, 2007, at 9:33 PM, Chris Lattner wrote:

> On Sun, 15 Jul 2007, Paolo Invernizzi wrote:
>> First, I'm not sure if deleting the ExecutionEngine is all I need to
>> clean-up... so I started with a minimal test just to check
>
> Is this llvm 2.0 or llvm svn head?  Several minor memory leaks have  
> been fixed since llvm 2.0.
>
> -Chris
>
>> int main( int argc, char **argv ){
>>     while( true ){
>>         Module *M = new Module("M");
>>         Function *F =  cast<Function>(M->getOrInsertFunction("F",
>> Type::Int32Ty, (Type*)0));
>>         BasicBlock *BB = new BasicBlock("BB",F);
>>         new ReturnInst( ConstantInt::get(Type::Int32Ty,1), BB );
>>         std::vector<GenericValue> Args(0);
>>         ExistingModuleProvider *MP = new ExistingModuleProvider(M);
>>         ExecutionEngine *EE = ExecutionEngine::create(MP, false);
>>         GenericValue GV = EE->runFunction(F, Args);
>>         delete EE;
>>     }
>> }
>>
>> The memory goes up, and OS X leaks command returns pages of:
>>
>> Leak: 0x011036b0  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01102988     .............)..
>> Leak: 0x011036a0  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01102988     .............)..
>> Leak: 0x01103680  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01102c78     ............x,..
>> Leak: 0x01103670  size=16
>>         0x00000000 0x00000000 0x00000000 0x01102728     ............
>> ('..
>> Leak: 0x01103640  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01102978     ............x)..
>> Leak: 0x01103630  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01103368     ............h3..
>> Leak: 0x01103620  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01102a58     ............X*..
>> Leak: 0x01103610  size=16
>>         0x00000000 0x00000000 0x00000000
>> 0x01103368     ............h3..
>>
>> So all I know is that It's a size 16 leak *grin*.
>>
>> But MallocDebug reports as leaks
>>
>> 2.4M start
>> 2.4M operator new(unsigned long)
>> 2.4M MDmalloc
>> 2.4M main
>> 2.4M llvm::PMDataManager::add(llvm::Pass*, bool)
>> 2.4M llvm::LoopPass::assignPassManager(llvm::PMStack&,
>> llvm::PassManagerType)
>> 2.4M llvm::LLVMTargetMachine::addPassesToEmitMachineCode
>> (llvm::FunctionPassManager&, llvm::MachineCodeEmitter&, bool)
>> 2.4M llvm::JIT::JIT[in-charge](llvm::ModuleProvider*,
>> llvm::TargetMachine&, llvm::TargetJITInfo&)
>> 2.4M llvm::JIT::create(llvm::ModuleProvider*, std::basic_string<char,
>> std::char_traits<char>, std::allocator<char> >*)
>> 2.4M llvm::FunctionPass::assignPassManager(llvm::PMStack&,
>> llvm::PassManagerType)
>> 2.4M llvm::ExecutionEngine::create(llvm::ModuleProvider*, bool,
>> std::basic_string<char, std::char_traits<char>,  
>> std::allocator<char> >*)
>> 2.4M _start
>> 2.4M 0x1
>> 2.4M llvm::PMTopLevelManager::schedulePass(llvm::Pass*)
>> 2.4M llvm::FunctionPassManagerImpl::addTopLevelPass(llvm::Pass*)
>>
>> I hope this is usefull, I'm pretty new to OS X.
>>
>> Cheers, Paolo Invernizzi
>>
>> On 14/lug/07, at 22:29, Gordon Henriksen wrote:
>>
>>> On 2007-07-14, at 13:56, Anton Korobeynikov wrote:
>>>
>>>>> You can find out what exactly leaks with the help of valgrind.
>>>>
>>>> It seems, that Paolo is on Mac OS X. No valgrind there :(
>>>
>>> All is not lost…
>>>
>>> http://developer.apple.com/documentation/Performance/Conceptual/
>>> ManagingMemory/Articles/FindingLeaks.html
>>>
>>> — Gordon
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> [hidden email]         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
> -Chris
>
> --
> http://nondot.org/sabre/
> http://llvm.org/_______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev