GC infrastructure checked in

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

GC infrastructure checked in

Gordon Henriksen-3
Everything described in GarbageCollection.html should now be live. Phew!

The collectors could use a review, but they compile, have no impact unless used, and tests would otherwise have had to be XFAILed waiting on them.

The ShadowStackCollector has one issue that may be of interest in that it adds constant globals in a runOnFunction context. This is bad in theory, but actually is entirely safe for both JIT and llc from at least a code generation perspective. (Can this invalidate any analyses used during code generation?) Since llc uses a FunctionPassManager, I don't see any way around this, though.

Enjoy!
— Gordon


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Chris Lattner

On Jan 6, 2008, at 6:32 PM, Gordon Henriksen wrote:

Everything described in GarbageCollection.html should now be live. Phew!

The collectors could use a review, but they compile, have no impact unless used, and tests would otherwise have had to be XFAILed waiting on them.

Woot!

The ShadowStackCollector has one issue that may be of interest in that it adds constant globals in a runOnFunction context. This is bad in theory, but actually is entirely safe for both JIT and llc from at least a code generation perspective. (Can this invalidate any analyses used during code generation?) Since llc uses a FunctionPassManager, I don't see any way around this, though.

This should be safe for now, in the future if it becomes a problem, we can figure out a solution.

-Chris

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Jon Harrop
In reply to this post by Gordon Henriksen-3
On Monday 07 January 2008 02:32:47 Gordon Henriksen wrote:

> Everything described in GarbageCollection.html should now be live. Phew!
>
> The collectors could use a review, but they compile, have no impact
> unless used, and tests would otherwise have had to be XFAILed waiting
> on them.
>
> The ShadowStackCollector has one issue that may be of interest in that
> it adds constant globals in a runOnFunction context. This is bad in
> theory, but actually is entirely safe for both JIT and llc from at
> least a code generation perspective. (Can this invalidate any analyses
> used during code generation?) Since llc uses a FunctionPassManager, I
> don't see any way around this, though.

This is wonderful news! Are there any example programs using these GCs?

--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Gordon Henriksen-3
On 2008-01-07, at 05:29, Jon Harrop wrote:

> On Monday 07 January 2008 02:32:47 Gordon Henriksen wrote:
>
>> Everything described in GarbageCollection.html should now be live.  
>> Phew!
>>
>
> This is wonderful news! Are there any example programs using these  
> GCs?

The division of labor is such that the user program must provide the  
stack walker (in addition to the GC), so complete GC programs are  
nontrivial. This is somewhat similar to the state of exception  
handling in LLVM. It would be an excellent project to provide  
reference implementations of these facilities.

That said, the PyPy group has llvmgcroot support on a branch; you  
could ask Armin Rigo <arigo at tunes dot org> for details about  
accessing it.

On his benchmarks, Armin saw an 8% speedup vs. a shadow stack. Their  
gcc backend still outperforms the LLVM backend, though—and llvm-gcc  
outperforms that further still. So perhaps those llvm-gcc GC  
extensions could be put to good use after all! :)

— Gordon


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Carl Friedrich Bolz
Gordon Henriksen wrote:

> On 2008-01-07, at 05:29, Jon Harrop wrote:
>
>> On Monday 07 January 2008 02:32:47 Gordon Henriksen wrote:
>>
>>> Everything described in GarbageCollection.html should now be live.  
>>> Phew!
>>>
>> This is wonderful news! Are there any example programs using these  
>> GCs?
>
> The division of labor is such that the user program must provide the  
> stack walker (in addition to the GC), so complete GC programs are  
> nontrivial. This is somewhat similar to the state of exception  
> handling in LLVM. It would be an excellent project to provide  
> reference implementations of these facilities.
>
> That said, the PyPy group has llvmgcroot support on a branch; you  
> could ask Armin Rigo <arigo at tunes dot org> for details about  
> accessing it.
>
> On his benchmarks, Armin saw an 8% speedup vs. a shadow stack.  Their
> gcc backend still outperforms the LLVM backend, though—and llvm-gcc  
> outperforms that further still.

The second part of this is not correct. I don't think we ever even tried
llvm-gcc on the output of our C backend. What we see on x86 machines is
that LLVM's own code generators produce slower code than using LLVM's C
backend and then running GCC on this. This however is then faster than
our own C backend using GCC. Hm, summary:

    pypy-llvm-backend with llvm's codegen
  < pypy-c-backend with GCC
  < pypy-llvm-backend using llvm's C backend and then GCC

The above is only true when using the Boehm GC, I don't really know the
numbers when using our own GC frameworks (and thus LLVM's GC support). I
guess what we would really like to have is LLVM's codegen to be better
to get rid of the shadow stack we are currently using with our own GCs.

> So perhaps those llvm-gcc GC  
> extensions could be put to good use after all! :)

Right now using LLVM's GC features is not really helpful for us, because
the benefits of having a proper stack walker disappear again due to the
worse code generator of LLVM.

Cheers,

Carl Friedrich Bolz
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Anton Korobeynikov
In reply to this post by Gordon Henriksen-3
Hello, Everyone

> That said, the PyPy group has llvmgcroot support on a branch; you  
> could ask Armin Rigo <arigo at tunes dot org> for details about  
> accessing it.
FYI: some status report can be seen here:
http://codespeak.net/pipermail/pypy-dev/2008q1/004441.html

--
WBR, Anton Korobeynikov
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Gordon Henriksen-3
In reply to this post by Carl Friedrich Bolz
On Jan 7, 2008, at 10:54, Carl Friedrich Bolz wrote:

Gordon Henriksen wrote:

That said, the PyPy group has llvmgcroot support on a branch; you could ask Armin Rigo <arigo at tunes dot org> for details about accessing it.

On his benchmarks, Armin saw an 8% speedup vs. a shadow stack.  Their gcc backend still outperforms the LLVM backend, though—and llvm-gcc outperforms that further still.

The second part of this is not correct. I don't think we ever even tried llvm-gcc on the output of our C backend. What we see on x86 machines is that LLVM's own code generators produce slower code than using LLVM's C backend and then running GCC on this. This however is then faster than our own C backend using GCC.

Thanks for the clarification, Carl.

Hm, summary:

   pypy-llvm-backend with llvm's codegen
 < pypy-c-backend with GCC
 < pypy-llvm-backend using llvm's C backend and then GCC

I wonder what the impact of using llvm-gcc instead of GCC is in this final pipeline. If that actually works, you could indeed eliminate the shadow stack here as well, since llvm-gcc has extensions to emit gcroot intrinsic calls. It's currently hardwired to the shadow-stack codegen, but that's very fixable.

— Gordon


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Carl Friedrich Bolz
Gordon Henriksen wrote:

>> Hm, summary:
>>
>>    pypy-llvm-backend with llvm's codegen
>>  < pypy-c-backend with GCC
>>  < pypy-llvm-backend using llvm's C backend and then GCC
>
> I wonder what the impact of using llvm-gcc instead of GCC is in this
> final pipeline. If that actually works, you could indeed eliminate the
> shadow stack here as well, since llvm-gcc has extensions to emit gcroot
> intrinsic calls. It's currently hardwired to the shadow-stack codegen,
> but that's very fixable.

How would that help? Using llvm-gcc would still give you LLVM's code
generator which seems to be not optimal for what PyPy does (for whatever
  reason).

Besides, PyPy cannot get rid of its own shadow-stack implementation as
long as LLVM is not the leading compiler on most platforms :-).

Cheers,

Carl Friedrich Bolz
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: GC infrastructure checked in

Gordon Henriksen-3
On 2008-01-08, at 08:12, Carl Friedrich Bolz wrote:

> Gordon Henriksen wrote:
>>> Hm, summary:
>>>
>>>   pypy-llvm-backend with llvm's codegen
>>> < pypy-c-backend with GCC
>>> < pypy-llvm-backend using llvm's C backend and then GCC
>>
>> I wonder what the impact of using llvm-gcc instead of GCC is in  
>> this final pipeline. If that actually works, you could indeed  
>> eliminate the shadow stack here as well, since llvm-gcc has  
>> extensions to emit gcroot intrinsic calls. It's currently hardwired  
>> to the shadow-stack codegen, but that's very fixable.
>
> How would that help? Using llvm-gcc would still give you LLVM's code  
> generator which seems to be not optimal for what PyPy does (for  
> whatever reason).

In the event that the translation via C is a significant factor.

— Gordon


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev