Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Chandler Carruth
(sorry for the cross post, but this issue effects lots of people)

Takumi diagnosed a problem when Zach switched LLVM to use compile-time multithreading, but it actually has little to do with the multithreading decisions.

LLVM is rapidly moving to use more C++11 library features and more multithreading features. I think this is a good thing. However, we're hitting a specific issue on cygwin when bootstrapping: Clang doesn't support the Cygwin __thread implementation (it's different... sadly...). As a consequence, a *huge* amount of libstdc++'s multithreading primitives when bootstrapping, including <mutex>.

I don't think the current situation is tenable -- we really shouldn't be increasingly using C++11 in LLVM while trying to keep all of the multithreading code away from the C++11 standard library. At some point, we will fail because some other part of the library needed <mutex> (or something), and get a really surprising breakage.

I see three options:
1) Give up on cygwin bootstrapping (I don't really like this, and I suspect there are users of it)
2) Switch cygwin bootstrap to libc++ and fix libc++ to not require __thread (seems rather gross and lots of work)
3) Implement __thread in cygwin mode ASAP.

Thoughts? I think we need to address this pretty quickly or we're just going to start running into more failure modes.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

C Bergström
On 06/25/14 06:14 PM, Chandler Carruth wrote:
>
> I see three options:
> 1) Give up on cygwin bootstrapping (I don't really like this, and I
> suspect there are users of it)
peanut gallery vote, but I'd go option 1 or possibly option 2. If you
compared the time/value of effort for improving something cygwin
specific vs say native compatibility.. Is cygwin really that important
of an investment?

If implementing a stable __thread for cygwin takes 1 day I doubt you'd
even ask - if you're talking about 2-3 weeks of work and you're asking
for feedback - I'd go with someone which has more value. I personally
don't see cygwin support being clang's sweetspot or "killer" feature on
Windows.

If there's no downside and it's just code churn - is there a deeper
question here?
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [cfe-dev] Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Chandler Carruth

On Wed, Jun 25, 2014 at 1:37 PM, "C. Bergström" <[hidden email]> wrote:
On 06/25/14 06:14 PM, Chandler Carruth wrote:

I see three options:
1) Give up on cygwin bootstrapping (I don't really like this, and I suspect there are users of it)
peanut gallery vote, but I'd go option 1 or possibly option 2. If you compared the time/value of effort for improving something cygwin specific vs say native compatibility.. Is cygwin really that important of an investment?

If it were just me, I would agree. I think the native windows work is far more important.

But I'm trying to respect the fact that there may well be a large set of existing users of cygwin.

To my mind, the real question is: do they really need to bootstrap? I don't know the answer, because I'm not one of the users.


(I also don't really know how long it would take, I don't even know how it works and have only found random documents psuedo describing it...)

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Renato Golin-2
In reply to this post by Chandler Carruth
On 25 June 2014 12:14, Chandler Carruth <[hidden email]> wrote:
> 3) Implement __thread in cygwin mode ASAP.

I wasn't involved in the original discussion, but this seems to be the
only solution that doesn't limit scope. Giving up the possibility to
use other C++ libraries, or Cygwin support altogether seems rather
limiting...

--renato
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Chandler Carruth

On Wed, Jun 25, 2014 at 2:07 PM, Renato Golin <[hidden email]> wrote:
On 25 June 2014 12:14, Chandler Carruth <[hidden email]> wrote:
> 3) Implement __thread in cygwin mode ASAP.

I wasn't involved in the original discussion, but this seems to be the
only solution that doesn't limit scope.

Yes, but who is going to step up to do the work? I don't use cygwin, and don't even have a system to test on. I'm a little worried about the project just being stalled here waiting on someone to implement
 
Giving up the possibility to
use other C++ libraries, or Cygwin support altogether seems rather
limiting...

The only limitation is one we've had for a long time and users on Cygwin are already living with...

The only new factor is that Clang hits the limitation in its bootstrap.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Renato Golin-2
On 25 June 2014 13:12, Chandler Carruth <[hidden email]> wrote:
> Yes, but who is going to step up to do the work? I don't use cygwin, and
> don't even have a system to test on. I'm a little worried about the project
> just being stalled here waiting on someone to implement

I was referring to the technical side, but yes, the answer is probably
the same for all seemingly unsupported features in the past: claim it
or lose it.

cheers,
--renato
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

NAKAMURA Takumi
In reply to this post by Chandler Carruth
I reverted <mutex>, since also clang-native-mingw32-win7 had been broken.
There were two ways of issues, in mingw32 (not w64) and cygwin.

For mingw32, it seems <mutex> is configured without __thread. Some
features, like "once", are unavailable.
When we reintroduce <mutex> in the tree, we should make mingw32
builder obsoleted or migrated to mingw-w64.

For Cygwin, I can suggest an option,

3a) Implement __thread in near future. Till then,

  3a-1) Give up bootstrapping. Cygwin-g++-4.8 can build the tree, though.
  3a-2) Apply local patch in the builder to suppress unsupported
features like <mutex>.

Conclusion: Go ahead!
I supposed, when I reverted <mutex>, it were trivial. I won't disturb
any more if we are going toward c++11.


2014-06-25 20:14 GMT+09:00 Chandler Carruth <[hidden email]>:

> (sorry for the cross post, but this issue effects lots of people)
>
> Takumi diagnosed a problem when Zach switched LLVM to use compile-time
> multithreading, but it actually has little to do with the multithreading
> decisions.
>
> LLVM is rapidly moving to use more C++11 library features and more
> multithreading features. I think this is a good thing. However, we're
> hitting a specific issue on cygwin when bootstrapping: Clang doesn't support
> the Cygwin __thread implementation (it's different... sadly...). As a
> consequence, a *huge* amount of libstdc++'s multithreading primitives when
> bootstrapping, including <mutex>.
>
> I don't think the current situation is tenable -- we really shouldn't be
> increasingly using C++11 in LLVM while trying to keep all of the
> multithreading code away from the C++11 standard library. At some point, we
> will fail because some other part of the library needed <mutex> (or
> something), and get a really surprising breakage.
>
> I see three options:
> 1) Give up on cygwin bootstrapping (I don't really like this, and I suspect
> there are users of it)
> 2) Switch cygwin bootstrap to libc++ and fix libc++ to not require __thread
> (seems rather gross and lots of work)
> 3) Implement __thread in cygwin mode ASAP.
>
> Thoughts? I think we need to address this pretty quickly or we're just going
> to start running into more failure modes.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [cfe-dev] Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Chandler Carruth

On Wed, Jun 25, 2014 at 4:01 PM, NAKAMURA Takumi <[hidden email]> wrote:
I reverted <mutex>, since also clang-native-mingw32-win7 had been broken.
There were two ways of issues, in mingw32 (not w64) and cygwin.

For mingw32, it seems <mutex> is configured without __thread. Some
features, like "once", are unavailable.
When we reintroduce <mutex> in the tree, we should make mingw32
builder obsoleted or migrated to mingw-w64.

For Cygwin, I can suggest an option,

3a) Implement __thread in near future. Till then,

  3a-1) Give up bootstrapping. Cygwin-g++-4.8 can build the tree, though.
  3a-2) Apply local patch in the builder to suppress unsupported
features like <mutex>.

Conclusion: Go ahead!
I supposed, when I reverted <mutex>, it were trivial. I won't disturb
any more if we are going toward c++11.

Ok. Unless someone pipes up, I'm going to say we go with mingw-w64 and 3a-1 (no bootstrapping on cygwin) until someone shows up to do 3a.

Galina, Takumi, let me know when your builders are OK for this change. No hurry here, I want to give folks time to request a different path as well, but I'd rather make the changes only after the builders are good to make sure there aren't other problems waiting.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [cfe-dev] Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Reid Kleckner-2
In reply to this post by NAKAMURA Takumi
On Wed, Jun 25, 2014 at 7:01 AM, NAKAMURA Takumi <[hidden email]> wrote:
I reverted <mutex>, since also clang-native-mingw32-win7 had been broken.
There were two ways of issues, in mingw32 (not w64) and cygwin.

For mingw32, it seems <mutex> is configured without __thread. Some
features, like "once", are unavailable.
When we reintroduce <mutex> in the tree, we should make mingw32
builder obsoleted or migrated to mingw-w64.

Other users reported that mingw64 doesn't support <mutex> either.  IMO it's important to support the most popular distribution of GCC on Windows, which I believe is mingw64 these days.  I'm fine with dropping Cygwin self-hosting support if that's the only thing blocking us from using std::mutex, but I think mingw64 is the real blocker.
 
For Cygwin, I can suggest an option,

3a) Implement __thread in near future. Till then,

  3a-1) Give up bootstrapping. Cygwin-g++-4.8 can build the tree, though.
  3a-2) Apply local patch in the builder to suppress unsupported
features like <mutex>.

Conclusion: Go ahead!
I supposed, when I reverted <mutex>, it were trivial. I won't disturb
any more if we are going toward c++11.

Thanks for that, it's important to keep the bots green.  The bot notifications were probably lost in the ether, and I didn't think to check them manually.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [cfe-dev] Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Óscar Fuentes
Reid Kleckner <[hidden email]> writes:

> Other users reported that mingw64 doesn't support <mutex> either.  IMO it's
> important to support the most popular distribution of GCC on Windows, which
> I believe is mingw64 these days.  I'm fine with dropping Cygwin
> self-hosting support if that's the only thing blocking us from using
> std::mutex, but I think mingw64 is the real blocker.

mingw-w64 comes on several combinations depending on exception handling
(sjlj/dwarf/etc), platform (32/64 bits) and thread support
(win32/posix). The threads-win32 variants lacks std::thread support but
the threads-posix variants uses the implementation provided by
libstdc++. So the reports you received about mingw-w64 not supporting
<mutex> most likely comes from people using the threads-win32 variants.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: Cygwin bootstrap is incompatible with <mutex> (and most other threading C++11 libraries)

Gabriel Dos Reis
In reply to this post by Chandler Carruth
4) since you are doing the native windows work, why not just go with mingw(64), and use that  as a native compiler under cygwin?


On Wed, Jun 25, 2014 at 4:14 AM, Chandler Carruth <[hidden email]> wrote:
(sorry for the cross post, but this issue effects lots of people)

Takumi diagnosed a problem when Zach switched LLVM to use compile-time multithreading, but it actually has little to do with the multithreading decisions.

LLVM is rapidly moving to use more C++11 library features and more multithreading features. I think this is a good thing. However, we're hitting a specific issue on cygwin when bootstrapping: Clang doesn't support the Cygwin __thread implementation (it's different... sadly...). As a consequence, a *huge* amount of libstdc++'s multithreading primitives when bootstrapping, including <mutex>.

I don't think the current situation is tenable -- we really shouldn't be increasingly using C++11 in LLVM while trying to keep all of the multithreading code away from the C++11 standard library. At some point, we will fail because some other part of the library needed <mutex> (or something), and get a really surprising breakage.

I see three options:
1) Give up on cygwin bootstrapping (I don't really like this, and I suspect there are users of it)
2) Switch cygwin bootstrap to libc++ and fix libc++ to not require __thread (seems rather gross and lots of work)
3) Implement __thread in cygwin mode ASAP.

Thoughts? I think we need to address this pretty quickly or we're just going to start running into more failure modes.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev