[llvm-dev] using emulated-tls on Darwin 8, 9, 10

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev
Please excuse hobbiest-level question.

Darwin 11+ enables thread_local variables using system-level supports.

I have an interest in enabling TLS on darwin < 11  using emulated-tls. This can be enabled with a few modest patches:

==========================
--- a/include/llvm/ADT/Triple.h.orig 2018-10-02 17:38:10.000000000 -0700
+++ b/include/llvm/ADT/Triple.h 2018-10-02 17:38:58.000000000 -0700
@@ -682,7 +682,7 @@

 /// Tests whether the target uses emulated TLS as default.
 bool hasDefaultEmulatedTLS() const {
-    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment();
+    return isAndroid() || isOSOpenBSD() || isWindowsCygwinEnvironment() || isMacOSXVersionLT(10, 7);
 }

 /// @}
==========================
--- a/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp.orig 2018-10-02 18:31:17.000000000 -0700
+++ b/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp 2018-10-02 18:32:35.000000000 -0700
@@ -2255,7 +2255,7 @@
 const char *Name = "__cxa_atexit";
 if (TLS) {
   const llvm::Triple &T = CGF.getTarget().getTriple();
-    Name = T.isOSDarwin() ?  "_tlv_atexit" : "__cxa_thread_atexit";
+    Name = (T.isOSDarwin() && !T.isMacOSXVersionLT(10, 7)) ?  "_tlv_atexit" : "__cxa_thread_atexit";
 }

 // We're assuming that the destructor function is something we can
--- a/tools/clang/lib/Basic/Targets/OSTargets.h.orig 2018-10-02 17:14:10.000000000 -0700
+++ b/tools/clang/lib/Basic/Targets/OSTargets.h 2018-10-02 17:14:41.000000000 -0700
@@ -93,7 +93,7 @@
   this->TLSSupported = false;

   if (Triple.isMacOSX())
-      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 7);
+      this->TLSSupported = !Triple.isMacOSXVersionLT(10, 4);
   else if (Triple.isiOS()) {
     // 64-bit iOS supported it from 8 onwards, 32-bit device from 9 onwards,
     // 32-bit simulator from 10 onwards.
==========================


and in doing so, when clang links against newer versions of libgcc , thread_local support is available. (In MacPorts, this is done using -stdlib=macports-libstdc++, which links against a new version of libgcc from gcc8).



I would like to enable this while using libc++ as well, however.

To do this, libc++abi is built with cxa_thread_atexit.cpp added, and emutls.c is built and used in libclang_rt.10.4.a.

However I am having an issue wherein the symbol ___emutls_get_address from lilbclang_rt_10.4.a is not visible to libc++abi.dylib at runtime.


"___emutls_get_address"  is built into clang_rt:

$ nm /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a  | grep emu
/opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a(emutls.c.o):
0000000000000000 T ___emutls_get_address
00000000000001a0 t _emutls_init
0000000000000210 d _emutls_init_once.once
00000000000001c0 t _emutls_key_destructor
0000000000000220 d _emutls_mutex
00000000000002f8 b _emutls_num_object
0000000000000300 b _emutls_pthread_key

The objects appear to be included in the built executables (a simple example using TLS 4.cpp made into "4"),  but the symbols are apparently not visible to libc++abi.dylib at runtime.

$ nm 4 | grep emu
0000000100007070 t ___emutls_get_address
0000000100009480 d ___emutls_v._ZGVZL18thread_with_accessPvE23counting_function_local
0000000100009400 d ___emutls_v._ZL15counting_static
0000000100009420 d ___emutls_v._ZN12_GLOBAL__N_128counting_anonymous_namespaceE
0000000100009460 d ___emutls_v._ZZL18thread_with_accessPvE23counting_function_local
00000001000094a0 d ___emutls_v.__tls_guard
0000000100009440 d ___emutls_v.counting_extern
0000000100007200 t _emutls_init
00000001000094c0 d _emutls_init_once.once
0000000100007220 t _emutls_key_destructor
00000001000094d0 d _emutls_mutex
0000000100009538 b _emutls_num_object
0000000100009540 b _emutls_pthread_key


$ ./4
info: testing pthread_create
dyld: lazy symbol binding failed: Symbol not found: ___emutls_get_address
 Referenced from: /usr/lib/libc++abi.dylib
 Expected in: flat namespace

dyld: Symbol not found: ___emutls_get_address
 Referenced from: /usr/lib/libc++abi.dylib
 Expected in: flat namespace

Trace/BPT trap


Adding -Wl,-flat_namespace does not make the symbols visible.


If I try to fix it like this by forcing the symbol to be exported, I get a clue:

$ clang++ -std=c++11 -stdlib=libc++ -Wl,-exported_symbol,___emutls_get_address  -o 4 4.cpp
ld: warning: cannot export hidden symbol ___emutls_get_address from /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.osx.a(emutls.c.o)


Things do work apparently properly however if I build emutls.c directly into libc++abi.dylib, using a slightly modified version of libcxxabi that includes the emutls.c source file in the build:

$ nm libc++abi.kenspecial.20181206.dylib | grep emutls_
0000000000020054 T ___emutls_get_address
0000000000025458 s ___emutls_t._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
000000000002a270 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_111dtors_aliveE
000000000002a290 d ___emutls_v._ZN10__cxxabiv112_GLOBAL__N_15dtorsE
000000000002a2b0 d _emutls_get_index.once
00000000000201c8 t _emutls_init
00000000000201ea t _emutls_key_destructor
000000000002a2c0 d _emutls_mutex
000000000002a360 b _emutls_num_object
000000000002a368 b _emutls_pthread_key


and with this modified libc++abi.dylib, all the symbols are found, things work correctly, and all the tests that should pass in the llvm "testit" suite do pass.


I presume that the visibility of ___emutls_get_address in libclang_rt.osx.a is being controlled by some mechanism (apple_versioning.c ?).

I notice that the library is built with -fvisibility=hidden

1) is my approach here generally reasonable?

2) is there a toggle to allow the ___emutls_get_address symbol in  libclang_rt.10.4.a to be made visible to libc++abi.dylib ?

Best,

Ken
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev
On 2018-12-07 22:30, Ken Cunningham via llvm-dev wrote:
> Please excuse hobbiest-level question.
>
> Darwin 11+ enables thread_local variables using system-level supports.
>
> I have an interest in enabling TLS on darwin < 11  using emulated-tls.

Is anyone still running macOS 10.6 or older?

--
/Jacob Carlborg

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev
> On 2018-12-07 22:30, Ken Cunningham via llvm-dev wrote:
>> Please excuse hobbiest-level question.
>> Darwin 11+ enables thread_local variables using system-level supports.
>> I have an interest in enabling TLS on darwin < 11  using emulated-tls.
>
> Is anyone still running macOS 10.6 or older?
>
> --
> /Jacob Carlborg
>


[off topic, apologies]
Yes, there remains a surprising amount of interest in these systems. MacPorts sees very frequent user tickets about them, many tickets lately about thread_local. All versions of clang/llvm run on 10.5+ Intel systems with minor tweaks, including the latest trunk versions, and  this enables almost all current opensource software to be used on these systems. Software (like browsers) geared to these systems see downloads in the low thousands per release. I guess people either have a need for these systems for some software version or workflow, or have otherwise workable hardware that as been obsoleted by newer os requirements.

Even PowerPC systems get frequent tickets at MacPorts, and we support them as we can. Some versions of clang will run on PowerPC Macs with workable coverage (in fact the only remaining hole is a persisting issue with a bad address for c++ exceptions being supplied).

[/on topic]



Making the emutls objects in libclang_rt visible to libcxxabi.dylib at runtime has been a hiccup, tho.

It works but seems a shame to hack emutls.c into the libc++abi code when the objects are already sitting there in the executable, but (AFAICT) have the wrong visibility setting to allow use.

At present I don't seem to know enough about the underpinnings to see how to change the visibility settings on the emutls objects from libclang_rt when placed into the executable.

Appreciate any thoughts on this, thanks for your interest, and sorry for the slight divergence above.

Ken (MacPorts dev)
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev

On 2018-12-08, at 8:06 AM, Ken Cunningham wrote:


> At present I don't seem to know enough about the underpinnings to see how to change the visibility settings on the emutls objects from libclang_rt when placed into the executable.

I think I'm possibly getting my head around this...

___emutls_get_address is not hidden in libclang_rt, it's visible:

$ nm /opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a  | grep emu
/opt/local/libexec/llvm-5.0/lib/clang/5.0.1/lib/darwin/libclang_rt.10.4.a(emutls.c.o):
0000000000000000 T ___emutls_get_address


but when that code is incorporated into the executable, the symbol is (by design) not exported beyond the executable, so libc++abi.dylib will never see it.

$ nm 4 | grep emu
0000000100007070 t ___emutls_get_address


and I am actually not sure that it is possible to have executables export symbols for linked libraries to call into. It would be probably bad, even if you could force it.

So perhaps having emutls.c in libclang_rt can never actually work...maybe the emutls.o code needs to exist in a separate dylib (like it does in libgcc).

So putting it into libc++abi.dylib might indeed be the only workable method, assuming each executable would get it's own copy in memory and they wouldn't all collide together.

Ken



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev
On 2018-12-08 19:10, Ken Cunningham via llvm-dev wrote:

> So putting it into libc++abi.dylib might indeed be the only workable method, assuming each executable would get it's own copy in memory and they wouldn't all collide together.

Can ibc++abi link with libclang_rt to resolve the symbol?

--
/Jacob Carlborg

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev

On 2018-12-08, at 11:51 AM, Jacob Carlborg via llvm-dev wrote:

> On 2018-12-08 19:10, Ken Cunningham via llvm-dev wrote:
>
>> So putting it into libc++abi.dylib might indeed be the only workable method, assuming each executable would get it's own copy in memory and they wouldn't all collide together.
>
> Can ibc++abi link with libclang_rt to resolve the symbol?
>

Thanks for the clue, and yes that led me to the answer.

We are building libc++abi with nodefaultlibs . The emutls.o symbols indeed belong right in libc++abi, and I will either compile them in with emutls.c or link them in manually by adding that library.

And then we should be in business, and all systems will have thread_local .

Thanks for the hint.

Ken

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev

On 2018-12-08, at 2:27 PM, Ken Cunningham wrote:

> And then we should be in business, and all systems will have thread_local .


All that being sorted out, we now have libc++ 7.0.0 and libc++abi  built and being used with emulated-tls support  on darwin < 11 now. Thanks!

The last bit of this I plan to sort out would be how to tweak the following patch so that __cxa_thread_atexit would be called based on the -femulated-tls flag rather than based on the MacOSXVersion. That way emulated-tls could be used/tested on any OS version. -- Ken



==========================
--- a/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp.orig 2018-10-02 18:31:17.000000000 -0700
+++ b/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp 2018-10-02 18:32:35.000000000 -0700
@@ -2255,7 +2255,7 @@
const char *Name = "__cxa_atexit";
if (TLS) {
  const llvm::Triple &T = CGF.getTarget().getTriple();
-    Name = T.isOSDarwin() ?  "_tlv_atexit" : "__cxa_thread_atexit";
+    Name = (T.isOSDarwin() && !T.isMacOSXVersionLT(10, 7)) ?  "_tlv_atexit" : "__cxa_thread_atexit";
}


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev
On 2018-12-10 07:52, Ken Cunningham via llvm-dev wrote:

> All that being sorted out, we now have libc++ 7.0.0 and libc++abi  built and being used with emulated-tls support  on darwin < 11 now. Thanks!

Cool. BTW, what about TLS in C, i.e. "__thread"?

--
/Jacob Carlborg

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] using emulated-tls on Darwin 8, 9, 10

Alex L via llvm-dev

On 2018-12-10, at 2:43 AM, Jacob Carlborg via llvm-dev wrote:

> On 2018-12-10 07:52, Ken Cunningham via llvm-dev wrote:
>
>> All that being sorted out, we now have libc++ 7.0.0 and libc++abi  built and being used with emulated-tls support  on darwin < 11 now. Thanks!
>
> Cool. BTW, what about TLS in C, i.e. "__thread"?
>

Yes, the various tests I've tried with this pass. I am a hobbyist, not a professional programmer, so it's possible I'm missing some aspect of the TLS coverage or the implementation. The darwin code in llvm generally assumes tlv_atexit for TLS, and I might be overlooking some detail that I haven't as yet discovered.

I wonder if there is any appetite to have darwin support the -femulated-tls flag in general, for testing the subsystems, etc. Right now if you use the flag, you still get tlv_atexit in the end.

I should be able to find a way to test if femulated-tls is set while in the scope of CodeGen/ItaniumCXXABI.cpp and branch on the flag to cxa_thread_atexit -- and if I sort that out, a general implementation would appear workable.

Ken



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev