[llvm-dev] (Thin)LTO llvm build

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
136 messages Options
1 ... 4567
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Tue, Dec 20, 2016 at 5:05 PM, Teresa Johnson <[hidden email]> wrote:
> Hi Carsten,
>
> A few responses below, but first, can you get the link command for
> lldb.so.3.9.1? Last time it was the lldb.so build that was using
> ld.bfd with the gold plugin which was exposing this issue.

Where would I find it in an otherwise already terminated process?

> On Tue, Dec 20, 2016 at 5:49 AM, Carsten Mattner <[hidden email]>

> No problem - I assumed no news was good news! Thanks for the
> explicit feedback though!

Sure, but it's important, and I've had my experience with never
hearing back even after asking for confirmation of success. Same as
drive-by patch drops.

> > - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS
> >   in favor of passing these exclusively as cmake's
> >   -Dsomething_something=<FLAGS>? Linux distros usually have an
> >   environment with CFLAGS etc. and use that building packages and
> >   this didn't interfere in the past in my builds, but I'm open to
> >   avoid this by unsetting any such variables first.
>
>
> The problem I've had in the past setting *FLAGS on the cmake command
> line is that sometimes the configure fails because it uses those
> flags in various configuration checking compiles. So typically I
> hand edit the resulting CMakeCache.txt if I want to pass specific
> options to the build. Note that there are various flavors of e.g.
> LINKER_FLAGS that get passed to different types of links though
> (SHARED vs EXE etc). I haven't tried setting CFLAGS etc env
> variables before kicking off the ninja build to pass args.

I set the variables before running cmake and I didn't expect for it to
be re-considered when running ninja, but it would of course more
closely mirror make's behavior if it does/dit.

> > - Is there a configuration in the CI matrix that builds all
> >   of LLVM with ThinLTO so that I can be reasonably sure
> >   any error is most likely local to my environment or
> >   because of operator (silly me) faults?
>
>
> There is an lld based ThinLTO buildbot. I've been really remiss in
> not setting up a gold based ThinLTO bot...note to self to get that
> done asap! But the existing lld will be head not 3.9.

Please have it build everything that's buildable in the llvm tree.

> > I also pass these in LDFLAGS but they're not visible
> > in the error trace below:
> >     -Wl,-plugin-opt,-function-sections \
> >     -Wl,-plugin-opt,-data-sections \
>
>
> Hmm, looks like LDFLAGS doesn't work for passing to the resulting link line?
> Try setting in the CMakeCache.txt as described above.

It's a mystery (aka haven't traced/debugged CMake).

> > Any idea why lldb-argdumper would fail to link reproducably?
>
> See suggestion above about looking at lldb.so.3.9.1 link line. Let's
> make sure that looks ok first.

Thanks.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Tue, Dec 20, 2016 at 11:05 AM, Carsten Mattner <[hidden email]> wrote:
On Tue, Dec 20, 2016 at 5:05 PM, Teresa Johnson <[hidden email]> wrote:
> Hi Carsten,
>
> A few responses below, but first, can you get the link command for
> lldb.so.3.9.1? Last time it was the lldb.so build that was using
> ld.bfd with the gold plugin which was exposing this issue.

Where would I find it in an otherwise already terminated process?

I think you can get this via "ninja -t commands bin/lldb-argdumper" (this will give you a lot of output, all of the compilation commands used to build that target). Or redo the build with -v to be sure.

Teresa


> On Tue, Dec 20, 2016 at 5:49 AM, Carsten Mattner <[hidden email]>

> No problem - I assumed no news was good news! Thanks for the
> explicit feedback though!

Sure, but it's important, and I've had my experience with never
hearing back even after asking for confirmation of success. Same as
drive-by patch drops.

> > - Should I avoid any and all CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS
> >   in favor of passing these exclusively as cmake's
> >   -Dsomething_something=<FLAGS>? Linux distros usually have an
> >   environment with CFLAGS etc. and use that building packages and
> >   this didn't interfere in the past in my builds, but I'm open to
> >   avoid this by unsetting any such variables first.
>
>
> The problem I've had in the past setting *FLAGS on the cmake command
> line is that sometimes the configure fails because it uses those
> flags in various configuration checking compiles. So typically I
> hand edit the resulting CMakeCache.txt if I want to pass specific
> options to the build. Note that there are various flavors of e.g.
> LINKER_FLAGS that get passed to different types of links though
> (SHARED vs EXE etc). I haven't tried setting CFLAGS etc env
> variables before kicking off the ninja build to pass args.

I set the variables before running cmake and I didn't expect for it to
be re-considered when running ninja, but it would of course more
closely mirror make's behavior if it does/dit.

> > - Is there a configuration in the CI matrix that builds all
> >   of LLVM with ThinLTO so that I can be reasonably sure
> >   any error is most likely local to my environment or
> >   because of operator (silly me) faults?
>
>
> There is an lld based ThinLTO buildbot. I've been really remiss in
> not setting up a gold based ThinLTO bot...note to self to get that
> done asap! But the existing lld will be head not 3.9.

Please have it build everything that's buildable in the llvm tree.

> > I also pass these in LDFLAGS but they're not visible
> > in the error trace below:
> >     -Wl,-plugin-opt,-function-sections \
> >     -Wl,-plugin-opt,-data-sections \
>
>
> Hmm, looks like LDFLAGS doesn't work for passing to the resulting link line?
> Try setting in the CMakeCache.txt as described above.

It's a mystery (aka haven't traced/debugged CMake).

> > Any idea why lldb-argdumper would fail to link reproducably?
>
> See suggestion above about looking at lldb.so.3.9.1 link line. Let's
> make sure that looks ok first.

Thanks.



--
Teresa Johnson | Software Engineer | [hidden email] | 408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Wed, Dec 21, 2016 at 2:55 PM, Teresa Johnson <[hidden email]> wrote:
> I think you can get this via "ninja -t commands bin/lldb-argdumper"
> (this will give you a lot of output, all of the compilation commands
> used to build that target). Or redo the build with -v to be sure.

Unfortunately the old build tree is gone for space reclaim reasons. Sorry,
I'll make sure to not nuke it the next time. I guess I thought the detailed
error message would be enough.

I didn't build again, but I did configure with the settings posted yesterday,
and there's a warning which I'm unsure how to interpret and if it's a
concern:

---
CMake Warning at tools/lldb/cmake/modules/LLDBConfig.cmake:409 (message):
  You appear to be linking to libstdc++ version lesser than 4.9 without
  exceptions enabled.  These versions of the library have an issue, which
  causes occasional lldb crashes.  See
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59656> for details.  Possible
  courses of action are:

  - use libstdc++ version 4.9 or newer

  - use libc++ (via LLVM_ENABLE_LIBCXX)

  - enable exceptions (via LLVM_ENABLE_EH)

  - ignore this warning and accept occasional instability
Call Stack (most recent call first):
  tools/lldb/CMakeLists.txt:4 (include)
---

libstdc++ is 6.2.1 so the CMake check seems wrong.

This prompted me to use -libstd=libc++ and -DLLVM_ENABLE_LIBCXX=ON,
but for some reason Arch Linux packages llvm and clang in version 3.9.0
but libc++ in 3.8.0, so I skipped it.

Unless the above libstdc++ warning is a problem, I will build and report
back when done.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Wed, Dec 21, 2016 at 9:00 AM, Carsten Mattner <[hidden email]> wrote:
On Wed, Dec 21, 2016 at 2:55 PM, Teresa Johnson <[hidden email]> wrote:
> I think you can get this via "ninja -t commands bin/lldb-argdumper"
> (this will give you a lot of output, all of the compilation commands
> used to build that target). Or redo the build with -v to be sure.

Unfortunately the old build tree is gone for space reclaim reasons. Sorry,
I'll make sure to not nuke it the next time. I guess I thought the detailed
error message would be enough.

I didn't build again, but I did configure with the settings posted yesterday,
and there's a warning which I'm unsure how to interpret and if it's a
concern:

---
CMake Warning at tools/lldb/cmake/modules/LLDBConfig.cmake:409 (message):
  You appear to be linking to libstdc++ version lesser than 4.9 without
  exceptions enabled.  These versions of the library have an issue, which
  causes occasional lldb crashes.  See
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59656> for details.  Possible
  courses of action are:

  - use libstdc++ version 4.9 or newer

  - use libc++ (via LLVM_ENABLE_LIBCXX)

  - enable exceptions (via LLVM_ENABLE_EH)

  - ignore this warning and accept occasional instability
Call Stack (most recent call first):
  tools/lldb/CMakeLists.txt:4 (include)
---

libstdc++ is 6.2.1 so the CMake check seems wrong.

Or maybe it is picking up a different libstdc++ from somewhere else on your system? Here's the check from the cmake file:

    # There doesn't seem to be an easy way to check the library version. Instead, we rely on the
    # fact that std::set did not have the allocator constructor available until version 4.9
    check_cxx_source_compiles("
            #include <set>
            std::set<int> s = std::set<int>(std::allocator<int>());
            int main() { return 0; }" 
LLDB_USING_LIBSTDCXX_4_9)

So the version check isn't precise, but unless libstdc++ 6.2.1 doesn't have this interface available, which seems unlikely, something else is going wrong.

Teresa
 

This prompted me to use -libstd=libc++ and -DLLVM_ENABLE_LIBCXX=ON,
but for some reason Arch Linux packages llvm and clang in version 3.9.0
but libc++ in 3.8.0, so I skipped it.

Unless the above libstdc++ warning is a problem, I will build and report
back when done.



--
Teresa Johnson | Software Engineer | [hidden email] | 408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Wed, Dec 21, 2016 at 6:10 PM, Teresa Johnson <[hidden email]> wrote:

> Or maybe it is picking up a different libstdc++ from somewhere else on your
> system?
> Here's the check from the cmake file:
>
>     # There doesn't seem to be an easy way to check the library version. Instead,
>     # we rely on the fact that std::set did not have the allocator constructor available
>     # until version 4.9
>     check_cxx_source_compiles("
>             #include <set>
>             std::set<int> s = std::set<int>(std::allocator<int>());
>             int main() { return 0; }"

I cannot make g++ 6.2.1 fail to build the test snippet with
or without the archlinux-default/custom CXXFLAGS and LDFLAGS.

Time to find the real error and what's happening.

> LLDB_USING_LIBSTDCXX_4_9)
>
> So the version check isn't precise, but unless libstdc++ 6.2.1 doesn't have this
> interface available, which seems unlikely, something else is going wrong.

That seems risky to ignore then. I guess I'll postpone the rebuild until this
can be resolved/explained.

Also because ideally I want to do it that way, and it's a goal for a
future default
build config of llvm as discussed recently here, I'd prefer to build
and then use
the in-tree libc++. But it doesn't seem like -DLLVM_ENABLE_LIBCXX has that
effect. Shouldn't it or if not why can't it? I would certainly welcome
and find natural
that the in-tree libc++ is used when LLVM_ENABLE_LIBCXX=ON. The need for
a pre-built and installed libc++ is less practical.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Thu, Dec 22, 2016 at 2:45 AM, Carsten Mattner <[hidden email]> wrote:
On Wed, Dec 21, 2016 at 6:10 PM, Teresa Johnson <[hidden email]> wrote:

> Or maybe it is picking up a different libstdc++ from somewhere else on your
> system?
> Here's the check from the cmake file:
>
>     # There doesn't seem to be an easy way to check the library version. Instead,
>     # we rely on the fact that std::set did not have the allocator constructor available
>     # until version 4.9
>     check_cxx_source_compiles("
>             #include <set>
>             std::set<int> s = std::set<int>(std::allocator<int>());
>             int main() { return 0; }"

I cannot make g++ 6.2.1 fail to build the test snippet with
or without the archlinux-default/custom CXXFLAGS and LDFLAGS.

Time to find the real error and what's happening.

> LLDB_USING_LIBSTDCXX_4_9)
>
> So the version check isn't precise, but unless libstdc++ 6.2.1 doesn't have this
> interface available, which seems unlikely, something else is going wrong.

That seems risky to ignore then. I guess I'll postpone the rebuild until this
can be resolved/explained.

Also because ideally I want to do it that way, and it's a goal for a
future default
build config of llvm as discussed recently here, I'd prefer to build
and then use
the in-tree libc++. But it doesn't seem like -DLLVM_ENABLE_LIBCXX has that
effect. Shouldn't it or if not why can't it? I would certainly welcome
and find natural
that the in-tree libc++ is used when LLVM_ENABLE_LIBCXX=ON. The need for
a pre-built and installed libc++ is less practical.

I'm really not sure, and I don't build or use libc++ myself. I was hoping someone else would answer this part, so maybe fork it into a different thread about that specifically so that the question gets more visibility.

Teresa 



--
Teresa Johnson | Software Engineer | [hidden email] | <a href="tel:(408)%20460-2413" value="+14084602413" target="_blank">408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Fri, Dec 23, 2016 at 5:20 PM, Teresa Johnson <[hidden email]> wrote:

> I'm really not sure, and I don't build or use libc++ myself. I was hoping
> someone else would answer this part, so maybe fork it into a different thread
> about that specifically so that the question gets more visibility.

Will do if I don't forget about it.

In the meantime, given no explanation for cmake warning about code that
builds and works fine if done manually with the toolchain to be used, I'll try
to build with the updates config and see how far it gets, since gold is not
used now. I want to say I doubt it will work, but I'll be optimistic.
Expect a build report sometime today or tomorrow......
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Sat, Dec 24, 2016 at 6:20 AM, Carsten Mattner <[hidden email]> wrote:
On Fri, Dec 23, 2016 at 5:20 PM, Teresa Johnson <[hidden email]> wrote:

> I'm really not sure, and I don't build or use libc++ myself. I was hoping
> someone else would answer this part, so maybe fork it into a different thread
> about that specifically so that the question gets more visibility.

Will do if I don't forget about it.

In the meantime, given no explanation for cmake warning about code that
builds and works fine if done manually with the toolchain to be used,

I assume you mean the libstdc++ issue - I wonder if cmake is actually picking up a different libstdc++ than expected. Can you look at the output under the produced CMakeFiles directory? Specifically, I would imagine there will be a more detailed error for this test in the CMakeError.log file, and there should be some output in CMakeOutput.log (in the latter search for LLVM_NO_OLD_LIBSTDCXX). What does the error output look like specifically? There may be debugging options to cmake to help figure out what is going on, not completely sure though.

I'll try to build with the updates config and see how far it gets, since gold is not
used now. I want to say I doubt it will work, but I'll be optimistic.
Expect a build report sometime today or tomorrow......

Ok sure. FYI I'm off for a few days now and later next week so my response times are slow right now.

Teresa 



--
Teresa Johnson | Software Engineer | [hidden email] | 408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
In reply to this post by Jonas Paulsson via llvm-dev
So, archlinux doesn't seem to package lld. Since the binaries from
llvm.org don't work here, maybe I can find another way to avoid
running two llvm builds (one to build lld, one to thinlto-lld rebuild).
TBD.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
After figuring out the fault in the configuration step and rebuilding,
and then rebuilding again by forcing it with `ninja -k 16`, I managed to
build everything but 12 ninja targets.

I have to sift through them before I can report more, and I don't
don't know if it's small enough to post here, but some of the more
interesting errors are:

llvm/projects/compiler-rt/lib/tsan/dd/dd_interceptors.cc:226:20:
error: redefinition of 'realpath'
INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {
                   ^
/usr/include/bits/stdlib.h:37:8: note: previous definition is here
__NTH (realpath (const char *__restrict __name, char *__restrict __resolved))

[...]

libomp.so
duplicate symbol __kmp_get_reduce_method in version script
duplicate symbol __kmp_itt_fini_ittlib in version script
duplicate symbol __kmp_itt_init_ittlib in version script
LLVM ERROR: A @@ version cannot be undefined
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Tue, Dec 27, 2016 at 5:23 AM, Carsten Mattner <[hidden email]> wrote:
After figuring out the fault in the configuration step and rebuilding,
and then rebuilding again by forcing it with `ninja -k 16`, I managed to
build everything but 12 ninja targets.

I have to sift through them before I can report more, and I don't
don't know if it's small enough to post here, but some of the more
interesting errors are:

llvm/projects/compiler-rt/lib/tsan/dd/dd_interceptors.cc:226:20:
error: redefinition of 'realpath'
INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {
                   ^
/usr/include/bits/stdlib.h:37:8: note: previous definition is here
__NTH (realpath (const char *__restrict __name, char *__restrict __resolved))

I've never seen this before. Looks like bits/stdlib.h gets pulled in only when _FORTIFY_SOURCE is enabled (which causes __USE_FORTIFY_LEVEL > 0). Do you have _FORTIFY_SOURCE set somewhere? Can you try with that not set?
Teresa

[...]

libomp.so
duplicate symbol __kmp_get_reduce_method in version script
duplicate symbol __kmp_itt_fini_ittlib in version script
duplicate symbol __kmp_itt_init_ittlib in version script
LLVM ERROR: A @@ version cannot be undefined



--
Teresa Johnson | Software Engineer | [hidden email] | <a href="tel:(408)%20460-2413" value="+14084602413" target="_blank">408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Tue, Dec 27, 2016 at 4:13 PM, Teresa Johnson <[hidden email]> wrote:

>
> On Tue, Dec 27, 2016 at 5:23 AM, Carsten Mattner <[hidden email]> wrote:
>>
>> After figuring out the fault in the configuration step and rebuilding,
>> and then rebuilding again by forcing it with `ninja -k 16`, I managed to
>> build everything but 12 ninja targets.
>>
>> I have to sift through them before I can report more, and I don't
>> don't know if it's small enough to post here, but some of the more
>> interesting errors are:
>>
>> llvm/projects/compiler-rt/lib/tsan/dd/dd_interceptors.cc:226:20:
>> error: redefinition of 'realpath'
>> INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {
>>                    ^
>> /usr/include/bits/stdlib.h:37:8: note: previous definition is here
>> __NTH (realpath (const char *__restrict __name, char *__restrict __resolved))
>
>
> I've never seen this before. Looks like bits/stdlib.h gets pulled in only
> when _FORTIFY_SOURCE is enabled (which causes
> __USE_FORTIFY_LEVEL > 0). Do you have _FORTIFY_SOURCE
> set somewhere?

I do, it's by default a part of hardening flags on most Linux distros,
and I'm just following what the distro packages are built with.

> Can you try with that not set?

I can try, but I would prefer not to since it's a wide-spread default
in Linux distro
as part of stack-protector use. I would use SafeStack but that's limited in
applicability right now. If I did we would still have the libomp
duplicate symbol
error.

Since 3.9.0 built fine with that macro, I'd blame either gcc or glibc headers or
compiler-rt 3.9.1.

Arch Linux's 3.9.1 recipe applies the following patch:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/msan-prevent-initialization-failure-with-newer-glibc.patch?h=packages/llvm

all the files
https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm&id=a7dd5d50ec82d6a35a99f9bf92b074d4aeab1f50
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Tue, Dec 27, 2016 at 8:30 AM, Carsten Mattner <[hidden email]> wrote:
On Tue, Dec 27, 2016 at 4:13 PM, Teresa Johnson <[hidden email]> wrote:
>
> On Tue, Dec 27, 2016 at 5:23 AM, Carsten Mattner <[hidden email]> wrote:
>>
>> After figuring out the fault in the configuration step and rebuilding,
>> and then rebuilding again by forcing it with `ninja -k 16`, I managed to
>> build everything but 12 ninja targets.
>>
>> I have to sift through them before I can report more, and I don't
>> don't know if it's small enough to post here, but some of the more
>> interesting errors are:
>>
>> llvm/projects/compiler-rt/lib/tsan/dd/dd_interceptors.cc:226:20:
>> error: redefinition of 'realpath'
>> INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {
>>                    ^
>> /usr/include/bits/stdlib.h:37:8: note: previous definition is here
>> __NTH (realpath (const char *__restrict __name, char *__restrict __resolved))
>
>
> I've never seen this before. Looks like bits/stdlib.h gets pulled in only
> when _FORTIFY_SOURCE is enabled (which causes
> __USE_FORTIFY_LEVEL > 0). Do you have _FORTIFY_SOURCE
> set somewhere?

I do, it's by default a part of hardening flags on most Linux distros,
and I'm just following what the distro packages are built with.

> Can you try with that not set?

I can try, but I would prefer not to since it's a wide-spread default
in Linux distro
as part of stack-protector use. I would use SafeStack but that's limited in
applicability right now. If I did we would still have the libomp
duplicate symbol
error.

Confirmed that when I build dd_interceptors.cc at head it normally builds fine, but when I add -D_FORTIFY_SOURCE=2 I get the same error. I poked around a little and saw that the sanitizers are not supported with FORTIFY_SOURCE (although the reasons given are different): https://sourceware.org/bugzilla/show_bug.cgi?id=20422. Looks like there was a patch proposed to emit a warning when they are both on (not sure if it went in).

So possibly building the tsan library itself is not tested/supported with FORTIFY_SOURCE. [hidden email] is probably the best person to ask, added him.

Teresa


Since 3.9.0 built fine with that macro, I'd blame either gcc or glibc headers or
compiler-rt 3.9.1.

Arch Linux's 3.9.1 recipe applies the following patch:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/msan-prevent-initialization-failure-with-newer-glibc.patch?h=packages/llvm

all the files
https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm&id=a7dd5d50ec82d6a35a99f9bf92b074d4aeab1f50



--
Teresa Johnson | Software Engineer | [hidden email] | <a href="tel:(408)%20460-2413" value="+14084602413" target="_blank">408-460-2413

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev


On Tue, Dec 27, 2016 at 8:49 AM, Teresa Johnson <[hidden email]> wrote:


On Tue, Dec 27, 2016 at 8:30 AM, Carsten Mattner <[hidden email]> wrote:
On Tue, Dec 27, 2016 at 4:13 PM, Teresa Johnson <[hidden email]> wrote:
>
> On Tue, Dec 27, 2016 at 5:23 AM, Carsten Mattner <[hidden email]> wrote:
>>
>> After figuring out the fault in the configuration step and rebuilding,
>> and then rebuilding again by forcing it with `ninja -k 16`, I managed to
>> build everything but 12 ninja targets.
>>
>> I have to sift through them before I can report more, and I don't
>> don't know if it's small enough to post here, but some of the more
>> interesting errors are:
>>
>> llvm/projects/compiler-rt/lib/tsan/dd/dd_interceptors.cc:226:20:
>> error: redefinition of 'realpath'
>> INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {
>>                    ^
>> /usr/include/bits/stdlib.h:37:8: note: previous definition is here
>> __NTH (realpath (const char *__restrict __name, char *__restrict __resolved))
>
>
> I've never seen this before. Looks like bits/stdlib.h gets pulled in only
> when _FORTIFY_SOURCE is enabled (which causes
> __USE_FORTIFY_LEVEL > 0). Do you have _FORTIFY_SOURCE
> set somewhere?

I do, it's by default a part of hardening flags on most Linux distros,
and I'm just following what the distro packages are built with.

> Can you try with that not set?

I can try, but I would prefer not to since it's a wide-spread default
in Linux distro
as part of stack-protector use. I would use SafeStack but that's limited in
applicability right now. If I did we would still have the libomp
duplicate symbol
error.

Confirmed that when I build dd_interceptors.cc at head it normally builds fine, but when I add -D_FORTIFY_SOURCE=2 I get the same error. I poked around a little and saw that the sanitizers are not supported with FORTIFY_SOURCE (although the reasons given are different): https://sourceware.org/bugzilla/show_bug.cgi?id=20422. Looks like there was a patch proposed to emit a warning when they are both on (not sure if it went in).

So possibly building the tsan library itself is not tested/supported with FORTIFY_SOURCE. [hidden email] is probably the best person to ask, added him.

mixing -D_FORTIFY_SOURCE=2 with the sanitizers may be confusing as mentioned above. 
building the sanitizers themselves with -D_FORTIFY_SOURCE=2 is not expected to work at all. 
 

Teresa


Since 3.9.0 built fine with that macro, I'd blame either gcc or glibc headers or
compiler-rt 3.9.1.

Arch Linux's 3.9.1 recipe applies the following patch:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/msan-prevent-initialization-failure-with-newer-glibc.patch?h=packages/llvm

all the files
https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/llvm&id=a7dd5d50ec82d6a35a99f9bf92b074d4aeab1f50



--
Teresa Johnson | Software Engineer | [hidden email] | <a href="tel:(408)%20460-2413" value="+14084602413" target="_blank">408-460-2413


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
On Tue, Dec 27, 2016 at 10:42 PM, Kostya Serebryany <[hidden email]> wrote:

> mixing -D_FORTIFY_SOURCE=2 with the sanitizers may be confusing as
> mentioned above. building the sanitizers themselves with
> -D_FORTIFY_SOURCE=2 is not expected to work at all.

Only to be clear, I didn't enable any -fsanitize for building llvm,
which I don't think you're saying, but allow me to be explicit for
clarity.

Arch Linux builds LLVM as follows and doesn't make an effort to
suppress the distro-default of using -fstack-protector=strong which
itself needs D_FORTIFY_SOURCE=2 macro:

https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/llvm

And flags are inherited from /etc/makepkg.conf which has this:

CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"

Which is exactly what I use in addition to the flags for LTO.

Given this and that 3.9.0 built with the above flags, I'm going to
suggest there's a different cause here. I'm not denying that
FORTIFY_SOURCE=2 is incompatible, but archlinux has already packaged
3.9.1 and the difference to my build is that I use clang and lld to
enable -flto=thin when building llvm as a whole.
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] (Thin)LTO llvm build

Jonas Paulsson via llvm-dev
In reply to this post by Jonas Paulsson via llvm-dev
​I've given up on this for the moment and will try again when
5.0 comes around, hoping CI already has such a successful
profile in the build matrix already.​


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
1 ... 4567