[llvm-dev] Hang generating sanitizer tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Hang generating sanitizer tests

U.Mutlu via llvm-dev
When I run "make check-all" with ToT I am seeing what looks like a hang
generating sanitizer tests:

[...]
[100%] Generating dynamic/Asan-x86_64-calls-Dynamic-Test
make[3]: Leaving directory '/build/debug'
[100%] Built target TAsan-x86_64-calls-Dynamic-Test
[100%] Generating default/Asan-x86_64-inline-Test
make[3]: Leaving directory '/build/debug'
[100%] Built target TAsan-x86_64-inline-Test
[100%] Generating dynamic/Asan-x86_64-inline-Dynamic-Test
make[3]: Leaving directory '/build/debug'
[100%] Built target TAsan-x86_64-inline-Dynamic-Test
make[3]: Entering directory '/build/debug'
Scanning dependencies of target AsanDynamicUnitTests
make[3]: Leaving directory '/build/debug'
[100%] Built target AsanDynamicUnitTests
[100%] Generating Sanitizer-x86_64-Test
make[3]: Leaving directory '/build/debug'
[100%] Built target TSanitizer-x86_64-Test
make[3]: Entering directory '/build/debug'
Scanning dependencies of target SanitizerUnitTests
make[3]: Leaving directory '/build/debug'
[100%] Built target SanitizerUnitTests

It seems hung up in lld linking Asan-x86_64-calls-Test among other
things.  It has been happening for at least a week and source updates
haven't changed the situation.  An strace reveals:

Process 408631 attached
futex(0x7ffdae5c7144, FUTEX_WAIT_PRIVATE, 1, NULL

It's been sitting there for hours, waiting on a mutex.  Rebuilds result
in the same behavior Has anyone else seen this?  Any tips to get things
going?

Thanks!

                          -David
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Hang generating sanitizer tests

U.Mutlu via llvm-dev


> On 31 May 2018, at 06:19, via llvm-dev <[hidden email]> wrote:
>
> When I run "make check-all" with ToT I am seeing what looks like a hang
> generating sanitizer tests:
>
> [...]
> [100%] Generating dynamic/Asan-x86_64-calls-Dynamic-Test
> make[3]: Leaving directory '/build/debug'
> [100%] Built target TAsan-x86_64-calls-Dynamic-Test
> [100%] Generating default/Asan-x86_64-inline-Test
> make[3]: Leaving directory '/build/debug'
> [100%] Built target TAsan-x86_64-inline-Test
> [100%] Generating dynamic/Asan-x86_64-inline-Dynamic-Test
> make[3]: Leaving directory '/build/debug'
> [100%] Built target TAsan-x86_64-inline-Dynamic-Test
> make[3]: Entering directory '/build/debug'
> Scanning dependencies of target AsanDynamicUnitTests
> make[3]: Leaving directory '/build/debug'
> [100%] Built target AsanDynamicUnitTests
> [100%] Generating Sanitizer-x86_64-Test
> make[3]: Leaving directory '/build/debug'
> [100%] Built target TSanitizer-x86_64-Test
> make[3]: Entering directory '/build/debug'
> Scanning dependencies of target SanitizerUnitTests
> make[3]: Leaving directory '/build/debug'
> [100%] Built target SanitizerUnitTests
>
> It seems hung up in lld linking Asan-x86_64-calls-Test among other
> things.  It has been happening for at least a week and source updates
> haven't changed the situation.  An strace reveals:
>
> Process 408631 attached
> futex(0x7ffdae5c7144, FUTEX_WAIT_PRIVATE, 1, NULL
>
> It's been sitting there for hours, waiting on a mutex.  Rebuilds result
> in the same behavior Has anyone else seen this?  Any tips to get things
> going?
>

Can you say:

- What platform/OS you’re doing this on?
- What your CMake configuration looks like (i.e. how you invoked cmake)
- Are you building this stand-alone or in-tree?
- Did you build lld or is that an lld that you got somewhere else?

That would be helpful.

-- Dean

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Hang generating sanitizer tests

U.Mutlu via llvm-dev
Just to follow-up, I'm now encountering this as well. I'm doing this on Linux.

It seems that when linking with the most-recently built clang, the
unit tests are taking a while to complete. It's notably not using more
than 1 thread, and if using `lld` I would have expected the linker to
still be running in parallel.

I'm doing this on a debug build, so that might help narrow it down.
On Thu, May 31, 2018 at 9:32 AM Dean Michael Berris
<[hidden email]> wrote:

>
>
>
> > On 31 May 2018, at 06:19, via llvm-dev <[hidden email]> wrote:
> >
> > When I run "make check-all" with ToT I am seeing what looks like a hang
> > generating sanitizer tests:
> >
> > [...]
> > [100%] Generating dynamic/Asan-x86_64-calls-Dynamic-Test
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target TAsan-x86_64-calls-Dynamic-Test
> > [100%] Generating default/Asan-x86_64-inline-Test
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target TAsan-x86_64-inline-Test
> > [100%] Generating dynamic/Asan-x86_64-inline-Dynamic-Test
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target TAsan-x86_64-inline-Dynamic-Test
> > make[3]: Entering directory '/build/debug'
> > Scanning dependencies of target AsanDynamicUnitTests
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target AsanDynamicUnitTests
> > [100%] Generating Sanitizer-x86_64-Test
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target TSanitizer-x86_64-Test
> > make[3]: Entering directory '/build/debug'
> > Scanning dependencies of target SanitizerUnitTests
> > make[3]: Leaving directory '/build/debug'
> > [100%] Built target SanitizerUnitTests
> >
> > It seems hung up in lld linking Asan-x86_64-calls-Test among other
> > things.  It has been happening for at least a week and source updates
> > haven't changed the situation.  An strace reveals:
> >
> > Process 408631 attached
> > futex(0x7ffdae5c7144, FUTEX_WAIT_PRIVATE, 1, NULL
> >
> > It's been sitting there for hours, waiting on a mutex.  Rebuilds result
> > in the same behavior Has anyone else seen this?  Any tips to get things
> > going?
> >
>
> Can you say:
>
> - What platform/OS you’re doing this on?
> - What your CMake configuration looks like (i.e. how you invoked cmake)
> - Are you building this stand-alone or in-tree?
> - Did you build lld or is that an lld that you got somewhere else?
>
> That would be helpful.
>
> -- Dean
>


--
Dean
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Hang generating sanitizer tests

U.Mutlu via llvm-dev
Glad I'm not the only one.  :)  I am building on Linux as well.  Here's
the cmake invocation I used:

cmake -G 'Unix Makefiles' -DCMAKE_C_COMPILER=/opt/gcc/6.1.0/bin/gcc -DCMAKE_CXX_COMPILER=/opt/gcc/6.1.0/bin/g++ -DC_INCLUDE_DIRS=/opt/gcc/6.1.0/include/g++:/opt/gcc/6.1.0/include/g++/x86_64-unknown-linux-gnu:/usr/include -DGCC_INSTALL_PREFIX=/opt/gcc/6.1.0 -DPYTHON_EXECUTABLE=/usr/bin/python -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/clang-install/debug -DCMAKE_CXX_LINK_FLAGS=-Wl,-rpath=/opt/gcc/6.1.0/lib64 -DLLVM_APPEND_VC_REV=ON -DLLVM_ENABLE_CXX1Y=ON -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_PARALLEL_COMPILE_JOBS=16 -DLLVM_PARALLEL_LINK_JOBS=1 -DLLVM_BUILD_DOCS=ON -DCLANG_INCLUDE_DOCS=ON -DLLVM_BUILD_TESTS=ON -DLLVM_LIT_ARGS=-j16 -DCLANG_DEFAULT_LINKER=lld '-DLLVM_ENABLE_PROJECTS=clang;compiler-rt;lld;debuginfo-tests' -DLLVM_EXTERNAL_CLANG_SOURCE_DIR=/src/clang -DLLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR=/src/compiler-rt -DLLVM_EXTERNAL_LLD_SOURCE_DIR=/src/lld -DLLVM_EXTERNAL_DEBUGINFO_TESTS_SOURCE_DIR=/src/debuginfo-tests '-DCMAKE_PREFIX_PATH=/usr;/usr' '-DCMAKE_LIBRARY_PATH=/usr/lib64;/usr/include;/usr/lib64;/usr/include' /src/llvm

This is a standalone (out of tree) build and I am building lld as part
of the build.

I don't think it's a matter of tests taking a long time.  It sat at the
mutex for hours.  Seems like a deadlock.

                              -David


Dean Michael Berris <[hidden email]> writes:

> Just to follow-up, I'm now encountering this as well. I'm doing this on Linux.
>
> It seems that when linking with the most-recently built clang, the
> unit tests are taking a while to complete. It's notably not using more
> than 1 thread, and if using `lld` I would have expected the linker to
> still be running in parallel.
>
> I'm doing this on a debug build, so that might help narrow it down.
> On Thu, May 31, 2018 at 9:32 AM Dean Michael Berris
> <[hidden email]> wrote:
>>
>>
>>
>> > On 31 May 2018, at 06:19, via llvm-dev <[hidden email]> wrote:
>> >
>> > When I run "make check-all" with ToT I am seeing what looks like a hang
>> > generating sanitizer tests:
>> >
>> > [...]
>> > [100%] Generating dynamic/Asan-x86_64-calls-Dynamic-Test
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target TAsan-x86_64-calls-Dynamic-Test
>> > [100%] Generating default/Asan-x86_64-inline-Test
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target TAsan-x86_64-inline-Test
>> > [100%] Generating dynamic/Asan-x86_64-inline-Dynamic-Test
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target TAsan-x86_64-inline-Dynamic-Test
>> > make[3]: Entering directory '/build/debug'
>> > Scanning dependencies of target AsanDynamicUnitTests
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target AsanDynamicUnitTests
>> > [100%] Generating Sanitizer-x86_64-Test
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target TSanitizer-x86_64-Test
>> > make[3]: Entering directory '/build/debug'
>> > Scanning dependencies of target SanitizerUnitTests
>> > make[3]: Leaving directory '/build/debug'
>> > [100%] Built target SanitizerUnitTests
>> >
>> > It seems hung up in lld linking Asan-x86_64-calls-Test among other
>> > things.  It has been happening for at least a week and source updates
>> > haven't changed the situation.  An strace reveals:
>> >
>> > Process 408631 attached
>> > futex(0x7ffdae5c7144, FUTEX_WAIT_PRIVATE, 1, NULL
>> >
>> > It's been sitting there for hours, waiting on a mutex.  Rebuilds result
>> > in the same behavior Has anyone else seen this?  Any tips to get things
>> > going?
>> >
>>
>> Can you say:
>>
>> - What platform/OS you’re doing this on?
>> - What your CMake configuration looks like (i.e. how you invoked cmake)
>> - Are you building this stand-alone or in-tree?
>> - Did you build lld or is that an lld that you got somewhere else?
>>
>> That would be helpful.
>>
>> -- Dean
>>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev