libiomp, not libgomp as default library linked with -fopenmp

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
56 messages Options
123
Reply | Threaded
Open this post in threaded view
|

libiomp, not libgomp as default library linked with -fopenmp

Andrey Bokhanko
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

For reference, the previous discussion is accessible there: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461

We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Last time Chandler objected against doing this, and gave a good (and proper!) scolding. Let me quote his objections along with updates on current state of things:

- This library is not being developed as an active part of the LLVM community, even if it is checked into SVN as part of the LLVM project and under its license. See r197914 where there is a code drop of many months worth of development with *no* change log, attribution, information, or other participation in any part of the community.


Now it is actively developed by several members of the community. Changes are committed in small patches. See "openmp-commits" mailing list (http://lists.cs.uiuc.edu/pipermail/openmp-commits/) for details.


- There has been essentially *zero* discussion with the rest of the clang or llvm community about this library. There are separate mailing lists which have nearly no traffic since the code drop.


Nowadays the traffic, while less than on llvm-dev mailing list (understandably so :-)) is far from being "zero". Discussions happen all the time. See the list archives for details (http://lists.cs.uiuc.edu/pipermail/openmp-dev/).


- There has been no effort to make this library even work properly with Clang as a host compiler. See the copious notes that only Clang 3.3 is supported, and that not full featured.


This is fixed.


- The build system is totally disjoint from LLVM's, in fact it is an entirely custom Perl build system that is unlike anything in use by the LLVM project.


The build system is full re-written and now cmake-based. The last remaining step (integration into overall llvm build system) is about to be done by any day now (http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500).


- There are *zero tests* in the open source repository!!!! This is even called out in the original submission and on the primary website. We simply *cannot* ship and link against a runtime library which has no tests!


UofHouston contributed a comprehensive test suite: http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/.


- No part of this library has gone through an LLVM release process either, not even as a "new" or "beta" project.


The whole library is a part of two last llvm releases: 3.5 and 3.6.


Thus, I believe all of Chandler's concerns are resolved, and it is finally a good time time to switch to libiomp as default OpenMP library.


Anyone who supports this?


Anyone who disagrees?


Chandler?


Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Hal Finkel
----- Original Message -----

> From: "Andrey Bokhanko" <[hidden email]>
> To: "cfe-dev" <[hidden email]>, "LLVM Developers Mailing List" <[hidden email]>, "Douglas Gregor"
> <[hidden email]>, "Hal Finkel" <[hidden email]>, "C Bergström" <[hidden email]>, "Michael Wong"
> <[hidden email]>, "Alexey Bataev" <[hidden email]>
> Sent: Thursday, April 30, 2015 8:49:30 AM
> Subject: libiomp, not libgomp as default library linked with -fopenmp
>
>
> All,
>
>
> I'd like to resurrect the discussion on replacing libgomp with
> libiomp as the default OpenMP runtime library linked with -fopenmp.
>
>
> For reference, the previous discussion is accessible there:
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>
>
> We are very close to getting *full* OpenMP 3.1 specification
> supported in clang (only one (!) clause is not implemented yet, and
> the patch is already sent for review today:
> http://reviews.llvm.org/D9370 ). This implementation generates Intel
> API library calls; thus, it can't be used with libgomp and it is
> simply logical to link a compatible runtime (libiomp) instead.

To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

>
>
> Last time Chandler objected against doing this, and gave a good (and
> proper!) scolding. Let me quote his objections along with updates on
> current state of things:
>
>
>
>
> - This library is not being developed as an active part of the LLVM
> community, even if it is checked into SVN as part of the LLVM
> project and under its license. See r197914 where there is a code
> drop of many months worth of development with *no* change log,
> attribution, information, or other participation in any part of the
> community.
>
>
>
>
> Now it is actively developed by several members of the community.
> Changes are committed in small patches. See "openmp-commits" mailing
> list ( http://lists.cs.uiuc.edu/pipermail/openmp-commits/ ) for
> details.
>
>
>
>
> - There has been essentially *zero* discussion with the rest of the
> clang or llvm community about this library. There are separate
> mailing lists which have nearly no traffic since the code drop.
>
>
>
>
> Nowadays the traffic, while less than on llvm-dev mailing list
> (understandably so :-)) is far from being "zero". Discussions happen
> all the time. See the list archives for details (
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/ ).
>
>
>
>
> - There has been no effort to make this library even work properly
> with Clang as a host compiler. See the copious notes that only Clang
> 3.3 is supported, and that not full featured.
>
>
>
>
> This is fixed.
>
>
>
>
> - The build system is totally disjoint from LLVM's, in fact it is an
> entirely custom Perl build system that is unlike anything in use by
> the LLVM project.
>
>
>
>
> The build system is full re-written and now cmake-based. The last
> remaining step (integration into overall llvm build system) is about
> to be done by any day now (
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500
> ).
>
>
>
>
> - There are *zero tests* in the open source repository!!!! This is
> even called out in the original submission and on the primary
> website. We simply *cannot* ship and link against a runtime library
> which has no tests!
>
>
>
>
> UofHouston contributed a comprehensive test suite:
> http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/ .
>
>
>
>
> - No part of this library has gone through an LLVM release process
> either, not even as a "new" or "beta" project.
>
>
>
>
> The whole library is a part of two last llvm releases: 3.5 and 3.6.
>
>
>
>
> Thus, I believe all of Chandler's concerns are resolved, and it is
> finally a good time time to switch to libiomp as default OpenMP
> library.
>
>
>
>
> Anyone who supports this?
>

I support this. Linking to libgomp will still be available by specifying -fopenmp=libgomp for those who need it. Our runtime supports a libgomp-compatible interface, so that's hopefully not necessary even for those with GCC-compiled OpenMP object files and libraries.

We need to get the build system integration committed and the buildbots updated to compile it, and the test suite setup (the build system integration and test-suite setup have patches under review on the openmp list, so I suspect this will happen soon). Practically, since we don't have OpenMP in the test suite (yet), none of these are really inflexible prerequisites. In my opinion, as soon as the build system integration is done (so that people have an easy way to use LLVM's runtime), we should flip the defaults -- the other things will follow shortly.

 -Hal

>
>
>
> Anyone who disagrees?
>
>
>
>
> Chandler?
>
>
> Yours,
> Andrey Bokhanko
> ==============
> Software Engineer
> Intel Compiler Team
> Intel
>
>

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2
In reply to this post by Andrey Bokhanko
     The current test suite status on x86_64-apple-darwin14, with the proposed patch from http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20150427/128219.html applied to current cfe trunk, is quite respectable....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              16
S Number of successful tests:          107
S + from this were verified:           106

Normal tests:
N Number of failed tests:              8
N + from this fail compilation:        1
N + from this timed out                0
N Number of successful tests:          54
N + from this were verified:           54

Orphaned tests:
O Number of failed tests:              8
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          53
O + from this were verified:           52

for 'make ctest' using "-fopenmp=libiomp5 -Xclang -fopenmp=libiomp5 -L/sw/opt/llvm-3.7.0/lib -lm -O3". Whereas for libgomp, the results are pitiful....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              30
S Number of successful tests:          93
S + from this were verified:           12

Normal tests:
N Number of failed tests:              15
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          47
N + from this were verified:           6

Orphaned tests:
O Number of failed tests:              15
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          46
O + from this were verified:           6

when using "-fopenmp=libgomp -Xclang -fopenmp=libgomp -L/sw/lib/gcc5/lib -lm -O3 -Wl,-no_compact_unwind". So we can't get much more broken that the current behavior with libgomp.
           Jack

On Thu, Apr 30, 2015 at 9:49 AM, Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

For reference, the previous discussion is accessible there: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461

We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Last time Chandler objected against doing this, and gave a good (and proper!) scolding. Let me quote his objections along with updates on current state of things:

- This library is not being developed as an active part of the LLVM community, even if it is checked into SVN as part of the LLVM project and under its license. See r197914 where there is a code drop of many months worth of development with *no* change log, attribution, information, or other participation in any part of the community.


Now it is actively developed by several members of the community. Changes are committed in small patches. See "openmp-commits" mailing list (http://lists.cs.uiuc.edu/pipermail/openmp-commits/) for details.


- There has been essentially *zero* discussion with the rest of the clang or llvm community about this library. There are separate mailing lists which have nearly no traffic since the code drop.


Nowadays the traffic, while less than on llvm-dev mailing list (understandably so :-)) is far from being "zero". Discussions happen all the time. See the list archives for details (http://lists.cs.uiuc.edu/pipermail/openmp-dev/).


- There has been no effort to make this library even work properly with Clang as a host compiler. See the copious notes that only Clang 3.3 is supported, and that not full featured.


This is fixed.


- The build system is totally disjoint from LLVM's, in fact it is an entirely custom Perl build system that is unlike anything in use by the LLVM project.


The build system is full re-written and now cmake-based. The last remaining step (integration into overall llvm build system) is about to be done by any day now (http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500).


- There are *zero tests* in the open source repository!!!! This is even called out in the original submission and on the primary website. We simply *cannot* ship and link against a runtime library which has no tests!


UofHouston contributed a comprehensive test suite: http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/.


- No part of this library has gone through an LLVM release process either, not even as a "new" or "beta" project.


The whole library is a part of two last llvm releases: 3.5 and 3.6.


Thus, I believe all of Chandler's concerns are resolved, and it is finally a good time time to switch to libiomp as default OpenMP library.


Anyone who supports this?


Anyone who disagrees?


Chandler?


Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

C Bergström
In reply to this post by Hal Finkel
On Thu, Apr 30, 2015 at 9:06 PM, Hal Finkel <[hidden email]> wrote:

> ----- Original Message -----
>> From: "Andrey Bokhanko" <[hidden email]>
>> To: "cfe-dev" <[hidden email]>, "LLVM Developers Mailing List" <[hidden email]>, "Douglas Gregor"
>> <[hidden email]>, "Hal Finkel" <[hidden email]>, "C Bergström" <[hidden email]>, "Michael Wong"
>> <[hidden email]>, "Alexey Bataev" <[hidden email]>
>> Sent: Thursday, April 30, 2015 8:49:30 AM
>> Subject: libiomp, not libgomp as default library linked with -fopenmp
>>
>>
>> All,
>>
>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

It doesn't really feel that way. I proposed a cmake patch and the only
person to review or comment was Intel. (This is coming from the person
who ported it to ARM)

To stay more agnostic I'd love to see a non-Intel owner. While Hal may
not be the most active contributor - his reviews are invaluable and
less biased. I don't know if Hal has the time or interest, but I'd
nominate him for "owner". I see Tom wants to assign more owners, but
I'd like to avoid this being an "Intel runtime owned and controlled by
Intel"

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2


On Thu, Apr 30, 2015 at 10:25 AM, C Bergström <[hidden email]> wrote:
On Thu, Apr 30, 2015 at 9:06 PM, Hal Finkel <[hidden email]> wrote:
> ----- Original Message -----
>> From: "Andrey Bokhanko" <[hidden email]>
>> To: "cfe-dev" <[hidden email]>, "LLVM Developers Mailing List" <[hidden email]>, "Douglas Gregor"
>> <[hidden email]>, "Hal Finkel" <[hidden email]>, "C Bergström" <[hidden email]>, "Michael Wong"
>> <[hidden email]>, "Alexey Bataev" <[hidden email]>
>> Sent: Thursday, April 30, 2015 8:49:30 AM
>> Subject: libiomp, not libgomp as default library linked with -fopenmp
>>
>>
>> All,
>>
>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

It doesn't really feel that way. I proposed a cmake patch and the only
person to review or comment was Intel. (This is coming from the person
who ported it to ARM)

To stay more agnostic I'd love to see a non-Intel owner. While Hal may
not be the most active contributor - his reviews are invaluable and
less biased. I don't know if Hal has the time or interest, but I'd
nominate him for "owner". I see Tom wants to assign more owners, but
I'd like to avoid this being an "Intel runtime owned and controlled by
Intel"


What results to you get from "-fopenmp=libgomp -Xclang -fopenmp=libgomp" on the OpenMP3.1_Validation test suite on x86_64 linux? On x86_64 darwin using libgomp, barely anything passes the verification unlike libiomp5 which only has one passed test which doesn't verify. Isn't that a serious defect for libgomp compared to libiomp5?


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

C Bergström
My apologies. I fully agree to switch the libs and I replied to the
wrong thread.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

John Leidel (jleidel)
In reply to this post by Hal Finkel
I definitely support Andrey’s proposal to move the default to libiomp.  Outside of the known ports for x86, ARM and PowerPC, I’m aware of at least one other port in progress.  More will likely follow.  

Per Hal’s notes, given the forthcoming commits of the CMake build system, flipping the default to libiomp may help spur/accelerate the progress of porting the runtime to new platforms (especially those that are already supported as LLVM backends).  

--John D. Leidel


On Apr 30, 2015, at 9:06 AM, Hal Finkel <[hidden email]> wrote:

> ----- Original Message -----
>> From: "Andrey Bokhanko" <[hidden email]>
>> To: "cfe-dev" <[hidden email]>, "LLVM Developers Mailing List" <[hidden email]>, "Douglas Gregor"
>> <[hidden email]>, "Hal Finkel" <[hidden email]>, "C Bergström" <[hidden email]>, "Michael Wong"
>> <[hidden email]>, "Alexey Bataev" <[hidden email]>
>> Sent: Thursday, April 30, 2015 8:49:30 AM
>> Subject: libiomp, not libgomp as default library linked with -fopenmp
>>
>>
>> All,
>>
>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).
>
>>
>>
>> Last time Chandler objected against doing this, and gave a good (and
>> proper!) scolding. Let me quote his objections along with updates on
>> current state of things:
>>
>>
>>
>>
>> - This library is not being developed as an active part of the LLVM
>> community, even if it is checked into SVN as part of the LLVM
>> project and under its license. See r197914 where there is a code
>> drop of many months worth of development with *no* change log,
>> attribution, information, or other participation in any part of the
>> community.
>>
>>
>>
>>
>> Now it is actively developed by several members of the community.
>> Changes are committed in small patches. See "openmp-commits" mailing
>> list ( http://lists.cs.uiuc.edu/pipermail/openmp-commits/ ) for
>> details.
>>
>>
>>
>>
>> - There has been essentially *zero* discussion with the rest of the
>> clang or llvm community about this library. There are separate
>> mailing lists which have nearly no traffic since the code drop.
>>
>>
>>
>>
>> Nowadays the traffic, while less than on llvm-dev mailing list
>> (understandably so :-)) is far from being "zero". Discussions happen
>> all the time. See the list archives for details (
>> http://lists.cs.uiuc.edu/pipermail/openmp-dev/ ).
>>
>>
>>
>>
>> - There has been no effort to make this library even work properly
>> with Clang as a host compiler. See the copious notes that only Clang
>> 3.3 is supported, and that not full featured.
>>
>>
>>
>>
>> This is fixed.
>>
>>
>>
>>
>> - The build system is totally disjoint from LLVM's, in fact it is an
>> entirely custom Perl build system that is unlike anything in use by
>> the LLVM project.
>>
>>
>>
>>
>> The build system is full re-written and now cmake-based. The last
>> remaining step (integration into overall llvm build system) is about
>> to be done by any day now (
>> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500
>> ).
>>
>>
>>
>>
>> - There are *zero tests* in the open source repository!!!! This is
>> even called out in the original submission and on the primary
>> website. We simply *cannot* ship and link against a runtime library
>> which has no tests!
>>
>>
>>
>>
>> UofHouston contributed a comprehensive test suite:
>> http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/ .
>>
>>
>>
>>
>> - No part of this library has gone through an LLVM release process
>> either, not even as a "new" or "beta" project.
>>
>>
>>
>>
>> The whole library is a part of two last llvm releases: 3.5 and 3.6.
>>
>>
>>
>>
>> Thus, I believe all of Chandler's concerns are resolved, and it is
>> finally a good time time to switch to libiomp as default OpenMP
>> library.
>>
>>
>>
>>
>> Anyone who supports this?
>>
>
> I support this. Linking to libgomp will still be available by specifying -fopenmp=libgomp for those who need it. Our runtime supports a libgomp-compatible interface, so that's hopefully not necessary even for those with GCC-compiled OpenMP object files and libraries.
>
> We need to get the build system integration committed and the buildbots updated to compile it, and the test suite setup (the build system integration and test-suite setup have patches under review on the openmp list, so I suspect this will happen soon). Practically, since we don't have OpenMP in the test suite (yet), none of these are really inflexible prerequisites. In my opinion, as soon as the build system integration is done (so that people have an easy way to use LLVM's runtime), we should flip the defaults -- the other things will follow shortly.
>
> -Hal
>
>>
>>
>>
>> Anyone who disagrees?
>>
>>
>>
>>
>> Chandler?
>>
>>
>> Yours,
>> Andrey Bokhanko
>> ==============
>> Software Engineer
>> Intel Compiler Team
>> Intel
>>
>>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2
In reply to this post by Jack Howarth-2
On Thu, Apr 30, 2015 at 10:23 AM, Jack Howarth <[hidden email]> wrote:
     The current test suite status on x86_64-apple-darwin14, with the proposed patch from http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20150427/128219.html applied to current cfe trunk, is quite respectable....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              16
S Number of successful tests:          107
S + from this were verified:           106

Normal tests:
N Number of failed tests:              8
N + from this fail compilation:        1
N + from this timed out                0
N Number of successful tests:          54
N + from this were verified:           54

Orphaned tests:
O Number of failed tests:              8
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          53
O + from this were verified:           52

for 'make ctest' using "-fopenmp=libiomp5 -Xclang -fopenmp=libiomp5 -L/sw/opt/llvm-3.7.0/lib -lm -O3". Whereas for libgomp, the results are pitiful....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              30
S Number of successful tests:          93
S + from this were verified:           12

Normal tests:
N Number of failed tests:              15
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          47
N + from this were verified:           6

Orphaned tests:
O Number of failed tests:              15
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          46
O + from this were verified:           6


For additional context, the OpenMP3.1_Validation results for libgomp using the FSF gcc 5.1.0  compiler on x86_64-apple-darwin14 are...

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              7
S Number of successful tests:          116
S + from this were verified:           95

Normal tests:
N Number of failed tests:              3
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          59
N + from this were verified:           48

Orphaned tests:
O Number of failed tests:              4
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          57
O + from this were verified:           47

So the current results for libiomp5 are very close to matching those for the FSF gcc compiler with libgomp in terms of the number of passed tests while libiomp5 already exceeds libgomp for the number of verified tests.

when using "-fopenmp=libgomp -Xclang -fopenmp=libgomp -L/sw/lib/gcc5/lib -lm -O3 -Wl,-no_compact_unwind". So we can't get much more broken that the current behavior with libgomp.
           Jack

On Thu, Apr 30, 2015 at 9:49 AM, Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

For reference, the previous discussion is accessible there: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461

We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Last time Chandler objected against doing this, and gave a good (and proper!) scolding. Let me quote his objections along with updates on current state of things:

- This library is not being developed as an active part of the LLVM community, even if it is checked into SVN as part of the LLVM project and under its license. See r197914 where there is a code drop of many months worth of development with *no* change log, attribution, information, or other participation in any part of the community.


Now it is actively developed by several members of the community. Changes are committed in small patches. See "openmp-commits" mailing list (http://lists.cs.uiuc.edu/pipermail/openmp-commits/) for details.


- There has been essentially *zero* discussion with the rest of the clang or llvm community about this library. There are separate mailing lists which have nearly no traffic since the code drop.


Nowadays the traffic, while less than on llvm-dev mailing list (understandably so :-)) is far from being "zero". Discussions happen all the time. See the list archives for details (http://lists.cs.uiuc.edu/pipermail/openmp-dev/).


- There has been no effort to make this library even work properly with Clang as a host compiler. See the copious notes that only Clang 3.3 is supported, and that not full featured.


This is fixed.


- The build system is totally disjoint from LLVM's, in fact it is an entirely custom Perl build system that is unlike anything in use by the LLVM project.


The build system is full re-written and now cmake-based. The last remaining step (integration into overall llvm build system) is about to be done by any day now (http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500).


- There are *zero tests* in the open source repository!!!! This is even called out in the original submission and on the primary website. We simply *cannot* ship and link against a runtime library which has no tests!


UofHouston contributed a comprehensive test suite: http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/.


- No part of this library has gone through an LLVM release process either, not even as a "new" or "beta" project.


The whole library is a part of two last llvm releases: 3.5 and 3.6.


Thus, I believe all of Chandler's concerns are resolved, and it is finally a good time time to switch to libiomp as default OpenMP library.


Anyone who supports this?


Anyone who disagrees?


Chandler?


Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev




_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Ed Maste
In reply to this post by Hal Finkel
On 30 April 2015 at 10:06, Hal Finkel <[hidden email]> wrote:

>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

It looks like openmp/runtime/README.txt could use an update to clarify
this status. The title is "README for the Intel(R) OpenMP* Runtime
Library" and there's a of supported Intel architectures -- this gives
a somewhat different impression than what's described in this thread.

That said, I would also be happy to see the default switched to libiomp.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2
In reply to this post by Jack Howarth-2
For comparison, the OpenMP3.1_Validation results on x86_64-apple-darwin14 for original clang-omp 3.5.2 are...

#Tested Directive         t ct ot oct
has_openmp                 100 100 100 100
omp_atomic                 100 100 100 100
omp_barrier               100 100 100 100
omp_critical               100 100 100 100
omp_flush                 100 100 100 100
omp_for_firstprivate       100 100 100 100
omp_for_lastprivate       100 100 100 80
omp_for_ordered           100 100 100 100
omp_for_private           100 100 100 100
omp_for_reduction         100 100 100 100
omp_for_schedule_dynamic   100 100 100 100
omp_for_schedule_guided   35 - 10 -
omp_for_schedule_static   100 100 100 100
omp_for_nowait             100 100 100 100
omp_get_num_threads       100 100 100 100
omp_get_wtick             100 100 100 100
omp_get_wtime             100 100 100 100
omp_in_parallel           100 100 100 100
omp_lock                   100 100 100 100
omp_master                 100 100 100 100
omp_nest_lock             100 100 100 100
omp_parallel_copyin       100 100 100 100
omp_parallel_for_firstprivate 100 100 100 100
omp_parallel_for_lastprivate 100 100 100 100
omp_parallel_for_ordered   100 100 100 100
omp_parallel_for_private   100 100 100 100
omp_parallel_for_reduction 100 100 100 100
omp_parallel_num_threads   100 100 100 100
omp_parallel_sections_firstprivate 100 100 100 100
omp_parallel_sections_lastprivate 100 100 100 100
omp_parallel_sections_private 100 100 100 100
omp_parallel_sections_reduction 100 95 100 90
omp_section_firstprivate   100 100 100 100
omp_section_lastprivate   100 100 100 100
omp_section_private       100 100 100 100
omp_sections_reduction     100 100 100 100
omp_sections_nowait       100 100 100 100
omp_parallel_for_if       100 100 100 100
omp_single_copyprivate     100 100 100 100
omp_single_nowait         100 100 100 100
omp_single_private         100 100 100 100
omp_single                 100 100 100 100
omp_test_lock             100 100 100 100
omp_test_nest_lock         100 100 100 100
omp_threadprivate         100 100 - -
omp_parallel_default       100 100 100 100
omp_parallel_shared       100 100 100 100
omp_parallel_private       100 100 100 100
omp_parallel_firstprivate 100 100 100 100
omp_parallel_if           100 100 100 100
omp_parallel_reduction     100 100 100 100
omp_for_collapse           100 100 100 100
omp_master_3               100 100 100 100
omp_task                   100 100 100 100
omp_task_if               100 100 100 100
omp_task_untied           0 - 0 -
omp_task_shared           100 100 100 100
omp_task_private           100 100 100 100
omp_task_firstprivate     100 100 100 100
omp_taskwait               100 100 100 100
omp_taskyield             100 100 10 -
omp_task_final             0 - 0 -


Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              7
S Number of successful tests:          116
S + from this were verified:           113

Normal tests:
N Number of failed tests:              3
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          59
N + from this were verified:           58

Orphaned tests:
O Number of failed tests:              4
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          57
O + from this were verified:           55

and for current llvm/cfe trunk with the two outstanding OPENMP patches applied are...

#Tested Directive         t ct ot oct
has_openmp                 100 100 100 100
omp_atomic                 100 100 100 100
omp_barrier               100 100 100 100
omp_critical               100 100 100 95
omp_flush                 100 100 100 95
omp_for_firstprivate       100 100 100 100
omp_for_lastprivate       100 100 100 95
omp_for_ordered           100 100 100 100
omp_for_private           100 100 100 100
omp_for_reduction         0 - 0 -
omp_for_schedule_dynamic   100 100 100 100
omp_for_schedule_guided   100 100 100 100
omp_for_schedule_static   100 100 100 100
omp_for_nowait             100 100 100 100
omp_get_num_threads       100 100 100 100
omp_get_wtick             100 100 100 100
omp_get_wtime             100 100 100 100
omp_in_parallel           100 100 100 100
omp_lock                   100 100 100 100
omp_master                 100 100 100 100
omp_nest_lock             100 100 100 100
omp_parallel_copyin       100 100 100 100
omp_parallel_for_firstprivate 100 100 100 100
omp_parallel_for_lastprivate 100 100 100 100
omp_parallel_for_ordered   100 100 100 100
omp_parallel_for_private   100 100 100 100
omp_parallel_for_reduction 0 - 0 -
omp_parallel_num_threads   100 100 100 100
omp_parallel_sections_firstprivate 100 100 100 100
omp_parallel_sections_lastprivate 100 100 100 100
omp_parallel_sections_private 100 100 100 100
omp_parallel_sections_reduction 0 - 0 -
omp_section_firstprivate   100 100 100 100
omp_section_lastprivate   100 100 100 100
omp_section_private       100 100 100 100
omp_sections_reduction     0 - 0 -
omp_sections_nowait       100 100 100 100
omp_parallel_for_if       100 100 100 100
omp_single_copyprivate     100 100 100 100
omp_single_nowait         100 100 100 100
omp_single_private         100 100 100 100
omp_single                 100 100 100 100
omp_test_lock             100 100 100 100
omp_test_nest_lock         100 100 100 100
omp_threadprivate         100 100 - -
omp_parallel_default       100 100 100 100
omp_parallel_shared       100 100 100 100
omp_parallel_private       100 100 100 100
omp_parallel_firstprivate 100 100 100 100
omp_parallel_if           100 100 100 100
omp_parallel_reduction     0 - 0 -
omp_for_collapse           100 100 100 100
omp_master_3               100 100 100 100
omp_task                   100 100 100 100
omp_task_if               100 100 100 100
omp_task_untied           0 - 0 -
omp_task_shared           100 100 100 100
omp_task_private           100 100 100 100
omp_task_firstprivate     100 100 100 100
omp_taskwait               100 100 100 100
omp_taskyield             100 100 10 -
omp_task_final             0 - 0 -


Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              15
S Number of successful tests:          108
S + from this were verified:           105

Normal tests:
N Number of failed tests:              7
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          55
N + from this were verified:           55

Orphaned tests:
O Number of failed tests:              8
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          53
O + from this were verified:           50

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Chandler Carruth-2
In reply to this post by Andrey Bokhanko
On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

Just for the record, I'm really excited to see this. =]
 
We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Is there no way to support libgomp here as well? I don't say this to hold up changing the defaults in any way, just curious. =]

Also, for the record, I'm really excited to see the progress here as well.
 
Chandler?

Hi! ;]

I totally agree, I think things are way better now. I generally support the direction. I think there are a few things I'd suggest we do as part of the process, but I think these are really small and just about "how" we switch.

1) I completely agree with the comments some others have made about us needing to make it clear that this isn't some Intel-only thing, its the LLVM OpenMP runtime. Some suggestions that I think would make sense to help here:
- I agree with finding some non-Intel folks to add as explicit code owners. I don't know who has been sufficiently involved, but if Hal makes sense, awesome.
- Clearly updating the readme and such would be appropriate.
- I suspect we should change the name of the installed library. 'libiomp' is pretty clearly the Intel library. We could continue in the grand tradition of LLVM naming conventions and use 'libllomp'? Of course, we should install symlinks under the name 'libiomp' if needed for existing users to not be broken.
- Any other changes?

2) I think we need to update the instructions for checking out LLVM and all the tools to include checking out the openmp project. I'm planning to try it out in a bit.

3) It would be nice to get at least one boring benchmark into the test-suite that uses OpenMP just so there's more coverage that the basic stuff all works. In particular, if we could get the benchmark that Phoronix and others keep pointing at, that'd be nice.


Speaking of which, have you checked the performance of some of the basic benchmarks using OpenMP with the two runtimes? Or looked at Clang vs GCC there? I'd be interested to see the numbers.
 

Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

C Bergström
I'm trimming everything - sorry too hard to manage the thread in my
current email client
--
1) I'd rather see a configure time option to change the name rather
than just switching it. It's not user facing and who really cares what
the name is. (I don't) It's one more change for others who have
established naming (Intel) to have to carry a patch. (I'd like to play
nice here.. UH, PathScale, foobar may also want to use a different
name.. etc)

2) Benchmarks - There's SPEC OMP2012, but I'm not sure if a gcc vs
llvm comparison is really going to stress the runtime sufficiently
enough to play the blame game. The retired OMP benchmark may use a
limited enough subset of features to work with llvm+gomp||iomp.. (That
way you could do a 1:1 comparison) - If that does work I'm not yet
confident it would expose any real performance issues in the runtime.
(You'd need to test it on a system with *A LOT* of cores)

There are lowering differences between the internal API as well..

Ancedotal evidence - Intel's runtime (essentially the same code as
"llvm omp") is used by Intel for OMP SPEC submissions as compared to
nobody publishing record setting performance with gomp...
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2
In reply to this post by Chandler Carruth-2


On Thu, Apr 30, 2015 at 5:51 PM, Chandler Carruth <[hidden email]> wrote:
On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

Just for the record, I'm really excited to see this. =]
 
We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Is there no way to support libgomp here as well? I don't say this to hold up changing the defaults in any way, just curious. =]

Also, for the record, I'm really excited to see the progress here as well.
 
Chandler?

Hi! ;]

I totally agree, I think things are way better now. I generally support the direction. I think there are a few things I'd suggest we do as part of the process, but I think these are really small and just about "how" we switch.

1) I completely agree with the comments some others have made about us needing to make it clear that this isn't some Intel-only thing, its the LLVM OpenMP runtime. Some suggestions that I think would make sense to help here:
- I agree with finding some non-Intel folks to add as explicit code owners. I don't know who has been sufficiently involved, but if Hal makes sense, awesome.
- Clearly updating the readme and such would be appropriate.
- I suspect we should change the name of the installed library. 'libiomp' is pretty clearly the Intel library. We could continue in the grand tradition of LLVM naming conventions and use 'libllomp'? Of course, we should install symlinks under the name 'libiomp' if needed for existing users to not be broken.
- Any other changes?

2) I think we need to update the instructions for checking out LLVM and all the tools to include checking out the openmp project. I'm planning to try it out in a bit.

3) It would be nice to get at least one boring benchmark into the test-suite that uses OpenMP just so there's more coverage that the basic stuff all works. In particular, if we could get the benchmark that Phoronix and others keep pointing at, that'd be nice.


Attached are two logs, taskbench_clang_3.7.log and taskbench_gcc_5.1.log, with the output from the task_bench, task_firstpriv_bench and taskwait_bench benchmarks run  on x86_64-apple-darwin14 using an early 2008 MacPro with dual 2.8 GHz quad-Xeon processors and 10 Gb of memory. The clang 3.7svn compiler was current trunk with the remaining two OPENMP patches on cfe-commits added to the build. The gcc compiler is release 5.1.0. In both cases, the -O3 -fopenmp flags were used. The taskbench-1.0-20110715 release was used for the benchmark sources.

Speaking of which, have you checked the performance of some of the basic benchmarks using OpenMP with the two runtimes? Or looked at Clang vs GCC there? I'd be interested to see the numbers.
 

Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

taskbench_clang_3.7.log (13K) Download Attachment
taskbench_gcc_5.1.log (13K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Jack Howarth-2
In reply to this post by Chandler Carruth-2


On Thu, Apr 30, 2015 at 5:51 PM, Chandler Carruth <[hidden email]> wrote:
On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

Just for the record, I'm really excited to see this. =]
 
We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Is there no way to support libgomp here as well? I don't say this to hold up changing the defaults in any way, just curious. =]

Also, for the record, I'm really excited to see the progress here as well.
 
Chandler?

Hi! ;]

I totally agree, I think things are way better now. I generally support the direction. I think there are a few things I'd suggest we do as part of the process, but I think these are really small and just about "how" we switch.

1) I completely agree with the comments some others have made about us needing to make it clear that this isn't some Intel-only thing, its the LLVM OpenMP runtime. Some suggestions that I think would make sense to help here:
- I agree with finding some non-Intel folks to add as explicit code owners. I don't know who has been sufficiently involved, but if Hal makes sense, awesome.
- Clearly updating the readme and such would be appropriate.
- I suspect we should change the name of the installed library. 'libiomp' is pretty clearly the Intel library. We could continue in the grand tradition of LLVM naming conventions and use 'libllomp'? Of course, we should install symlinks under the name 'libiomp' if needed for existing users to not be broken.
- Any other changes?

2) I think we need to update the instructions for checking out LLVM and all the tools to include checking out the openmp project. I'm planning to try it out in a bit.

3) It would be nice to get at least one boring benchmark into the test-suite that uses OpenMP just so there's more coverage that the basic stuff all works. In particular, if we could get the benchmark that Phoronix and others keep pointing at, that'd be nice.


Speaking of which, have you checked the performance of some of the basic benchmarks using OpenMP with the two runtimes? Or looked at Clang vs GCC there? I'd be interested to see the numbers.

Another set of benchmark numbers are attached in openmpbench_C_v3_clang-3.7.log and openmpbench_C_v3_gcc_5.1.log using the schedbench, syncbench and taskbench compiled with the default -O1 optimization using the clang 3.7svn trunk with the two remaining OPENMP patches from cfe-commits and the gcc 5.1.0 release compiler. The OMP_NUM_THREADS=4 is set before each set of runs which were performed on x86_64-apple-darwin14 using a early 2008 MacPro with dual 2.8 GHz quad-core Xeon processors and 10 Gb of memory. The benchmark is detailed at https://www.epcc.ed.ac.uk/research/computing/performance-characterisation-and-benchmarking/epcc-openmp-micro-benchmark-suite.
 

Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

openmpbench_C_v3_clang-3.7.log (24K) Download Attachment
openmpbench_C_v3_gcc_5.1.log (24K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

C Bergström
While OpenMP is very explicit in nature - some compilers may not turn
on advanced analysis and become really aggressive with performance
until -O2 or -O3 (all depends on the compiler).

Benchmarks at -O1 may not have any value.. be careful of using weak comparisons.

On Fri, May 1, 2015 at 6:29 AM, Jack Howarth
<[hidden email]> wrote:

>
>
> On Thu, Apr 30, 2015 at 5:51 PM, Chandler Carruth <[hidden email]>
> wrote:
>>
>> On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <[hidden email]>
>> wrote:
>>>
>>> All,
>>>
>>> I'd like to resurrect the discussion on replacing libgomp with libiomp as
>>> the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> Just for the record, I'm really excited to see this. =]
>>
>>>
>>> We are very close to getting *full* OpenMP 3.1 specification supported in
>>> clang (only one (!) clause is not implemented yet, and the patch is already
>>> sent for review today: http://reviews.llvm.org/D9370). This implementation
>>> generates Intel API library calls; thus, it can't be used with libgomp and
>>> it is simply logical to link a compatible runtime (libiomp) instead.
>>
>>
>> Is there no way to support libgomp here as well? I don't say this to hold
>> up changing the defaults in any way, just curious. =]
>>
>> Also, for the record, I'm really excited to see the progress here as well.
>>
>>>
>>> Chandler?
>>
>>
>> Hi! ;]
>>
>> I totally agree, I think things are way better now. I generally support
>> the direction. I think there are a few things I'd suggest we do as part of
>> the process, but I think these are really small and just about "how" we
>> switch.
>>
>> 1) I completely agree with the comments some others have made about us
>> needing to make it clear that this isn't some Intel-only thing, its the LLVM
>> OpenMP runtime. Some suggestions that I think would make sense to help here:
>> - I agree with finding some non-Intel folks to add as explicit code
>> owners. I don't know who has been sufficiently involved, but if Hal makes
>> sense, awesome.
>> - Clearly updating the readme and such would be appropriate.
>> - I suspect we should change the name of the installed library. 'libiomp'
>> is pretty clearly the Intel library. We could continue in the grand
>> tradition of LLVM naming conventions and use 'libllomp'? Of course, we
>> should install symlinks under the name 'libiomp' if needed for existing
>> users to not be broken.
>> - Any other changes?
>>
>> 2) I think we need to update the instructions for checking out LLVM and
>> all the tools to include checking out the openmp project. I'm planning to
>> try it out in a bit.
>>
>> 3) It would be nice to get at least one boring benchmark into the
>> test-suite that uses OpenMP just so there's more coverage that the basic
>> stuff all works. In particular, if we could get the benchmark that Phoronix
>> and others keep pointing at, that'd be nice.
>>
>>
>> Speaking of which, have you checked the performance of some of the basic
>> benchmarks using OpenMP with the two runtimes? Or looked at Clang vs GCC
>> there? I'd be interested to see the numbers.
>
>
> Another set of benchmark numbers are attached in
> openmpbench_C_v3_clang-3.7.log and openmpbench_C_v3_gcc_5.1.log using the
> schedbench, syncbench and taskbench compiled with the default -O1
> optimization using the clang 3.7svn trunk with the two remaining OPENMP
> patches from cfe-commits and the gcc 5.1.0 release compiler. The
> OMP_NUM_THREADS=4 is set before each set of runs which were performed on
> x86_64-apple-darwin14 using a early 2008 MacPro with dual 2.8 GHz quad-core
> Xeon processors and 10 Gb of memory. The benchmark is detailed at
> https://www.epcc.ed.ac.uk/research/computing/performance-characterisation-and-benchmarking/epcc-openmp-micro-benchmark-suite.
>>
>>
>>>
>>>
>>> Yours,
>>> Andrey Bokhanko
>>> ==============
>>> Software Engineer
>>> Intel Compiler Team
>>> Intel
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> [hidden email]         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Andrey Bokhanko
In reply to this post by Hal Finkel
Hal,

On Thu, Apr 30, 2015 at 5:06 PM, Hal Finkel <[hidden email]> wrote:
We need to get the build system integration committed and the buildbots updated to compile it

Andrey


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Andrey Bokhanko
In reply to this post by Jack Howarth-2
Jack,

Thanks for running the test suite!

I wonder how a *single* test passed with libgomp, given that no real OMP runtime calls are generated if clang targets libgomp... My educated guess is that compiler simply ignores all OMP pragmas, a test still compiles and runs correctly (which is a property of OMP specification) and the test says "oh! omp parallel for is working just fine!".

Andrey


On Thu, Apr 30, 2015 at 5:23 PM, Jack Howarth <[hidden email]> wrote:
     The current test suite status on x86_64-apple-darwin14, with the proposed patch from http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20150427/128219.html applied to current cfe trunk, is quite respectable....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              16
S Number of successful tests:          107
S + from this were verified:           106

Normal tests:
N Number of failed tests:              8
N + from this fail compilation:        1
N + from this timed out                0
N Number of successful tests:          54
N + from this were verified:           54

Orphaned tests:
O Number of failed tests:              8
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          53
O + from this were verified:           52

for 'make ctest' using "-fopenmp=libiomp5 -Xclang -fopenmp=libiomp5 -L/sw/opt/llvm-3.7.0/lib -lm -O3". Whereas for libgomp, the results are pitiful....

Summary:
S Number of tested Open MP constructs: 62
S Number of used tests:                123
S Number of failed tests:              30
S Number of successful tests:          93
S + from this were verified:           12

Normal tests:
N Number of failed tests:              15
N + from this fail compilation:        0
N + from this timed out                0
N Number of successful tests:          47
N + from this were verified:           6

Orphaned tests:
O Number of failed tests:              15
O + from this fail compilation:        0
O + from this timed out                0
O Number of successful tests:          46
O + from this were verified:           6

when using "-fopenmp=libgomp -Xclang -fopenmp=libgomp -L/sw/lib/gcc5/lib -lm -O3 -Wl,-no_compact_unwind". So we can't get much more broken that the current behavior with libgomp.
           Jack

On Thu, Apr 30, 2015 at 9:49 AM, Andrey Bokhanko <[hidden email]> wrote:
All,

I'd like to resurrect the discussion on replacing libgomp with libiomp as the default OpenMP runtime library linked with -fopenmp.

For reference, the previous discussion is accessible there: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461

We are very close to getting *full* OpenMP 3.1 specification supported in clang (only one (!) clause is not implemented yet, and the patch is already sent for review today: http://reviews.llvm.org/D9370). This implementation generates Intel API library calls; thus, it can't be used with libgomp and it is simply logical to link a compatible runtime (libiomp) instead.

Last time Chandler objected against doing this, and gave a good (and proper!) scolding. Let me quote his objections along with updates on current state of things:

- This library is not being developed as an active part of the LLVM community, even if it is checked into SVN as part of the LLVM project and under its license. See r197914 where there is a code drop of many months worth of development with *no* change log, attribution, information, or other participation in any part of the community.


Now it is actively developed by several members of the community. Changes are committed in small patches. See "openmp-commits" mailing list (http://lists.cs.uiuc.edu/pipermail/openmp-commits/) for details.


- There has been essentially *zero* discussion with the rest of the clang or llvm community about this library. There are separate mailing lists which have nearly no traffic since the code drop.


Nowadays the traffic, while less than on llvm-dev mailing list (understandably so :-)) is far from being "zero". Discussions happen all the time. See the list archives for details (http://lists.cs.uiuc.edu/pipermail/openmp-dev/).


- There has been no effort to make this library even work properly with Clang as a host compiler. See the copious notes that only Clang 3.3 is supported, and that not full featured.


This is fixed.


- The build system is totally disjoint from LLVM's, in fact it is an entirely custom Perl build system that is unlike anything in use by the LLVM project.


The build system is full re-written and now cmake-based. The last remaining step (integration into overall llvm build system) is about to be done by any day now (http://lists.cs.uiuc.edu/pipermail/openmp-dev/2015-April/thread.html#500).


- There are *zero tests* in the open source repository!!!! This is even called out in the original submission and on the primary website. We simply *cannot* ship and link against a runtime library which has no tests!


UofHouston contributed a comprehensive test suite: http://llvm.org/viewvc/llvm-project/openmp/trunk/testsuite/.


- No part of this library has gone through an LLVM release process either, not even as a "new" or "beta" project.


The whole library is a part of two last llvm releases: 3.5 and 3.6.


Thus, I believe all of Chandler's concerns are resolved, and it is finally a good time time to switch to libiomp as default OpenMP library.


Anyone who supports this?


Anyone who disagrees?


Chandler?


Yours,
Andrey Bokhanko
==============
Software Engineer
Intel Compiler Team
Intel


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev




_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Andrey Bokhanko
In reply to this post by Ed Maste
Adding openmp-dev list and Andrey [Churbanov].

Andrey, please see Ed's message below.

Andrey


On Thu, Apr 30, 2015 at 6:59 PM, Ed Maste <[hidden email]> wrote:
On 30 April 2015 at 10:06, Hal Finkel <[hidden email]> wrote:
>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

It looks like openmp/runtime/README.txt could use an update to clarify
this status. The title is "README for the Intel(R) OpenMP* Runtime
Library" and there's a of supported Intel architectures -- this gives
a somewhat different impression than what's described in this thread.

That said, I would also be happy to see the default switched to libiomp.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: libiomp, not libgomp as default library linked with -fopenmp

Andrey Bokhanko
In reply to this post by Chandler Carruth-2
Chandler,

Thanks for the reply -- I always included you in libiomp supporters camp; it is good to see I wasn't mistaken! ;-)

On Fri, May 1, 2015 at 12:51 AM, Chandler Carruth <[hidden email]> wrote:
Is there no way to support libgomp here as well? I don't say this to hold up changing the defaults in any way, just curious. =]

No, sorry. libgomp doesn't support Intel API and clang generates Intel API calls only -- as simple as that. Someday someone may implement generation of GNU API calls as well, but this is a separate big task that, IMHO, doesn't serve any real purpose -- and potentially introduces nasty GPL-related legal issues.

There is an option to choose what library clang links (-fopenmp={libiomp|libgomp}), though.
 
I totally agree, I think things are way better now. I generally support the direction. I think there are a few things I'd suggest we do as part of the process, but I think these are really small and just about "how" we switch.

1) I completely agree with the comments some others have made about us needing to make it clear that this isn't some Intel-only thing, its the LLVM OpenMP runtime. Some suggestions that I think would make sense to help here:
- I agree with finding some non-Intel folks to add as explicit code owners. I don't know who has been sufficiently involved, but if Hal makes sense, awesome.

This really belongs to a separate thread (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-April/085037.html); see my answer there in a couple of minutes.
 
- Clearly updating the readme and such would be appropriate.
- I suspect we should change the name of the installed library. 'libiomp' is pretty clearly the Intel library. We could continue in the grand tradition of LLVM naming conventions and use 'libllomp'? Of course, we should install symlinks under the name 'libiomp' if needed for existing users to not be broken.
- Any other changes?

Adding openmp-dev list (in retrospect, should have been done at the very start...), Jim Cownie and Andrey Churbanov.
 
2) I think we need to update the instructions for checking out LLVM and all the tools to include checking out the openmp project. I'm planning to try it out in a bit.

Cool! Thank you!
 
3) It would be nice to get at least one boring benchmark into the test-suite that uses OpenMP just so there's more coverage that the basic stuff all works. In particular, if we could get the benchmark that Phoronix and others keep pointing at, that'd be nice.

Speaking of which, have you checked the performance of some of the basic benchmarks using OpenMP with the two runtimes? Or looked at Clang vs GCC there? I'd be interested to see the numbers.

This is very tricky for me -- I'm employed by a CPU vendor (Intel), and we have very strict rules and long processes for publishing benchmark results. I simply can't run a benchmark and say: "hey! clang has this number and gcc has that number".

The only thing I can share is that we do tested SPEC OMP2012 (https://www.spec.org/omp2012/), which is the industry standard for OMP benchmarks, on a non-server class Darwin machine, and the results are quite good and comparable with other compilers.

Speaking on Phoronix, two benchmarks where clang always lose due to lack of OpenMP are "John the Ripper" (http://www.phoronix.com/scan.php?page=article&item=clang-gcc-broadwell&num=3) and ImageMagick -- though latter is not included in most recent "clang vs gcc" comparison.

Is there a generous soul (not employed by a CPU vendor :-)) willing to run "John the Ripper" with "clang -fopenmp=libiomp5 -Xclang -fopenmp=libiomp5 -lm -O3" and compare results with "clang -O3"?

Also, Jack Howarth did testing with some other benchmarks, and it is nice to see that clang + libiomp compare quite well (to say it mildly ;-)) with gcc + libgomp!
 
Andrey


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [Openmp-dev] libiomp, not libgomp as default library linked with -fopenmp

Cownie, James H
In reply to this post by Andrey Bokhanko
> It looks like openmp/runtime/README.txt could use an update to clarify
> this status. The title is "README for the Intel(R) OpenMP* Runtime
> Library" and there's a of supported Intel architectures -- this gives
> a somewhat different impression than what's described in this thread.

Of course we'd be very happy to accept patches that update the document to include the relevant support matrices for non-Intel processors (and to change the misleading documentation titles).

Since this *is* a collaborative development, I'd hope that IBM can contribute information
for their machines, since they did the port, and any input from the folks who worked on the ARM code would also be great.

In the more general discussion, I think one critical point was made less forcefully than it merits, which is that the LLVN OpenMP runtime (libiomp5) *is binary compatible* with the GCC one (libgomp); libiomp5 includes the entrypoints used by GCC. Therefore replacing libgomp in the distribution with libiomp5 should not cause problems for people who are using the libgomp interfaces. Their code should continue to link and run without a problem.

This compatibility means that you can link code compiled with gcc -fopenmp against the LLVM runtime (and mix it with code compiled by clang's OpenMP support). What you *cannot* do is compile code with clang/Openmp and link it against the GCC OpenMP runtime.

So, replacing the GCC OpenMP runtime as the default runtime for the clang distribution should not cause any compatibility issues for anyone who was using the libgomp that was previously distributed, but does ensure that the runtime is compatible with LLVM. (Which is the whole point, surely!)

(Full disclosure: There is one incompatibility between the libraries because the LLVM runtime does not support one interface that gcc -openmp uses for a tasking feature that is not in OpenMP 3.1. The interface gcc uses has significant performance implications for code that never uses tasking, and that seemed unreasonable. The same limitation applies to the Intel OpenMP runtime, and we have had no complaints about it yet :-). This incompatibility only arose with gcc 4.8 (IIRC); code compiled by earlier gcc's cannot hit the problem [because they don't support the language feature in question!].

-- Jim

James Cownie <[hidden email]>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)
Tel: +44 117 9071438

From: [hidden email] [mailto:[hidden email]] On Behalf Of Andrey Bokhanko
Sent: Friday, May 1, 2015 9:16 AM
To: Ed Maste; [hidden email]
Cc: cfe-dev; LLVM Developers Mailing List
Subject: Re: [Openmp-dev] [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp

Adding openmp-dev list and Andrey [Churbanov].
Andrey, please see Ed's message below.
Andrey

On Thu, Apr 30, 2015 at 6:59 PM, Ed Maste <[hidden email]> wrote:
On 30 April 2015 at 10:06, Hal Finkel <[hidden email]> wrote:

>>
>> I'd like to resurrect the discussion on replacing libgomp with
>> libiomp as the default OpenMP runtime library linked with -fopenmp.
>>
>>
>> For reference, the previous discussion is accessible there:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/thread.html#99461
>>
>>
>> We are very close to getting *full* OpenMP 3.1 specification
>> supported in clang (only one (!) clause is not implemented yet, and
>> the patch is already sent for review today:
>> http://reviews.llvm.org/D9370 ). This implementation generates Intel
>> API library calls; thus, it can't be used with libgomp and it is
>> simply logical to link a compatible runtime (libiomp) instead.
>
> To be clear, this is now LLVM's OpenMP runtime (not just Intel's), and has been ported to several platforms in addition to x86 (PowerPC, ARM).

It looks like openmp/runtime/README.txt could use an update to clarify
this status. The title is "README for the Intel(R) OpenMP* Runtime
Library" and there's a of supported Intel architectures -- this gives
a somewhat different impression than what's described in this thread.

That said, I would also be happy to see the default switched to libiomp.

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
123