[llvm-dev] FYI: Ninja-build user may use CMake-3.9

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
This is great news! Do we know who contributed the changes to cut the extra library dependencies?

Do you think we should remove ENABLE_OBJLIB to simplify our CMake files in the near future? It seems to me that anyone who cares about highly parallel build throughput can upgrade CMake to get the good behavior. It's probably easier and less error-prone than maintaining a special build configuration.

On Thu, Jul 20, 2017 at 8:31 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
Hey Reid,

I tracked down the relevant gitlab issue, merge request and commits on kitware.
https://gitlab.kitware.com/cmake/cmake/issues/15555
https://gitlab.kitware.com/cmake/cmake/merge_requests/430
Ben from kitware seems to have done the heavy lifting here

Best,
Martell


On Thu, Jul 20, 2017 at 5:16 PM, Reid Kleckner via llvm-dev <[hidden email]> wrote:
This is great news! Do we know who contributed the changes to cut the extra library dependencies?

Do you think we should remove ENABLE_OBJLIB to simplify our CMake files in the near future? It seems to me that anyone who cares about highly parallel build throughput can upgrade CMake to get the good behavior. It's probably easier and less error-prone than maintaining a special build configuration.

On Thu, Jul 20, 2017 at 8:31 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
In reply to this post by George Karpenkov via llvm-dev
Very cool. Thanks Nakamura!

Best,
Tobias

On Thu, Jul 20, 2017, at 05:31 PM, NAKAMURA Takumi via llvm-dev wrote:

> This is useful for developer who uses multicore builder.
> https://cmake.org/cmake/help/v3.9/release/3.9.html#other-changes
>
>
>    - The Ninja
>    <https://cmake.org/cmake/help/v3.9/generator/Ninja.html#generator:Ninja>
> generator
>    has loosened the dependencies of object compilation. Object
>    compilation now
>    depends only on custom targets and custom commands associated with
>    libraries on which the object’s target depends and no longer depends
>    on the
>    libraries themselves. Source files in dependent targets may now
>    compile
>    without waiting for their targets’ dependencies to link.
>
> With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared
> libs.
> See also; http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux
> Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't
> wait for preceding libraries,
> but targets by add_dependencies().
>
> It doesn't break anything in llvm tree. Assume;
>   add_executable(foo foo.cpp)
>   target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
> Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
> Linking foo waits for LLVMCore.
>
> I have been working for cutting dependencies to increase parallelism.
> For example, I introduced ENABLE_OBJLIB.
> See also, https://reviews.llvm.org/rL305635
> Ninja with CMake-3.9 doesn't require parallelize with objlib.
>
> I love ninja-build.
>
> Thanks,
> Takumi
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
In reply to this post by George Karpenkov via llvm-dev


On Fri, Jul 21, 2017 at 1:16 AM Reid Kleckner <[hidden email]> wrote:
This is great news! Do we know who contributed the changes to cut the extra library dependencies?

Do you think we should remove ENABLE_OBJLIB to simplify our CMake files in the near future? It seems to me that anyone who cares about highly parallel build throughput can upgrade CMake to get the good behavior. It's probably easier and less error-prone than maintaining a special build configuration.

At the moment, this facility (loose deps) is specific to Ninja generator.
There are a couple of users of ENABLE_OBJLIB.
- Tablegen (except for ninja).
- LIBCLANG_BUILD_STATIC. It doesn't use LLVM_ENABLE_OBJLIB, but uses it internally.

Thanks,
Takumi

 
On Thu, Jul 20, 2017 at 8:31 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
Hey Takumi ,

Seen as this is related to 3.9. What are your thoughts on how to update the variables to solve this
Will we just add the policy to cmake or change RPATH on macos behave differently?

```
CMake Warning (dev):
  Policy CMP0068 is not set: RPATH settings on macOS do not affect
  install_name.  Run "cmake --help-policy CMP0068" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.
  For compatibility with older versions of CMake, the install_name fields for
  the following targets are still affected by RPATH settings:
   LTO
   libclang
This warning is for project developers.  Use -Wno-dev to suppress it.
```

This is from an MacOS host building clang with cmake 3.9 and ninja

On Fri, Jul 21, 2017 at 12:07 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:


On Fri, Jul 21, 2017 at 1:16 AM Reid Kleckner <[hidden email]> wrote:
This is great news! Do we know who contributed the changes to cut the extra library dependencies?

Do you think we should remove ENABLE_OBJLIB to simplify our CMake files in the near future? It seems to me that anyone who cares about highly parallel build throughput can upgrade CMake to get the good behavior. It's probably easier and less error-prone than maintaining a special build configuration.

At the moment, this facility (loose deps) is specific to Ninja generator.
There are a couple of users of ENABLE_OBJLIB.
- Tablegen (except for ninja).
- LIBCLANG_BUILD_STATIC. It doesn't use LLVM_ENABLE_OBJLIB, but uses it internally.

Thanks,
Takumi

 
On Thu, Jul 20, 2017 at 8:31 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] FYI: Ninja-build user may use CMake-3.9

George Karpenkov via llvm-dev
Looks like subsequent calls to cmake_minimum_required(VERSION 3.4.3) resets policies.  You can test this by duplicating the call in llvm/CMakeLists.txt after setting a policy.

Perhaps clang, et al, including libcxx, etc., should do something like this:

if( CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR )
  cmake_minimum_required(VERSION 3.4.3)
endif()

Clang already does this check, so moving the call to cmake_minimum_required inside the check would help.

Also, it might be a good idea to remove all the obsolete cmake_policy calls that set the policy to NEW, since that's the default once the minumum version has been set.  Perhaps we could even put all of this in a macro that all projects could call without the need to duplicate the same logic.

On Fri, Aug 4, 2017 at 3:04 AM, Martell Malone via llvm-dev <[hidden email]> wrote:
Hey Takumi ,

Seen as this is related to 3.9. What are your thoughts on how to update the variables to solve this
Will we just add the policy to cmake or change RPATH on macos behave differently?

```
CMake Warning (dev):
  Policy CMP0068 is not set: RPATH settings on macOS do not affect
  install_name.  Run "cmake --help-policy CMP0068" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.
  For compatibility with older versions of CMake, the install_name fields for
  the following targets are still affected by RPATH settings:
   LTO
   libclang
This warning is for project developers.  Use -Wno-dev to suppress it.
```

This is from an MacOS host building clang with cmake 3.9 and ninja

On Fri, Jul 21, 2017 at 12:07 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:


On Fri, Jul 21, 2017 at 1:16 AM Reid Kleckner <[hidden email]> wrote:
This is great news! Do we know who contributed the changes to cut the extra library dependencies?

Do you think we should remove ENABLE_OBJLIB to simplify our CMake files in the near future? It seems to me that anyone who cares about highly parallel build throughput can upgrade CMake to get the good behavior. It's probably easier and less error-prone than maintaining a special build configuration.

At the moment, this facility (loose deps) is specific to Ninja generator.
There are a couple of users of ENABLE_OBJLIB.
- Tablegen (except for ninja).
- LIBCLANG_BUILD_STATIC. It doesn't use LLVM_ENABLE_OBJLIB, but uses it internally.

Thanks,
Takumi

 
On Thu, Jul 20, 2017 at 8:31 AM, NAKAMURA Takumi via llvm-dev <[hidden email]> wrote:
This is useful for developer who uses multicore builder.

  • The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object’s target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets’ dependencies to link.
With BUILD_SHARED_LIBS, compiling units don't wait for preceding shared libs.
Regardless of BUILD_SHARED_LIBS, compile units in add_executable() don't wait for preceding libraries,
but targets by add_dependencies().

It doesn't break anything in llvm tree. Assume;
  add_executable(foo foo.cpp)
  target_link_libraries(foo LLVMCore) # depends on intrinsics_gen
Compiling foo.cpp doesn't wait for LLVMCore, but intrinsics_gen.
Linking foo waits for LLVMCore.

I have been working for cutting dependencies to increase parallelism.
For example, I introduced ENABLE_OBJLIB.
Ninja with CMake-3.9 doesn't require parallelize with objlib.

I love ninja-build.

Thanks,
Takumi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Loading...