[llvm-dev] RFC: LLVM Build System Future Direction

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
On Mon, 21 Oct 2019 at 11:26, Chris Bieneman via llvm-dev
<[hidden email]> wrote:
>
> Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.
>
> This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.
>
> I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.
>
> Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

Thanks for sharing this RFC Chris. I'd also like to highlight that the
LLVM test-suite still has parallel Make and CMake build systems - it
might be useful to consider if it's practical to retire the
Makefile-based build system?

Best,

Alex
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
Hi Chris,

This is a great initiative and it feels like the right direction.

I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
Currently it works pretty good, but some parts can be improved (somehow). E.g.:
 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
 - getting other configuration options (for example whether LLVM is built with or without exceptions)
 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject

I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.

Cheers,
Alex.

On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev


On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email]> wrote:

Hi Chris,

This is a great initiative and it feels like the right direction.

I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
Currently it works pretty good, but some parts can be improved (somehow). E.g.:

I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.

Louis

 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
 - getting other configuration options (for example whether LLVM is built with or without exceptions)
 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject

I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.

Cheers,
Alex.

On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
That works perfectly. 

But I’m referring to the case when I want to add LLVM as a sub project, which would allow me to debug/modify LLVM as if it is my code.
As I said, this also works, but some parts are not straightforward.

On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:


On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email]> wrote:

Hi Chris,

This is a great initiative and it feels like the right direction.

I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
Currently it works pretty good, but some parts can be improved (somehow). E.g.:

I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.


Louis

 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
 - getting other configuration options (for example whether LLVM is built with or without exceptions)
 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject

I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.

Cheers,
Alex.

On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
Sorry for the delay in writing this up and sending it out, but I wanted to recap the discussion from the roundtable on October 23rd. The roundtable ran for almost two hours and we discussed at most of the main points in my RFC. Thank you everyone who participated and contributed to the discussion!

TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all` really `all`. We should strive to reduce complexity and remove options, specifically options that aren't relevant to the monorepo. We should work to standardize workflows. We need to keep thinking about how to build runtime projects.

In my recap I'd like to fully disclose that I was standing up through most of the roundtable directing the conversation so I don't have much in the way of notes to go off. I may get details or things wrong, so please chime in to correct me.

We had a brief discussion around raising the minimum required CMake version. The general consensus was that since CMake provides binary packages for most common OSs, and building CMake from source has lower system requirements than LLVM and is very simple, nobody saw any barrier to adopting new versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it was James Knight who said the actual cost of updating the bots is the hard part in raising the version, so maybe we should just take the newest. That reasoning made sense to everyone at the roundtable and there were no objections at the roundtable to moving to CMake 3.15. Look for an RFC from me shortly that will propose that and lay out a timeline.

We spent a lot of time talking about a handful of other topics that all spring out from my RFC. There was a general agreement that the number of options in the build system is unwieldy and the combinatoric effect of the options is making the build system difficult to maintain. There was a general agreement at least in principal that we should work to reduce the complexity of the build system.

We did discuss specific build system functionality that could be simplified or removed, and on some points there was agreement, and on others there is need for more discussion. A few examples:

There was general agreement on the concept from my RFC that the `all` target should always really be `all`. That would mean removing the `LLVM_TOOL_*_BUILD` options.

There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.

There was disagreement over whether or not standalone builds of non-runtime sub-projects should remain. This specifically would relate to clang and lldb, and whether or not they can be built against installed or separately built copies of LLVM. There were points on both side of this discussion, and it will require more discussion to resolve.

The roundtable also focused a lot of time on the runtime libraries, and how they should be built. It is clear from the discussion that the runtime libraries need to support being built standalone (separately from LLVM) in order to fit into the various distribution strategies. Specifically many of the runtime libraries are sometimes shipped as OS components rather than as part of the toolchain, so you need to be able to build them separately from the toolchain. Those same libraries can also be shipped as part of a toolchain, so we need to support that workflow too. The need to support these disparate workflows has led to much of the complexity. The proposal from my RFC of standardizing runtimes builds as "standalone", and using the LLVM runtimes directory (which uses CMake's ExternalProject) to configure and build runtimes had pretty wide support in the discussion. I suspect this is the direction we should move in, but with some slight changes.

One of the points that came up was that building runtimes for multiple platforms at once can be a configuration nightmare requiring hundreds of build settings. It was suggested that it might be easier if you could build all of the runtime libraries with a single CMake invocation separately from LLVM. There are some dependency complications around the ordering of the builtin libraries build, but otherwise this is largely doable.

There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.

The last topic that was brought up at the end of the roundtable was about supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio ships a CMake integration that generates Ninja builds and supports the IDE UI interactions as if it were a Visual Studio project without being a project generator. Not supporting IDE build systems would clean up a lot of complexity in our build system. As was pointed out during the discussion, Xcode has no such support and is used by members of the community. There was discussion of whether or not an Xcode+Ninja generator could be created which would use Xcode's external build system mechanism to create an Xcode project for browsing, editing, and debugging, while using Ninja to build. The conversation had no real resolution other than "that would be cool" and "it would be nice to remove all the `if (XCODE)` blocks".

Thank you everyone who participated in the roundtable! If there is anything I missed please help fill in the blanks.

Thanks,
-Chris

On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev <[hidden email]> wrote:

That works perfectly. 

But I’m referring to the case when I want to add LLVM as a sub project, which would allow me to debug/modify LLVM as if it is my code.
As I said, this also works, but some parts are not straightforward.

On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:


On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email]> wrote:

Hi Chris,

This is a great initiative and it feels like the right direction.

I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
Currently it works pretty good, but some parts can be improved (somehow). E.g.:

I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.


Louis

 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
 - getting other configuration options (for example whether LLVM is built with or without exceptions)
 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject

I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.

Cheers,
Alex.

On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev

LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external clang, etc., and I agree that with the monorepo, there’s one canonical location for those sub-projects to live, and we don’t need to support it for those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other projects you want to include in your build but don’t necessarily want to place beside the llvm directory, e.g. Swift. We’ll continue supporting it for the latter use case, correct?

 

From: llvm-dev <[hidden email]> on behalf of Chris Bieneman via llvm-dev <[hidden email]>
Reply-To: Chris Bieneman <[hidden email]>
Date: Tuesday, October 29, 2019 at 10:10 AM
To: llvm-dev <[hidden email]>
Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction

 

Sorry for the delay in writing this up and sending it out, but I wanted to recap the discussion from the roundtable on October 23rd. The roundtable ran for almost two hours and we discussed at most of the main points in my RFC. Thank you everyone who participated and contributed to the discussion!

 

TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all` really `all`. We should strive to reduce complexity and remove options, specifically options that aren't relevant to the monorepo. We should work to standardize workflows. We need to keep thinking about how to build runtime projects.

 

In my recap I'd like to fully disclose that I was standing up through most of the roundtable directing the conversation so I don't have much in the way of notes to go off. I may get details or things wrong, so please chime in to correct me.

 

We had a brief discussion around raising the minimum required CMake version. The general consensus was that since CMake provides binary packages for most common OSs, and building CMake from source has lower system requirements than LLVM and is very simple, nobody saw any barrier to adopting new versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it was James Knight who said the actual cost of updating the bots is the hard part in raising the version, so maybe we should just take the newest. That reasoning made sense to everyone at the roundtable and there were no objections at the roundtable to moving to CMake 3.15. Look for an RFC from me shortly that will propose that and lay out a timeline.

 

We spent a lot of time talking about a handful of other topics that all spring out from my RFC. There was a general agreement that the number of options in the build system is unwieldy and the combinatoric effect of the options is making the build system difficult to maintain. There was a general agreement at least in principal that we should work to reduce the complexity of the build system.

 

We did discuss specific build system functionality that could be simplified or removed, and on some points there was agreement, and on others there is need for more discussion. A few examples:

 

There was general agreement on the concept from my RFC that the `all` target should always really be `all`. That would mean removing the `LLVM_TOOL_*_BUILD` options.

 

There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.

 

There was disagreement over whether or not standalone builds of non-runtime sub-projects should remain. This specifically would relate to clang and lldb, and whether or not they can be built against installed or separately built copies of LLVM. There were points on both side of this discussion, and it will require more discussion to resolve.

 

The roundtable also focused a lot of time on the runtime libraries, and how they should be built. It is clear from the discussion that the runtime libraries need to support being built standalone (separately from LLVM) in order to fit into the various distribution strategies. Specifically many of the runtime libraries are sometimes shipped as OS components rather than as part of the toolchain, so you need to be able to build them separately from the toolchain. Those same libraries can also be shipped as part of a toolchain, so we need to support that workflow too. The need to support these disparate workflows has led to much of the complexity. The proposal from my RFC of standardizing runtimes builds as "standalone", and using the LLVM runtimes directory (which uses CMake's ExternalProject) to configure and build runtimes had pretty wide support in the discussion. I suspect this is the direction we should move in, but with some slight changes.

 

One of the points that came up was that building runtimes for multiple platforms at once can be a configuration nightmare requiring hundreds of build settings. It was suggested that it might be easier if you could build all of the runtime libraries with a single CMake invocation separately from LLVM. There are some dependency complications around the ordering of the builtin libraries build, but otherwise this is largely doable.

 

There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.

 

The last topic that was brought up at the end of the roundtable was about supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio ships a CMake integration that generates Ninja builds and supports the IDE UI interactions as if it were a Visual Studio project without being a project generator. Not supporting IDE build systems would clean up a lot of complexity in our build system. As was pointed out during the discussion, Xcode has no such support and is used by members of the community. There was discussion of whether or not an Xcode+Ninja generator could be created which would use Xcode's external build system mechanism to create an Xcode project for browsing, editing, and debugging, while using Ninja to build. The conversation had no real resolution other than "that would be cool" and "it would be nice to remove all the `if (XCODE)` blocks".

 

Thank you everyone who participated in the roundtable! If there is anything I missed please help fill in the blanks.

 

Thanks,

-Chris



On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev <[hidden email]> wrote:

 

That works perfectly. 

 

But I’m referring to the case when I want to add LLVM as a sub project, which would allow me to debug/modify LLVM as if it is my code.

As I said, this also works, but some parts are not straightforward.

 

On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:

 



On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email]> wrote:

 

Hi Chris,

 

This is a great initiative and it feels like the right direction.

 

I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.

Currently it works pretty good, but some parts can be improved (somehow). E.g.:

 

I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.

 

 

Louis



 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future

 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me

 - getting other configuration options (for example whether LLVM is built with or without exceptions)

 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject

 

I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.

 

Cheers,

Alex.

 

On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:

Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

 

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev


On Oct 29, 2019, at 11:45 AM, Shoaib Meenai <[hidden email]> wrote:

LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external clang, etc., and I agree that with the monorepo, there’s one canonical location for those sub-projects to live, and we don’t need to support it for those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other projects you want to include in your build but don’t necessarily want to place beside the llvm directory, e.g. Swift. We’ll continue supporting it for the latter use case, correct?

Yes. I see no reason not to.

-Chris

 
From: llvm-dev <[hidden email]> on behalf of Chris Bieneman via llvm-dev <[hidden email]>
Reply-To: Chris Bieneman <[hidden email]>
Date: Tuesday, October 29, 2019 at 10:10 AM
To: llvm-dev <[hidden email]>
Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction
 
Sorry for the delay in writing this up and sending it out, but I wanted to recap the discussion from the roundtable on October 23rd. The roundtable ran for almost two hours and we discussed at most of the main points in my RFC. Thank you everyone who participated and contributed to the discussion!
 
TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all` really `all`. We should strive to reduce complexity and remove options, specifically options that aren't relevant to the monorepo. We should work to standardize workflows. We need to keep thinking about how to build runtime projects.
 
In my recap I'd like to fully disclose that I was standing up through most of the roundtable directing the conversation so I don't have much in the way of notes to go off. I may get details or things wrong, so please chime in to correct me.
 
We had a brief discussion around raising the minimum required CMake version. The general consensus was that since CMake provides binary packages for most common OSs, and building CMake from source has lower system requirements than LLVM and is very simple, nobody saw any barrier to adopting new versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it was James Knight who said the actual cost of updating the bots is the hard part in raising the version, so maybe we should just take the newest. That reasoning made sense to everyone at the roundtable and there were no objections at the roundtable to moving to CMake 3.15. Look for an RFC from me shortly that will propose that and lay out a timeline.
 
We spent a lot of time talking about a handful of other topics that all spring out from my RFC. There was a general agreement that the number of options in the build system is unwieldy and the combinatoric effect of the options is making the build system difficult to maintain. There was a general agreement at least in principal that we should work to reduce the complexity of the build system.
 
We did discuss specific build system functionality that could be simplified or removed, and on some points there was agreement, and on others there is need for more discussion. A few examples:
 
There was general agreement on the concept from my RFC that the `all` target should always really be `all`. That would mean removing the `LLVM_TOOL_*_BUILD` options.
 
There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
 
There was disagreement over whether or not standalone builds of non-runtime sub-projects should remain. This specifically would relate to clang and lldb, and whether or not they can be built against installed or separately built copies of LLVM. There were points on both side of this discussion, and it will require more discussion to resolve.
 
The roundtable also focused a lot of time on the runtime libraries, and how they should be built. It is clear from the discussion that the runtime libraries need to support being built standalone (separately from LLVM) in order to fit into the various distribution strategies. Specifically many of the runtime libraries are sometimes shipped as OS components rather than as part of the toolchain, so you need to be able to build them separately from the toolchain. Those same libraries can also be shipped as part of a toolchain, so we need to support that workflow too. The need to support these disparate workflows has led to much of the complexity. The proposal from my RFC of standardizing runtimes builds as "standalone", and using the LLVM runtimes directory (which uses CMake's ExternalProject) to configure and build runtimes had pretty wide support in the discussion. I suspect this is the direction we should move in, but with some slight changes.
 
One of the points that came up was that building runtimes for multiple platforms at once can be a configuration nightmare requiring hundreds of build settings. It was suggested that it might be easier if you could build all of the runtime libraries with a single CMake invocation separately from LLVM. There are some dependency complications around the ordering of the builtin libraries build, but otherwise this is largely doable.
 
There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.
 
The last topic that was brought up at the end of the roundtable was about supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio ships a CMake integration that generates Ninja builds and supports the IDE UI interactions as if it were a Visual Studio project without being a project generator. Not supporting IDE build systems would clean up a lot of complexity in our build system. As was pointed out during the discussion, Xcode has no such support and is used by members of the community. There was discussion of whether or not an Xcode+Ninja generator could be created which would use Xcode's external build system mechanism to create an Xcode project for browsing, editing, and debugging, while using Ninja to build. The conversation had no real resolution other than "that would be cool" and "it would be nice to remove all the `if (XCODE)` blocks".
 
Thank you everyone who participated in the roundtable! If there is anything I missed please help fill in the blanks.
 
Thanks,
-Chris


On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev <[hidden email]> wrote:
 
That works perfectly. 
 
But I’m referring to the case when I want to add LLVM as a sub project, which would allow me to debug/modify LLVM as if it is my code.
As I said, this also works, but some parts are not straightforward.
 
On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:
 


On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email]> wrote:
 
Hi Chris,
 
This is a great initiative and it feels like the right direction.
 
I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
Currently it works pretty good, but some parts can be improved (somehow). E.g.:
 
I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.
 
 
Louis


 - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
 - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
 - getting other configuration options (for example whether LLVM is built with or without exceptions)
 - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject
 
I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.
 
Cheers,
Alex.
 
On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email]> wrote:
Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.

This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.

I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.

Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.

## The Problem
Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.

This manifests itself in many unfortunate ways. Some examples are:

(1) There are three different ways to build compiler-rt as part of LLVM
(2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)

As the build system gains complexity maintaining the build system is getting more expensive to the community.

## Future Directions
The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.

### Updating CMake More Regularly
In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.

As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.

In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.

Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.

### Reducing the Test Matrix
The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations. 

Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.

We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.

Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.

### Adopting Conventions
Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.

Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.

One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:

#### Avoid Order Dependent Behavior
CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.

#### Avoid Options to Enable/Disable Configuration
If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.

#### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.

Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.

#### Making Sense of Runtime Builds
Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.

We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.

It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.

My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.

Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
 
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
On 10/29/2019 10:09 AM, Chris Bieneman via llvm-dev wrote:
> Sorry for the delay in writing this up and sending it out, but I wanted to recap the discussion from the roundtable on October 23rd. The roundtable ran for almost two hours and we discussed at most of the main points in my RFC. Thank you everyone who participated and contributed to the discussion!
>
> TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all` really `all`. We should strive to reduce complexity and remove options, specifically options that aren't relevant to the monorepo. We should work to standardize workflows. We need to keep thinking about how to build runtime projects.
>
> In my recap I'd like to fully disclose that I was standing up through most of the roundtable directing the conversation so I don't have much in the way of notes to go off. I may get details or things wrong, so please chime in to correct me.
>

We also briefly discussed removing llvm-config and distributing
pkg-config files on Linux.

-Tom

> We had a brief discussion around raising the minimum required CMake version. The general consensus was that since CMake provides binary packages for most common OSs, and building CMake from source has lower system requirements than LLVM and is very simple, nobody saw any barrier to adopting new versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it was James Knight who said the actual cost of updating the bots is the hard part in raising the version, so maybe we should just take the newest. That reasoning made sense to everyone at the roundtable and there were no objections at the roundtable to moving to CMake 3.15. Look for an RFC from me shortly that will propose that and lay out a timeline.
>
> We spent a lot of time talking about a handful of other topics that all spring out from my RFC. There was a general agreement that the number of options in the build system is unwieldy and the combinatoric effect of the options is making the build system difficult to maintain. There was a general agreement at least in principal that we should work to reduce the complexity of the build system.
>
> We did discuss specific build system functionality that could be simplified or removed, and on some points there was agreement, and on others there is need for more discussion. A few examples:
>
> There was general agreement on the concept from my RFC that the `all` target should always really be `all`. That would mean removing the `LLVM_TOOL_*_BUILD` options.
>
> There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
>
> There was disagreement over whether or not standalone builds of non-runtime sub-projects should remain. This specifically would relate to clang and lldb, and whether or not they can be built against installed or separately built copies of LLVM. There were points on both side of this discussion, and it will require more discussion to resolve.
>
> The roundtable also focused a lot of time on the runtime libraries, and how they should be built. It is clear from the discussion that the runtime libraries need to support being built standalone (separately from LLVM) in order to fit into the various distribution strategies. Specifically many of the runtime libraries are sometimes shipped as OS components rather than as part of the toolchain, so you need to be able to build them separately from the toolchain. Those same libraries can also be shipped as part of a toolchain, so we need to support that workflow too. The need to support these disparate workflows has led to much of the complexity. The proposal from my RFC of standardizing runtimes builds as "standalone", and using the LLVM runtimes directory (which uses CMake's ExternalProject) to configure and build runtimes had pretty wide support in the discussion. I suspect this is the direction we should move in, but with some slight changes.
>
> One of the points that came up was that building runtimes for multiple platforms at once can be a configuration nightmare requiring hundreds of build settings. It was suggested that it might be easier if you could build all of the runtime libraries with a single CMake invocation separately from LLVM. There are some dependency complications around the ordering of the builtin libraries build, but otherwise this is largely doable.
>
> There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.
>
> The last topic that was brought up at the end of the roundtable was about supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio ships a CMake integration that generates Ninja builds and supports the IDE UI interactions as if it were a Visual Studio project without being a project generator. Not supporting IDE build systems would clean up a lot of complexity in our build system. As was pointed out during the discussion, Xcode has no such support and is used by members of the community. There was discussion of whether or not an Xcode+Ninja generator could be created which would use Xcode's external build system mechanism to create an Xcode project for browsing, editing, and debugging, while using Ninja to build. The conversation had no real resolution other than "that would be cool" and "it would be nice to remove all the `if (XCODE)` blocks".
>
> Thank you everyone who participated in the roundtable! If there is anything I missed please help fill in the blanks.
>
> Thanks,
> -Chris
>
>> On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev <[hidden email] <mailto:[hidden email]>> wrote:
>>
>> That works perfectly.
>>
>> But I’m referring to the case when I want to add LLVM as a sub project, which would allow me to debug/modify LLVM as if it is my code.
>> As I said, this also works, but some parts are not straightforward.
>>
>> On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>
>>
>>>     On Oct 24, 2019, at 02:17, Alex Denisov via llvm-dev <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>     Hi Chris,
>>>
>>>     This is a great initiative and it feels like the right direction.
>>>
>>>     I'd like to add another point to the list: using LLVM as a library, i.e. being able to add it as a CMake subproject.
>>>     Currently it works pretty good, but some parts can be improved (somehow). E.g.:
>>
>>     I believe the CMake-correct way of doing that is to produce LLVM export files when installing LLVM, which would allow find_package to work out of the box. You’d get LLVM targets you can link against and all dependencies would be propagated by CMake properly.
>>
>>
>>     Louis
>>
>>>      - getting LLVM's version: the only way I've found is to parse the CMakeLists.txt with regexes and hope it doesn't break in the future
>>>      - getting include dirs info: IIRC currently it works transitively when one links against some library, but it feels like a weak point to me
>>>      - getting other configuration options (for example whether LLVM is built with or without exceptions)
>>>      - controlling how LLVM is built: the only way I found is to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON CACHE BOOL "" FORCE)) before adding LLVM as a subproject
>>>
>>>     I think my list is not exhaustive since it covers only my use cases, so maybe there are other opinions/requests from others.
>>>
>>>     Cheers,
>>>     Alex.
>>>
>>>     On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via llvm-dev <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>         Over the past few years the LLVM CMake build system has gained a lot of new features, which have powered new workflows and capabilities. This development has largely been individual efforts without focus or long-term direction. The build system is of incredible importance to LLVM as it is a primary interface for contributors to build and test their changes.
>>>
>>>         This year, LLVM is making a huge infrastructure shift to GitHub. Along with that shift many of the previously supported build options may not make sense. This is a prefect opportunity to revisit the state of our build infrastructure with an eye toward the future.
>>>
>>>         I would like this RFC, and any discussion it sparks, to serve as a starting point for a conversation we can continue at the LLVM Developer Meeting to set goals and conventions for the development of the build system.
>>>
>>>         Tom Stellard has scheduled a roundtable on CMake from 10:45-11:55 on Wednesday, Oct 23.
>>>
>>>         ## The Problem
>>>         Lacking clear direction and oversight the build system is evolving in rapidly divergent ways. Further since we don't have a formalized process for unifying workflows and deprecating old behaviors LLVM's build system has a convoluted feature set.
>>>
>>>         This manifests itself in many unfortunate ways. Some examples are:
>>>
>>>         (1) There are three different ways to build compiler-rt as part of LLVM
>>>         (2) There are lots of incompatible build configurations that need to be accounted for, some that aren't (like -DLLVM_BUILD_LLVM_DYLIB=On -DBUILD_SHARED_LIBS=On, which will explode at runtime)
>>>
>>>         As the build system gains complexity maintaining the build system is getting more expensive to the community.
>>>
>>>         ## Future Directions
>>>         The following are proposals to enable the build system to better facilitate LLVM development and provide a usable, extensible, and stable build system for LLVM and all sub-projects.
>>>
>>>         ### Updating CMake More Regularly
>>>         In the past we have clung to old versions of CMake for extended periods of time. This has resulted in significant checking `CMAKE_VERSION` to enable some features or being completely unable to use others. In particular recent CMake development extending generator expressions could provide substantial benefit to the project. If we stick to current upgrade policies, we may not be able to use the current CMake release for another few years.
>>>
>>>         As an alternative proposal, we should consider CMake upgrades independent of OS package manager versioning. That is not to say we should take every new version of CMake, however we should upgrade for compelling reasons.
>>>
>>>         In tree right now we have code gated on CMake 3.7 which enables CMake to generate Ninja rules for tablegen that process tablegen's dep files. This allows accurately rebuilding tablegen when included files change. I propose that this change's benefit should be sufficient to justify moving to CMake 3.7 for the project.
>>>
>>>         Additionally, building LLDB.framework on Darwin requires CMake 3.8 due to bugs in earlier versions of CMake. This could also be a justification for updating.
>>>         Lastly, getting updated versions of CMake is very easy. Kitware provides Windows, Mac and Linux builds on cmake.org <http://cmake.org/> as well as an Ubuntu apt source. If that is insufficient building CMake from source is simple, and has minimal system requirements. Visual Studio contains reasonably up-to-date CMake bundled. As such we should not allow OS release or support cycles to dictate when we upgrade CMake.
>>>
>>>         ### Reducing the Test Matrix
>>>         The most important guiding principal for development of the LLVM build system must be to reduce the matrix of configurations. The more possible configurations the build system supports the wider the test matrix. This is not to say the build system should support doing less, but rather to support less unique configurations.
>>>
>>>         Many configuration options in the build system just turn on or off different parts of the project. For example, the `LLVM_BUILD_LLVM_DYLIB` option just disables configuring libLLVM. An alternative approach would be to always configure libLLVM, and leave it up to users of the build system to determine whether or not to build it.
>>>
>>>         We also have options to enable and disable configuring individual tools in the LLVM and Clang projects. I believe we should eliminate those options, which will result in the `all` target always being everything. We have explicit clean dependencies for the `check-*` targets so most developer workflows should be un-impacted. Distribution workflows can use the `LLVM_DISTRIBUTION_COMPONENTS` option to hand tailor which parts of LLVM to build and install with better control without as much complication.
>>>
>>>         Many other options exist to support a wide variety of divergent workflows. For example, the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR` options exist to allow users to specify custom paths for projects so that, historically, they didn't need to nest clang inside LLVM. With the move to the mono-repo we should define consistent workflows and eliminate options that support divergent workflows.
>>>
>>>         ### Adopting Conventions
>>>         Much of LLVM's build system is not idiomatic CMake. Some of those differences make sense because LLVM is not a typical software project. I'm unaware of any build configuration system that was designed specifically to build compilers and handle the complex dependency chains that come with that territory.
>>>
>>>         Some of our divergences come from history. We have a great many features implemented in our CMake because CMake didn't support them at the time. We also have patterns that were appropriate before CMake added new features, and have never cleaned them up.
>>>
>>>         One big thing our build system needs is a set of guiding conventions to direct future development. Some key conventions that I believe are crucial:
>>>
>>>         #### Avoid Order Dependent Behavior
>>>         CMake generator expressions provide the ability to defer logic until after script processing. This allows the build system to avoid direct dependence on the order in which targets are processed. We should not use the `if(TARGET ...)` or `get_target_property` interfaces unless it is completely impossible to avoid.
>>>
>>>         #### Avoid Options to Enable/Disable Configuration
>>>         If we reduce the test matrix, having a convention to keep it reduced is of vital importance so that we don't find ourselves needing to clean up again in a few years.
>>>
>>>         #### Avoid Caching, Use `mark_as_advanced` and `INTERNAL` Liberally
>>>         CMake has no strategy for cache invalidation. As such, cached variables add additional maintenance burden because they can break builds sometimes in hard to diagnose ways. That said they are useful. In particular for things like configuration checks that are slow caching the result makes incremental re-configuration much faster. We should use cached values sparingly and only where they provide benefit.
>>>
>>>         Additionally, every cached CMake variable is a configuration point. Variables not marked `INTERNAL` show up in `ccmake` and `cmake-gui`, and variables not `mark_as_advanced` show up to all users. We should use the `INTERNAL` and `mark_as_advanced` options wherever appropriate to limit our supported configuration interface.
>>>
>>>         #### Making Sense of Runtime Builds
>>>         Right now, there are three different ways to build compiler-rt as part of LLVM and two different ways to build most of the other runtime libraries (libcxxabi, libcxx, libunwind, etc). This situation is confusing even for long time contributors.
>>>
>>>         We need a clearer story for building runtime libraries to reduce the number of different ways they are built and provide simplified workflows for users.
>>>
>>>         It is my opinion that if you are building a runtime library as part of an LLVM/Clang build, it should be configured and built with the in-tree clang as it would be for distribution. If you don't want to build with the in-tree clang, we should encourage people to build the runtime libraries independently of the compiler.
>>>
>>>         My reasoning for this is that distributions of clang are generally built from the default settings in the build and configuration process, and distributions (or installs by new users) which include the runtime libraries should have runtimes built with the just-built compiler. To align these two situations we need the default build configuration of LLVM+Clang+Runtimes to be using the just-built compiler.
>>>
>>>         Adopting this change would mean runtime library projects would only contain build system support for building "standalone" meaning not in the same configuration as LLVM. We would then support runtime libraries built as individual projects or using the LLVM runtimes directory, which separately configures and builds runtime libraries using the just-built clang.
>>>         _______________________________________________
>>>         LLVM Developers mailing list
>>>         [hidden email] <mailto:[hidden email]>
>>>         https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>     _______________________________________________
>>>     LLVM Developers mailing list
>>>     [hidden email] <mailto:[hidden email]>
>>>     https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email] <mailto:[hidden email]>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
Thanks for organizing the discussion and summarizing it. I haven't been involved, but it all seems like it's going in the right direction.

On Tue, Oct 29, 2019 at 10:10 AM Chris Bieneman via llvm-dev <[hidden email]> wrote:
There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.

Why split out builtins from compiler-rt? They were the original compiler runtime library. Should we go the other direction instead? For a long time, I've wondered if we should have split the sanitizer runtimes out of compiler-rt. GCC maintains a forked copy of some of the sanitizer runtimes in a libsanitizer subdirectory next to libgcc:
I think we put the sanitizers in compiler-rt mainly because it is a "compiler runtime", a runtime to which the compiler generates calls, but the same applies to libcxxabi, OpenMP, I assume libcl, and other things. The sanitizers are the biggest piece of compiler-rt by far, so maybe they deserve their own top level directory.

Here are the contents of compiler-rt/lib, with my categorizations:

Platform building blocks:
- crt
- builtins
- BlocksRuntime

Instrumentation tools:
- profile
- xray

Sanitizer or security things:
- cfi
- asan
- tsan
- ubsan
- ubsan_minimal
- dfsan
- fuzzer
- gwp_asan
- hwasan
- interception
- lsan
- msan
- safestack
- sanitizer_common
- stats
- scudo

---

All of the above said, I don't see any particularly strong reasons to change anything at the moment, but maybe you have a good reason to separate builtins and instrumentation that I haven't thought of.

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
Reid, I think your breakdown makes a lot of sense. Generally I think instrumentation tools (profile & xray) could be grouped with the sanitizers. More broadly those are all "developer tool" libraries.

Pulling crt, builtins, and blocks into a separate top-level project is, IMO, the right goal.

-Chris

On Oct 29, 2019, at 1:37 PM, Reid Kleckner <[hidden email]> wrote:

Thanks for organizing the discussion and summarizing it. I haven't been involved, but it all seems like it's going in the right direction.

On Tue, Oct 29, 2019 at 10:10 AM Chris Bieneman via llvm-dev <[hidden email]> wrote:
There are also problems related to cross-runtime dependencies which have come up recently. Some of these problems can be addressed more cleanly with modern CMake generator expressions, and other issues may require a larger restructuring of code. For years there have been discussions thrown around about breaking the builtins libraries out of compiler-rt. Maybe now is the time to consider doing that.

Why split out builtins from compiler-rt? They were the original compiler runtime library. Should we go the other direction instead? For a long time, I've wondered if we should have split the sanitizer runtimes out of compiler-rt. GCC maintains a forked copy of some of the sanitizer runtimes in a libsanitizer subdirectory next to libgcc:
I think we put the sanitizers in compiler-rt mainly because it is a "compiler runtime", a runtime to which the compiler generates calls, but the same applies to libcxxabi, OpenMP, I assume libcl, and other things. The sanitizers are the biggest piece of compiler-rt by far, so maybe they deserve their own top level directory.

Here are the contents of compiler-rt/lib, with my categorizations:

Platform building blocks:
- crt
- builtins
- BlocksRuntime

Instrumentation tools:
- profile
- xray

Sanitizer or security things:
- cfi
- asan
- tsan
- ubsan
- ubsan_minimal
- dfsan
- fuzzer
- gwp_asan
- hwasan
- interception
- lsan
- msan
- safestack
- sanitizer_common
- stats
- scudo

---

All of the above said, I don't see any particularly strong reasons to change anything at the moment, but maybe you have a good reason to separate builtins and instrumentation that I haven't thought of.


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
Speaking on how to locate/enable projects (like clang/lldb) - even with
the monorepo, I'm still enabling clang/lldb by symlinking them into
llvm/tools, for one specific reason: CMake and relative paths.

As long as all the source that is being built is below the toplevel CMake
directory, and the build directory itself is within the toplevel CMake
project directory, CMake uses relative paths in all build files.
(https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346 
contains some mention of this.)

This means, my build layout is that I build in llvm-project/llvm/build. In
this setup, CMake produces build files with paths e.g.
../tools/clang/. If I use LLVM_ENABLE_PROJECTS for enabling clang/lldb,
all clang/lldb source files are referenced by absolute path (instead of
../../clang).

Using relative paths everywhere has the nice (and for me, at this point,
essential) side effect of making the build output fully independent of the
path to the source tree, allowing ccache to share build artefacts across
two source trees at different locations. This allows builds to run a
couple order of magnitudes faster, if I've built the same code version in
a different code tree on the same machine.

I guess this could be overcome by adding another top level CMake file at
the top of the monorepo, allowing that to be the main source root of the
build. I'm not familiar enough with CMake to know if it's possible to just
make this a couple-lines CMakeLists.txt that includes llvm/CMakeLists.txt,
or if something else is needed to make this work.

// Martin

On Tue, 29 Oct 2019, Shoaib Meenai via llvm-dev wrote:

>
> LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external
> clang, etc., and I agree that with the monorepo, there’s one canonical
> location for those sub-projects to live, and we don’t need to support it for
> those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in
> conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other
> projects you want to include in your build but don’t necessarily want to
> place beside the llvm directory, e.g. Swift. We’ll continue supporting it
> for the latter use case, correct?
>
>  
>
> From: llvm-dev <[hidden email]> on behalf of Chris Bieneman
> via llvm-dev <[hidden email]>
> Reply-To: Chris Bieneman <[hidden email]>
> Date: Tuesday, October 29, 2019 at 10:10 AM
> To: llvm-dev <[hidden email]>
> Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction
>
>  
>
> Sorry for the delay in writing this up and sending it out, but I wanted to
> recap the discussion from the roundtable on October 23rd. The roundtable ran
> for almost two hours and we discussed at most of the main points in my RFC.
> Thank you everyone who participated and contributed to the discussion!
>
>  
>
> TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all`
> really `all`. We should strive to reduce complexity and remove options,
> specifically options that aren't relevant to the monorepo. We should work to
> standardize workflows. We need to keep thinking about how to build runtime
> projects.
>
>  
>
> In my recap I'd like to fully disclose that I was standing up through most
> of the roundtable directing the conversation so I don't have much in the way
> of notes to go off. I may get details or things wrong, so please chime in to
> correct me.
>
>  
>
> We had a brief discussion around raising the minimum required CMake version.
> The general consensus was that since CMake provides binary packages for most
> common OSs, and building CMake from source has lower system requirements
> than LLVM and is very simple, nobody saw any barrier to adopting new
> versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it
> was James Knight who said the actual cost of updating the bots is the hard
> part in raising the version, so maybe we should just take the newest. That
> reasoning made sense to everyone at the roundtable and there were no
> objections at the roundtable to moving to CMake 3.15. Look for an RFC from
> me shortly that will propose that and lay out a timeline.
>
>  
>
> We spent a lot of time talking about a handful of other topics that all
> spring out from my RFC. There was a general agreement that the number of
> options in the build system is unwieldy and the combinatoric effect of the
> options is making the build system difficult to maintain. There was a
> general agreement at least in principal that we should work to reduce the
> complexity of the build system.
>
>  
>
> We did discuss specific build system functionality that could be simplified
> or removed, and on some points there was agreement, and on others there is
> need for more discussion. A few examples:
>
>  
>
> There was general agreement on the concept from my RFC that the `all` target
> should always really be `all`. That would mean removing the
> `LLVM_TOOL_*_BUILD` options.
>
>  
>
> There was also agreement that in the monorepo it no longer makes sense to
> have an option to specify the source locations of sub-projects. That means
> we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
>
>  
>
> There was disagreement over whether or not standalone builds of non-runtime
> sub-projects should remain. This specifically would relate to clang and
> lldb, and whether or not they can be built against installed or separately
> built copies of LLVM. There were points on both side of this discussion, and
> it will require more discussion to resolve.
>
>  
>
> The roundtable also focused a lot of time on the runtime libraries, and how
> they should be built. It is clear from the discussion that the runtime
> libraries need to support being built standalone (separately from LLVM) in
> order to fit into the various distribution strategies. Specifically many of
> the runtime libraries are sometimes shipped as OS components rather than as
> part of the toolchain, so you need to be able to build them separately from
> the toolchain. Those same libraries can also be shipped as part of a
> toolchain, so we need to support that workflow too. The need to support
> these disparate workflows has led to much of the complexity. The proposal
> from my RFC of standardizing runtimes builds as "standalone", and using the
> LLVM runtimes directory (which uses CMake's ExternalProject) to configure
> and build runtimes had pretty wide support in the discussion. I suspect this
> is the direction we should move in, but with some slight changes.
>
>  
>
> One of the points that came up was that building runtimes for multiple
> platforms at once can be a configuration nightmare requiring hundreds of
> build settings. It was suggested that it might be easier if you could build
> all of the runtime libraries with a single CMake invocation separately from
> LLVM. There are some dependency complications around the ordering of the
> builtin libraries build, but otherwise this is largely doable.
>
>  
>
> There are also problems related to cross-runtime dependencies which have
> come up recently. Some of these problems can be addressed more cleanly with
> modern CMake generator expressions, and other issues may require a larger
> restructuring of code. For years there have been discussions thrown around
> about breaking the builtins libraries out of compiler-rt. Maybe now is the
> time to consider doing that.
>
>  
>
> The last topic that was brought up at the end of the roundtable was about
> supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio
> ships a CMake integration that generates Ninja builds and supports the IDE
> UI interactions as if it were a Visual Studio project without being a
> project generator. Not supporting IDE build systems would clean up a lot of
> complexity in our build system. As was pointed out during the discussion,
> Xcode has no such support and is used by members of the community. There was
> discussion of whether or not an Xcode+Ninja generator could be created which
> would use Xcode's external build system mechanism to create an Xcode project
> for browsing, editing, and debugging, while using Ninja to build. The
> conversation had no real resolution other than "that would be cool" and "it
> would be nice to remove all the `if (XCODE)` blocks".
>
>  
>
> Thank you everyone who participated in the roundtable! If there is anything
> I missed please help fill in the blanks.
>
>  
>
> Thanks,
>
> -Chris
>
>
>
>       On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev
>       <[hidden email]> wrote:
>
>  
>
> That works perfectly. 
>
>  
>
> But I’m referring to the case when I want to add LLVM as a sub
> project, which would allow me to debug/modify LLVM as if it is my
> code.
>
> As I said, this also works, but some parts are not straightforward.
>
>  
>
> On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:
>
>        
>
>
>
>             On Oct 24, 2019, at 02:17, Alex Denisov via
>             llvm-dev <[hidden email]> wrote:
>
>  
>
> Hi Chris,
>
>  
>
> This is a great initiative and it feels like the right
> direction.
>
>  
>
> I'd like to add another point to the list: using LLVM as a
> library, i.e. being able to add it as a CMake subproject.
>
> Currently it works pretty good, but some parts can be
> improved (somehow). E.g.:
>
>  
>
> I believe the CMake-correct way of doing that is to produce LLVM
> export files when installing LLVM, which would allow
> find_package to work out of the box. You’d get LLVM targets you
> can link against and all dependencies would be propagated by
> CMake properly.
>
>  
>
>  
>
> Louis
>
>
>
>        - getting LLVM's version: the only way I've found
>       is to parse the CMakeLists.txt with regexes and hope
>       it doesn't break in the future
>
>  - getting include dirs info: IIRC currently it works
> transitively when one links against some library, but it
> feels like a weak point to me
>
>  - getting other configuration options (for example
> whether LLVM is built with or without exceptions)
>
>  - controlling how LLVM is built: the only way I found is
> to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON
> CACHE BOOL "" FORCE)) before adding LLVM as a subproject
>
>  
>
> I think my list is not exhaustive since it covers only my
> use cases, so maybe there are other opinions/requests from
> others.
>
>  
>
> Cheers,
>
> Alex.
>
>  
>
> On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via
> llvm-dev <[hidden email]> wrote:
>
>       Over the past few years the LLVM CMake build
>       system has gained a lot of new features, which
>       have powered new workflows and capabilities.
>       This development has largely been individual
>       efforts without focus or long-term direction.
>       The build system is of incredible importance
>       to LLVM as it is a primary interface for
>       contributors to build and test their changes.
>
>       This year, LLVM is making a huge
>       infrastructure shift to GitHub. Along with
>       that shift many of the previously supported
>       build options may not make sense. This is a
>       prefect opportunity to revisit the state of
>       our build infrastructure with an eye toward
>       the future.
>
>       I would like this RFC, and any discussion it
>       sparks, to serve as a starting point for a
>       conversation we can continue at the LLVM
>       Developer Meeting to set goals and conventions
>       for the development of the build system.
>
>       Tom Stellard has scheduled a roundtable on
>       CMake from 10:45-11:55 on Wednesday, Oct 23.
>
>       ## The Problem
>       Lacking clear direction and oversight the
>       build system is evolving in rapidly divergent
>       ways. Further since we don't have a formalized
>       process for unifying workflows and deprecating
>       old behaviors LLVM's build system has a
>       convoluted feature set.
>
>       This manifests itself in many unfortunate
>       ways. Some examples are:
>
>       (1) There are three different ways to build
>       compiler-rt as part of LLVM
>       (2) There are lots of incompatible build
>       configurations that need to be accounted for,
>       some that aren't (like
>       -DLLVM_BUILD_LLVM_DYLIB=On
>       -DBUILD_SHARED_LIBS=On, which will explode at
>       runtime)
>
>       As the build system gains complexity
>       maintaining the build system is getting more
>       expensive to the community.
>
>       ## Future Directions
>       The following are proposals to enable the
>       build system to better facilitate LLVM
>       development and provide a usable, extensible,
>       and stable build system for LLVM and all
>       sub-projects.
>
>       ### Updating CMake More Regularly
>       In the past we have clung to old versions of
>       CMake for extended periods of time. This has
>       resulted in significant checking
>       `CMAKE_VERSION` to enable some features or
>       being completely unable to use others. In
>       particular recent CMake development extending
>       generator expressions could provide
>       substantial benefit to the project. If we
>       stick to current upgrade policies, we may not
>       be able to use the current CMake release for
>       another few years.
>
>       As an alternative proposal, we should consider
>       CMake upgrades independent of OS package
>       manager versioning. That is not to say we
>       should take every new version of CMake,
>       however we should upgrade for compelling
>       reasons.
>
>       In tree right now we have code gated on CMake
>       3.7 which enables CMake to generate Ninja
>       rules for tablegen that process tablegen's dep
>       files. This allows accurately rebuilding
>       tablegen when included files change. I propose
>       that this change's benefit should be
>       sufficient to justify moving to CMake 3.7 for
>       the project.
>
>       Additionally, building LLDB.framework on
>       Darwin requires CMake 3.8 due to bugs in
>       earlier versions of CMake. This could also be
>       a justification for updating.
>       Lastly, getting updated versions of CMake is
>       very easy. Kitware provides Windows, Mac and
>       Linux builds on cmake.org as well as an Ubuntu
>       apt source. If that is insufficient building
>       CMake from source is simple, and has minimal
>       system requirements. Visual Studio contains
>       reasonably up-to-date CMake bundled. As such
>       we should not allow OS release or support
>       cycles to dictate when we upgrade CMake.
>
>       ### Reducing the Test Matrix
>       The most important guiding principal for
>       development of the LLVM build system must be
>       to reduce the matrix of configurations. The
>       more possible configurations the build system
>       supports the wider the test matrix. This is
>       not to say the build system should support
>       doing less, but rather to support less unique
>       configurations.
>
>       Many configuration options in the build system
>       just turn on or off different parts of the
>       project. For example, the
>       `LLVM_BUILD_LLVM_DYLIB` option just disables
>       configuring libLLVM. An alternative approach
>       would be to always configure libLLVM, and
>       leave it up to users of the build system to
>       determine whether or not to build it.
>
>       We also have options to enable and disable
>       configuring individual tools in the LLVM and
>       Clang projects. I believe we should eliminate
>       those options, which will result in the `all`
>       target always being everything. We have
>       explicit clean dependencies for the `check-*`
>       targets so most developer workflows should be
>       un-impacted. Distribution workflows can use
>       the `LLVM_DISTRIBUTION_COMPONENTS` option to
>       hand tailor which parts of LLVM to build and
>       install with better control without as much
>       complication.
>
>       Many other options exist to support a wide
>       variety of divergent workflows. For example,
>       the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR`
>       options exist to allow users to specify custom
>       paths for projects so that, historically, they
>       didn't need to nest clang inside LLVM. With
>       the move to the mono-repo we should define
>       consistent workflows and eliminate options
>       that support divergent workflows.
>
>       ### Adopting Conventions
>       Much of LLVM's build system is not idiomatic
>       CMake. Some of those differences make sense
>       because LLVM is not a typical software
>       project. I'm unaware of any build
>       configuration system that was designed
>       specifically to build compilers and handle the
>       complex dependency chains that come with that
>       territory.
>
>       Some of our divergences come from history. We
>       have a great many features implemented in our
>       CMake because CMake didn't support them at the
>       time. We also have patterns that were
>       appropriate before CMake added new features,
>       and have never cleaned them up.
>
>       One big thing our build system needs is a set
>       of guiding conventions to direct future
>       development. Some key conventions that I
>       believe are crucial:
>
>       #### Avoid Order Dependent Behavior
>       CMake generator expressions provide the
>       ability to defer logic until after script
>       processing. This allows the build system to
>       avoid direct dependence on the order in which
>       targets are processed. We should not use the
>       `if(TARGET ...)` or `get_target_property`
>       interfaces unless it is completely impossible
>       to avoid.
>
>       #### Avoid Options to Enable/Disable
>       Configuration
>       If we reduce the test matrix, having a
>       convention to keep it reduced is of vital
>       importance so that we don't find ourselves
>       needing to clean up again in a few years.
>
>       #### Avoid Caching, Use `mark_as_advanced` and
>       `INTERNAL` Liberally
>       CMake has no strategy for cache invalidation.
>       As such, cached variables add additional
>       maintenance burden because they can break
>       builds sometimes in hard to diagnose ways.
>       That said they are useful. In particular for
>       things like configuration checks that are slow
>       caching the result makes incremental
>       re-configuration much faster. We should use
>       cached values sparingly and only where they
>       provide benefit.
>
>       Additionally, every cached CMake variable is a
>       configuration point. Variables not marked
>       `INTERNAL` show up in `ccmake` and
>       `cmake-gui`, and variables not
>       `mark_as_advanced` show up to all users. We
>       should use the `INTERNAL` and
>       `mark_as_advanced` options wherever
>       appropriate to limit our supported
>       configuration interface.
>
>       #### Making Sense of Runtime Builds
>       Right now, there are three different ways to
>       build compiler-rt as part of LLVM and two
>       different ways to build most of the other
>       runtime libraries (libcxxabi, libcxx,
>       libunwind, etc). This situation is confusing
>       even for long time contributors.
>
>       We need a clearer story for building runtime
>       libraries to reduce the number of different
>       ways they are built and provide simplified
>       workflows for users.
>
>       It is my opinion that if you are building a
>       runtime library as part of an LLVM/Clang
>       build, it should be configured and built with
>       the in-tree clang as it would be for
>       distribution. If you don't want to build with
>       the in-tree clang, we should encourage people
>       to build the runtime libraries independently
>       of the compiler.
>
>       My reasoning for this is that distributions of
>       clang are generally built from the default
>       settings in the build and configuration
>       process, and distributions (or installs by new
>       users) which include the runtime libraries
>       should have runtimes built with the just-built
>       compiler. To align these two situations we
>       need the default build configuration of
>       LLVM+Clang+Runtimes to be using the just-built
>       compiler.
>
>       Adopting this change would mean runtime
>       library projects would only contain build
>       system support for building "standalone"
>       meaning not in the same configuration as LLVM.
>       We would then support runtime libraries built
>       as individual projects or using the LLVM
>       runtimes directory, which separately
>       configures and builds runtime libraries using
>       the just-built clang.
>       _______________________________________________
>       LLVM Developers mailing list
>       [hidden email]
>       https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>  
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
On Tue, 29 Oct 2019, Reid Kleckner via llvm-dev wrote:

> Why split out builtins from compiler-rt? They were the original compiler
> runtime library. Should we go the other direction instead? For a long time,
> I've wondered if we should have split the sanitizer runtimes out of
> compiler-rt. GCC maintains a forked copy of some of the sanitizer runtimes
> in a libsanitizer subdirectory next to libgcc:
> https://github.com/gcc-mirror/gcc/tree/master/libsanitizer
> I think we put the sanitizers in compiler-rt mainly because it is a
> "compiler runtime", a runtime to which the compiler generates calls, but the
> same applies to libcxxabi, OpenMP, I assume libcl, and other things. The
> sanitizers are the biggest piece of compiler-rt by far, so maybe they
> deserve their own top level directory.

When building a cross toolchain from scratch, with no preexisting
libraries for the target, I end up building compiler-rt twice: First I
build only the builtins, which are necessary for a basic working C
environment. With builtins in place, I build libunwind/libcxxabi/libcxx,
to get a working C++ environment. After that, I build compiler-rt again,
building all of it, as the sanitizers now can be built as the C++
environment is complete.

(I haven't dug in very deep to see how much of a C++ standard library
actually is required by the sanitizers, or if it's just a case where I
could tell CMake to ignore the check for a "working" C++ compiler.)

// Martin

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev


> On Oct 29, 2019, at 1:50 PM, Martin Storsjö <[hidden email]> wrote:
>
> On Tue, 29 Oct 2019, Reid Kleckner via llvm-dev wrote:
>
>> Why split out builtins from compiler-rt? They were the original compiler
>> runtime library. Should we go the other direction instead? For a long time,
>> I've wondered if we should have split the sanitizer runtimes out of
>> compiler-rt. GCC maintains a forked copy of some of the sanitizer runtimes
>> in a libsanitizer subdirectory next to libgcc:
>> https://github.com/gcc-mirror/gcc/tree/master/libsanitizer
>> I think we put the sanitizers in compiler-rt mainly because it is a
>> "compiler runtime", a runtime to which the compiler generates calls, but the
>> same applies to libcxxabi, OpenMP, I assume libcl, and other things. The
>> sanitizers are the biggest piece of compiler-rt by far, so maybe they
>> deserve their own top level directory.
>
> When building a cross toolchain from scratch, with no preexisting libraries for the target, I end up building compiler-rt twice: First I build only the builtins, which are necessary for a basic working C environment. With builtins in place, I build libunwind/libcxxabi/libcxx, to get a working C++ environment. After that, I build compiler-rt again, building all of it, as the sanitizers now can be built as the C++ environment is complete.

This is the problem the llvm/runtimes directory seeks to solve. It builds the builtins separately from the rest of the runtimes, then builds everything else together.

>
> (I haven't dug in very deep to see how much of a C++ standard library actually is required by the sanitizers, or if it's just a case where I could tell CMake to ignore the check for a "working" C++ compiler.)

My understanding is the sanitizers only require the c++ headers, not the library.

-Chris

>
> // Martin
>

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev


On Tue, Oct 29, 2019 at 1:46 PM Martin Storsjö via llvm-dev <[hidden email]> wrote:
Speaking on how to locate/enable projects (like clang/lldb) - even with
the monorepo, I'm still enabling clang/lldb by symlinking them into
llvm/tools, for one specific reason: CMake and relative paths.

As long as all the source that is being built is below the toplevel CMake
directory, and the build directory itself is within the toplevel CMake
project directory, CMake uses relative paths in all build files.
(https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346
contains some mention of this.)

This means, my build layout is that I build in llvm-project/llvm/build. In
this setup, CMake produces build files with paths e.g.
../tools/clang/. If I use LLVM_ENABLE_PROJECTS for enabling clang/lldb,
all clang/lldb source files are referenced by absolute path (instead of
../../clang).

Using relative paths everywhere has the nice (and for me, at this point,
essential) side effect of making the build output fully independent of the
path to the source tree, allowing ccache to share build artefacts across
two source trees at different locations. This allows builds to run a
couple order of magnitudes faster, if I've built the same code version in
a different code tree on the same machine.

I guess this could be overcome by adding another top level CMake file at
the top of the monorepo, allowing that to be the main source root of the
build.

This seems to be a natural evolution of the monorepo to me to have a top-level CMake instead of pointing at the one nested in the `llvm/` directory.
I wasn't at the round table, was this mentioned?

-- 
Mehdi


 
I'm not familiar enough with CMake to know if it's possible to just
make this a couple-lines CMakeLists.txt that includes llvm/CMakeLists.txt,
or if something else is needed to make this work.

// Martin

On Tue, 29 Oct 2019, Shoaib Meenai via llvm-dev wrote:

>
> LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external
> clang, etc., and I agree that with the monorepo, there’s one canonical
> location for those sub-projects to live, and we don’t need to support it for
> those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in
> conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other
> projects you want to include in your build but don’t necessarily want to
> place beside the llvm directory, e.g. Swift. We’ll continue supporting it
> for the latter use case, correct?
>
>  
>
> From: llvm-dev <[hidden email]> on behalf of Chris Bieneman
> via llvm-dev <[hidden email]>
> Reply-To: Chris Bieneman <[hidden email]>
> Date: Tuesday, October 29, 2019 at 10:10 AM
> To: llvm-dev <[hidden email]>
> Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction
>
>  
>
> Sorry for the delay in writing this up and sending it out, but I wanted to
> recap the discussion from the roundtable on October 23rd. The roundtable ran
> for almost two hours and we discussed at most of the main points in my RFC.
> Thank you everyone who participated and contributed to the discussion!
>
>  
>
> TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all`
> really `all`. We should strive to reduce complexity and remove options,
> specifically options that aren't relevant to the monorepo. We should work to
> standardize workflows. We need to keep thinking about how to build runtime
> projects.
>
>  
>
> In my recap I'd like to fully disclose that I was standing up through most
> of the roundtable directing the conversation so I don't have much in the way
> of notes to go off. I may get details or things wrong, so please chime in to
> correct me.
>
>  
>
> We had a brief discussion around raising the minimum required CMake version.
> The general consensus was that since CMake provides binary packages for most
> common OSs, and building CMake from source has lower system requirements
> than LLVM and is very simple, nobody saw any barrier to adopting new
> versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it
> was James Knight who said the actual cost of updating the bots is the hard
> part in raising the version, so maybe we should just take the newest. That
> reasoning made sense to everyone at the roundtable and there were no
> objections at the roundtable to moving to CMake 3.15. Look for an RFC from
> me shortly that will propose that and lay out a timeline.
>
>  
>
> We spent a lot of time talking about a handful of other topics that all
> spring out from my RFC. There was a general agreement that the number of
> options in the build system is unwieldy and the combinatoric effect of the
> options is making the build system difficult to maintain. There was a
> general agreement at least in principal that we should work to reduce the
> complexity of the build system.
>
>  
>
> We did discuss specific build system functionality that could be simplified
> or removed, and on some points there was agreement, and on others there is
> need for more discussion. A few examples:
>
>  
>
> There was general agreement on the concept from my RFC that the `all` target
> should always really be `all`. That would mean removing the
> `LLVM_TOOL_*_BUILD` options.
>
>  
>
> There was also agreement that in the monorepo it no longer makes sense to
> have an option to specify the source locations of sub-projects. That means
> we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
>
>  
>
> There was disagreement over whether or not standalone builds of non-runtime
> sub-projects should remain. This specifically would relate to clang and
> lldb, and whether or not they can be built against installed or separately
> built copies of LLVM. There were points on both side of this discussion, and
> it will require more discussion to resolve.
>
>  
>
> The roundtable also focused a lot of time on the runtime libraries, and how
> they should be built. It is clear from the discussion that the runtime
> libraries need to support being built standalone (separately from LLVM) in
> order to fit into the various distribution strategies. Specifically many of
> the runtime libraries are sometimes shipped as OS components rather than as
> part of the toolchain, so you need to be able to build them separately from
> the toolchain. Those same libraries can also be shipped as part of a
> toolchain, so we need to support that workflow too. The need to support
> these disparate workflows has led to much of the complexity. The proposal
> from my RFC of standardizing runtimes builds as "standalone", and using the
> LLVM runtimes directory (which uses CMake's ExternalProject) to configure
> and build runtimes had pretty wide support in the discussion. I suspect this
> is the direction we should move in, but with some slight changes.
>
>  
>
> One of the points that came up was that building runtimes for multiple
> platforms at once can be a configuration nightmare requiring hundreds of
> build settings. It was suggested that it might be easier if you could build
> all of the runtime libraries with a single CMake invocation separately from
> LLVM. There are some dependency complications around the ordering of the
> builtin libraries build, but otherwise this is largely doable.
>
>  
>
> There are also problems related to cross-runtime dependencies which have
> come up recently. Some of these problems can be addressed more cleanly with
> modern CMake generator expressions, and other issues may require a larger
> restructuring of code. For years there have been discussions thrown around
> about breaking the builtins libraries out of compiler-rt. Maybe now is the
> time to consider doing that.
>
>  
>
> The last topic that was brought up at the end of the roundtable was about
> supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio
> ships a CMake integration that generates Ninja builds and supports the IDE
> UI interactions as if it were a Visual Studio project without being a
> project generator. Not supporting IDE build systems would clean up a lot of
> complexity in our build system. As was pointed out during the discussion,
> Xcode has no such support and is used by members of the community. There was
> discussion of whether or not an Xcode+Ninja generator could be created which
> would use Xcode's external build system mechanism to create an Xcode project
> for browsing, editing, and debugging, while using Ninja to build. The
> conversation had no real resolution other than "that would be cool" and "it
> would be nice to remove all the `if (XCODE)` blocks".
>
>  
>
> Thank you everyone who participated in the roundtable! If there is anything
> I missed please help fill in the blanks.
>
>  
>
> Thanks,
>
> -Chris
>
>
>
>       On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev
>       <[hidden email]> wrote:
>
>  
>
> That works perfectly. 
>
>  
>
> But I’m referring to the case when I want to add LLVM as a sub
> project, which would allow me to debug/modify LLVM as if it is my
> code.
>
> As I said, this also works, but some parts are not straightforward.
>
>  
>
> On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:
>
>        
>
>
>
>             On Oct 24, 2019, at 02:17, Alex Denisov via
>             llvm-dev <[hidden email]> wrote:
>
>  
>
> Hi Chris,
>
>  
>
> This is a great initiative and it feels like the right
> direction.
>
>  
>
> I'd like to add another point to the list: using LLVM as a
> library, i.e. being able to add it as a CMake subproject.
>
> Currently it works pretty good, but some parts can be
> improved (somehow). E.g.:
>
>  
>
> I believe the CMake-correct way of doing that is to produce LLVM
> export files when installing LLVM, which would allow
> find_package to work out of the box. You’d get LLVM targets you
> can link against and all dependencies would be propagated by
> CMake properly.
>
>  
>
>  
>
> Louis
>
>
>
>        - getting LLVM's version: the only way I've found
>       is to parse the CMakeLists.txt with regexes and hope
>       it doesn't break in the future
>
>  - getting include dirs info: IIRC currently it works
> transitively when one links against some library, but it
> feels like a weak point to me
>
>  - getting other configuration options (for example
> whether LLVM is built with or without exceptions)
>
>  - controlling how LLVM is built: the only way I found is
> to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON
> CACHE BOOL "" FORCE)) before adding LLVM as a subproject
>
>  
>
> I think my list is not exhaustive since it covers only my
> use cases, so maybe there are other opinions/requests from
> others.
>
>  
>
> Cheers,
>
> Alex.
>
>  
>
> On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via
> llvm-dev <[hidden email]> wrote:
>
>       Over the past few years the LLVM CMake build
>       system has gained a lot of new features, which
>       have powered new workflows and capabilities.
>       This development has largely been individual
>       efforts without focus or long-term direction.
>       The build system is of incredible importance
>       to LLVM as it is a primary interface for
>       contributors to build and test their changes.
>
>       This year, LLVM is making a huge
>       infrastructure shift to GitHub. Along with
>       that shift many of the previously supported
>       build options may not make sense. This is a
>       prefect opportunity to revisit the state of
>       our build infrastructure with an eye toward
>       the future.
>
>       I would like this RFC, and any discussion it
>       sparks, to serve as a starting point for a
>       conversation we can continue at the LLVM
>       Developer Meeting to set goals and conventions
>       for the development of the build system.
>
>       Tom Stellard has scheduled a roundtable on
>       CMake from 10:45-11:55 on Wednesday, Oct 23.
>
>       ## The Problem
>       Lacking clear direction and oversight the
>       build system is evolving in rapidly divergent
>       ways. Further since we don't have a formalized
>       process for unifying workflows and deprecating
>       old behaviors LLVM's build system has a
>       convoluted feature set.
>
>       This manifests itself in many unfortunate
>       ways. Some examples are:
>
>       (1) There are three different ways to build
>       compiler-rt as part of LLVM
>       (2) There are lots of incompatible build
>       configurations that need to be accounted for,
>       some that aren't (like
>       -DLLVM_BUILD_LLVM_DYLIB=On
>       -DBUILD_SHARED_LIBS=On, which will explode at
>       runtime)
>
>       As the build system gains complexity
>       maintaining the build system is getting more
>       expensive to the community.
>
>       ## Future Directions
>       The following are proposals to enable the
>       build system to better facilitate LLVM
>       development and provide a usable, extensible,
>       and stable build system for LLVM and all
>       sub-projects.
>
>       ### Updating CMake More Regularly
>       In the past we have clung to old versions of
>       CMake for extended periods of time. This has
>       resulted in significant checking
>       `CMAKE_VERSION` to enable some features or
>       being completely unable to use others. In
>       particular recent CMake development extending
>       generator expressions could provide
>       substantial benefit to the project. If we
>       stick to current upgrade policies, we may not
>       be able to use the current CMake release for
>       another few years.
>
>       As an alternative proposal, we should consider
>       CMake upgrades independent of OS package
>       manager versioning. That is not to say we
>       should take every new version of CMake,
>       however we should upgrade for compelling
>       reasons.
>
>       In tree right now we have code gated on CMake
>       3.7 which enables CMake to generate Ninja
>       rules for tablegen that process tablegen's dep
>       files. This allows accurately rebuilding
>       tablegen when included files change. I propose
>       that this change's benefit should be
>       sufficient to justify moving to CMake 3.7 for
>       the project.
>
>       Additionally, building LLDB.framework on
>       Darwin requires CMake 3.8 due to bugs in
>       earlier versions of CMake. This could also be
>       a justification for updating.
>       Lastly, getting updated versions of CMake is
>       very easy. Kitware provides Windows, Mac and
>       Linux builds on cmake.org as well as an Ubuntu
>       apt source. If that is insufficient building
>       CMake from source is simple, and has minimal
>       system requirements. Visual Studio contains
>       reasonably up-to-date CMake bundled. As such
>       we should not allow OS release or support
>       cycles to dictate when we upgrade CMake.
>
>       ### Reducing the Test Matrix
>       The most important guiding principal for
>       development of the LLVM build system must be
>       to reduce the matrix of configurations. The
>       more possible configurations the build system
>       supports the wider the test matrix. This is
>       not to say the build system should support
>       doing less, but rather to support less unique
>       configurations.
>
>       Many configuration options in the build system
>       just turn on or off different parts of the
>       project. For example, the
>       `LLVM_BUILD_LLVM_DYLIB` option just disables
>       configuring libLLVM. An alternative approach
>       would be to always configure libLLVM, and
>       leave it up to users of the build system to
>       determine whether or not to build it.
>
>       We also have options to enable and disable
>       configuring individual tools in the LLVM and
>       Clang projects. I believe we should eliminate
>       those options, which will result in the `all`
>       target always being everything. We have
>       explicit clean dependencies for the `check-*`
>       targets so most developer workflows should be
>       un-impacted. Distribution workflows can use
>       the `LLVM_DISTRIBUTION_COMPONENTS` option to
>       hand tailor which parts of LLVM to build and
>       install with better control without as much
>       complication.
>
>       Many other options exist to support a wide
>       variety of divergent workflows. For example,
>       the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR`
>       options exist to allow users to specify custom
>       paths for projects so that, historically, they
>       didn't need to nest clang inside LLVM. With
>       the move to the mono-repo we should define
>       consistent workflows and eliminate options
>       that support divergent workflows.
>
>       ### Adopting Conventions
>       Much of LLVM's build system is not idiomatic
>       CMake. Some of those differences make sense
>       because LLVM is not a typical software
>       project. I'm unaware of any build
>       configuration system that was designed
>       specifically to build compilers and handle the
>       complex dependency chains that come with that
>       territory.
>
>       Some of our divergences come from history. We
>       have a great many features implemented in our
>       CMake because CMake didn't support them at the
>       time. We also have patterns that were
>       appropriate before CMake added new features,
>       and have never cleaned them up.
>
>       One big thing our build system needs is a set
>       of guiding conventions to direct future
>       development. Some key conventions that I
>       believe are crucial:
>
>       #### Avoid Order Dependent Behavior
>       CMake generator expressions provide the
>       ability to defer logic until after script
>       processing. This allows the build system to
>       avoid direct dependence on the order in which
>       targets are processed. We should not use the
>       `if(TARGET ...)` or `get_target_property`
>       interfaces unless it is completely impossible
>       to avoid.
>
>       #### Avoid Options to Enable/Disable
>       Configuration
>       If we reduce the test matrix, having a
>       convention to keep it reduced is of vital
>       importance so that we don't find ourselves
>       needing to clean up again in a few years.
>
>       #### Avoid Caching, Use `mark_as_advanced` and
>       `INTERNAL` Liberally
>       CMake has no strategy for cache invalidation.
>       As such, cached variables add additional
>       maintenance burden because they can break
>       builds sometimes in hard to diagnose ways.
>       That said they are useful. In particular for
>       things like configuration checks that are slow
>       caching the result makes incremental
>       re-configuration much faster. We should use
>       cached values sparingly and only where they
>       provide benefit.
>
>       Additionally, every cached CMake variable is a
>       configuration point. Variables not marked
>       `INTERNAL` show up in `ccmake` and
>       `cmake-gui`, and variables not
>       `mark_as_advanced` show up to all users. We
>       should use the `INTERNAL` and
>       `mark_as_advanced` options wherever
>       appropriate to limit our supported
>       configuration interface.
>
>       #### Making Sense of Runtime Builds
>       Right now, there are three different ways to
>       build compiler-rt as part of LLVM and two
>       different ways to build most of the other
>       runtime libraries (libcxxabi, libcxx,
>       libunwind, etc). This situation is confusing
>       even for long time contributors.
>
>       We need a clearer story for building runtime
>       libraries to reduce the number of different
>       ways they are built and provide simplified
>       workflows for users.
>
>       It is my opinion that if you are building a
>       runtime library as part of an LLVM/Clang
>       build, it should be configured and built with
>       the in-tree clang as it would be for
>       distribution. If you don't want to build with
>       the in-tree clang, we should encourage people
>       to build the runtime libraries independently
>       of the compiler.
>
>       My reasoning for this is that distributions of
>       clang are generally built from the default
>       settings in the build and configuration
>       process, and distributions (or installs by new
>       users) which include the runtime libraries
>       should have runtimes built with the just-built
>       compiler. To align these two situations we
>       need the default build configuration of
>       LLVM+Clang+Runtimes to be using the just-built
>       compiler.
>
>       Adopting this change would mean runtime
>       library projects would only contain build
>       system support for building "standalone"
>       meaning not in the same configuration as LLVM.
>       We would then support runtime libraries built
>       as individual projects or using the LLVM
>       runtimes directory, which separately
>       configures and builds runtime libraries using
>       the just-built clang.
>       _______________________________________________
>       LLVM Developers mailing list
>       [hidden email]
>       https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>  
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
>_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev

This page will show which cmake versions install with the different Ubuntu releases.

xenial 16.04 cmake 3.5.1

bionic 18.04 cmake 3.10.2

cosmic 18.10 cmake 3.12.1

disco 19.04 cmake 3.13.4

eoan 19.10 cmake 3.13.4

19.10 is a new release as of 10/20. And it is common to use the LTS releases with the last one being 18.04, cmake 3.10.2.

For around the last decade or so it has been bad form to override the standard release packages, a rule I expect is well known to those on the LTS releases. Of course those on different OS's will have different standards but as with all these software package dependencies there is likely some release level that does not overly burden a primary number of users, while still burdening the remainder, but where nevertheless a reasonable line has to be drawn.

Drawing the line at everyone, which could well be the case here, may be more than what is required.

Neil Nelson 

On 10/29/19 11:09 AM, Chris Bieneman via llvm-dev wrote:
...

TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all` really `all`. We should strive to reduce complexity and remove options, specifically options that aren't relevant to the monorepo. We should work to standardize workflows. We need to keep thinking about how to build runtime projects.
...

We had a brief discussion around raising the minimum required CMake version. The general consensus was that since CMake provides binary packages for most common OSs, and building CMake from source has lower system requirements than LLVM and is very simple, nobody saw any barrier to adopting new versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it was James Knight who said the actual cost of updating the bots is the hard part in raising the version, so maybe we should just take the newest. That reasoning made sense to everyone at the roundtable and there were no objections at the roundtable to moving to CMake 3.15. Look for an RFC from me shortly that will propose that and lay out a timeline.

...

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev


On Oct 29, 2019, at 7:09 PM, Chris Bieneman via llvm-dev <[hidden email]> wrote:

There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.


Hi Chris,

What about genuine external projects? For example, I build Swift “unified” as a part of LLVM but I can’t keep the Swift source inside of the llvm-project directory due to how Git submodules work. In other words:

$ cd my_project
$ ls toolchain
cmark
llvm-project
swift
$ git submodule status
 bfa95d55b535fa178f75484b5e8f82ae3d8517af toolchain/cmark
 ea51a1ab2980d055c68820888ce255242f387c91 toolchain/llvm-project
+cec05255029d0ce354825e5530a47a5affb03975 toolchain/swift 

Dave

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev


On Oct 29, 2019, at 8:09 PM, Mehdi AMINI via llvm-dev <[hidden email]> wrote:



On Tue, Oct 29, 2019 at 1:46 PM Martin Storsjö via llvm-dev <[hidden email]> wrote:
Speaking on how to locate/enable projects (like clang/lldb) - even with
the monorepo, I'm still enabling clang/lldb by symlinking them into
llvm/tools, for one specific reason: CMake and relative paths.

As long as all the source that is being built is below the toplevel CMake
directory, and the build directory itself is within the toplevel CMake
project directory, CMake uses relative paths in all build files.
(https://github.com/Kitware/CMake/blob/master/Source/cmLocalGenerator.h#L338-L346
contains some mention of this.)

This means, my build layout is that I build in llvm-project/llvm/build. In
this setup, CMake produces build files with paths e.g.
../tools/clang/. If I use LLVM_ENABLE_PROJECTS for enabling clang/lldb,
all clang/lldb source files are referenced by absolute path (instead of
../../clang).

Using relative paths everywhere has the nice (and for me, at this point,
essential) side effect of making the build output fully independent of the
path to the source tree, allowing ccache to share build artefacts across
two source trees at different locations. This allows builds to run a
couple order of magnitudes faster, if I've built the same code version in
a different code tree on the same machine.

I guess this could be overcome by adding another top level CMake file at
the top of the monorepo, allowing that to be the main source root of the
build.

This seems to be a natural evolution of the monorepo to me to have a top-level CMake instead of pointing at the one nested in the `llvm/` directory.
I wasn't at the round table, was this mentioned?

Yes, briefly, although there are some things we'll need to fix to make that work. LLVM does some unconventional things in CMake to make the binary directory structure roughly match the install directory structure. Some of those don't behave well if LLVM isn't the top-level directory. This is all resolvable by cleaning up usage of various CMake-defined variables.

-Chris



-- 
Mehdi


 
I'm not familiar enough with CMake to know if it's possible to just
make this a couple-lines CMakeLists.txt that includes llvm/CMakeLists.txt,
or if something else is needed to make this work.

// Martin

On Tue, 29 Oct 2019, Shoaib Meenai via llvm-dev wrote:

>
> LLVM_EXTERNAL_*_SOURCE_DIR can be used for specifying paths to external
> clang, etc., and I agree that with the monorepo, there’s one canonical
> location for those sub-projects to live, and we don’t need to support it for
> those subprojects. However, LLVM_EXTERNAL_*_SOURCE_DIR can also be used in
> conjunction with LLVM_EXTERNAL_PROJECTS to specify the paths to other
> projects you want to include in your build but don’t necessarily want to
> place beside the llvm directory, e.g. Swift. We’ll continue supporting it
> for the latter use case, correct?
>
>  
>
> From: llvm-dev <[hidden email]> on behalf of Chris Bieneman
> via llvm-dev <[hidden email]>
> Reply-To: Chris Bieneman <[hidden email]>
> Date: Tuesday, October 29, 2019 at 10:10 AM
> To: llvm-dev <[hidden email]>
> Subject: Re: [llvm-dev] RFC: LLVM Build System Future Direction
>
>  
>
> Sorry for the delay in writing this up and sending it out, but I wanted to
> recap the discussion from the roundtable on October 23rd. The roundtable ran
> for almost two hours and we discussed at most of the main points in my RFC.
> Thank you everyone who participated and contributed to the discussion!
>
>  
>
> TL;DR: We should move to CMake 3.15 (RFC incoming). We should make `all`
> really `all`. We should strive to reduce complexity and remove options,
> specifically options that aren't relevant to the monorepo. We should work to
> standardize workflows. We need to keep thinking about how to build runtime
> projects.
>
>  
>
> In my recap I'd like to fully disclose that I was standing up through most
> of the roundtable directing the conversation so I don't have much in the way
> of notes to go off. I may get details or things wrong, so please chime in to
> correct me.
>
>  
>
> We had a brief discussion around raising the minimum required CMake version.
> The general consensus was that since CMake provides binary packages for most
> common OSs, and building CMake from source has lower system requirements
> than LLVM and is very simple, nobody saw any barrier to adopting new
> versions. Initially I suggested moving to CMake 3.11 or 3.12, I believe it
> was James Knight who said the actual cost of updating the bots is the hard
> part in raising the version, so maybe we should just take the newest. That
> reasoning made sense to everyone at the roundtable and there were no
> objections at the roundtable to moving to CMake 3.15. Look for an RFC from
> me shortly that will propose that and lay out a timeline.
>
>  
>
> We spent a lot of time talking about a handful of other topics that all
> spring out from my RFC. There was a general agreement that the number of
> options in the build system is unwieldy and the combinatoric effect of the
> options is making the build system difficult to maintain. There was a
> general agreement at least in principal that we should work to reduce the
> complexity of the build system.
>
>  
>
> We did discuss specific build system functionality that could be simplified
> or removed, and on some points there was agreement, and on others there is
> need for more discussion. A few examples:
>
>  
>
> There was general agreement on the concept from my RFC that the `all` target
> should always really be `all`. That would mean removing the
> `LLVM_TOOL_*_BUILD` options.
>
>  
>
> There was also agreement that in the monorepo it no longer makes sense to
> have an option to specify the source locations of sub-projects. That means
> we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.
>
>  
>
> There was disagreement over whether or not standalone builds of non-runtime
> sub-projects should remain. This specifically would relate to clang and
> lldb, and whether or not they can be built against installed or separately
> built copies of LLVM. There were points on both side of this discussion, and
> it will require more discussion to resolve.
>
>  
>
> The roundtable also focused a lot of time on the runtime libraries, and how
> they should be built. It is clear from the discussion that the runtime
> libraries need to support being built standalone (separately from LLVM) in
> order to fit into the various distribution strategies. Specifically many of
> the runtime libraries are sometimes shipped as OS components rather than as
> part of the toolchain, so you need to be able to build them separately from
> the toolchain. Those same libraries can also be shipped as part of a
> toolchain, so we need to support that workflow too. The need to support
> these disparate workflows has led to much of the complexity. The proposal
> from my RFC of standardizing runtimes builds as "standalone", and using the
> LLVM runtimes directory (which uses CMake's ExternalProject) to configure
> and build runtimes had pretty wide support in the discussion. I suspect this
> is the direction we should move in, but with some slight changes.
>
>  
>
> One of the points that came up was that building runtimes for multiple
> platforms at once can be a configuration nightmare requiring hundreds of
> build settings. It was suggested that it might be easier if you could build
> all of the runtime libraries with a single CMake invocation separately from
> LLVM. There are some dependency complications around the ordering of the
> builtin libraries build, but otherwise this is largely doable.
>
>  
>
> There are also problems related to cross-runtime dependencies which have
> come up recently. Some of these problems can be addressed more cleanly with
> modern CMake generator expressions, and other issues may require a larger
> restructuring of code. For years there have been discussions thrown around
> about breaking the builtins libraries out of compiler-rt. Maybe now is the
> time to consider doing that.
>
>  
>
> The last topic that was brought up at the end of the roundtable was about
> supporting IDE generators. Saleem Abdulrasool pointed out that Visual Studio
> ships a CMake integration that generates Ninja builds and supports the IDE
> UI interactions as if it were a Visual Studio project without being a
> project generator. Not supporting IDE build systems would clean up a lot of
> complexity in our build system. As was pointed out during the discussion,
> Xcode has no such support and is used by members of the community. There was
> discussion of whether or not an Xcode+Ninja generator could be created which
> would use Xcode's external build system mechanism to create an Xcode project
> for browsing, editing, and debugging, while using Ninja to build. The
> conversation had no real resolution other than "that would be cool" and "it
> would be nice to remove all the `if (XCODE)` blocks".
>
>  
>
> Thank you everyone who participated in the roundtable! If there is anything
> I missed please help fill in the blanks.
>
>  
>
> Thanks,
>
> -Chris
>
>
>
>       On Oct 24, 2019, at 10:28 AM, Alex Denisov via llvm-dev
>       <[hidden email]> wrote:
>
>  
>
> That works perfectly. 
>
>  
>
> But I’m referring to the case when I want to add LLVM as a sub
> project, which would allow me to debug/modify LLVM as if it is my
> code.
>
> As I said, this also works, but some parts are not straightforward.
>
>  
>
> On Thu 24. Oct 2019 at 18:15, Louis Dionne <[hidden email]> wrote:
>
>        
>
>
>
>             On Oct 24, 2019, at 02:17, Alex Denisov via
>             llvm-dev <[hidden email]> wrote:
>
>  
>
> Hi Chris,
>
>  
>
> This is a great initiative and it feels like the right
> direction.
>
>  
>
> I'd like to add another point to the list: using LLVM as a
> library, i.e. being able to add it as a CMake subproject.
>
> Currently it works pretty good, but some parts can be
> improved (somehow). E.g.:
>
>  
>
> I believe the CMake-correct way of doing that is to produce LLVM
> export files when installing LLVM, which would allow
> find_package to work out of the box. You’d get LLVM targets you
> can link against and all dependencies would be propagated by
> CMake properly.
>
>  
>
>  
>
> Louis
>
>
>
>        - getting LLVM's version: the only way I've found
>       is to parse the CMakeLists.txt with regexes and hope
>       it doesn't break in the future
>
>  - getting include dirs info: IIRC currently it works
> transitively when one links against some library, but it
> feels like a weak point to me
>
>  - getting other configuration options (for example
> whether LLVM is built with or without exceptions)
>
>  - controlling how LLVM is built: the only way I found is
> to force set a variable (i.e. set (LLVM_BUILD_32_BITS ON
> CACHE BOOL "" FORCE)) before adding LLVM as a subproject
>
>  
>
> I think my list is not exhaustive since it covers only my
> use cases, so maybe there are other opinions/requests from
> others.
>
>  
>
> Cheers,
>
> Alex.
>
>  
>
> On Mon, Oct 21, 2019 at 8:26 PM Chris Bieneman via
> llvm-dev <[hidden email]> wrote:
>
>       Over the past few years the LLVM CMake build
>       system has gained a lot of new features, which
>       have powered new workflows and capabilities.
>       This development has largely been individual
>       efforts without focus or long-term direction.
>       The build system is of incredible importance
>       to LLVM as it is a primary interface for
>       contributors to build and test their changes.
>
>       This year, LLVM is making a huge
>       infrastructure shift to GitHub. Along with
>       that shift many of the previously supported
>       build options may not make sense. This is a
>       prefect opportunity to revisit the state of
>       our build infrastructure with an eye toward
>       the future.
>
>       I would like this RFC, and any discussion it
>       sparks, to serve as a starting point for a
>       conversation we can continue at the LLVM
>       Developer Meeting to set goals and conventions
>       for the development of the build system.
>
>       Tom Stellard has scheduled a roundtable on
>       CMake from 10:45-11:55 on Wednesday, Oct 23.
>
>       ## The Problem
>       Lacking clear direction and oversight the
>       build system is evolving in rapidly divergent
>       ways. Further since we don't have a formalized
>       process for unifying workflows and deprecating
>       old behaviors LLVM's build system has a
>       convoluted feature set.
>
>       This manifests itself in many unfortunate
>       ways. Some examples are:
>
>       (1) There are three different ways to build
>       compiler-rt as part of LLVM
>       (2) There are lots of incompatible build
>       configurations that need to be accounted for,
>       some that aren't (like
>       -DLLVM_BUILD_LLVM_DYLIB=On
>       -DBUILD_SHARED_LIBS=On, which will explode at
>       runtime)
>
>       As the build system gains complexity
>       maintaining the build system is getting more
>       expensive to the community.
>
>       ## Future Directions
>       The following are proposals to enable the
>       build system to better facilitate LLVM
>       development and provide a usable, extensible,
>       and stable build system for LLVM and all
>       sub-projects.
>
>       ### Updating CMake More Regularly
>       In the past we have clung to old versions of
>       CMake for extended periods of time. This has
>       resulted in significant checking
>       `CMAKE_VERSION` to enable some features or
>       being completely unable to use others. In
>       particular recent CMake development extending
>       generator expressions could provide
>       substantial benefit to the project. If we
>       stick to current upgrade policies, we may not
>       be able to use the current CMake release for
>       another few years.
>
>       As an alternative proposal, we should consider
>       CMake upgrades independent of OS package
>       manager versioning. That is not to say we
>       should take every new version of CMake,
>       however we should upgrade for compelling
>       reasons.
>
>       In tree right now we have code gated on CMake
>       3.7 which enables CMake to generate Ninja
>       rules for tablegen that process tablegen's dep
>       files. This allows accurately rebuilding
>       tablegen when included files change. I propose
>       that this change's benefit should be
>       sufficient to justify moving to CMake 3.7 for
>       the project.
>
>       Additionally, building LLDB.framework on
>       Darwin requires CMake 3.8 due to bugs in
>       earlier versions of CMake. This could also be
>       a justification for updating.
>       Lastly, getting updated versions of CMake is
>       very easy. Kitware provides Windows, Mac and
>       Linux builds on cmake.org as well as an Ubuntu
>       apt source. If that is insufficient building
>       CMake from source is simple, and has minimal
>       system requirements. Visual Studio contains
>       reasonably up-to-date CMake bundled. As such
>       we should not allow OS release or support
>       cycles to dictate when we upgrade CMake.
>
>       ### Reducing the Test Matrix
>       The most important guiding principal for
>       development of the LLVM build system must be
>       to reduce the matrix of configurations. The
>       more possible configurations the build system
>       supports the wider the test matrix. This is
>       not to say the build system should support
>       doing less, but rather to support less unique
>       configurations.
>
>       Many configuration options in the build system
>       just turn on or off different parts of the
>       project. For example, the
>       `LLVM_BUILD_LLVM_DYLIB` option just disables
>       configuring libLLVM. An alternative approach
>       would be to always configure libLLVM, and
>       leave it up to users of the build system to
>       determine whether or not to build it.
>
>       We also have options to enable and disable
>       configuring individual tools in the LLVM and
>       Clang projects. I believe we should eliminate
>       those options, which will result in the `all`
>       target always being everything. We have
>       explicit clean dependencies for the `check-*`
>       targets so most developer workflows should be
>       un-impacted. Distribution workflows can use
>       the `LLVM_DISTRIBUTION_COMPONENTS` option to
>       hand tailor which parts of LLVM to build and
>       install with better control without as much
>       complication.
>
>       Many other options exist to support a wide
>       variety of divergent workflows. For example,
>       the `LLVM_EXTERNAL_${PROJECT}_SOURCE_DIR`
>       options exist to allow users to specify custom
>       paths for projects so that, historically, they
>       didn't need to nest clang inside LLVM. With
>       the move to the mono-repo we should define
>       consistent workflows and eliminate options
>       that support divergent workflows.
>
>       ### Adopting Conventions
>       Much of LLVM's build system is not idiomatic
>       CMake. Some of those differences make sense
>       because LLVM is not a typical software
>       project. I'm unaware of any build
>       configuration system that was designed
>       specifically to build compilers and handle the
>       complex dependency chains that come with that
>       territory.
>
>       Some of our divergences come from history. We
>       have a great many features implemented in our
>       CMake because CMake didn't support them at the
>       time. We also have patterns that were
>       appropriate before CMake added new features,
>       and have never cleaned them up.
>
>       One big thing our build system needs is a set
>       of guiding conventions to direct future
>       development. Some key conventions that I
>       believe are crucial:
>
>       #### Avoid Order Dependent Behavior
>       CMake generator expressions provide the
>       ability to defer logic until after script
>       processing. This allows the build system to
>       avoid direct dependence on the order in which
>       targets are processed. We should not use the
>       `if(TARGET ...)` or `get_target_property`
>       interfaces unless it is completely impossible
>       to avoid.
>
>       #### Avoid Options to Enable/Disable
>       Configuration
>       If we reduce the test matrix, having a
>       convention to keep it reduced is of vital
>       importance so that we don't find ourselves
>       needing to clean up again in a few years.
>
>       #### Avoid Caching, Use `mark_as_advanced` and
>       `INTERNAL` Liberally
>       CMake has no strategy for cache invalidation.
>       As such, cached variables add additional
>       maintenance burden because they can break
>       builds sometimes in hard to diagnose ways.
>       That said they are useful. In particular for
>       things like configuration checks that are slow
>       caching the result makes incremental
>       re-configuration much faster. We should use
>       cached values sparingly and only where they
>       provide benefit.
>
>       Additionally, every cached CMake variable is a
>       configuration point. Variables not marked
>       `INTERNAL` show up in `ccmake` and
>       `cmake-gui`, and variables not
>       `mark_as_advanced` show up to all users. We
>       should use the `INTERNAL` and
>       `mark_as_advanced` options wherever
>       appropriate to limit our supported
>       configuration interface.
>
>       #### Making Sense of Runtime Builds
>       Right now, there are three different ways to
>       build compiler-rt as part of LLVM and two
>       different ways to build most of the other
>       runtime libraries (libcxxabi, libcxx,
>       libunwind, etc). This situation is confusing
>       even for long time contributors.
>
>       We need a clearer story for building runtime
>       libraries to reduce the number of different
>       ways they are built and provide simplified
>       workflows for users.
>
>       It is my opinion that if you are building a
>       runtime library as part of an LLVM/Clang
>       build, it should be configured and built with
>       the in-tree clang as it would be for
>       distribution. If you don't want to build with
>       the in-tree clang, we should encourage people
>       to build the runtime libraries independently
>       of the compiler.
>
>       My reasoning for this is that distributions of
>       clang are generally built from the default
>       settings in the build and configuration
>       process, and distributions (or installs by new
>       users) which include the runtime libraries
>       should have runtimes built with the just-built
>       compiler. To align these two situations we
>       need the default build configuration of
>       LLVM+Clang+Runtimes to be using the just-built
>       compiler.
>
>       Adopting this change would mean runtime
>       library projects would only contain build
>       system support for building "standalone"
>       meaning not in the same configuration as LLVM.
>       We would then support runtime libraries built
>       as individual projects or using the LLVM
>       runtimes directory, which separately
>       configures and builds runtime libraries using
>       the just-built clang.
>       _______________________________________________
>       LLVM Developers mailing list
>       [hidden email]
>       https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>  
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
>_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
In reply to this post by Doerfert, Johannes via llvm-dev
Dave,

We would keep the ability to specify source paths for things that live outside the mono-repo, just not for things inside the repository. This would have the impact of supporting `LLVM_EXTERNAL_*_SOURCE_DIR` for projects specified in `LLVM_EXTERNAL_PROJECTS` but not projects specified with `LLVM_ENABLE_PROJECTS`.

Does that address your concerns?

-Chris

On Oct 30, 2019, at 6:37 AM, David Zarzycki via llvm-dev <[hidden email]> wrote:



On Oct 29, 2019, at 7:09 PM, Chris Bieneman via llvm-dev <[hidden email]> wrote:

There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.


Hi Chris,

What about genuine external projects? For example, I build Swift “unified” as a part of LLVM but I can’t keep the Swift source inside of the llvm-project directory due to how Git submodules work. In other words:

$ cd my_project
$ ls toolchain
cmark
llvm-project
swift
$ git submodule status
 bfa95d55b535fa178f75484b5e8f82ae3d8517af toolchain/cmark
 ea51a1ab2980d055c68820888ce255242f387c91 toolchain/llvm-project
+cec05255029d0ce354825e5530a47a5affb03975 toolchain/swift 

Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: LLVM Build System Future Direction

Doerfert, Johannes via llvm-dev
Yes. Thanks!

On Nov 3, 2019, at 8:09 PM, Chris Bieneman <[hidden email]> wrote:

Dave,

We would keep the ability to specify source paths for things that live outside the mono-repo, just not for things inside the repository. This would have the impact of supporting `LLVM_EXTERNAL_*_SOURCE_DIR` for projects specified in `LLVM_EXTERNAL_PROJECTS` but not projects specified with `LLVM_ENABLE_PROJECTS`.

Does that address your concerns?

-Chris

On Oct 30, 2019, at 6:37 AM, David Zarzycki via llvm-dev <[hidden email]> wrote:



On Oct 29, 2019, at 7:09 PM, Chris Bieneman via llvm-dev <[hidden email]> wrote:

There was also agreement that in the monorepo it no longer makes sense to have an option to specify the source locations of sub-projects. That means we can remove the `LLVM_EXTERNAL_*_SOURCE_DIR` variables.


Hi Chris,

What about genuine external projects? For example, I build Swift “unified” as a part of LLVM but I can’t keep the Swift source inside of the llvm-project directory due to how Git submodules work. In other words:

$ cd my_project
$ ls toolchain
cmark
llvm-project
swift
$ git submodule status
 bfa95d55b535fa178f75484b5e8f82ae3d8517af toolchain/cmark
 ea51a1ab2980d055c68820888ce255242f387c91 toolchain/llvm-project
+cec05255029d0ce354825e5530a47a5affb03975 toolchain/swift 

Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
12