[RFC] Ideas on improving Compiler-RT CMake

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

[RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman
LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake build system feature equivalent to the autotools one. One of the biggest outstanding tasks is PR 21562, which I've been viewing more as a "We really need to fix how we build Compiler-RT."

I've thought about this quite a bit, and I wanted to send out some of my ideas to get some feedback and have some discussions about the way forward.

The complication with Compiler-RT is that it is inherently a cross-compile. When you build LLVM, Clang and Compiler-RT, you really want to build LLVM, Clang, and n Compiler-RTs where n is the valid combinations of target architectures and/or platforms your clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host
(2) Building a cross-targeting Clang that runs on one OS or architecture and targets another
(3) Building a Clang that both self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to handle case 1 as an in-tree build similar to how it is today, case 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS, when I configure my build directory there will be an in-tree build of Compiler-RT (via add_llvm_external_project) to generate the OS X runtime libraries, and an out-of-tree-style cross-compile generated for building the iOS runtime libraries for all supported architectures. This would all be driven via command line settings like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to target only FreeBSD when you configured your build directory there would only be an out-of-tree-style cross-compile generated for your target architecture against FreeBSD. The command line setting here would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an in-tree build with outputs:
libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file settings, so if you were cross-targeting and need to set compiler options like -isysroot there will need to be a toolchain file defined for your target platform.

Using toolchain files and cross-compile configurations solves one of the problems with our current CMake system where the Compiler-RT builds don't re-run the CMake checks. Not re-running the CMake checks causes significant problems if your host and target OS are different.

So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a plan.

The first thing I'd like to do is try to make a bunch of the Darwin-specific code go away. Obviously there will always need to be some at a lower level because Darwin is different, but some of the lower-level functions like add_compiler_rt_darwin_object_library probably don't need to exist.

One example of the type of thing I want to get rid of is the ASan CMakeLists.txt. The top-level CMake files for libraries and executables shouldn't need to care about the targets. Today we loop over OSs for darwin and call add_compiler_rt_darwin_object_library, and for other operating systems we loop over archs and call add_compiler_rt_object_library. I want to push the looping as low down into the utility functions as possible. Doing this will reduce the amount of Darwin-specific code because we replicate this exact looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to generate binaries for just OSX or just iOS. Today the Darwin build tries to build everything all the time, and that really isn't the behavior we need if we're going to treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt combined build use cross-compilation for building the iOS runtime libraries. I want to do this on Darwin only because that's the platform I'm most familiar with. At some point if we want to move other targets to cross-compiling I will need help from other people in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and trying to streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks,
-Chris
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman
Looping in cfe-dev because I should have send this there too from the start.

-Chris

> On Jun 1, 2015, at 3:21 PM, Chris Bieneman <[hidden email]> wrote:
>
> LLVMDev,
>
> PR 15732 is the umbrella tracking the progress of making the CMake build system feature equivalent to the autotools one. One of the biggest outstanding tasks is PR 21562, which I've been viewing more as a "We really need to fix how we build Compiler-RT."
>
> I've thought about this quite a bit, and I wanted to send out some of my ideas to get some feedback and have some discussions about the way forward.
>
> The complication with Compiler-RT is that it is inherently a cross-compile. When you build LLVM, Clang and Compiler-RT, you really want to build LLVM, Clang, and n Compiler-RTs where n is the valid combinations of target architectures and/or platforms your clang supports.
>
> There are three basic configurations for building Compiler-RT.
>
> (1) Building a "pure" Clang supporting targeting your host
> (2) Building a cross-targeting Clang that runs on one OS or architecture and targets another
> (3) Building a Clang that both self-targets and cross-targets
>
> The overall change I want to make to Compiler-RT is that I want to handle case 1 as an in-tree build similar to how it is today, case 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.
>
> Let me walk through an example of how I think this should work.
>
> If my host is OS X and I'm building Clang to target OS X and iOS, when I configure my build directory there will be an in-tree build of Compiler-RT (via add_llvm_external_project) to generate the OS X runtime libraries, and an out-of-tree-style cross-compile generated for building the iOS runtime libraries for all supported architectures. This would all be driven via command line settings like:
>
> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
>
> This would generate universal binary outputs in the format:
> libclangrt_<lib>.<os>.<ext>
>
> As a second example, if you were on Windows building a Clang to target only FreeBSD when you configured your build directory there would only be an out-of-tree-style cross-compile generated for your target architecture against FreeBSD. The command line setting here would be:
>
> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
>
> This would generate binary outputs in the format:
> libclangrt_<lib>.<arch>.<ext>
>
> Similarly if you were on linux targeting linux you’d only get an in-tree build with outputs:
> libclangrt_<lib>.<arch>.<ext>
>
> The cross-compile builds will be driven via CMake's toolchain file settings, so if you were cross-targeting and need to set compiler options like -isysroot there will need to be a toolchain file defined for your target platform.
>
> Using toolchain files and cross-compile configurations solves one of the problems with our current CMake system where the Compiler-RT builds don't re-run the CMake checks. Not re-running the CMake checks causes significant problems if your host and target OS are different.
>
> So, how do we get to this state?
>
> Honestly I don't have a complete plan, but I have some parts of a plan.
>
> The first thing I'd like to do is try to make a bunch of the Darwin-specific code go away. Obviously there will always need to be some at a lower level because Darwin is different, but some of the lower-level functions like add_compiler_rt_darwin_object_library probably don't need to exist.
>
> One example of the type of thing I want to get rid of is the ASan CMakeLists.txt. The top-level CMake files for libraries and executables shouldn't need to care about the targets. Today we loop over OSs for darwin and call add_compiler_rt_darwin_object_library, and for other operating systems we loop over archs and call add_compiler_rt_object_library. I want to push the looping as low down into the utility functions as possible. Doing this will reduce the amount of Darwin-specific code because we replicate this exact looping behavior all over Compiler-RT's build.
>
> The second thing I want to do is make the Darwin build able to generate binaries for just OSX or just iOS. Today the Darwin build tries to build everything all the time, and that really isn't the behavior we need if we're going to treat iOS as a cross-compile.
>
> The third step is to make the Darwin llvm/clang/compiler-rt combined build use cross-compilation for building the iOS runtime libraries. I want to do this on Darwin only because that's the platform I'm most familiar with. At some point if we want to move other targets to cross-compiling I will need help from other people in the community who are more familiar with the specific targets.
>
> With each step I'll be trying to remove more OS-specific code and trying to streamline building.
>
> Questions/Comments/Concerns/Cries of insanity?
>
> Thanks,
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Jonathan Roelofs


On 6/1/15 6:42 PM, Chris Bieneman wrote:

> Looping in cfe-dev because I should have send this there too from the
> start.
>
> -Chris
>
>> On Jun 1, 2015, at 3:21 PM, Chris Bieneman <[hidden email]>
>> wrote:
>>
>> LLVMDev,
>>
>> PR 15732 is the umbrella tracking the progress of making the CMake
>> build system feature equivalent to the autotools one. One of the
>> biggest outstanding tasks is PR 21562, which I've been viewing more
>> as a "We really need to fix how we build Compiler-RT."

+1

>>
>> I've thought about this quite a bit, and I wanted to send out some
>> of my ideas to get some feedback and have some discussions about
>> the way forward.
>>
>> The complication with Compiler-RT is that it is inherently a
>> cross-compile. When you build LLVM, Clang and Compiler-RT, you
>> really want to build LLVM, Clang, and n Compiler-RTs where n is the
>> valid combinations of target architectures and/or platforms your
>> clang supports.
>>
>> There are three basic configurations for building Compiler-RT.
>>
>> (1) Building a "pure" Clang supporting targeting your host (2)
>> Building a cross-targeting Clang that runs on one OS or
>> architecture and targets another (3) Building a Clang that both
>> self-targets and cross-targets
>>
>> The overall change I want to make to Compiler-RT is that I want to
>> handle case 1 as an in-tree build similar to how it is today, case
>> 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Why treat $host as a special case? IM-not-so-humble-O, $host is just
another $target.

I think the ideal case is to have a table of:

   (triple, compiler, flags, sysroot)

tuples that say which configurations of the library to build.

If Clang is compiler-rt's compiler, then all of the compiler entries in
that table can point to the same binary. For in-tree builds, we *should*
be using the "just built" Clang anyway (but are not currently, without
enabling the flag to do so). Either way, I think we ought to move away
from thinking that a native build is in some way "special".

If GCC is compiler-rt's compiler, then there would have to have a
different binary per triple.

If we drop support for building compiler-rt with GCC, this gets even
simpler. Compiler-rt is *Clang's* runtime library, after all.

The plus side of treating *every* build as a cross build is that then
there will be less risk of builds differing between native builds and
cross builds when the target is the same (for example, linux targeting
linux vs darwin targeting linux).

>>
>> Let me walk through an example of how I think this should work.
>>
>> If my host is OS X and I'm building Clang to target OS X and iOS,
>> when I configure my build directory there will be an in-tree build
>> of Compiler-RT (via add_llvm_external_project) to generate the OS X
>> runtime libraries, and an out-of-tree-style cross-compile generated
>> for building the iOS runtime libraries for all supported
>> architectures. This would all be driven via command line settings
>> like:
>>
>> LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
>> LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"
>>
>> This would generate universal binary outputs in the format:
>> libclangrt_<lib>.<os>.<ext>
>>
>> As a second example, if you were on Windows building a Clang to
>> target only FreeBSD when you configured your build directory there
>> would only be an out-of-tree-style cross-compile generated for your
>> target architecture against FreeBSD. The command line setting here
>> would be:
>>
>> LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"
>>
>> This would generate binary outputs in the format:
>> libclangrt_<lib>.<arch>.<ext>
>>
>> Similarly if you were on linux targeting linux you’d only get an
>> in-tree build with outputs: libclangrt_<lib>.<arch>.<ext>
>>
>> The cross-compile builds will be driven via CMake's toolchain file
>> settings, so if you were cross-targeting and need to set compiler
>> options like -isysroot there will need to be a toolchain file
>> defined for your target platform.
>>
>> Using toolchain files and cross-compile configurations solves one
>> of the problems with our current CMake system where the Compiler-RT
>> builds don't re-run the CMake checks. Not re-running the CMake
>> checks causes significant problems if your host and target OS are
>> different.

Can you elaborate on that last sentence? I'm not very familiar with
CMake, and that's a bit of a surprise to me.

>>
>> So, how do we get to this state?
>>
>> Honestly I don't have a complete plan, but I have some parts of a
>> plan.
>>
>> The first thing I'd like to do is try to make a bunch of the
>> Darwin-specific code go away. Obviously there will always need to
>> be some at a lower level because Darwin is different, but some of
>> the lower-level functions like
>> add_compiler_rt_darwin_object_library probably don't need to
>> exist.
>>
>> One example of the type of thing I want to get rid of is the ASan
>> CMakeLists.txt. The top-level CMake files for libraries and
>> executables shouldn't need to care about the targets. Today we loop
>> over OSs for darwin and call add_compiler_rt_darwin_object_library,
>> and for other operating systems we loop over archs and call
>> add_compiler_rt_object_library. I want to push the looping as low
>> down into the utility functions as possible. Doing this will reduce
>> the amount of Darwin-specific code because we replicate this exact
>> looping behavior all over Compiler-RT's build.
>>
>> The second thing I want to do is make the Darwin build able to
>> generate binaries for just OSX or just iOS. Today the Darwin build
>> tries to build everything all the time, and that really isn't the
>> behavior we need if we're going to treat iOS as a cross-compile.
>>
>> The third step is to make the Darwin llvm/clang/compiler-rt
>> combined build use cross-compilation for building the iOS runtime
>> libraries. I want to do this on Darwin only because that's the
>> platform I'm most familiar with. At some point if we want to move
>> other targets to cross-compiling I will need help from other people
>> in the community who are more familiar with the specific targets.
>>
>> With each step I'll be trying to remove more OS-specific code and
>> trying to streamline building.
>>
>> Questions/Comments/Concerns/Cries of insanity?

Thanks for taking the driver's seat on the discussions of all this CMake
transition stuff, BTW.


Cheers,

Jon

>>
>> Thanks, -Chris _______________________________________________ LLVM
>> Developers mailing list [hidden email]
>> http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
> _______________________________________________ LLVM Developers
> mailing list [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman

On Jun 1, 2015, at 6:26 PM, Jonathan Roelofs <[hidden email]> wrote:



On 6/1/15 6:42 PM, Chris Bieneman wrote:
Looping in cfe-dev because I should have send this there too from the
start.

-Chris

On Jun 1, 2015, at 3:21 PM, Chris Bieneman <[hidden email]>
wrote:

LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake
build system feature equivalent to the autotools one. One of the
biggest outstanding tasks is PR 21562, which I've been viewing more
as a "We really need to fix how we build Compiler-RT."

+1


I've thought about this quite a bit, and I wanted to send out some
of my ideas to get some feedback and have some discussions about
the way forward.

The complication with Compiler-RT is that it is inherently a
cross-compile. When you build LLVM, Clang and Compiler-RT, you
really want to build LLVM, Clang, and n Compiler-RTs where n is the
valid combinations of target architectures and/or platforms your
clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host (2)
Building a cross-targeting Clang that runs on one OS or
architecture and targets another (3) Building a Clang that both
self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to
handle case 1 as an in-tree build similar to how it is today, case
2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Why treat $host as a special case? IM-not-so-humble-O, $host is just another $target.

Conceptually I agree, but treating host differently here is a build-time optimization. Treating it as a cross-compile bears CMake-induced build time overhead, which can be avoided in the simple case.


I think the ideal case is to have a table of:

 (triple, compiler, flags, sysroot)

tuples that say which configurations of the library to build.

Basing this off tuples is fine. I don’t think you want a table of (triple, compiler, flags, sysroot) though. I think you want a mapping of tuple->toolchain, and the toolchain file defines compiler, flags and sysroot. Using the CMAKE_TOOLCHAIN_FILE mechanism makes leveraging CMake’s cross targeting support easier. We also really don’t want a 1:1 mapping of triples to toolchains because that would be too much redundancy. We should define flavors of toolchains. For example, the iOS toolchain that exists in LLVM today could be generalized to a Darwin toolchain with minimal changes.


If Clang is compiler-rt's compiler, then all of the compiler entries in that table can point to the same binary. For in-tree builds, we *should* be using the "just built" Clang anyway (but are not currently, without enabling the flag to do so). Either way, I think we ought to move away from thinking that a native build is in some way "special".

If GCC is compiler-rt's compiler, then there would have to have a different binary per triple.

If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.

I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.


The plus side of treating *every* build as a cross build is that then there will be less risk of builds differing between native builds and cross builds when the target is the same (for example, linux targeting linux vs darwin targeting linux).

I think there are other things we can do to mitigate risk in this regard. I’m not opposed to making all compiler-rt builds cross-builds, but it does carry some downsides.



Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS,
when I configure my build directory there will be an in-tree build
of Compiler-RT (via add_llvm_external_project) to generate the OS X
runtime libraries, and an out-of-tree-style cross-compile generated
for building the iOS runtime libraries for all supported
architectures. This would all be driven via command line settings
like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to
target only FreeBSD when you configured your build directory there
would only be an out-of-tree-style cross-compile generated for your
target architecture against FreeBSD. The command line setting here
would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an
in-tree build with outputs: libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file
settings, so if you were cross-targeting and need to set compiler
options like -isysroot there will need to be a toolchain file
defined for your target platform.

Using toolchain files and cross-compile configurations solves one
of the problems with our current CMake system where the Compiler-RT
builds don't re-run the CMake checks. Not re-running the CMake
checks causes significant problems if your host and target OS are
different.

Can you elaborate on that last sentence? I'm not very familiar with CMake, and that's a bit of a surprise to me.

CMake acts like a configure script determining what libraries, APIs, compiler features, etc are available. One of the limitations in our CMake system today is if you wanted to build compiler-rt for Linux on Darwin… good luck. The problem is that when you configure your build it runs all the availability checks against the OS X toolchain capabilities, which may be very different from the Linux you are targeting. For that matter, it can be different even in the much more common case of OS X targeting iOS.

Treating compiler-rt builds as single platform cross-targeted builds allows us to have CMake re-run the checks for every triple that you want to build compiler-rt for. Thus smoothing all that away.

For added context. When I talk about toolchain files I’m referring to the CMake variable CMAKE_TOOLCHAIN_FILE (http://www.cmake.org/cmake/help/v3.0/variable/CMAKE_TOOLCHAIN_FILE.html). It is basically a little blurb of CMake that gets processed before any of the CMake platform checks or any other CMake file in your project. It can be used to define settings that need to influence the try_compile invocations that CMake uses for availability checks.

Thanks,
-Chris



So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a
plan.

The first thing I'd like to do is try to make a bunch of the
Darwin-specific code go away. Obviously there will always need to
be some at a lower level because Darwin is different, but some of
the lower-level functions like
add_compiler_rt_darwin_object_library probably don't need to
exist.

One example of the type of thing I want to get rid of is the ASan
CMakeLists.txt. The top-level CMake files for libraries and
executables shouldn't need to care about the targets. Today we loop
over OSs for darwin and call add_compiler_rt_darwin_object_library,
and for other operating systems we loop over archs and call
add_compiler_rt_object_library. I want to push the looping as low
down into the utility functions as possible. Doing this will reduce
the amount of Darwin-specific code because we replicate this exact
looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to
generate binaries for just OSX or just iOS. Today the Darwin build
tries to build everything all the time, and that really isn't the
behavior we need if we're going to treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt
combined build use cross-compilation for building the iOS runtime
libraries. I want to do this on Darwin only because that's the
platform I'm most familiar with. At some point if we want to move
other targets to cross-compiling I will need help from other people
in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and
trying to streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks for taking the driver's seat on the discussions of all this CMake transition stuff, BTW.


Cheers,

Jon


Thanks, -Chris _______________________________________________ LLVM
Developers mailing list [hidden email]
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________ LLVM Developers
mailing list [hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Duncan P. N. Exon Smith

> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>
>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>
> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.

Weird, I'd assumed building compiler-rt with something other than
clang was unsupported.  Maybe I'm missing something, but shouldn't
the only supported configuration be building with the just-built
clang?
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Jonathan Roelofs


On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:

>
>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>>
>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>>
>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>
> Weird, I'd assumed building compiler-rt with something other than
> clang was unsupported.  Maybe I'm missing something, but shouldn't
> the only supported configuration be building with the just-built
> clang?

The current default for an in-tree build is to build compiler-rt with
whatever compiler is being used to build Clang... sometimes that
compiler is GCC.

I agree though. We should always use the just-built Clang, and have that
behavior be opt-out (if folks need it), instead of opt-in as it is now.


Jon

> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

--
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Duncan P. N. Exon Smith

> On 2015-Jun-02, at 14:03, Jonathan Roelofs <[hidden email]> wrote:
>
>
>
> On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:
>>
>>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>>>
>>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>>>
>>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>>
>> Weird, I'd assumed building compiler-rt with something other than
>> clang was unsupported.  Maybe I'm missing something, but shouldn't
>> the only supported configuration be building with the just-built
>> clang?
>
> The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.
>
> I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.

Right, it sounds like a bug, not a feature.

(I think configure+make gets this right already.)
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Richard Smith-33
In reply to this post by Jonathan Roelofs

On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:
>
>
>
> On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:
>>
>>
>>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>>>
>>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>>>
>>>
>>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>>
>>
>> Weird, I'd assumed building compiler-rt with something other than
>> clang was unsupported.  Maybe I'm missing something, but shouldn't
>> the only supported configuration be building with the just-built
>> clang?
>
>
> The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.
>
> I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.

What would the build system do for a cross compile of Clang?

>
> Jon
>
>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
> --
> Jon Roelofs
> [hidden email]
> CodeSourcery / Mentor Embedded
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman

On Jun 2, 2015, at 6:29 PM, Richard Smith <[hidden email]> wrote:

On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:
>
>
>
> On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:
>>
>>
>>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>>>
>>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>>>
>>>
>>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>>
>>
>> Weird, I'd assumed building compiler-rt with something other than
>> clang was unsupported.  Maybe I'm missing something, but shouldn't
>> the only supported configuration be building with the just-built
>> clang?
>
>
> The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.
>
> I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.

What would the build system do for a cross compile of Clang?

This is the big question right? If we decide to only support building compiler-rt with the just-built clang that inherently means that you always have to build a clang that can run on host, so cross-compiling clang means building for host first, then the target.

I don’t think that is the behavior we want.

I suspect the behavior we probably want is to use the just-built clang by default unless you are cross-compiling, in which case use the host compiler.

Whether or not the host compiler needs to be clang, or any version restrictions we want to apply is a separate issue.

-Chris

>
> Jon
>
>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
> --
> Jon Roelofs
> [hidden email]
> CodeSourcery / Mentor Embedded
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Jim Grosbach
In reply to this post by Chris Bieneman

On Jun 1, 2015, at 7:47 PM, Chris Bieneman <[hidden email]> wrote:


On Jun 1, 2015, at 6:26 PM, Jonathan Roelofs <[hidden email]> wrote:



On 6/1/15 6:42 PM, Chris Bieneman wrote:
Looping in cfe-dev because I should have send this there too from the
start.

-Chris

On Jun 1, 2015, at 3:21 PM, Chris Bieneman <[hidden email]>
wrote:

LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake
build system feature equivalent to the autotools one. One of the
biggest outstanding tasks is PR 21562, which I've been viewing more
as a "We really need to fix how we build Compiler-RT."

+1


I've thought about this quite a bit, and I wanted to send out some
of my ideas to get some feedback and have some discussions about
the way forward.

The complication with Compiler-RT is that it is inherently a
cross-compile. When you build LLVM, Clang and Compiler-RT, you
really want to build LLVM, Clang, and n Compiler-RTs where n is the
valid combinations of target architectures and/or platforms your
clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host (2)
Building a cross-targeting Clang that runs on one OS or
architecture and targets another (3) Building a Clang that both
self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to
handle case 1 as an in-tree build similar to how it is today, case
2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Why treat $host as a special case? IM-not-so-humble-O, $host is just another $target.

Conceptually I agree, but treating host differently here is a build-time optimization. Treating it as a cross-compile bears CMake-induced build time overhead, which can be avoided in the simple case.

Interesting. Maybe start by treating everything as cross compile and then come back to optimizing for $host==$target later if it’s worthwhile?



I think the ideal case is to have a table of:

 (triple, compiler, flags, sysroot)

tuples that say which configurations of the library to build.

Basing this off tuples is fine. I don’t think you want a table of (triple, compiler, flags, sysroot) though. I think you want a mapping of tuple->toolchain, and the toolchain file defines compiler, flags and sysroot. Using the CMAKE_TOOLCHAIN_FILE mechanism makes leveraging CMake’s cross targeting support easier. We also really don’t want a 1:1 mapping of triples to toolchains because that would be too much redundancy. We should define flavors of toolchains. For example, the iOS toolchain that exists in LLVM today could be generalized to a Darwin toolchain with minimal changes.


If Clang is compiler-rt's compiler, then all of the compiler entries in that table can point to the same binary. For in-tree builds, we *should* be using the "just built" Clang anyway (but are not currently, without enabling the flag to do so). Either way, I think we ought to move away from thinking that a native build is in some way "special".

If GCC is compiler-rt's compiler, then there would have to have a different binary per triple.

If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.

I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.

Start a thread about that question specifically and see? Nothing ventured… :)



The plus side of treating *every* build as a cross build is that then there will be less risk of builds differing between native builds and cross builds when the target is the same (for example, linux targeting linux vs darwin targeting linux).

I think there are other things we can do to mitigate risk in this regard. I’m not opposed to making all compiler-rt builds cross-builds, but it does carry some downsides.



Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS,
when I configure my build directory there will be an in-tree build
of Compiler-RT (via add_llvm_external_project) to generate the OS X
runtime libraries, and an out-of-tree-style cross-compile generated
for building the iOS runtime libraries for all supported
architectures. This would all be driven via command line settings
like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to
target only FreeBSD when you configured your build directory there
would only be an out-of-tree-style cross-compile generated for your
target architecture against FreeBSD. The command line setting here
would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an
in-tree build with outputs: libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file
settings, so if you were cross-targeting and need to set compiler
options like -isysroot there will need to be a toolchain file
defined for your target platform.

Using toolchain files and cross-compile configurations solves one
of the problems with our current CMake system where the Compiler-RT
builds don't re-run the CMake checks. Not re-running the CMake
checks causes significant problems if your host and target OS are
different.

Can you elaborate on that last sentence? I'm not very familiar with CMake, and that's a bit of a surprise to me.

CMake acts like a configure script determining what libraries, APIs, compiler features, etc are available. One of the limitations in our CMake system today is if you wanted to build compiler-rt for Linux on Darwin… good luck. The problem is that when you configure your build it runs all the availability checks against the OS X toolchain capabilities, which may be very different from the Linux you are targeting. For that matter, it can be different even in the much more common case of OS X targeting iOS.

Treating compiler-rt builds as single platform cross-targeted builds allows us to have CMake re-run the checks for every triple that you want to build compiler-rt for. Thus smoothing all that away.

For added context. When I talk about toolchain files I’m referring to the CMake variable CMAKE_TOOLCHAIN_FILE (http://www.cmake.org/cmake/help/v3.0/variable/CMAKE_TOOLCHAIN_FILE.html). It is basically a little blurb of CMake that gets processed before any of the CMake platform checks or any other CMake file in your project. It can be used to define settings that need to influence the try_compile invocations that CMake uses for availability checks.

Thanks,
-Chris



So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a
plan.

The first thing I'd like to do is try to make a bunch of the
Darwin-specific code go away. Obviously there will always need to
be some at a lower level because Darwin is different, but some of
the lower-level functions like
add_compiler_rt_darwin_object_library probably don't need to
exist.

One example of the type of thing I want to get rid of is the ASan
CMakeLists.txt. The top-level CMake files for libraries and
executables shouldn't need to care about the targets. Today we loop
over OSs for darwin and call add_compiler_rt_darwin_object_library,
and for other operating systems we loop over archs and call
add_compiler_rt_object_library. I want to push the looping as low
down into the utility functions as possible. Doing this will reduce
the amount of Darwin-specific code because we replicate this exact
looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to
generate binaries for just OSX or just iOS. Today the Darwin build
tries to build everything all the time, and that really isn't the
behavior we need if we're going to treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt
combined build use cross-compilation for building the iOS runtime
libraries. I want to do this on Darwin only because that's the
platform I'm most familiar with. At some point if we want to move
other targets to cross-compiling I will need help from other people
in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and
trying to streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks for taking the driver's seat on the discussions of all this CMake transition stuff, BTW.


Cheers,

Jon


Thanks, -Chris _______________________________________________ LLVM
Developers mailing list [hidden email]
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________ LLVM Developers
mailing list [hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


-- 
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

-- 
Jon Roelofs
[hidden email]
CodeSourcery / Mentor Embedded

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Duncan P. N. Exon Smith
In reply to this post by Chris Bieneman

> On 2015-Jun-03, at 12:57, Chris Bieneman <[hidden email]> wrote:
>
>
>> On Jun 2, 2015, at 6:29 PM, Richard Smith <[hidden email]> wrote:
>>
>> On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:
>> >
>> >
>> >
>> > On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:
>> >>
>> >>
>> >>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>> >>>
>> >>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>> >>>
>> >>>
>> >>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>> >>
>> >>
>> >> Weird, I'd assumed building compiler-rt with something other than
>> >> clang was unsupported.  Maybe I'm missing something, but shouldn't
>> >> the only supported configuration be building with the just-built
>> >> clang?
>> >
>> >
>> > The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.
>> >
>> > I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.
>>
>> What would the build system do for a cross compile of Clang?
>>
> This is the big question right? If we decide to only support building compiler-rt with the just-built clang that inherently means that you always have to build a clang that can run on host, so cross-compiling clang means building for host first, then the target.
>
> I don’t think that is the behavior we want.

Why not?

> I suspect the behavior we probably want is to use the just-built clang by default unless you are cross-compiling, in which case use the host compiler.

This sounds reasonable (and would obviously be a big improvement), but
I'm interested in what the arguments are against the above/below.

Another option that seems superficially reasonable to me:

  - Only support building compiler-RT with the "matching" clang.
  - When cross-compiling clang, only support the same version of clang
    as the host (and use the host compiler).  In other words, when
    cross-compiling, only support a two-stage build, where the first
    stage builds a host clang, and the second stage builds the cross
    clang.
  - When not cross-compiling, always use the just-built clang.

> Whether or not the host compiler needs to be clang, or any version restrictions we want to apply is a separate issue.



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman
[re-sending my response to Duncan because I didn’t hit reply-all :-)]

On Jun 3, 2015, at 1:42 PM, Duncan P. N. Exon Smith <[hidden email]> wrote:


On 2015-Jun-03, at 12:57, Chris Bieneman <[hidden email]> wrote:


On Jun 2, 2015, at 6:29 PM, Richard Smith <[hidden email]> wrote:

On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:



On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:


On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:

If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.


I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.


Weird, I'd assumed building compiler-rt with something other than
clang was unsupported.  Maybe I'm missing something, but shouldn't
the only supported configuration be building with the just-built
clang?


The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.

I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.

What would the build system do for a cross compile of Clang?

This is the big question right? If we decide to only support building compiler-rt with the just-built clang that inherently means that you always have to build a clang that can run on host, so cross-compiling clang means building for host first, then the target.

I don’t think that is the behavior we want.

Why not?

I don’t think you want to force anyone cross-compiling clang to also build a host clang for compiler-rt. If there were a good reason for it I could see it being a necessary evil, but I don’t think we have a technical reason for making this a requirement and doubling the time of a clang cross-build seems like a poor trade-off if we’re not getting something good in return.


I suspect the behavior we probably want is to use the just-built clang by default unless you are cross-compiling, in which case use the host compiler.

This sounds reasonable (and would obviously be a big improvement), but
I'm interested in what the arguments are against the above/below.

Another option that seems superficially reasonable to me:

 - Only support building compiler-RT with the "matching" clang.
 - When cross-compiling clang, only support the same version of clang
   as the host (and use the host compiler).  In other words, when
   cross-compiling, only support a two-stage build, where the first
   stage builds a host clang, and the second stage builds the cross
   clang.
 - When not cross-compiling, always use the just-built clang.

This is kinda what I was getting at with the end of my email.

We could possibly say compiler-rt can only be built with clang, and is only expected to work with clang (not as a libgcc replacement for gcc). And we could add an additional statement that if you are cross-compiling clang you must have a host clang that “matches” (for some definition of matches) the one you’re building.

-Chris


Whether or not the host compiler needs to be clang, or any version restrictions we want to apply is a separate issue.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman
Since nobody seems to disagree with the basic idea here I’ve sent out my first cleanup patch:


-Chris

On Jun 4, 2015, at 11:32 AM, Chris Bieneman <[hidden email]> wrote:

[re-sending my response to Duncan because I didn’t hit reply-all :-)]

On Jun 3, 2015, at 1:42 PM, Duncan P. N. Exon Smith <[hidden email]> wrote:


On 2015-Jun-03, at 12:57, Chris Bieneman <[hidden email]> wrote:


On Jun 2, 2015, at 6:29 PM, Richard Smith <[hidden email]> wrote:

On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:



On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:


On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:

If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.


I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.


Weird, I'd assumed building compiler-rt with something other than
clang was unsupported.  Maybe I'm missing something, but shouldn't
the only supported configuration be building with the just-built
clang?


The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.

I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.

What would the build system do for a cross compile of Clang?

This is the big question right? If we decide to only support building compiler-rt with the just-built clang that inherently means that you always have to build a clang that can run on host, so cross-compiling clang means building for host first, then the target.

I don’t think that is the behavior we want.

Why not?

I don’t think you want to force anyone cross-compiling clang to also build a host clang for compiler-rt. If there were a good reason for it I could see it being a necessary evil, but I don’t think we have a technical reason for making this a requirement and doubling the time of a clang cross-build seems like a poor trade-off if we’re not getting something good in return.


I suspect the behavior we probably want is to use the just-built clang by default unless you are cross-compiling, in which case use the host compiler.

This sounds reasonable (and would obviously be a big improvement), but
I'm interested in what the arguments are against the above/below.

Another option that seems superficially reasonable to me:

 - Only support building compiler-RT with the "matching" clang.
 - When cross-compiling clang, only support the same version of clang
   as the host (and use the host compiler).  In other words, when
   cross-compiling, only support a two-stage build, where the first
   stage builds a host clang, and the second stage builds the cross
   clang.
 - When not cross-compiling, always use the just-built clang.

This is kinda what I was getting at with the end of my email.

We could possibly say compiler-rt can only be built with clang, and is only expected to work with clang (not as a libgcc replacement for gcc). And we could add an additional statement that if you are cross-compiling clang you must have a host clang that “matches” (for some definition of matches) the one you’re building.

-Chris


Whether or not the host compiler needs to be clang, or any version restrictions we want to apply is a separate issue.


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Duncan P. N. Exon Smith
In reply to this post by Chris Bieneman

> On 2015-Jun-04, at 11:32, Chris Bieneman <[hidden email]> wrote:
>
> [re-sending my response to Duncan because I didn’t hit reply-all :-)]
>
>> On Jun 3, 2015, at 1:42 PM, Duncan P. N. Exon Smith <[hidden email]> wrote:
>>
>>>
>>> On 2015-Jun-03, at 12:57, Chris Bieneman <[hidden email]> wrote:
>>>
>>>
>>>> On Jun 2, 2015, at 6:29 PM, Richard Smith <[hidden email]> wrote:
>>>>
>>>> On 2 Jun 2015 2:04 pm, "Jonathan Roelofs" <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 6/2/15 2:38 PM, Duncan P. N. Exon Smith wrote:
>>>>>>
>>>>>>
>>>>>>> On 2015-Jun-01, at 19:47, Chris Bieneman <[hidden email]> wrote:
>>>>>>>
>>>>>>>> If we drop support for building compiler-rt with GCC, this gets even simpler. Compiler-rt is *Clang's* runtime library, after all.
>>>>>>>
>>>>>>>
>>>>>>> I don’t know if it is on the table to drop supporting compiler-rt with GCC, but that would dramatically simplify things.
>>>>>>
>>>>>>
>>>>>> Weird, I'd assumed building compiler-rt with something other than
>>>>>> clang was unsupported.  Maybe I'm missing something, but shouldn't
>>>>>> the only supported configuration be building with the just-built
>>>>>> clang?
>>>>>
>>>>>
>>>>> The current default for an in-tree build is to build compiler-rt with whatever compiler is being used to build Clang... sometimes that compiler is GCC.
>>>>>
>>>>> I agree though. We should always use the just-built Clang, and have that behavior be opt-out (if folks need it), instead of opt-in as it is now.
>>>>
>>>> What would the build system do for a cross compile of Clang?
>>>>
>>> This is the big question right? If we decide to only support building compiler-rt with the just-built clang that inherently means that you always have to build a clang that can run on host, so cross-compiling clang means building for host first, then the target.
>>>
>>> I don’t think that is the behavior we want.
>>
>> Why not?
>
> I don’t think you want to force anyone cross-compiling clang to also build a host clang for compiler-rt. If there were a good reason for it I could see it being a necessary evil, but I don’t think we have a technical reason for making this a requirement and doubling the time of a clang cross-build seems like a poor trade-off if we’re not getting something good in return.
>
>>
>>> I suspect the behavior we probably want is to use the just-built clang by default unless you are cross-compiling, in which case use the host compiler.
>>
>> This sounds reasonable (and would obviously be a big improvement), but
>> I'm interested in what the arguments are against the above/below.
>>
>> Another option that seems superficially reasonable to me:
>>
>>  - Only support building compiler-RT with the "matching" clang.
>>  - When cross-compiling clang, only support the same version of clang
>>    as the host (and use the host compiler).  In other words, when
>>    cross-compiling, only support a two-stage build, where the first
>>    stage builds a host clang, and the second stage builds the cross
>>    clang.
>>  - When not cross-compiling, always use the just-built clang.
>
> This is kinda what I was getting at with the end of my email.
>
> We could possibly say compiler-rt can only be built with clang, and is only expected to work with clang (not as a libgcc replacement for gcc). And we could add an additional statement that if you are cross-compiling clang you must have a host clang that “matches” (for some definition of matches) the one you’re building.

This makes sense to me.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Reid Kleckner-2
In reply to this post by Chris Bieneman
I'm actually pretty leery of using the CMake cross-compiling support because it requires using multiple build directories with split build systems. We end up a situation where the parent build shells out to cmake and then recursively reinvokes the parent build system on the new build directory. Running cmake again in the first place is really slow (lots of test compiles in serial), and recursively invoking the build system leads to really bad job scheduling on all build systems other than 'make'. In particular, ninja shelling out to ninja will oversaturate the system, and I don't expect it to ever add job server functionality like make has.

How much are we really getting from using the upstream cmake cross-compiling support? Most of the header and symbol checks that we have are for the sanitizer test suite, which I doubt needs generalized cross-compilation support. Only the <unwind.h> check appears to matter for lib/builtins. Maybe we should roll our own custom rules for invoking the just-built clang compiler like we already do for the sanitizer test suite? I cc'd Alexey, who I believe wrote that support.

Anyway, thanks for tackling this, you're the one doing the work, and I'm not volunteering. :) Do what you think is best. I mostly wanted to present a possible alternative approach.

On Mon, Jun 1, 2015 at 3:21 PM, Chris Bieneman <[hidden email]> wrote:
LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake build system feature equivalent to the autotools one. One of the biggest outstanding tasks is PR 21562, which I've been viewing more as a "We really need to fix how we build Compiler-RT."

I've thought about this quite a bit, and I wanted to send out some of my ideas to get some feedback and have some discussions about the way forward.

The complication with Compiler-RT is that it is inherently a cross-compile. When you build LLVM, Clang and Compiler-RT, you really want to build LLVM, Clang, and n Compiler-RTs where n is the valid combinations of target architectures and/or platforms your clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host
(2) Building a cross-targeting Clang that runs on one OS or architecture and targets another
(3) Building a Clang that both self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to handle case 1 as an in-tree build similar to how it is today, case 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS, when I configure my build directory there will be an in-tree build of Compiler-RT (via add_llvm_external_project) to generate the OS X runtime libraries, and an out-of-tree-style cross-compile generated for building the iOS runtime libraries for all supported architectures. This would all be driven via command line settings like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to target only FreeBSD when you configured your build directory there would only be an out-of-tree-style cross-compile generated for your target architecture against FreeBSD. The command line setting here would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an in-tree build with outputs:
libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file settings, so if you were cross-targeting and need to set compiler options like -isysroot there will need to be a toolchain file defined for your target platform.

Using toolchain files and cross-compile configurations solves one of the problems with our current CMake system where the Compiler-RT builds don't re-run the CMake checks. Not re-running the CMake checks causes significant problems if your host and target OS are different.

So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a plan.

The first thing I'd like to do is try to make a bunch of the Darwin-specific code go away. Obviously there will always need to be some at a lower level because Darwin is different, but some of the lower-level functions like add_compiler_rt_darwin_object_library probably don't need to exist.

One example of the type of thing I want to get rid of is the ASan CMakeLists.txt. The top-level CMake files for libraries and executables shouldn't need to care about the targets. Today we loop over OSs for darwin and call add_compiler_rt_darwin_object_library, and for other operating systems we loop over archs and call add_compiler_rt_object_library. I want to push the looping as low down into the utility functions as possible. Doing this will reduce the amount of Darwin-specific code because we replicate this exact looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to generate binaries for just OSX or just iOS. Today the Darwin build tries to build everything all the time, and that really isn't the behavior we need if we're going to treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt combined build use cross-compilation for building the iOS runtime libraries. I want to do this on Darwin only because that's the platform I'm most familiar with. At some point if we want to move other targets to cross-compiling I will need help from other people in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and trying to streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks,
-Chris
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman
This is definitely a concern. That’s why I had proposed not treating “host” as a cross-target. Handling building compiler-rt for host differently makes one of the common cases better.

WRT Ninja.

Ninja’s pooling capabilities actually should solve some of the problems you’re voicing. In particular, we can make the cross-builds fall into the “console” pool to prevent ninja from over-saturating the system (https://martine.github.io/ninja/manual.html#_the_literal_console_literal_pool).

I also expect that we could engage with the CMake community to come up with better tooling for cross-compiling to help mitigate some of this. In particular I would really love to have a way import a configured project into the project you’re configuring, and have it generate a single build configuration that doesn’t just call out to CMake to build the other project.

I honestly don’t know a whole lot about the compiler-rt codebase. I’ve stepped into this project largely just as an extension of my hatred for autoconf. I don’t know how much we’ll get from CMake’s cross-compilation, but I do know that the way we handle cross-targeting CMake today for iOS leaves a lot to be desired.

One of my concerns is actually that the custom rules we have for the sanitizer test suite doesn’t work at all on Darwin unless you have the OS headers installed to / (which isn’t the default anymore). I think that teaching CMake to be smart enough about how it invokes the compiler to be able to support all our platforms will be a lot of work duplicating things CMake itself already does.

-Chris


On Jun 4, 2015, at 2:25 PM, Reid Kleckner <[hidden email]> wrote:

I'm actually pretty leery of using the CMake cross-compiling support because it requires using multiple build directories with split build systems. We end up a situation where the parent build shells out to cmake and then recursively reinvokes the parent build system on the new build directory. Running cmake again in the first place is really slow (lots of test compiles in serial), and recursively invoking the build system leads to really bad job scheduling on all build systems other than 'make'. In particular, ninja shelling out to ninja will oversaturate the system, and I don't expect it to ever add job server functionality like make has.

How much are we really getting from using the upstream cmake cross-compiling support? Most of the header and symbol checks that we have are for the sanitizer test suite, which I doubt needs generalized cross-compilation support. Only the <unwind.h> check appears to matter for lib/builtins. Maybe we should roll our own custom rules for invoking the just-built clang compiler like we already do for the sanitizer test suite? I cc'd Alexey, who I believe wrote that support.

Anyway, thanks for tackling this, you're the one doing the work, and I'm not volunteering. :) Do what you think is best. I mostly wanted to present a possible alternative approach.

On Mon, Jun 1, 2015 at 3:21 PM, Chris Bieneman <[hidden email]> wrote:
LLVMDev,

PR 15732 is the umbrella tracking the progress of making the CMake build system feature equivalent to the autotools one. One of the biggest outstanding tasks is PR 21562, which I've been viewing more as a "We really need to fix how we build Compiler-RT."

I've thought about this quite a bit, and I wanted to send out some of my ideas to get some feedback and have some discussions about the way forward.

The complication with Compiler-RT is that it is inherently a cross-compile. When you build LLVM, Clang and Compiler-RT, you really want to build LLVM, Clang, and n Compiler-RTs where n is the valid combinations of target architectures and/or platforms your clang supports.

There are three basic configurations for building Compiler-RT.

(1) Building a "pure" Clang supporting targeting your host
(2) Building a cross-targeting Clang that runs on one OS or architecture and targets another
(3) Building a Clang that both self-targets and cross-targets

The overall change I want to make to Compiler-RT is that I want to handle case 1 as an in-tree build similar to how it is today, case 2 as a cross-compile, and case 3 as a combination of cases 1 & 2.

Let me walk through an example of how I think this should work.

If my host is OS X and I'm building Clang to target OS X and iOS, when I configure my build directory there will be an in-tree build of Compiler-RT (via add_llvm_external_project) to generate the OS X runtime libraries, and an out-of-tree-style cross-compile generated for building the iOS runtime libraries for all supported architectures. This would all be driven via command line settings like:

LLVM_COMPILER_RT_IOS_ARCHS="armv7;armv7s;arm64"
LLVM_COMPILER_RT_OSX_ARCHS="x86;x86_64"

This would generate universal binary outputs in the format:
libclangrt_<lib>.<os>.<ext>

As a second example, if you were on Windows building a Clang to target only FreeBSD when you configured your build directory there would only be an out-of-tree-style cross-compile generated for your target architecture against FreeBSD. The command line setting here would be:

LLVM_COMPILER_RT_FREEBSD_ARCHS="x86;x86_64"

This would generate binary outputs in the format:
libclangrt_<lib>.<arch>.<ext>

Similarly if you were on linux targeting linux you’d only get an in-tree build with outputs:
libclangrt_<lib>.<arch>.<ext>

The cross-compile builds will be driven via CMake's toolchain file settings, so if you were cross-targeting and need to set compiler options like -isysroot there will need to be a toolchain file defined for your target platform.

Using toolchain files and cross-compile configurations solves one of the problems with our current CMake system where the Compiler-RT builds don't re-run the CMake checks. Not re-running the CMake checks causes significant problems if your host and target OS are different.

So, how do we get to this state?

Honestly I don't have a complete plan, but I have some parts of a plan.

The first thing I'd like to do is try to make a bunch of the Darwin-specific code go away. Obviously there will always need to be some at a lower level because Darwin is different, but some of the lower-level functions like add_compiler_rt_darwin_object_library probably don't need to exist.

One example of the type of thing I want to get rid of is the ASan CMakeLists.txt. The top-level CMake files for libraries and executables shouldn't need to care about the targets. Today we loop over OSs for darwin and call add_compiler_rt_darwin_object_library, and for other operating systems we loop over archs and call add_compiler_rt_object_library. I want to push the looping as low down into the utility functions as possible. Doing this will reduce the amount of Darwin-specific code because we replicate this exact looping behavior all over Compiler-RT's build.

The second thing I want to do is make the Darwin build able to generate binaries for just OSX or just iOS. Today the Darwin build tries to build everything all the time, and that really isn't the behavior we need if we're going to treat iOS as a cross-compile.

The third step is to make the Darwin llvm/clang/compiler-rt combined build use cross-compilation for building the iOS runtime libraries. I want to do this on Darwin only because that's the platform I'm most familiar with. At some point if we want to move other targets to cross-compiling I will need help from other people in the community who are more familiar with the specific targets.

With each step I'll be trying to remove more OS-specific code and trying to streamline building.

Questions/Comments/Concerns/Cries of insanity?

Thanks,
-Chris
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Reid Kleckner-2
On Thu, Jun 4, 2015 at 3:02 PM, Chris Bieneman <[hidden email]> wrote:
One of my concerns is actually that the custom rules we have for the sanitizer test suite doesn’t work at all on Darwin unless you have the OS headers installed to / (which isn’t the default anymore). I think that teaching CMake to be smart enough about how it invokes the compiler to be able to support all our platforms will be a lot of work duplicating things CMake itself already does.

Right, for problems like this, we'd end up writing lots of cmake code to go and find headers and libraries like this, effectively replacing the toolchain knowledge that we could get from cmake. It's a tradeoff that may or may not be worth it, and I'm not sure what the answer is. =/

Maybe if we use the standard cmake cross-compilation approach someone will come along later and solve this problem at the cmake level.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Alexey Samsonov-2
Sorry for the late response.

You should check out CMAKE_TOOLCHAIN_FILE logic for Android platform - IIRC build rules for Android are much simpler than for Darwin/Linux,
and assume that build tree will be properly configured to build a single compiler-rt runtime for a specific triple. Probably this is the approach you want
to eventually use for another platforms.

Also check out LLVM_BUILD_EXTERNAL_COMPILER_RT flag, which is an attempt to use ExternalProject to recursively call CMake from LLVM/Clang
build tree. It doesn't iterate over all the target triples we want/can support, though. It had some problems with Ninja, though.

I'd also prefer to not move host==target to a special case, at least at the beginning.

I definitely support the effort of simplifying things and generalizing the OS-specific code in CMake. Patches are more than welcome here =)

We want to support building (at least part of) compiler-rt libraries with GCC. I'm mostly speaking about sanitizers here:
sanitizer runtimes shipped with GCC are built from the same sources we have in compiler-rt. However, I agree that it
should not be the default (just-built Clang should be the default) - we can just setup a buildbot that would build
standalone compiler-rt with GCC.



On Thu, Jun 4, 2015 at 3:45 PM, Reid Kleckner <[hidden email]> wrote:
On Thu, Jun 4, 2015 at 3:02 PM, Chris Bieneman <[hidden email]> wrote:
One of my concerns is actually that the custom rules we have for the sanitizer test suite doesn’t work at all on Darwin unless you have the OS headers installed to / (which isn’t the default anymore). I think that teaching CMake to be smart enough about how it invokes the compiler to be able to support all our platforms will be a lot of work duplicating things CMake itself already does.

Right, for problems like this, we'd end up writing lots of cmake code to go and find headers and libraries like this, effectively replacing the toolchain knowledge that we could get from cmake. It's a tradeoff that may or may not be worth it, and I'm not sure what the answer is. =/

Maybe if we use the standard cmake cross-compilation approach someone will come along later and solve this problem at the cmake level.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev




--
Alexey Samsonov
[hidden email]

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Chris Bieneman

On Jun 4, 2015, at 6:18 PM, Alexey Samsonov <[hidden email]> wrote:

Sorry for the late response.

You should check out CMAKE_TOOLCHAIN_FILE logic for Android platform - IIRC build rules for Android are much simpler than for Darwin/Linux,
and assume that build tree will be properly configured to build a single compiler-rt runtime for a specific triple. Probably this is the approach you want
to eventually use for another platforms.

This is the basic idea. I don’t know that I’d say the Android toolchain is materially simpler than the iOS one. They do basically the same thing. The difference is that on OS X we have tools (xcrun and xcodebuild) that have the ability to find sysroots and toolchain tools. This results in the iOS toolchain running tools find the right paths to things, and thus supporting multiple versions of Xcode and the corresponding SDKs.

The Android toolchain is more fragile. It relies on clang being found via a relative path, and requires the sysroot path be explicitly set as an input variable. While the Cmake file is technically less complex, using it is more complex. I can cross-compile LLVM for iOS using the relatively simple CMake command:

 cmake -G "Ninja" -DCMAKE_OSX_ARCHITECTURES=“armv7;armv7s;arm64"
  -DCMAKE_TOOLCHAIN_FILE=<PATH_TO_LLVM>/cmake/platforms/iOS.cmake <path to source>

I think that if we do this how I’m suggesting and base this all off a mapping of triples to toolchain files, we’ll want the Android toolchain file to get more complex so that it has less hard coded assumptions.


Also check out LLVM_BUILD_EXTERNAL_COMPILER_RT flag, which is an attempt to use ExternalProject to recursively call CMake from LLVM/Clang
build tree. It doesn't iterate over all the target triples we want/can support, though. It had some problems with Ninja, though.

LLVM_BUILD_EXTERNAL_COMPILER_RT is basically the way I think we want to go forward, but there are some changes that need to be made to deal with toolchains and properly mapping targets.

I also think we can work around the Ninja issues by taking advantage of Ninja’s job pooling capabilities.


I'd also prefer to not move host==target to a special case, at least at the beginning.

When I talked to Chandler about this last night, my impression was that he agreed with me that at least having the option to not treat compiler-rt built for host as an external project was important for iterative development.


I definitely support the effort of simplifying things and generalizing the OS-specific code in CMake. Patches are more than welcome here =)

We want to support building (at least part of) compiler-rt libraries with GCC. I'm mostly speaking about sanitizers here:
sanitizer runtimes shipped with GCC are built from the same sources we have in compiler-rt. However, I agree that it
should not be the default (just-built Clang should be the default) - we can just setup a buildbot that would build
standalone compiler-rt with GCC.

It is good to know the sanitizers are used with GCC. I think the goal should be just-built clang is the default unless you are cross-compiling clang, in which case the compiler used to build clang is used (could be the system compiler or an override one).

-Chris




On Thu, Jun 4, 2015 at 3:45 PM, Reid Kleckner <[hidden email]> wrote:
On Thu, Jun 4, 2015 at 3:02 PM, Chris Bieneman <[hidden email]> wrote:
One of my concerns is actually that the custom rules we have for the sanitizer test suite doesn’t work at all on Darwin unless you have the OS headers installed to / (which isn’t the default anymore). I think that teaching CMake to be smart enough about how it invokes the compiler to be able to support all our platforms will be a lot of work duplicating things CMake itself already does.

Right, for problems like this, we'd end up writing lots of cmake code to go and find headers and libraries like this, effectively replacing the toolchain knowledge that we could get from cmake. It's a tradeoff that may or may not be worth it, and I'm not sure what the answer is. =/

Maybe if we use the standard cmake cross-compilation approach someone will come along later and solve this problem at the cmake level.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev




--
Alexey Samsonov
[hidden email]


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: [RFC] Ideas on improving Compiler-RT CMake

Alexey Samsonov-2

On Fri, Jun 5, 2015 at 10:57 AM, Chris Bieneman <[hidden email]> wrote:

On Jun 4, 2015, at 6:18 PM, Alexey Samsonov <[hidden email]> wrote:

Sorry for the late response.

You should check out CMAKE_TOOLCHAIN_FILE logic for Android platform - IIRC build rules for Android are much simpler than for Darwin/Linux,
and assume that build tree will be properly configured to build a single compiler-rt runtime for a specific triple. Probably this is the approach you want
to eventually use for another platforms.

This is the basic idea. I don’t know that I’d say the Android toolchain is materially simpler than the iOS one. They do basically the same thing. The difference is that on OS X we have tools (xcrun and xcodebuild) that have the ability to find sysroots and toolchain tools. This results in the iOS toolchain running tools find the right paths to things, and thus supporting multiple versions of Xcode and the corresponding SDKs.

The Android toolchain is more fragile. It relies on clang being found via a relative path, and requires the sysroot path be explicitly set as an input variable. While the Cmake file is technically less complex, using it is more complex. I can cross-compile LLVM for iOS using the relatively simple CMake command:

 cmake -G "Ninja" -DCMAKE_OSX_ARCHITECTURES=“armv7;armv7s;arm64"
  -DCMAKE_TOOLCHAIN_FILE=<PATH_TO_LLVM>/cmake/platforms/iOS.cmake <path to source>

I think that if we do this how I’m suggesting and base this all off a mapping of triples to toolchain files, we’ll want the Android toolchain file to get more complex so that it has less hard coded assumptions.


Also check out LLVM_BUILD_EXTERNAL_COMPILER_RT flag, which is an attempt to use ExternalProject to recursively call CMake from LLVM/Clang
build tree. It doesn't iterate over all the target triples we want/can support, though. It had some problems with Ninja, though.

LLVM_BUILD_EXTERNAL_COMPILER_RT is basically the way I think we want to go forward, but there are some changes that need to be made to deal with toolchains and properly mapping targets.

I also think we can work around the Ninja issues by taking advantage of Ninja’s job pooling capabilities.


I'd also prefer to not move host==target to a special case, at least at the beginning.

When I talked to Chandler about this last night, my impression was that he agreed with me that at least having the option to not treat compiler-rt built for host as an external project was important for iterative development.

Why? So that when you hack on Clang and run "make check-all" you won't recompile all the compiler-rt? I'd say it's not a big deal, as we currently use just-built Clang to recompile all
compiler-rt unit tests, and this compilation is slow and in general comparable to the time you'd need to rebuild the runtimes as well.


I definitely support the effort of simplifying things and generalizing the OS-specific code in CMake. Patches are more than welcome here =)

We want to support building (at least part of) compiler-rt libraries with GCC. I'm mostly speaking about sanitizers here:
sanitizer runtimes shipped with GCC are built from the same sources we have in compiler-rt. However, I agree that it
should not be the default (just-built Clang should be the default) - we can just setup a buildbot that would build
standalone compiler-rt with GCC.

It is good to know the sanitizers are used with GCC. I think the goal should be just-built clang is the default unless you are cross-compiling clang, in which case the compiler used to build clang is used (could be the system compiler or an override one).

Sure, we already have the option to build compiler-rt as a standalone CMake project with any host compiler we like.


On Thu, Jun 4, 2015 at 3:45 PM, Reid Kleckner <[hidden email]> wrote:
On Thu, Jun 4, 2015 at 3:02 PM, Chris Bieneman <[hidden email]> wrote:
One of my concerns is actually that the custom rules we have for the sanitizer test suite doesn’t work at all on Darwin unless you have the OS headers installed to / (which isn’t the default anymore). I think that teaching CMake to be smart enough about how it invokes the compiler to be able to support all our platforms will be a lot of work duplicating things CMake itself already does.

Right, for problems like this, we'd end up writing lots of cmake code to go and find headers and libraries like this, effectively replacing the toolchain knowledge that we could get from cmake. It's a tradeoff that may or may not be worth it, and I'm not sure what the answer is. =/

Maybe if we use the standard cmake cross-compilation approach someone will come along later and solve this problem at the cmake level.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev




--
Alexey Samsonov
[hidden email]




--
Alexey Samsonov
[hidden email]

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev