[llvm-dev] Trying to cross-compile LLVM runtimes to Windows

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
I've tried building libc++ for Windows on Linux by setting clang-cl as
the compiler and lld-link as the linker, but I immediately hit
roadblocks. The first issue seems to be the fact that CMake appears to
ignore the value of CMAKE_RC_COMPILER variable and hardcodes "rc" as
the resource compiler. This got resolved in upstream 12 days ago
(https://github.com/Kitware/CMake/commit/bd9bfc644954a48b1bf7ea18fc260a1231840671)

I managed to workaround this issue by symlinking llvm-rc as rc.
However, that's not sufficient because CMake uses following invocation
(https://github.com/Kitware/CMake/blob/master/Source/cmcmd.cxx#L1815):

rc /foCMakeFiles/cmTC_2c376.dir/manifest.res
CMakeFiles/cmTC_2c376.dir/manifest.rc

llvm-rc option parses requires /fo option argument to be separate.
I've modified the flag to support both joined and separate value which
helped getting past that error, but I immediately hit another error:
https://reviews.llvm.org/P8113

The problem seems to be that the .rc file generated by CMake contains
an absolute path to the resource, but llvm-rc expects the path to be
always relative. However, I couldn't fine any documentation that would
explain whether that's a requirement and it's CMake's fault or whether
llvm-rc should be handling absolute paths as well. Is anyone familiar
with Windows resource file semantics?

All of these errors seem to suggest that this cross-compiling for
Windows with Clang using CMake isn't a well explored area. Has anyone
had more success with cross-compiling LLVM runtimes (libc++,
compiler-rt) to Windows?
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
I have been using this project with great success to achieve this :
https://github.com/mstorsjo/llvm-mingw

It generates a linux-based windows cross-compiler with clang, libc++, etc.

I was able to build a large part of Qt with it afterwards for instance.


Best,

Jean-Michaël

On 06/11/2018 04:21, Petr Hosek via llvm-dev wrote:

> I've tried building libc++ for Windows on Linux by setting clang-cl as
> the compiler and lld-link as the linker, but I immediately hit
> roadblocks. The first issue seems to be the fact that CMake appears to
> ignore the value of CMAKE_RC_COMPILER variable and hardcodes "rc" as
> the resource compiler. This got resolved in upstream 12 days ago
> (https://github.com/Kitware/CMake/commit/bd9bfc644954a48b1bf7ea18fc260a1231840671)
>
> I managed to workaround this issue by symlinking llvm-rc as rc.
> However, that's not sufficient because CMake uses following invocation
> (https://github.com/Kitware/CMake/blob/master/Source/cmcmd.cxx#L1815):
>
> rc /foCMakeFiles/cmTC_2c376.dir/manifest.res
> CMakeFiles/cmTC_2c376.dir/manifest.rc
>
> llvm-rc option parses requires /fo option argument to be separate.
> I've modified the flag to support both joined and separate value which
> helped getting past that error, but I immediately hit another error:
> https://reviews.llvm.org/P8113
>
> The problem seems to be that the .rc file generated by CMake contains
> an absolute path to the resource, but llvm-rc expects the path to be
> always relative. However, I couldn't fine any documentation that would
> explain whether that's a requirement and it's CMake's fault or whether
> llvm-rc should be handling absolute paths as well. Is anyone familiar
> with Windows resource file semantics?
>
> All of these errors seem to suggest that this cross-compiling for
> Windows with Clang using CMake isn't a well explored area. Has anyone
> had more success with cross-compiling LLVM runtimes (libc++,
> compiler-rt) to Windows?
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
In reply to this post by David Jones via llvm-dev
On Mon, 5 Nov 2018, Petr Hosek via llvm-dev wrote:

> I've tried building libc++ for Windows on Linux by setting clang-cl as
> the compiler and lld-link as the linker, but I immediately hit
> roadblocks. The first issue seems to be the fact that CMake appears to
> ignore the value of CMAKE_RC_COMPILER variable and hardcodes "rc" as
> the resource compiler. This got resolved in upstream 12 days ago
> (https://github.com/Kitware/CMake/commit/bd9bfc644954a48b1bf7ea18fc260a1231840671)
>
> I managed to workaround this issue by symlinking llvm-rc as rc.
> However, that's not sufficient because CMake uses following invocation
> (https://github.com/Kitware/CMake/blob/master/Source/cmcmd.cxx#L1815):
>
> rc /foCMakeFiles/cmTC_2c376.dir/manifest.res
> CMakeFiles/cmTC_2c376.dir/manifest.rc
>
> llvm-rc option parses requires /fo option argument to be separate.
> I've modified the flag to support both joined and separate value which
> helped getting past that error,
Please send that patch for review (add me, zturner, amccarth and thakis as
reviewers) - it's probably useful irrespective of the other issues you're
running into.

> but I immediately hit another error: https://reviews.llvm.org/P8113
>
> The problem seems to be that the .rc file generated by CMake contains
> an absolute path to the resource, but llvm-rc expects the path to be
> always relative. However, I couldn't fine any documentation that would
> explain whether that's a requirement and it's CMake's fault or whether
> llvm-rc should be handling absolute paths as well. Is anyone familiar
> with Windows resource file semantics?

As the existing cmake behaviour works with rc.exe, llvm-rc should adapt to
handle it as well.

It seems like we look in the INCLUDE environment for files included in
resource files, but as that assert points out, doing that doesn't makse
sense if the file name looked for actually turns out to be an absolute
path. I'd just add an "if (!path::is_absolute(FileName))" before the
FindInEnvPath call.

> All of these errors seem to suggest that this cross-compiling for
> Windows with Clang using CMake isn't a well explored area. Has anyone
> had more success with cross-compiling LLVM runtimes (libc++,
> compiler-rt) to Windows?

I do this, successfully - see https://github.com/mstorsjo/llvm-mingw (as
Jean-Michaël already linked). I do this in a mingw setup though, so cmake
doesn't use llvm-rc directly, but instead I wrap llvm-rc in a GNU windres
like frontend, that takes care of preprocessing (in case there are
#include etc) and option handling (wrangling options from GNU windres
style to what llvm-rc expects).

It's definitely not a well-explored area though - as build-compiler-rt.sh
and build-libcxx.sh in my repo shows, I go through quite a bit of manual
fiddling to get it all to build in a bootstrap setup. There might be
opportunities for simplifying all of it somewhat, but this setup works for
me at least.

// Martin

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
In reply to this post by David Jones via llvm-dev
On Tue, 6 Nov 2018, Jean-Michaël Celerier via llvm-dev wrote:

> I have been using this project with great success to achieve this :
> https://github.com/mstorsjo/llvm-mingw
>
> It generates a linux-based windows cross-compiler with clang, libc++, etc.

FWIW, lately I've added prebuilt packages of it that you can run on actual
windows as well, even though cross compilation was my first/main focus.

> I was able to build a large part of Qt with it afterwards for instance.

Any particular part of Qt which you weren't able to build with it? I
regularly build Qt (qtbase) for i686/x86_64/armv7/aarch64, and as part of
VLC, I also build a few more Qt modules (qtdeclarative, qtquickcontrols2)
for i686 and x86_64.

// Martin

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
I suspect mingw is going to be incompatible with Petr’s needs. Adding Nico as he probably does the most cross compiling.  I did it a little several weeks ago, just not with libc++, and I don’t remember running into any problems.

One thing that’s not clear to me is why the resource compiler is even being invoked at all

On Tue, Nov 6, 2018 at 5:46 AM Martin Storsjö via llvm-dev <[hidden email]> wrote:
On Tue, 6 Nov 2018, Jean-Michaël Celerier via llvm-dev wrote:

> I have been using this project with great success to achieve this :
> https://github.com/mstorsjo/llvm-mingw
>
> It generates a linux-based windows cross-compiler with clang, libc++, etc.

FWIW, lately I've added prebuilt packages of it that you can run on actual
windows as well, even though cross compilation was my first/main focus.

> I was able to build a large part of Qt with it afterwards for instance.

Any particular part of Qt which you weren't able to build with it? I
regularly build Qt (qtbase) for i686/x86_64/armv7/aarch64, and as part of
VLC, I also build a few more Qt modules (qtdeclarative, qtquickcontrols2)
for i686 and x86_64.

// Martin
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
Yes, I don't think mingw will fit his needs, but it's at least some
datapoint wrt "does anybody do this". But as most of the tool interfaces
are different it probably won't hit the same issues anyway.

// Martin


On Tue, 6 Nov 2018, Zachary Turner wrote:

> I suspect mingw is going to be incompatible with Petr’s needs. Adding Nico
> as he probably does the most cross compiling.  I did it a little several
> weeks ago, just not with libc++, and I don’t remember running into any
> problems.
>
> One thing that’s not clear to me is why the resource compiler is even being
> invoked at all
>
> On Tue, Nov 6, 2018 at 5:46 AM Martin Storsjö via llvm-dev
> <[hidden email]> wrote:
>       On Tue, 6 Nov 2018, Jean-Michaël Celerier via llvm-dev wrote:
>
>       > I have been using this project with great success to achieve
>       this :
>       > https://github.com/mstorsjo/llvm-mingw
>       >
>       > It generates a linux-based windows cross-compiler with clang,
>       libc++, etc.
>
>       FWIW, lately I've added prebuilt packages of it that you can run
>       on actual
>       windows as well, even though cross compilation was my first/main
>       focus.
>
>       > I was able to build a large part of Qt with it afterwards for
>       instance.
>
>       Any particular part of Qt which you weren't able to build with
>       it? I
>       regularly build Qt (qtbase) for i686/x86_64/armv7/aarch64, and
>       as part of
>       VLC, I also build a few more Qt modules (qtdeclarative,
>       qtquickcontrols2)
>       for i686 and x86_64.
>
>       // Martin
>       _______________________________________________
>       LLVM Developers mailing list
>       [hidden email]
>       http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
In reply to this post by David Jones via llvm-dev
No, it's just that I didn't try to build qtwebkit / qtwebengine / qt3d
since I don't use them. Everything else seemed to build fine (including
the qml stuff as you mentioned).

On 06/11/2018 14:46, Martin Storsjö wrote:
>
> Any particular part of Qt which you weren't able to build with it?
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
Ah, right - yeah, I haven't tried any of those either. Those sound big and
complex enough that there probably are some issues to be expected in them.
(I had to do half a dozen patches on qtbase itself just to make it work
originally.)

// Martin

On Tue, 6 Nov 2018, Jean-Michaël Celerier wrote:

> No, it's just that I didn't try to build qtwebkit / qtwebengine / qt3d since
> I don't use them. Everything else seemed to build fine (including the qml
> stuff as you mentioned).
>
> On 06/11/2018 14:46, Martin Storsjö wrote:
>>
>> Any particular part of Qt which you weren't able to build with it?
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
In reply to this post by David Jones via llvm-dev

We cross-compile clang and compiler-rt for Windows on Linux. For clang, we use LLVM's WinMsvc toolchain file (https://reviews.llvm.org/diffusion/L/browse/llvm/trunk/cmake/platforms/WinMsvc.cmake). We add /manifest:no to our link flags in that toolchain file, which prevents cmake from attempting to do any mt or rc calls IIRC. I don't know if not embedding a manifest is an acceptable workaround for your use case, but it works for us. For compiler-rt, we have this weird setup (for historic reasons) where we build for Windows with an msvc triple but using the gcc-style driver, which I think ends up saving us from CMake's rc calls (though we then need some other CMake shenanigans to make lld-link happy). I wouldn't recommend going that route, but even with clang-cl the /manifest:no might save you?

 

From: llvm-dev <[hidden email]> on behalf of Petr Hosek via llvm-dev <[hidden email]>
Reply-To: Petr Hosek <[hidden email]>
Date: Monday, November 5, 2018 at 7:22 PM
To: llvm-dev <[hidden email]>
Subject: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

 

I've tried building libc++ for Windows on Linux by setting clang-cl as

the compiler and lld-link as the linker, but I immediately hit

roadblocks. The first issue seems to be the fact that CMake appears to

ignore the value of CMAKE_RC_COMPILER variable and hardcodes "rc" as

the resource compiler. This got resolved in upstream 12 days ago

 

I managed to workaround this issue by symlinking llvm-rc as rc.

However, that's not sufficient because CMake uses following invocation

 

rc /foCMakeFiles/cmTC_2c376.dir/manifest.res

CMakeFiles/cmTC_2c376.dir/manifest.rc

 

llvm-rc option parses requires /fo option argument to be separate.

I've modified the flag to support both joined and separate value which

helped getting past that error, but I immediately hit another error:

 

The problem seems to be that the .rc file generated by CMake contains

an absolute path to the resource, but llvm-rc expects the path to be

always relative. However, I couldn't fine any documentation that would

explain whether that's a requirement and it's CMake's fault or whether

llvm-rc should be handling absolute paths as well. Is anyone familiar

with Windows resource file semantics?

 

All of these errors seem to suggest that this cross-compiling for

Windows with Clang using CMake isn't a well explored area. Has anyone

had more success with cross-compiling LLVM runtimes (libc++,

compiler-rt) to Windows?

_______________________________________________

LLVM Developers mailing list

 


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
In reply to this post by David Jones via llvm-dev
I haven't cross-compiled LLVM. We have a working cross build of chrome/win, some notes on it at https://cs.chromium.org/chromium/src/docs/win_cross.md?type=cs&q=win_cross.md&sq=package:chromium&l=34

I don't have any experience cross-compiling with cmake.

For chrome, we still use the rc.cc at https://github.com/nico/hack/tree/master/res (through this driver: https://cs.chromium.org/chromium/src/build/toolchain/win/rc/rc.py?q=file:rc.py&sq=package:chromium&dr) which was the prototype for llvm-rc. We haven't switched to llvm-rc yet; my impression was that llvm-rc isn't super production quality yet (but some people do use it successfully).

On Tue, Nov 6, 2018 at 9:04 AM Martin Storsjö <[hidden email]> wrote:
Yes, I don't think mingw will fit his needs, but it's at least some
datapoint wrt "does anybody do this". But as most of the tool interfaces
are different it probably won't hit the same issues anyway.

// Martin


On Tue, 6 Nov 2018, Zachary Turner wrote:

> I suspect mingw is going to be incompatible with Petr’s needs. Adding Nico
> as he probably does the most cross compiling.  I did it a little several
> weeks ago, just not with libc++, and I don’t remember running into any
> problems.
>
> One thing that’s not clear to me is why the resource compiler is even being
> invoked at all
>
> On Tue, Nov 6, 2018 at 5:46 AM Martin Storsjö via llvm-dev
> <[hidden email]> wrote:
>       On Tue, 6 Nov 2018, Jean-Michaël Celerier via llvm-dev wrote:
>
>       > I have been using this project with great success to achieve
>       this :
>       > https://github.com/mstorsjo/llvm-mingw
>       >
>       > It generates a linux-based windows cross-compiler with clang,
>       libc++, etc.
>
>       FWIW, lately I've added prebuilt packages of it that you can run
>       on actual
>       windows as well, even though cross compilation was my first/main
>       focus.
>
>       > I was able to build a large part of Qt with it afterwards for
>       instance.
>
>       Any particular part of Qt which you weren't able to build with
>       it? I
>       regularly build Qt (qtbase) for i686/x86_64/armv7/aarch64, and
>       as part of
>       VLC, I also build a few more Qt modules (qtdeclarative,
>       qtquickcontrols2)
>       for i686 and x86_64.
>
>       // Martin
>       _______________________________________________
>       LLVM Developers mailing list
>       [hidden email]
>       http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Trying to cross-compile LLVM runtimes to Windows

David Jones via llvm-dev
Yup, re llvm-rc, I use it for a bunch of projects (with mostly
unspectacular resources), where it works fine, but I do know of a few
corner case resource types that are unimplemented. As long as you don't
use them (menuex is one that I have noted somewhere), it might work fine
though.

And if you have resource files with c preprocessor includes, one needs to
manually run the preprocessing outside of llvm-rc (contrary to rc.exe
which invokes the preprocessor internally).

// Martin

On Tue, 6 Nov 2018, Nico Weber wrote:

> I haven't cross-compiled LLVM. We have a working cross build of chrome/win,
> some notes on it athttps://cs.chromium.org/chromium/src/docs/win_cross.md?type=cs&q=win_cross.
> md&sq=package:chromium&l=34
>
> I don't have any experience cross-compiling with cmake.
>
> For chrome, we still use the rc.cc at
> https://github.com/nico/hack/tree/master/res (through thisdriver: https://cs.chromium.org/chromium/src/build/toolchain/win/rc/rc.py?q
> =file:rc.py&sq=package:chromium&dr) which was the prototype for llvm-rc. We
> haven't switched to llvm-rc yet; my impression was that llvm-rc isn't super
> production quality yet (but some people do use it successfully).
>
> On Tue, Nov 6, 2018 at 9:04 AM Martin Storsjö <[hidden email]> wrote:
>       Yes, I don't think mingw will fit his needs, but it's at least
>       some
>       datapoint wrt "does anybody do this". But as most of the tool
>       interfaces
>       are different it probably won't hit the same issues anyway.
>
>       // Martin
>
>
>       On Tue, 6 Nov 2018, Zachary Turner wrote:
>
>       > I suspect mingw is going to be incompatible with Petr’s needs.
>       Adding Nico
>       > as he probably does the most cross compiling.  I did it a
>       little several
>       > weeks ago, just not with libc++, and I don’t remember running
>       into any
>       > problems.
>       >
>       > One thing that’s not clear to me is why the resource compiler
>       is even being
>       > invoked at all
>       >
>       > On Tue, Nov 6, 2018 at 5:46 AM Martin Storsjö via llvm-dev
>       > <[hidden email]> wrote:
>       >       On Tue, 6 Nov 2018, Jean-Michaël Celerier via llvm-dev
>       wrote:
>       >
>       >       > I have been using this project with great success to
>       achieve
>       >       this :
>       >       > https://github.com/mstorsjo/llvm-mingw
>       >       >
>       >       > It generates a linux-based windows cross-compiler with
>       clang,
>       >       libc++, etc.
>       >
>       >       FWIW, lately I've added prebuilt packages of it that you
>       can run
>       >       on actual
>       >       windows as well, even though cross compilation was my
>       first/main
>       >       focus.
>       >
>       >       > I was able to build a large part of Qt with it
>       afterwards for
>       >       instance.
>       >
>       >       Any particular part of Qt which you weren't able to
>       build with
>       >       it? I
>       >       regularly build Qt (qtbase) for
>       i686/x86_64/armv7/aarch64, and
>       >       as part of
>       >       VLC, I also build a few more Qt modules (qtdeclarative,
>       >       qtquickcontrols2)
>       >       for i686 and x86_64.
>       >
>       >       // Martin
>       >       _______________________________________________
>       >       LLVM Developers mailing list
>       >       [hidden email]
>       >       http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>       >
>       >
>       >
>
>
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev