[llvm-dev] Windows build strangeness

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
I have the git monorepo, and Visual Studio 2015.  I am finding that
running a build from the command line with msbuild (as a nightly job)
invariably fails on the first try, and succeeds on a retry.

The first msbuild command looks like this:

msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 /t:Rebuild

This appears to compile everything okay, but invariably fails with
some set of link-time errors, reporting missing symbols.  Then I run
msbuild again, but without the /t:Rebuild (thus it's a dirty build the
second time) and this succeeds.

Is there some inadequacy in the CMake dependencies somewhere, so that
some links are trying to happen before the libraries are completely
finished?  Or just an msbuild issue?

I made my nightly script check for errors and run the second round,
because it has been happening so consistently, but a proper build
system shouldn't require such a workaround.

For reference, I generated the sln/project files like this:

cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
  -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
  -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
  ..\llvm-project\llvm

(Yes I know the optimized tablegen comes from a separate tree,
that's built separately and always works.  It's just the main
build that fails.)

Thanks,
--paulr

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
What kind of missing symbols are you getting? 
I had to work around dependencies for a Mingw32 build. See https://reviews.llvm.org/D44650

On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
I have the git monorepo, and Visual Studio 2015.  I am finding that
running a build from the command line with msbuild (as a nightly job)
invariably fails on the first try, and succeeds on a retry.

The first msbuild command looks like this:

msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 /t:Rebuild

This appears to compile everything okay, but invariably fails with
some set of link-time errors, reporting missing symbols.  Then I run
msbuild again, but without the /t:Rebuild (thus it's a dirty build the
second time) and this succeeds.

Is there some inadequacy in the CMake dependencies somewhere, so that
some links are trying to happen before the libraries are completely
finished?  Or just an msbuild issue?

I made my nightly script check for errors and run the second round,
because it has been happening so consistently, but a proper build
system shouldn't require such a workaround.

For reference, I generated the sln/project files like this:

cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
  -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
  -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
  ..\llvm-project\llvm

(Yes I know the optimized tablegen comes from a separate tree,
that's built separately and always works.  It's just the main
build that fails.)

Thanks,
--paulr

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev

Here are a couple of representative errors.  C:\Dev\upstream\gitmono is where I keep my clone.

 

       "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target) (1) ->

       "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj" (default target) (170:2) ->

         c1xx : fatal error C1083: Cannot open source file: 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor

       t\DynamicLibrary\PipSqueak.cxx': No such file or directory [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn

       amicLibrary\SecondLib.vcxproj]

 

       "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target) (1) ->

       "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj" (default target) (339:2) ->

       (Link target) ->

         Backend.obj : error LNK2019: unresolved external symbol "public: virtual bool __cdecl mca::FetchStage::isReady

       (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced in function "public: void __cdecl mca::Backend::run

       (void)" (?run@Backend@mca@@QEAAXXZ) [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]

 

 

 

From: Valentin Churavy [mailto:[hidden email]]
Sent: Wednesday, May 16, 2018 1:54 PM
To: Robinson, Paul
Cc: [hidden email]
Subject: Re: [llvm-dev] Windows build strangeness

 

What kind of missing symbols are you getting? 

I had to work around dependencies for a Mingw32 build. See https://reviews.llvm.org/D44650

On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:

I have the git monorepo, and Visual Studio 2015.  I am finding that
running a build from the command line with msbuild (as a nightly job)
invariably fails on the first try, and succeeds on a retry.

The first msbuild command looks like this:

msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 /t:Rebuild

This appears to compile everything okay, but invariably fails with
some set of link-time errors, reporting missing symbols.  Then I run
msbuild again, but without the /t:Rebuild (thus it's a dirty build the
second time) and this succeeds.

Is there some inadequacy in the CMake dependencies somewhere, so that
some links are trying to happen before the libraries are completely
finished?  Or just an msbuild issue?

I made my nightly script check for errors and run the second round,
because it has been happening so consistently, but a proper build
system shouldn't require such a workaround.

For reference, I generated the sln/project files like this:

cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
  -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
  -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
  ..\llvm-project\llvm

(Yes I know the optimized tablegen comes from a separate tree,
that's built separately and always works.  It's just the main
build that fails.)

Thanks,
--paulr

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
I think MSBuild isn't capable of re-running cmake and then reloading the project files when CMakeLists.txt changes. It re-runs cmake, but then continues the build with the stale projects. That probably explains the "PipSqueek.cxx doesn't exist" errors. As for the link error, it could also be caused by things like a file rename not getting picked up by MSBuild.

The fix is probably to always run cmake before MSBuild in the nightly job. This isn't necessary if you are using ninja or make, if those are options.

On Wed, May 16, 2018 at 12:59 PM via llvm-dev <[hidden email]> wrote:

Here are a couple of representative errors.  C:\Dev\upstream\gitmono is where I keep my clone.

 

       "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target) (1) ->

       "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj" (default target) (170:2) ->

         c1xx : fatal error C1083: Cannot open source file: 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor

       t\DynamicLibrary\PipSqueak.cxx': No such file or directory [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn

       amicLibrary\SecondLib.vcxproj]

 

       "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target) (1) ->

       "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj" (default target) (339:2) ->

       (Link target) ->

         Backend.obj : error LNK2019: unresolved external symbol "public: virtual bool __cdecl mca::FetchStage::isReady

       (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced in function "public: void __cdecl mca::Backend::run

       (void)" (?run@Backend@mca@@QEAAXXZ) [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]

 

 

 

From: Valentin Churavy [mailto:[hidden email]]
Sent: Wednesday, May 16, 2018 1:54 PM
To: Robinson, Paul
Cc: [hidden email]
Subject: Re: [llvm-dev] Windows build strangeness

 

What kind of missing symbols are you getting? 

I had to work around dependencies for a Mingw32 build. See https://reviews.llvm.org/D44650

On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:

I have the git monorepo, and Visual Studio 2015.  I am finding that
running a build from the command line with msbuild (as a nightly job)
invariably fails on the first try, and succeeds on a retry.

The first msbuild command looks like this:

msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 /t:Rebuild

This appears to compile everything okay, but invariably fails with
some set of link-time errors, reporting missing symbols.  Then I run
msbuild again, but without the /t:Rebuild (thus it's a dirty build the
second time) and this succeeds.

Is there some inadequacy in the CMake dependencies somewhere, so that
some links are trying to happen before the libraries are completely
finished?  Or just an msbuild issue?

I made my nightly script check for errors and run the second round,
because it has been happening so consistently, but a proper build
system shouldn't require such a workaround.

For reference, I generated the sln/project files like this:

cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
  -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
  -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
  ..\llvm-project\llvm

(Yes I know the optimized tablegen comes from a separate tree,
that's built separately and always works.  It's just the main
build that fails.)

Thanks,
--paulr

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake
adds a special project "ZERO_CHECK" that does this. However, I am not
sure it runs when invoking on the individual projects instead of the
solution.

Try the cmake --build command, which should output the following:

> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]

Michael



2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <[hidden email]>:

> I think MSBuild isn't capable of re-running cmake and then reloading the
> project files when CMakeLists.txt changes. It re-runs cmake, but then
> continues the build with the stale projects. That probably explains the
> "PipSqueek.cxx doesn't exist" errors. As for the link error, it could also
> be caused by things like a file rename not getting picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev <[hidden email]>
> wrote:
>>
>> Here are a couple of representative errors.  C:\Dev\upstream\gitmono is
>> where I keep my clone.
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target)
>> (1) ->
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>>          c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>>        t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>>        amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild target)
>> (1) ->
>>
>>        "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>>        (Link target) ->
>>
>>          Backend.obj : error LNK2019: unresolved external symbol "public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>>        (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced in
>> function "public: void __cdecl mca::Backend::run
>>
>>        (void)" (?run@Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:[hidden email]]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: [hidden email]
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015.  I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols.  Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build the
>> second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished?  Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>>   -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>>   -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>>   ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree,
>> that's built separately and always works.  It's just the main
>> build that fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run successfully and the build to also continue successfully, but since I switched to VS2015 the CMake re-run occurs - apparently successfully, but more often than not the build failed afterwards from either the IDE and from MSBuild.  Since I seldom change the CMakeLists.txt files, I simply do a clean CMake configuration anytime I do change them.  I also use the "Build" rather than "Rebuild" for incremental development.

        MartinO

-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Michael Kruse via llvm-dev
Sent: 16 May 2018 21:26
To: Reid Kleckner <[hidden email]>
Cc: llvm-dev <[hidden email]>; Valentin Churavy <[hidden email]>
Subject: Re: [llvm-dev] Windows build strangeness

msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a special project "ZERO_CHECK" that does this. However, I am not sure it runs when invoking on the individual projects instead of the solution.

Try the cmake --build command, which should output the following:

> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]

Michael



2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <[hidden email]>:

> I think MSBuild isn't capable of re-running cmake and then reloading
> the project files when CMakeLists.txt changes. It re-runs cmake, but
> then continues the build with the stale projects. That probably
> explains the "PipSqueek.cxx doesn't exist" errors. As for the link
> error, it could also be caused by things like a file rename not getting picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev
> <[hidden email]>
> wrote:
>>
>> Here are a couple of representative errors.  C:\Dev\upstream\gitmono
>> is where I keep my clone.
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>>          c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>>        t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>>        amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>        "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>>        (Link target) ->
>>
>>          Backend.obj : error LNK2019: unresolved external symbol "public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>>        (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced
>> in function "public: void __cdecl mca::Backend::run
>>
>>        (void)" (?run@Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:[hidden email]]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: [hidden email]
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015.  I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols.  Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build
>> the second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished?  Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>>   -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>>   -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>>   ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree, that's
>> built separately and always works.  It's just the main build that
>> fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev
From my own experience this is what I think happens when building the whole solution through Visual Studio's UI. This also happens for building individual projects. I assume something similar happens when building via the command-line, but I rarely do that, so I can't be certain.

1) Visual Studio/MSBuild (I don't know which, but probably MSBuild) determines the dependency graph of the projects that are to be built.
2) The "ZeroCheck" project runs and cmake is rerun if any of the CMakeLists.txt files have been updated.
3) The build continues for each of the projects, using the dependency graph from 1). If the projects from 1) were updated due to 2) then the updated projects will be built in the updated way (e.g. new files will be included in that build).
4) Since any new projects created by re-running cmake in 2) were not in the dependency graph in 1), projects that now rely on them will likely get link errors.

There are two solutions I've used in my development: 1) always run cmake explicitly after doing an svn/git fetch before building or 2) build twice. The second build is usually a lot faster, so it doesn't really matter.

James

On 16 May 2018 at 22:07, Martin J. O'Riordan via llvm-dev <[hidden email]> wrote:
With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run successfully and the build to also continue successfully, but since I switched to VS2015 the CMake re-run occurs - apparently successfully, but more often than not the build failed afterwards from either the IDE and from MSBuild.  Since I seldom change the CMakeLists.txt files, I simply do a clean CMake configuration anytime I do change them.  I also use the "Build" rather than "Rebuild" for incremental development.

        MartinO

-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Michael Kruse via llvm-dev
Sent: 16 May 2018 21:26
To: Reid Kleckner <[hidden email]>
Cc: llvm-dev <[hidden email]>; Valentin Churavy <[hidden email]>
Subject: Re: [llvm-dev] Windows build strangeness

msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a special project "ZERO_CHECK" that does this. However, I am not sure it runs when invoking on the individual projects instead of the solution.

Try the cmake --build command, which should output the following:

> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]

Michael



2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <[hidden email]>:
> I think MSBuild isn't capable of re-running cmake and then reloading
> the project files when CMakeLists.txt changes. It re-runs cmake, but
> then continues the build with the stale projects. That probably
> explains the "PipSqueek.cxx doesn't exist" errors. As for the link
> error, it could also be caused by things like a file rename not getting picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev
> <[hidden email]>
> wrote:
>>
>> Here are a couple of representative errors.  C:\Dev\upstream\gitmono
>> is where I keep my clone.
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>>          c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>>        t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>>        amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>        "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>>        (Link target) ->
>>
>>          Backend.obj : error LNK2019: unresolved external symbol "public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>>        (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced
>> in function "public: void __cdecl mca::Backend::run
>>
>>        (void)" (?run@Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:[hidden email]]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: [hidden email]
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015.  I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols.  Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build
>> the second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished?  Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>>   -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>>   -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>>   ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree, that's
>> built separately and always works.  It's just the main build that
>> fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev

It looks like building ZERO_CHECK first to reconstruct the project files as needed, then running msbuild a second time to do the actual build, has solved the problem.  At least, last night's run didn't take the usual two tries.  Running msbuild twice is a little bit simpler than running cmake explicitly, the way I have my scripts set up, but I'm sure that would work as well.

Thanks everybody!

--paulr

 

From: llvm-dev [mailto:[hidden email]] On Behalf Of James Henderson via llvm-dev
Sent: Thursday, May 17, 2018 5:54 AM
To: Martin J. O'Riordan
Cc: LLVM Developers; Valentin Churavy
Subject: Re: [llvm-dev] Windows build strangeness

 

From my own experience this is what I think happens when building the whole solution through Visual Studio's UI. This also happens for building individual projects. I assume something similar happens when building via the command-line, but I rarely do that, so I can't be certain.

1) Visual Studio/MSBuild (I don't know which, but probably MSBuild) determines the dependency graph of the projects that are to be built.

2) The "ZeroCheck" project runs and cmake is rerun if any of the CMakeLists.txt files have been updated.

3) The build continues for each of the projects, using the dependency graph from 1). If the projects from 1) were updated due to 2) then the updated projects will be built in the updated way (e.g. new files will be included in that build).

4) Since any new projects created by re-running cmake in 2) were not in the dependency graph in 1), projects that now rely on them will likely get link errors.

There are two solutions I've used in my development: 1) always run cmake explicitly after doing an svn/git fetch before building or 2) build twice. The second build is usually a lot faster, so it doesn't really matter.

James

 

On 16 May 2018 at 22:07, Martin J. O'Riordan via llvm-dev <[hidden email]> wrote:

With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run successfully and the build to also continue successfully, but since I switched to VS2015 the CMake re-run occurs - apparently successfully, but more often than not the build failed afterwards from either the IDE and from MSBuild.  Since I seldom change the CMakeLists.txt files, I simply do a clean CMake configuration anytime I do change them.  I also use the "Build" rather than "Rebuild" for incremental development.

        MartinO


-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Michael Kruse via llvm-dev
Sent: 16 May 2018 21:26
To: Reid Kleckner <[hidden email]>
Cc: llvm-dev <[hidden email]>; Valentin Churavy <[hidden email]>
Subject: Re: [llvm-dev] Windows build strangeness

msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a special project "ZERO_CHECK" that does this. However, I am not sure it runs when invoking on the individual projects instead of the solution.

Try the cmake --build command, which should output the following:

> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]

Michael



2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <[hidden email]>:
> I think MSBuild isn't capable of re-running cmake and then reloading
> the project files when CMakeLists.txt changes. It re-runs cmake, but
> then continues the build with the stale projects. That probably
> explains the "PipSqueek.cxx doesn't exist" errors. As for the link
> error, it could also be caused by things like a file rename not getting picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev
> <[hidden email]>
> wrote:
>>
>> Here are a couple of representative errors.  C:\Dev\upstream\gitmono
>> is where I keep my clone.
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>>          c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>>        t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>>        amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>        "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>>        (Link target) ->
>>
>>          Backend.obj : error LNK2019: unresolved external symbol "public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>>        (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced
>> in function "public: void __cdecl mca::Backend::run
>>
>>        (void)" (?run@Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:[hidden email]]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: [hidden email]
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015.  I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols.  Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build
>> the second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished?  Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>>   -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>>   -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>>   ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree, that's
>> built separately and always works.  It's just the main build that
>> fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

 


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows build strangeness

Sam McCall via llvm-dev

FTR, this has *mostly* solved the problem.  I think there is still some kind of bug in msbuild's internal coordination of parallel jobs, because I do sometimes see errors about multiple processes trying to access the same "lastbuildstate" file.  But it's way better than it was.

And probably someday I'll try ninja on Windows.

--paulr

 

From: llvm-dev [mailto:[hidden email]] On Behalf Of via llvm-dev
Sent: Thursday, May 17, 2018 10:20 AM
To: [hidden email]; [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: [llvm-dev] Windows build strangeness

 

It looks like building ZERO_CHECK first to reconstruct the project files as needed, then running msbuild a second time to do the actual build, has solved the problem.  At least, last night's run didn't take the usual two tries.  Running msbuild twice is a little bit simpler than running cmake explicitly, the way I have my scripts set up, but I'm sure that would work as well.

Thanks everybody!

--paulr

 

From: llvm-dev [mailto:[hidden email]] On Behalf Of James Henderson via llvm-dev
Sent: Thursday, May 17, 2018 5:54 AM
To: Martin J. O'Riordan
Cc: LLVM Developers; Valentin Churavy
Subject: Re: [llvm-dev] Windows build strangeness

 

From my own experience this is what I think happens when building the whole solution through Visual Studio's UI. This also happens for building individual projects. I assume something similar happens when building via the command-line, but I rarely do that, so I can't be certain.

1) Visual Studio/MSBuild (I don't know which, but probably MSBuild) determines the dependency graph of the projects that are to be built.

2) The "ZeroCheck" project runs and cmake is rerun if any of the CMakeLists.txt files have been updated.

3) The build continues for each of the projects, using the dependency graph from 1). If the projects from 1) were updated due to 2) then the updated projects will be built in the updated way (e.g. new files will be included in that build).

4) Since any new projects created by re-running cmake in 2) were not in the dependency graph in 1), projects that now rely on them will likely get link errors.

There are two solutions I've used in my development: 1) always run cmake explicitly after doing an svn/git fetch before building or 2) build twice. The second build is usually a lot faster, so it doesn't really matter.

James

 

On 16 May 2018 at 22:07, Martin J. O'Riordan via llvm-dev <[hidden email]> wrote:

With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run successfully and the build to also continue successfully, but since I switched to VS2015 the CMake re-run occurs - apparently successfully, but more often than not the build failed afterwards from either the IDE and from MSBuild.  Since I seldom change the CMakeLists.txt files, I simply do a clean CMake configuration anytime I do change them.  I also use the "Build" rather than "Rebuild" for incremental development.

        MartinO


-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Michael Kruse via llvm-dev
Sent: 16 May 2018 21:26
To: Reid Kleckner <[hidden email]>
Cc: llvm-dev <[hidden email]>; Valentin Churavy <[hidden email]>
Subject: Re: [llvm-dev] Windows build strangeness

msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a special project "ZERO_CHECK" that does this. However, I am not sure it runs when invoking on the individual projects instead of the solution.

Try the cmake --build command, which should output the following:

> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]

Michael



2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <[hidden email]>:
> I think MSBuild isn't capable of re-running cmake and then reloading
> the project files when CMakeLists.txt changes. It re-runs cmake, but
> then continues the build with the stale projects. That probably
> explains the "PipSqueek.cxx doesn't exist" errors. As for the link
> error, it could also be caused by things like a file rename not getting picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev
> <[hidden email]>
> wrote:
>>
>> Here are a couple of representative errors.  C:\Dev\upstream\gitmono
>> is where I keep my clone.
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>>          c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>>        t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>>        amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>>        "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild
>> target)
>> (1) ->
>>
>>        "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>>        (Link target) ->
>>
>>          Backend.obj : error LNK2019: unresolved external symbol "public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>>        (void)const " (?isReady@FetchStage@mca@@UEBA_NXZ) referenced
>> in function "public: void __cdecl mca::Backend::run
>>
>>        (void)" (?run@Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:[hidden email]]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: [hidden email]
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <[hidden email]> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015.  I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols.  Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build
>> the second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished?  Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>>   -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>>   -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>>   ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree, that's
>> built separately and always works.  It's just the main build that
>> fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

 


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev