make check rebuilds the project?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

make check rebuilds the project?

Arnold Schwaighofer-3
It appears the behavior of "make check" has changed recently (configure style build).  I believe it used to just run the tests. Now, it seems to recompile the project just like a "make" would and then run the tests. Is it just me or are others seeing this too? Is this a intended behavior change?

Best,
Arnold
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Bob Wilson-3
Eric changed it in svn 179116.  He didn't provide any explanation in the commit message.  Eric, why did you do that?

On Apr 17, 2013, at 8:30 AM, Arnold Schwaighofer <[hidden email]> wrote:

> It appears the behavior of "make check" has changed recently (configure style build).  I believe it used to just run the tests. Now, it seems to recompile the project just like a "make" would and then run the tests. Is it just me or are others seeing this too? Is this a intended behavior change?
>
> Best,
> Arnold
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eric Christopher
Because almost no other piece of software out there doesn't have
'check' depend on rebuilding the target it's testing and it's bitten
people whereas other than the lameness of our build system needing to
recurse all the directories to figure out that all has already been
rebuilt there's no penalty to requiring the dependency. :)

-eric

On Wed, Apr 17, 2013 at 9:35 AM, Bob Wilson <[hidden email]> wrote:

> Eric changed it in svn 179116.  He didn't provide any explanation in the commit message.  Eric, why did you do that?
>
> On Apr 17, 2013, at 8:30 AM, Arnold Schwaighofer <[hidden email]> wrote:
>
>> It appears the behavior of "make check" has changed recently (configure style build).  I believe it used to just run the tests. Now, it seems to recompile the project just like a "make" would and then run the tests. Is it just me or are others seeing this too? Is this a intended behavior change?
>>
>> Best,
>> Arnold
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Krzysztof Parzyszek
On 4/17/2013 11:50 AM, Eric Christopher wrote:
> Because almost no other piece of software out there doesn't have
> 'check' depend on rebuilding the target it's testing and it's bitten
> people whereas other than the lameness of our build system needing to
> recurse all the directories to figure out that all has already been
> rebuilt there's no penalty to requiring the dependency. :)

Given this explanation my vote goes for reverting it.  :)

-Krzysztof

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eric Christopher
On Wed, Apr 17, 2013 at 10:05 AM, Krzysztof Parzyszek
<[hidden email]> wrote:

> On 4/17/2013 11:50 AM, Eric Christopher wrote:
>>
>> Because almost no other piece of software out there doesn't have
>> 'check' depend on rebuilding the target it's testing and it's bitten
>> people whereas other than the lameness of our build system needing to
>> recurse all the directories to figure out that all has already been
>> rebuilt there's no penalty to requiring the dependency. :)
>
>
> Given this explanation my vote goes for reverting it.  :)
>

Give a compelling argument and I might :)

-eric
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Krzysztof Parzyszek
On 4/17/2013 12:15 PM, Eric Christopher wrote:
>
> Give a compelling argument and I might :)

The cost is the extra time spent verifying that the binaries are fresh.
  In my experience it has never happened that I ran "make check" on a
wrong set of binaries.  On the other hand, it does happen that I run
"make check" repeatedly, when working on a testcase (or a set of
testcases), or when I fixed something that previously caused some
testcases to be XFAILed.

-Krzysztof


--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Krzysztof Parzyszek
On 4/17/2013 12:23 PM, Krzysztof Parzyszek wrote:
>
> The cost is the extra time spent verifying that the binaries are fresh.
>   In my experience it has never happened that I ran "make check" on a
> wrong set of binaries.  On the other hand, it does happen that I run
> "make check" repeatedly, when working on a testcase (or a set of
> testcases), or when I fixed something that previously caused some
> testcases to be XFAILed.

I don't know if this is compelling enough, since it hinges on my own
working habits.  It would be interesting to see the more opinions about it.

-Krzysztof

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eric Christopher
In reply to this post by Krzysztof Parzyszek
On Wed, Apr 17, 2013 at 10:23 AM, Krzysztof Parzyszek
<[hidden email]> wrote:

> On 4/17/2013 12:15 PM, Eric Christopher wrote:
>>
>>
>> Give a compelling argument and I might :)
>
>
> The cost is the extra time spent verifying that the binaries are fresh.  In
> my experience it has never happened that I ran "make check" on a wrong set
> of binaries.  On the other hand, it does happen that I run "make check"
> repeatedly, when working on a testcase (or a set of testcases), or when I
> fixed something that previously caused some testcases to be XFAILed.
>

The extra cost is pretty trivial and you can run individual test cases
by using lit explicitly. We've had annoying bot failures because
someone ran "make check" but it didn't rebuild the targets that check
depends upon. On my laptop it's a total of 4 seconds to get to
check... you probably would save more time by running lit explicitly
on a testcase you're changing :)

-eric
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eli Bendersky
In reply to this post by Krzysztof Parzyszek



On Wed, Apr 17, 2013 at 10:23 AM, Krzysztof Parzyszek <[hidden email]> wrote:
On 4/17/2013 12:15 PM, Eric Christopher wrote:

Give a compelling argument and I might :)

The cost is the extra time spent verifying that the binaries are fresh.  In my experience it has never happened that I ran "make check" on a wrong set of binaries.  On the other hand, it does happen that I run "make check" repeatedly, when working on a testcase (or a set of testcases), or when I fixed something that previously caused some testcases to be XFAILed.

If development speed / turnaround time is important for you, 'make' is the wrong solution in the first plane. Use a cmake+ninja build, which is way faster, especially for null builds (where all the build system does is make sure there's nothing new to build, or very few things to build). 

Here's a ninja build run on a fully built debug LLVM+Clang checkout:

$ time ninja
ninja: no work to do.

real 0m0.088s
user 0m0.070s
sys 0m0.010s

You can't beat that with 'make' no matter what you do ;-)

Eli


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Nick Lewycky
In reply to this post by Eric Christopher
On 04/17/2013 09:50 AM, Eric Christopher wrote:
> Because almost no other piece of software out there doesn't have
> 'check' depend on rebuilding the target it's testing and it's bitten
> people whereas other than the lameness of our build system needing to
> recurse all the directories to figure out that all has already been
> rebuilt there's no penalty to requiring the dependency. :)

As a single datapoint, I've been relying on the fact that make check
doesn't rebuild. Sometimes I work on underpowered machines and will
continue to reformat my code after the .o file has been built. Relinking
can take more than five minutes. I do not like this change, but my
argument is also rather silly, per http://xkcd.com/1172 .

Nick

>
> -eric
>
> On Wed, Apr 17, 2013 at 9:35 AM, Bob Wilson<[hidden email]>  wrote:
>> Eric changed it in svn 179116.  He didn't provide any explanation in the commit message.  Eric, why did you do that?
>>
>> On Apr 17, 2013, at 8:30 AM, Arnold Schwaighofer<[hidden email]>  wrote:
>>
>>> It appears the behavior of "make check" has changed recently (configure style build).  I believe it used to just run the tests. Now, it seems to recompile the project just like a "make" would and then run the tests. Is it just me or are others seeing this too? Is this a intended behavior change?
>>>
>>> Best,
>>> Arnold
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> [hidden email]         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Jim Grosbach
In reply to this post by Eli Bendersky

On Apr 17, 2013, at 10:43 AM, Eli Bendersky <[hidden email]> wrote:




On Wed, Apr 17, 2013 at 10:23 AM, Krzysztof Parzyszek <[hidden email]> wrote:
On 4/17/2013 12:15 PM, Eric Christopher wrote:

Give a compelling argument and I might :)

The cost is the extra time spent verifying that the binaries are fresh.  In my experience it has never happened that I ran "make check" on a wrong set of binaries.  On the other hand, it does happen that I run "make check" repeatedly, when working on a testcase (or a set of testcases), or when I fixed something that previously caused some testcases to be XFAILed.

If development speed / turnaround time is important for you, 'make' is the wrong solution in the first plane. Use a cmake+ninja build, which is way faster, especially for null builds (where all the build system does is make sure there's nothing new to build, or very few things to build). 

Here's a ninja build run on a fully built debug LLVM+Clang checkout:

$ time ninja
ninja: no work to do.

real 0m0.088s
user 0m0.070s
sys 0m0.010s

You can't beat that with 'make' no matter what you do ;-)


cmake+ninja is a non-option for many developers. Fixing that is a separate problem. We can’t just ignore problems in our configure+make build system in the meantime.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eli Bendersky



On Wed, Apr 17, 2013 at 10:53 AM, Jim Grosbach <[hidden email]> wrote:

On Apr 17, 2013, at 10:43 AM, Eli Bendersky <[hidden email]> wrote:




On Wed, Apr 17, 2013 at 10:23 AM, Krzysztof Parzyszek <[hidden email]> wrote:
On 4/17/2013 12:15 PM, Eric Christopher wrote:

Give a compelling argument and I might :)

The cost is the extra time spent verifying that the binaries are fresh.  In my experience it has never happened that I ran "make check" on a wrong set of binaries.  On the other hand, it does happen that I run "make check" repeatedly, when working on a testcase (or a set of testcases), or when I fixed something that previously caused some testcases to be XFAILed.

If development speed / turnaround time is important for you, 'make' is the wrong solution in the first plane. Use a cmake+ninja build, which is way faster, especially for null builds (where all the build system does is make sure there's nothing new to build, or very few things to build). 

Here's a ninja build run on a fully built debug LLVM+Clang checkout:

$ time ninja
ninja: no work to do.

real 0m0.088s
user 0m0.070s
sys 0m0.010s

You can't beat that with 'make' no matter what you do ;-)


cmake+ninja is a non-option for many developers. Fixing that is a separate problem. We can’t just ignore problems in our configure+make build system in the meantime.

Understood, and this is why I didn't express an opinion on the original issue. I just thought it may help a fellow developer to point this out if he really cares about the speed of null builds. Personally I've saved a lot of time since switching to ninja a few months ago.

Eli




_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Krzysztof Parzyszek
On 4/17/2013 1:02 PM, Eli Bendersky wrote:
>
> Understood, and this is why I didn't express an opinion on the original
> issue. I just thought it may help a fellow developer to point this out
> if he really cares about the speed of null builds. Personally I've saved
> a lot of time since switching to ninja a few months ago.

For me it really boils down to my personal working habits.  Something
that has worked for me just fine, now is about to become more
time-consuming, that is pretty much it.  I wouldn't want that preference
of mine to be the sole cause for any further changes, but I wanted to
share my thoughts about it.

-Krzysztof

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Reid Kleckner-2
In reply to this post by Jim Grosbach
FWIW, the cmake build has the "check" target depend on the binaries under test.

On Apr 17, 2013, at 10:43 AM, Eli Bendersky <[hidden email]> wrote:
> If development speed / turnaround time is important for you, 'make' is the
> wrong solution in the first plane. Use a cmake+ninja build, which is way
> faster, especially for null builds (where all the build system does is make
> sure there's nothing new to build, or very few things to build).

I would 100% recommend it for that reason, but one thing to note is
that ninja buffers subprocess output.  Usually this is good because
you can avoid interleaved diagnostics from multiple source files, but
it means "ninja check" buffers lit's output until it's done.  That's
OK for me when I'm doing final testing for a change, but it means it's
not quite a drop-in replacement for "make check".
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Daniel Berlin
In reply to this post by Jim Grosbach
On Wed, Apr 17, 2013 at 10:53 AM, Jim Grosbach <[hidden email]> wrote:

>
> On Apr 17, 2013, at 10:43 AM, Eli Bendersky <[hidden email]> wrote:
>
>
>
>
> On Wed, Apr 17, 2013 at 10:23 AM, Krzysztof Parzyszek
> <[hidden email]> wrote:
>>
>> On 4/17/2013 12:15 PM, Eric Christopher wrote:
>>>
>>>
>>> Give a compelling argument and I might :)
>>
>>
>> The cost is the extra time spent verifying that the binaries are fresh.
>> In my experience it has never happened that I ran "make check" on a wrong
>> set of binaries.  On the other hand, it does happen that I run "make check"
>> repeatedly, when working on a testcase (or a set of testcases), or when I
>> fixed something that previously caused some testcases to be XFAILed.
>
>
> If development speed / turnaround time is important for you, 'make' is the
> wrong solution in the first plane. Use a cmake+ninja build, which is way
> faster, especially for null builds (where all the build system does is make
> sure there's nothing new to build, or very few things to build).
>
> Here's a ninja build run on a fully built debug LLVM+Clang checkout:
>
> $ time ninja
> ninja: no work to do.
>
> real 0m0.088s
> user 0m0.070s
> sys 0m0.010s
>
> You can't beat that with 'make' no matter what you do ;-)
>
>
> cmake+ninja is a non-option for many developers.


By the way, i'm curious what this means. It can be interpreted in a
number of ways, but it would be good to know what problems you are
specifically saying making it a non-option for many developers
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Jim Grosbach


cmake+ninja is a non-option for many developers.


By the way, i'm curious what this means. It can be interpreted in a
number of ways, but it would be good to know what problems you are
specifically saying making it a non-option for many developers

Hi Danny,

Sorry for being unclear. My fault for posting when I’m in a hurry.

I don’t think there’s any unsolvable problems. Well, not technical ones anyway. It’s a question of supporting a broad enough cross section of environments and workflows that devs not only can move to cmake+ninja, but actively want to do so.

For me personally, it’s more a question of supported workflows being inconvenient than it being a strict non-starter. That is, it’s a question of which system has fewer inconveniences for me in my daily work. Right now, that’s configure+make. I’m very much looking forward to the day when the balance of that equation changes.

Note, I’m explicitly excluding the whole “remove autoconf+make” thing from this. That’s another topic with a whole extra set of issues. Likewise, I’m not going to make any sorts of claims about exhaustiveness here, but rather just relate mostly what I’ve encountered.

AFAIK ninja still only has very preliminary support for Windows, so anyone working there can’t use it. Perhaps that’s changed or I’m misinformed? Last I tried it, OSX support was better, but still very much “try it and see if it works for you.” I personally found ninja’s OSX support to be fine, but I didn’t really beat on it to put it through its paces, either. Relatedly, some work environments have varying ability to install and use “non-standard” tools. I’ve been in workplaces before that I’d have had to fight tooth and nail to get something like cmake or ninja available to me on my workstation. Even now when I don’t have that policy sort of problem, it’s still a pain when every time I install a new system (which I do disturbingly often), I need to install additional tools before I can start using llvm. Not a big problem on its own of course, but an inconvenience.

cmake+ninja doesn’t support building and running the llvm test suite. That’s pure configure+make still. I rely on configure automatically setting up the test suite to run on whatever compiler I just built.

For example, I build Debug+Asserts, Release+Asserts and Release configurations all in the same build directory and rely on the configure+make machinery to create separate build results subdirectories named for those. That is, it’s pretty common for me to do something that boils down to (with lots of steps in between of course), “<src>/configure && make && make check && make ENABLE_OPTIMIZED=1 && make check ENABLE_OPTIMIZED=1 && make ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1 && make check  ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1”. Depending on what I’m doing, I bounce back and forth between them. With  cmake, I have to have completely separate configured build directories. I already have 10+ llvm build directories lying around that I regularly use. I’d prefer to keep that number from growing too horribly. When I last tried using cmake+ninja, this is the one that I kept running into and getting annoyed with.

Do the cmake scripts support cross-compiling? Last I tried, I had trouble with that. Many moons ago, I had to beat the plain Makefiles into some sort of submission for that. I not infrequently build llvm to run on ARM, for example.

Somewhat relatedly, I like to keep my development builds as close as reasonably possible to production builds. There are few things that make my stomach churn more than bugs that only reproduce on the production builders. That’s somewhat off-topic, though, as that’s getting into “what would it take to remove configure+make support” territory.

In short, I’m a big fan of ninja and really want to use it. CMake, I’m less thrilled about, but I’ll put up with it if it gets me ninja. Right now though, it’s a close but not quite a fully baked replacement for my uses.

Anyways, none of this should be construed to be a slight to the work people have put into cmake+ninja support for llvm/clang. Quite the opposite. I’m excited enough about what’s been done that I’m frustrated it doesn’t fit my use-cases well enough for me to switch.

My point with that comment, which Eli correctly and effectively addressed in his reply, is that we can’t assume we can define away problems with the configure+make system by just telling people it’s deprecated and they should use cmake+ninja instead. We’re not there yet.

Thanks,
  Jim

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Nico Rieck
On 17.04.2013 21:46, Jim Grosbach wrote:
> AFAIK ninja still only has very preliminary support for Windows, so anyone
> working there can’t use it. Perhaps that’s changed or I’m misinformed?

Ninja works fine for me on Windows. I've been using it to compile LLVM
and co. for a few months. It's also really easy to compile Ninja itself
on Windows.
There's just one issue: When a *.td file is changed, I have to invoke
ninja twice. But I haven't investigated yet wether that's an issue with
Windows, ninja, cmake or LLVM's build scripts.


-Nico
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Jim Grosbach

On Apr 17, 2013, at 1:03 PM, Nico Rieck <[hidden email]> wrote:

On 17.04.2013 21:46, Jim Grosbach wrote:
AFAIK ninja still only has very preliminary support for Windows, so anyone
working there can’t use it. Perhaps that’s changed or I’m misinformed?

Ninja works fine for me on Windows. I've been using it to compile LLVM and co. for a few months. It's also really easy to compile Ninja itself on Windows.
There's just one issue: When a *.td file is changed, I have to invoke ninja twice. But I haven't investigated yet wether that's an issue with Windows, ninja, cmake or LLVM's build scripts.

Nice! Sounds like things have indeed moved in the right direction since I last looked at that. Most excellent.

-Jim

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Hal Finkel
In reply to this post by Jim Grosbach
----- Original Message -----

> From: "Jim Grosbach" <[hidden email]>
> To: "Daniel Berlin" <[hidden email]>
> Cc: [hidden email]
> Sent: Wednesday, April 17, 2013 2:46:49 PM
> Subject: Re: [LLVMdev] make check rebuilds the project?
>
>
>
>
>
>
>
>
>
>
> cmake+ninja is a non-option for many developers.
>
>
> By the way, i'm curious what this means. It can be interpreted in a
> number of ways, but it would be good to know what problems you are
> specifically saying making it a non-option for many developers
>
>
> Hi Danny,
>
>
> Sorry for being unclear. My fault for posting when I’m in a hurry.
>
>
> I don’t think there’s any unsolvable problems. Well, not technical
> ones anyway. It’s a question of supporting a broad enough cross
> section of environments and workflows that devs not only can move to
> cmake+ninja, but actively want to do so.
>
>
> For me personally, it’s more a question of supported workflows being
> inconvenient than it being a strict non-starter. That is, it’s a
> question of which system has fewer inconveniences for me in my daily
> work. Right now, that’s configure+make. I’m very much looking
> forward to the day when the balance of that equation changes.
>
> Note, I’m explicitly excluding the whole “remove autoconf+make” thing
> from this. That’s another topic with a whole extra set of issues.
> Likewise, I’m not going to make any sorts of claims about
> exhaustiveness here, but rather just relate mostly what I’ve
> encountered.
>
>
> AFAIK ninja still only has very preliminary support for Windows, so
> anyone working there can’t use it. Perhaps that’s changed or I’m
> misinformed? Last I tried it, OSX support was better, but still very
> much “try it and see if it works for you.” I personally found
> ninja’s OSX support to be fine, but I didn’t really beat on it to
> put it through its paces, either. Relatedly, some work environments
> have varying ability to install and use “non-standard” tools. I’ve
> been in workplaces before that I’d have had to fight tooth and nail
> to get something like cmake or ninja available to me on my
> workstation. Even now when I don’t have that policy sort of problem,
> it’s still a pain when every time I install a new system (which I do
> disturbingly often), I need to install additional tools before I can
> start using llvm. Not a big problem on its own of course, but an
> inconvenience.
>
>
> cmake+ninja doesn’t support building and running the llvm test suite.
> That’s pure configure+make still. I rely on configure automatically
> setting up the test suite to run on whatever compiler I just built.
>
>
> For example, I build Debug+Asserts, Release+Asserts and Release
> configurations all in the same build directory and rely on the
> configure+make machinery to create separate build results
> subdirectories named for those. That is, it’s pretty common for me
> to do something that boils down to (with lots of steps in between of
> course), “<src>/configure && make && make check && make
> ENABLE_OPTIMIZED=1 && make check ENABLE_OPTIMIZED=1 && make
> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1 && make check
> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1”. Depending on what I’m
> doing, I bounce back and forth between them. With cmake, I have to
> have completely separate configured build directories. I already
> have 10+ llvm build directories lying around that I regularly use.
> I’d prefer to keep that number from growing too horribly. When I
> last tried using cmake+ninja, this is the one that I kept running
> into and getting annoyed with.
>
>
> Do the cmake scripts support cross-compiling? Last I tried, I had
> trouble with that. Many moons ago, I had to beat the plain Makefiles
> into some sort of submission for that. I not infrequently build llvm
> to run on ARM, for example.
>
>
> Somewhat relatedly, I like to keep my development builds as close as
> reasonably possible to production builds. There are few things that
> make my stomach churn more than bugs that only reproduce on the
> production builders. That’s somewhat off-topic, though, as that’s
> getting into “what would it take to remove configure+make support”
> territory.
>
>
> In short, I’m a big fan of ninja and really want to use it. CMake,
> I’m less thrilled about, but I’ll put up with it if it gets me
> ninja. Right now though, it’s a close but not quite a fully baked
> replacement for my uses.
>
>
> Anyways, none of this should be construed to be a slight to the work
> people have put into cmake+ninja support for llvm/clang. Quite the
> opposite. I’m excited enough about what’s been done that I’m
> frustrated it doesn’t fit my use-cases well enough for me to switch.
>
>
> My point with that comment, which Eli correctly and effectively
> addressed in his reply, is that we can’t assume we can define away
> problems with the configure+make system by just telling people it’s
> deprecated and they should use cmake+ninja instead. We’re not there
> yet.

I'd like to second all of this.

Also, I'll add that I also sometimes build on machines where linking a Debug+Asserts clang takes several minutes, and I like to be able to cut off that process to run make check. However, I discovered that if you 'cd test' before running make check, then there are no dependency problems and it will just run the tests with your current build as-is. So long as that works, I'm fine with keeping the top-level make check dependent on the rest of the build.

 -Hal

>
>
> Thanks,
> Jim
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: make check rebuilds the project?

Eric Christopher
On Wed, Apr 17, 2013 at 1:05 PM, Hal Finkel <[hidden email]> wrote:

> ----- Original Message -----
>> From: "Jim Grosbach" <[hidden email]>
>> To: "Daniel Berlin" <[hidden email]>
>> Cc: [hidden email]
>> Sent: Wednesday, April 17, 2013 2:46:49 PM
>> Subject: Re: [LLVMdev] make check rebuilds the project?
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> cmake+ninja is a non-option for many developers.
>>
>>
>> By the way, i'm curious what this means. It can be interpreted in a
>> number of ways, but it would be good to know what problems you are
>> specifically saying making it a non-option for many developers
>>
>>
>> Hi Danny,
>>
>>
>> Sorry for being unclear. My fault for posting when I’m in a hurry.
>>
>>
>> I don’t think there’s any unsolvable problems. Well, not technical
>> ones anyway. It’s a question of supporting a broad enough cross
>> section of environments and workflows that devs not only can move to
>> cmake+ninja, but actively want to do so.
>>
>>
>> For me personally, it’s more a question of supported workflows being
>> inconvenient than it being a strict non-starter. That is, it’s a
>> question of which system has fewer inconveniences for me in my daily
>> work. Right now, that’s configure+make. I’m very much looking
>> forward to the day when the balance of that equation changes.
>>
>> Note, I’m explicitly excluding the whole “remove autoconf+make” thing
>> from this. That’s another topic with a whole extra set of issues.
>> Likewise, I’m not going to make any sorts of claims about
>> exhaustiveness here, but rather just relate mostly what I’ve
>> encountered.
>>
>>
>> AFAIK ninja still only has very preliminary support for Windows, so
>> anyone working there can’t use it. Perhaps that’s changed or I’m
>> misinformed? Last I tried it, OSX support was better, but still very
>> much “try it and see if it works for you.” I personally found
>> ninja’s OSX support to be fine, but I didn’t really beat on it to
>> put it through its paces, either. Relatedly, some work environments
>> have varying ability to install and use “non-standard” tools. I’ve
>> been in workplaces before that I’d have had to fight tooth and nail
>> to get something like cmake or ninja available to me on my
>> workstation. Even now when I don’t have that policy sort of problem,
>> it’s still a pain when every time I install a new system (which I do
>> disturbingly often), I need to install additional tools before I can
>> start using llvm. Not a big problem on its own of course, but an
>> inconvenience.
>>
>>
>> cmake+ninja doesn’t support building and running the llvm test suite.
>> That’s pure configure+make still. I rely on configure automatically
>> setting up the test suite to run on whatever compiler I just built.
>>
>>
>> For example, I build Debug+Asserts, Release+Asserts and Release
>> configurations all in the same build directory and rely on the
>> configure+make machinery to create separate build results
>> subdirectories named for those. That is, it’s pretty common for me
>> to do something that boils down to (with lots of steps in between of
>> course), “<src>/configure && make && make check && make
>> ENABLE_OPTIMIZED=1 && make check ENABLE_OPTIMIZED=1 && make
>> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1 && make check
>> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1”. Depending on what I’m
>> doing, I bounce back and forth between them. With cmake, I have to
>> have completely separate configured build directories. I already
>> have 10+ llvm build directories lying around that I regularly use.
>> I’d prefer to keep that number from growing too horribly. When I
>> last tried using cmake+ninja, this is the one that I kept running
>> into and getting annoyed with.
>>
>>
>> Do the cmake scripts support cross-compiling? Last I tried, I had
>> trouble with that. Many moons ago, I had to beat the plain Makefiles
>> into some sort of submission for that. I not infrequently build llvm
>> to run on ARM, for example.
>>
>>
>> Somewhat relatedly, I like to keep my development builds as close as
>> reasonably possible to production builds. There are few things that
>> make my stomach churn more than bugs that only reproduce on the
>> production builders. That’s somewhat off-topic, though, as that’s
>> getting into “what would it take to remove configure+make support”
>> territory.
>>
>>
>> In short, I’m a big fan of ninja and really want to use it. CMake,
>> I’m less thrilled about, but I’ll put up with it if it gets me
>> ninja. Right now though, it’s a close but not quite a fully baked
>> replacement for my uses.
>>
>>
>> Anyways, none of this should be construed to be a slight to the work
>> people have put into cmake+ninja support for llvm/clang. Quite the
>> opposite. I’m excited enough about what’s been done that I’m
>> frustrated it doesn’t fit my use-cases well enough for me to switch.
>>
>>
>> My point with that comment, which Eli correctly and effectively
>> addressed in his reply, is that we can’t assume we can define away
>> problems with the configure+make system by just telling people it’s
>> deprecated and they should use cmake+ninja instead. We’re not there
>> yet.
>
> I'd like to second all of this.
>
> Also, I'll add that I also sometimes build on machines where linking a Debug+Asserts clang takes several minutes, and I like to be able to cut off that process to run make check. However, I discovered that if you 'cd test' before running make check, then there are no dependency problems and it will just run the tests with your current build as-is. So long as that works, I'm fine with keeping the top-level make check dependent on the rest of the build.
>
>

Yeah, check is just: make -C test so that'll work as well.

The point was that if you've made source changes you should really
probably retest what you've changed and the dependency highlights that
in the normal use case. There are many ways around it, but the general
goodness should still be there.

-eric

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
12