[llvm-dev] Changing behavior of lit.py's -v flag

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Changing behavior of lit.py's -v flag

Jeremy Morse via llvm-dev
Hi,

I think lit.py’s current behavior is somewhat unintuitive in the presence of -v (and not
using -vv), so I’m proposing that it be changed.

Current behavior:
-v: Prints all the substituted lines, not clear which line failed :(
-vv: Prints all the lines up to and including the line that failed :)

Option 1:
-v: Prints only the failing line :)
-vv: Same as today. :)
-vvv (new): Prints all the substituted lines, highlighting* the line that failed :)

Option 2 (less deviation from status quo):
-v: Adopts behavior of -vvv from Option 1. :)
-vv: Same as today. :)

changed slightly since my comment there)

Varun

* highlighting — could be bold and/or underlined or something else. Based on some simple
testing on a Mac terminal, this doesn’t break copy-pasting even if you’re using the default
formatting-aware copy-paste.


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Changing behavior of lit.py's -v flag

Jeremy Morse via llvm-dev
Hi Varun,

I'm definitely in favour of making -v more useful like this.

On Thu, 12 Sep 2019 at 19:31, Varun Gandhi via llvm-dev
<[hidden email]> wrote:
> Option 2 (less deviation from status quo):
> -v: Adopts behavior of -vvv from Option 1. :)
> -vv: Same as today. :)

I think I prefer this one. There's a significant minority of tests
where a the single failing line isn't enough to work out what's
actually happened, if the FileCheck is a separate RUN for example.
Under this scheme no-one even needs to know -vv exists (which is good,
because I've been working with LLVM for many years and didn't until
this thread).

Cheers.

Tim.
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Changing behavior of lit.py's -v flag

Jeremy Morse via llvm-dev
Tim Northover via llvm-dev <[hidden email]> writes:

> Hi Varun,
>
> I'm definitely in favour of making -v more useful like this.
>
> On Thu, 12 Sep 2019 at 19:31, Varun Gandhi via llvm-dev
> <[hidden email]> wrote:
>> Option 2 (less deviation from status quo):
>> -v: Adopts behavior of -vvv from Option 1. :)
>> -vv: Same as today. :)
>
> I think I prefer this one. There's a significant minority of tests
> where a the single failing line isn't enough to work out what's
> actually happened, if the FileCheck is a separate RUN for example.
> Under this scheme no-one even needs to know -vv exists (which is good,
> because I've been working with LLVM for many years and didn't until
> this thread).

+1.

                      -David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Changing behavior of lit.py's -v flag

Jeremy Morse via llvm-dev


On Mon, Sep 16, 2019 at 10:01 AM David Greene via llvm-dev <[hidden email]> wrote:
Tim Northover via llvm-dev <[hidden email]> writes:

> Hi Varun,
>
> I'm definitely in favour of making -v more useful like this.
>
> On Thu, 12 Sep 2019 at 19:31, Varun Gandhi via llvm-dev
> <[hidden email]> wrote:
>> Option 2 (less deviation from status quo):
>> -v: Adopts behavior of -vvv from Option 1. :)
>> -vv: Same as today. :)
>
> I think I prefer this one. There's a significant minority of tests
> where a the single failing line isn't enough to work out what's
> actually happened, if the FileCheck is a separate RUN for example.
> Under this scheme no-one even needs to know -vv exists (which is good,
> because I've been working with LLVM for many years and didn't until
> this thread).

+1.

+1 as well.

Perhaps just alias -v to -vv.  The -v option is pretty much worthless.

Making -v behave like -vv would be huge win for anyone trying to debug a failure on a bot you don't own, and wouldn't require bot owners to change anything.

I'll be happy to review the patch, or write it myself if that helps.

thanks...
don
 

                      -David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Changing behavior of lit.py's -v flag

Jeremy Morse via llvm-dev
In the mean time, how about changing "ninja check" to run lit with -vv
instead of -v by default?
https://reviews.llvm.org/D72389

This gives me output like:


******************** TEST 'LLVM :: CodeGen/AMDGPU/fneg-combines.ll'
FAILED ********************
Script:
--
: 'RUN: at line 1';   /home/jayfoad2/llvm-debug/bin/llc -march=amdgcn
-mcpu=hawaii -start-after=sink -mattr=+flat-for-global
-verify-machineinstrs <
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
| /home/jayfoad2/llvm-debug/bin/FileCheck -enable-var-scope
-check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=SI
-check-prefix=FUNC
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
: 'RUN: at line 2';   /home/jayfoad2/llvm-debug/bin/llc
-enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=hawaii
-mattr=+flat-for-global -start-after=sink -verify-machineinstrs <
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
| /home/jayfoad2/llvm-debug/bin/FileCheck -enable-var-scope
-check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=SI
-check-prefix=FUNC
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
: 'RUN: at line 4';   /home/jayfoad2/llvm-debug/bin/llc -march=amdgcn
-mcpu=fiji -start-after=sink --verify-machineinstrs <
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
| /home/jayfoad2/llvm-debug/bin/FileCheck -enable-var-scope
-check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=VI
-check-prefix=FUNC
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
: 'RUN: at line 5';   /home/jayfoad2/llvm-debug/bin/llc
-enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=fiji
-start-after=sink -verify-machineinstrs <
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
| /home/jayfoad2/llvm-debug/bin/FileCheck -enable-var-scope
-check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=VI
-check-prefix=FUNC
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
--
Exit Code: 1

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /home/jayfoad2/llvm-debug/bin/FileCheck -enable-var-scope
-check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=SI
-check-prefix=FUNC
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll
+ /home/jayfoad2/llvm-debug/bin/llc -march=amdgcn -mcpu=hawaii
-start-after=sink -mattr=+flat-for-global -verify-machineinstrs
/home/jayfoad2/git/llvm-project/llvm/test/CodeGen/AMDGPU/fneg-combines.ll:1330:13:
error: GCN-SAFE: expected string not found in input
; GCN-SAFE: v_fma_f32 [[FMA:v[0-9]+]]
            ^
<stdin>:3835:2: note: scanning from here
 v_xor_b32_e32 v3, 0x80000000, v3
 ^
<stdin>:3876:14: note: possible intended match here
 .globl v_fneg_fmad_f32 ; -- Begin function v_fneg_fmad_f32
             ^

--


... from which I can deduce that the first RUN line failed. I still
find it a bit verbose. I'd prefer if it printed the "Script:" but only
up to the first line that failed, so that I could easily copy'n'paste
the last RUN line it printed to reproduce the failure (assuming it was
independent of the previous RUN lines).

Jay.

On Thu, 19 Sep 2019 at 03:38, Don Hinton via llvm-dev
<[hidden email]> wrote:

>
>
>
> On Mon, Sep 16, 2019 at 10:01 AM David Greene via llvm-dev <[hidden email]> wrote:
>>
>> Tim Northover via llvm-dev <[hidden email]> writes:
>>
>> > Hi Varun,
>> >
>> > I'm definitely in favour of making -v more useful like this.
>> >
>> > On Thu, 12 Sep 2019 at 19:31, Varun Gandhi via llvm-dev
>> > <[hidden email]> wrote:
>> >> Option 2 (less deviation from status quo):
>> >> -v: Adopts behavior of -vvv from Option 1. :)
>> >> -vv: Same as today. :)
>> >
>> > I think I prefer this one. There's a significant minority of tests
>> > where a the single failing line isn't enough to work out what's
>> > actually happened, if the FileCheck is a separate RUN for example.
>> > Under this scheme no-one even needs to know -vv exists (which is good,
>> > because I've been working with LLVM for many years and didn't until
>> > this thread).
>>
>> +1.
>
>
> +1 as well.
>
> Perhaps just alias -v to -vv.  The -v option is pretty much worthless.
>
> Making -v behave like -vv would be huge win for anyone trying to debug a failure on a bot you don't own, and wouldn't require bot owners to change anything.
>
> I'll be happy to review the patch, or write it myself if that helps.
>
> thanks...
> don
>
>>
>>
>>                       -David
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev