[llvm-dev] Using -runs consistently in libFuzzer tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Using -runs consistently in libFuzzer tests

Robin Eklind via llvm-dev
Timeouts when running tests are annoying,
but for many libFuzzer tests lacking -runs argument that’s an expected failure mode.
Should we go through all the tests making sure that -runs is supplied?

Motivation: https://bugs.llvm.org/show_bug.cgi?id=38034
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using -runs consistently in libFuzzer tests

Robin Eklind via llvm-dev
This sounds reasonable to me.  Some tests could probably use a seed as well.

On Thu, Jul 5, 2018 at 3:55 PM George Karpenkov <[hidden email]> wrote:
Timeouts when running tests are annoying,
but for many libFuzzer tests lacking -runs argument that’s an expected failure mode.
Should we go through all the tests making sure that -runs is supplied?

Motivation: https://bugs.llvm.org/show_bug.cgi?id=38034

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using -runs consistently in libFuzzer tests

Robin Eklind via llvm-dev
Not sure. My answer would depend on a particular test. 
for some tests -runs should not be required. 

On Fri, Jul 6, 2018 at 8:57 AM Matt Morehouse <[hidden email]> wrote:
This sounds reasonable to me.  Some tests could probably use a seed as well.

On Thu, Jul 5, 2018 at 3:55 PM George Karpenkov <[hidden email]> wrote:
Timeouts when running tests are annoying,
but for many libFuzzer tests lacking -runs argument that’s an expected failure mode.
Should we go through all the tests making sure that -runs is supplied?

Motivation: https://bugs.llvm.org/show_bug.cgi?id=38034

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using -runs consistently in libFuzzer tests

Robin Eklind via llvm-dev

On Jul 6, 2018, at 9:31 AM, Kostya Serebryany <[hidden email]> wrote:

Not sure. My answer would depend on a particular test. 
for some tests -runs should not be required. 

For which ones?
My point is that failing tests should not hang, and while -runs *should* not be requires when the test passes,
it comes in handy when it does not.


On Fri, Jul 6, 2018 at 8:57 AM Matt Morehouse <[hidden email]> wrote:
This sounds reasonable to me.  Some tests could probably use a seed as well.

On Thu, Jul 5, 2018 at 3:55 PM George Karpenkov <[hidden email]> wrote:
Timeouts when running tests are annoying,
but for many libFuzzer tests lacking -runs argument that’s an expected failure mode.
Should we go through all the tests making sure that -runs is supplied?

Motivation: https://bugs.llvm.org/show_bug.cgi?id=38034


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev