Re: 3.3 Testers!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: 3.3 Testers!

Tyler Hardin
First, let me ask if there's a page that documents the testing process in full. If there is, just give me a link to that and save yourself the time of answering this. (I did Google it, all I found was docs on writing tests for LLVM.)

Sorry if this is way off, but I have no idea what to download and unpack to start testing. I know it's stuff from here: http://llvm.org/pre-releases/3.3/rc1/.

I've built LLVM and Clang before, for my own use. IIRC, I put clang in llvm/tools/ and just ran ./configure and make. So what I need to know is: which of those files do I need, and which llvm subdirectories the go in?

And I do this for both 3.3rc1 and 3.2, then run:
utils/release/test-release.sh --release 3.2 in the llvm 3.2 directory, and
utils/release/test-release.sh --release 3.3 --rc 1 in the llvm 3.3rc1 directory, right?

For clarification, by llvm 3.x directory, I mean the extracted llvm 3.x tarball and all the 3.x sub-tools extracted in their respective subdirectories.

Then what after that?

Sorry for all the extra clarification. This is the first testing I've ever done. I really like LLVM and accompanying tools. I'd like to become more involved in the LLVM project, but I don't think I have the necessary skills yet.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: 3.3 Testers!

Sebastian Dreßler
Hi Tyler,

On 05/09/2013 07:03 AM, Tyler Hardin wrote:

> First, let me ask if there's a page that documents the testing process in
> full. If there is, just give me a link to that and save yourself the time
> of answering this. (I did Google it, all I found was docs on writing tests
> for LLVM.)
>
> Sorry if this is way off, but I have no idea what to download and unpack to
> start testing. I know it's stuff from here:
> http://llvm.org/pre-releases/3.3/rc1/.
>
> I've built LLVM and Clang before, for my own use. IIRC, I put clang in
> llvm/tools/ and just ran ./configure and make. So what I need to know is:
> which of those files do I need, and which llvm subdirectories the go in?
>
> And I do this for both 3.3rc1 and 3.2, then run:
> utils/release/test-release.sh --release 3.2 in the llvm 3.2 directory, and
> utils/release/test-release.sh --release 3.3 --rc 1 in the llvm 3.3rc1
> directory, right?
>
> For clarification, by llvm 3.x directory, I mean the extracted llvm 3.x
> tarball and all the 3.x sub-tools extracted in their respective
> subdirectories.
>
> Then what after that?
>
> Sorry for all the extra clarification. This is the first testing I've ever
> done. I really like LLVM and accompanying tools. I'd like to become more
> involved in the LLVM project, but I don't think I have the necessary skills
> yet.
>

Disclaimer: This is also my very first testing session, but I think that
I've figured it out ... at least the procedure makes sense to me ;)


The full procedure for getting the nightly test results of *a single RC*
is as follows:

1. Get the sources for LLVM + Clang from SVN (i.e. checkout from the
specific tags, e.g. RELEASE_33/rc1)

2. Run the script from utils/release/test-release.sh with appropriate
arguments, e.g. to build 3.3 RC1:

    test-release.sh -release 3.3 -rc 1

Adapt -build-dir and -j as needed

3. Once 2 is finished, download the llvm-test-suite, install LNT (if not
done already) and run it, don't forget to set --cc and --cxx to the
newly compiled clang/clang++ binaries. When this step is finished,
you'll get a directory containing a log file (report.json)


Now for the important: you have to do this for 3.2final *and* 3.3rc1m so
simply repeat steps 1 to 3. Once you have the result.json files for both
releases, you can compare them with the mentioned
findRegressions-{nightly,simple}.py scripts.


I hope that clarifies some things.

Cheers,
Sebastian

--
Mit freundlichen Grüßen / Kind regards

Sebastian Dreßler

Zuse Institute Berlin (ZIB)
Takustraße 7
D-14195 Berlin-Dahlem
Germany

[hidden email]
Phone: +49 30 84185-261

http://www.zib.de/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: 3.3 Testers!

Sebastian Dreßler
Sorry, I forgot to mention, that, if you download the sources yourself,
you should append "-no-checkout" to test-release.sh.


Cheers,
Sebastian

On 05/09/2013 08:39 AM, Sebastian Dreßler wrote:

> Hi Tyler,
>
> On 05/09/2013 07:03 AM, Tyler Hardin wrote:
>> First, let me ask if there's a page that documents the testing process in
>> full. If there is, just give me a link to that and save yourself the time
>> of answering this. (I did Google it, all I found was docs on writing tests
>> for LLVM.)
>>
>> Sorry if this is way off, but I have no idea what to download and unpack to
>> start testing. I know it's stuff from here:
>> http://llvm.org/pre-releases/3.3/rc1/.
>>
>> I've built LLVM and Clang before, for my own use. IIRC, I put clang in
>> llvm/tools/ and just ran ./configure and make. So what I need to know is:
>> which of those files do I need, and which llvm subdirectories the go in?
>>
>> And I do this for both 3.3rc1 and 3.2, then run:
>> utils/release/test-release.sh --release 3.2 in the llvm 3.2 directory, and
>> utils/release/test-release.sh --release 3.3 --rc 1 in the llvm 3.3rc1
>> directory, right?
>>
>> For clarification, by llvm 3.x directory, I mean the extracted llvm 3.x
>> tarball and all the 3.x sub-tools extracted in their respective
>> subdirectories.
>>
>> Then what after that?
>>
>> Sorry for all the extra clarification. This is the first testing I've ever
>> done. I really like LLVM and accompanying tools. I'd like to become more
>> involved in the LLVM project, but I don't think I have the necessary skills
>> yet.
>>
>
> Disclaimer: This is also my very first testing session, but I think that
> I've figured it out ... at least the procedure makes sense to me ;)
>
>
> The full procedure for getting the nightly test results of *a single RC*
> is as follows:
>
> 1. Get the sources for LLVM + Clang from SVN (i.e. checkout from the
> specific tags, e.g. RELEASE_33/rc1)
>
> 2. Run the script from utils/release/test-release.sh with appropriate
> arguments, e.g. to build 3.3 RC1:
>
>     test-release.sh -release 3.3 -rc 1
>
> Adapt -build-dir and -j as needed
>
> 3. Once 2 is finished, download the llvm-test-suite, install LNT (if not
> done already) and run it, don't forget to set --cc and --cxx to the
> newly compiled clang/clang++ binaries. When this step is finished,
> you'll get a directory containing a log file (report.json)
>
>
> Now for the important: you have to do this for 3.2final *and* 3.3rc1m so
> simply repeat steps 1 to 3. Once you have the result.json files for both
> releases, you can compare them with the mentioned
> findRegressions-{nightly,simple}.py scripts.
>
>
> I hope that clarifies some things.
>
> Cheers,
> Sebastian
>


--
Mit freundlichen Grüßen / Kind regards

Sebastian Dreßler

Zuse Institute Berlin (ZIB)
Takustraße 7
D-14195 Berlin-Dahlem
Germany

[hidden email]
Phone: +49 30 84185-261

http://www.zib.de/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: 3.3 Testers!

Sebastian Dreßler
In reply to this post by Sebastian Dreßler
Hi,

On 05/09/2013 08:39 AM, Sebastian Dreßler wrote:
> Now for the important: you have to do this for 3.2final *and* 3.3rc1m so
> simply repeat steps 1 to 3. Once you have the result.json files for both
> releases, you can compare them with the mentioned
> findRegressions-{nightly,simple}.py scripts.

sorry, I'll have to correct this. It's not the result.json the script
needs. I think it's "test.log" but this gives me unexpected output:
findRegressions-nightly.py writes "ERROR!" in particular.

Maybe someone could confirm that it is "test.log" or correct me?


Cheers,
Sebastian

--
Mit freundlichen Grüßen / Kind regards

Sebastian Dreßler

Zuse Institute Berlin (ZIB)
Takustraße 7
D-14195 Berlin-Dahlem
Germany

[hidden email]
Phone: +49 30 84185-261

http://www.zib.de/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev