1<html lang="en"> 2<head> 3<title>Installing GCC: Testing</title> 4<meta http-equiv="Content-Type" content="text/html"> 5<meta name="description" content="Installing GCC: Testing"> 6<meta name="generator" content="makeinfo 4.12"> 7<link title="Top" rel="top" href="#Top"> 8<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage"> 9<!-- 10Copyright (C) 1988-2015 Free Software Foundation, Inc. 11 12 Permission is granted to copy, distribute and/or modify this document 13under the terms of the GNU Free Documentation License, Version 1.3 or 14any later version published by the Free Software Foundation; with no 15Invariant Sections, the Front-Cover texts being (a) (see below), and 16with the Back-Cover Texts being (b) (see below). A copy of the 17license is included in the section entitled "GNU Free Documentation License". 18 19(a) The FSF's Front-Cover Text is: 20 21 A GNU Manual 22 23(b) The FSF's Back-Cover Text is: 24 25 You have freedom to copy and modify this GNU Manual, like GNU 26 software. Copies published by the Free Software Foundation raise 27 funds for GNU development.--> 28<meta http-equiv="Content-Style-Type" content="text/css"> 29<style type="text/css"><!-- 30 pre.display { font-family:inherit } 31 pre.format { font-family:inherit } 32 pre.smalldisplay { font-family:inherit; font-size:smaller } 33 pre.smallformat { font-family:inherit; font-size:smaller } 34 pre.smallexample { font-size:smaller } 35 pre.smalllisp { font-size:smaller } 36 span.sc { font-variant:small-caps } 37 span.roman { font-family:serif; font-weight:normal; } 38 span.sansserif { font-family:sans-serif; font-weight:normal; } 39--></style> 40</head> 41<body> 42<h1 class="settitle">Installing GCC: Testing</h1> 43<a name="index-Testing-1"></a><a name="index-Installing-GCC_003a-Testing-2"></a><a name="index-Testsuite-3"></a> 44Before you install GCC, we encourage you to run the testsuites and to 45compare your results with results from a similar configuration that have 46been submitted to the 47<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>. 48Some of these archived results are linked from the build status lists 49at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who 50reports a successful build runs the testsuites and submits the results. 51This step is optional and may require you to download additional software, 52but it can give you confidence in your new GCC installation or point out 53problems before you install and start using your new GCC. 54 55 <p>First, you must have <a href="download.html">downloaded the testsuites</a>. 56These are part of the full distribution, but if you downloaded the 57“core” compiler plus any front ends, you must download the testsuites 58separately. 59 60 <p>Second, you must have the testing tools installed. This includes 61<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect; 62the DejaGnu site has links to these. 63 64 <p>If the directories where <samp><span class="command">runtest</span></samp> and <samp><span class="command">expect</span></samp> were 65installed are not in the <samp><span class="env">PATH</span></samp>, you may need to set the following 66environment variables appropriately, as in the following example (which 67assumes that DejaGnu has been installed under <samp><span class="file">/usr/local</span></samp>): 68 69<pre class="smallexample"> TCL_LIBRARY = /usr/local/share/tcl8.0 70 DEJAGNULIBS = /usr/local/share/dejagnu 71</pre> 72 <p>(On systems such as Cygwin, these paths are required to be actual 73paths, not mounts or links; presumably this is due to some lack of 74portability in the DejaGnu code.) 75 76 <p>Finally, you can run the testsuite (which may take a long time): 77<pre class="smallexample"> cd <var>objdir</var>; make -k check 78</pre> 79 <p>This will test various components of GCC, such as compiler 80front ends and runtime libraries. While running the testsuite, DejaGnu 81might emit some harmless messages resembling 82‘<samp><span class="samp">WARNING: Couldn't find the global config file.</span></samp>’ or 83‘<samp><span class="samp">WARNING: Couldn't find tool init file</span></samp>’ that can be ignored. 84 85 <p>If you are testing a cross-compiler, you may want to run the testsuite 86on a simulator as described at <a href="http://gcc.gnu.org/simtest-howto.html">http://gcc.gnu.org/simtest-howto.html</a>. 87 88<h3 class="section"><a name="TOC0"></a>How can you run the testsuite on selected tests?</h3> 89 90<p>In order to run sets of tests selectively, there are targets 91‘<samp><span class="samp">make check-gcc</span></samp>’ and language specific ‘<samp><span class="samp">make check-c</span></samp>’, 92‘<samp><span class="samp">make check-c++</span></samp>’, ‘<samp><span class="samp">make check-fortran</span></samp>’, ‘<samp><span class="samp">make check-java</span></samp>’, 93‘<samp><span class="samp">make check-ada</span></samp>’, ‘<samp><span class="samp">make check-objc</span></samp>’, ‘<samp><span class="samp">make check-obj-c++</span></samp>’, 94‘<samp><span class="samp">make check-lto</span></samp>’ 95in the <samp><span class="file">gcc</span></samp> subdirectory of the object directory. You can also 96just run ‘<samp><span class="samp">make check</span></samp>’ in a subdirectory of the object directory. 97 98 <p>A more selective way to just run all <samp><span class="command">gcc</span></samp> execute tests in the 99testsuite is to use 100 101<pre class="smallexample"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>" 102</pre> 103 <p>Likewise, in order to run only the <samp><span class="command">g++</span></samp> “old-deja” tests in 104the testsuite with filenames matching ‘<samp><span class="samp">9805*</span></samp>’, you would use 105 106<pre class="smallexample"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>" 107</pre> 108 <p>The <samp><span class="file">*.exp</span></samp> files are located in the testsuite directories of the GCC 109source, the most important ones being <samp><span class="file">compile.exp</span></samp>, 110<samp><span class="file">execute.exp</span></samp>, <samp><span class="file">dg.exp</span></samp> and <samp><span class="file">old-deja.exp</span></samp>. 111To get a list of the possible <samp><span class="file">*.exp</span></samp> files, pipe the 112output of ‘<samp><span class="samp">make check</span></samp>’ into a file and look at the 113‘<samp><span class="samp">Running ... .exp</span></samp>’ lines. 114 115<h3 class="section"><a name="TOC1"></a>Passing options and running multiple testsuites</h3> 116 117<p>You can pass multiple options to the testsuite using the 118‘<samp><span class="samp">--target_board</span></samp>’ option of DejaGNU, either passed as part of 119‘<samp><span class="samp">RUNTESTFLAGS</span></samp>’, or directly to <samp><span class="command">runtest</span></samp> if you prefer to 120work outside the makefiles. For example, 121 122<pre class="smallexample"> make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants" 123</pre> 124 <p>will run the standard <samp><span class="command">g++</span></samp> testsuites (“unix” is the target name 125for a standard native testsuite situation), passing 126‘<samp><span class="samp">-O3 -fmerge-constants</span></samp>’ to the compiler on every test, i.e., 127slashes separate options. 128 129 <p>You can run the testsuites multiple times using combinations of options 130with a syntax similar to the brace expansion of popular shells: 131 132<pre class="smallexample"> ..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}" 133</pre> 134 <p>(Note the empty option caused by the trailing comma in the final group.) 135The following will run each testsuite eight times using the ‘<samp><span class="samp">arm-sim</span></samp>’ 136target, as if you had specified all possible combinations yourself: 137 138<pre class="smallexample"> --target_board='arm-sim/-mhard-float/-O1 \ 139 arm-sim/-mhard-float/-O2 \ 140 arm-sim/-mhard-float/-O3 \ 141 arm-sim/-mhard-float \ 142 arm-sim/-msoft-float/-O1 \ 143 arm-sim/-msoft-float/-O2 \ 144 arm-sim/-msoft-float/-O3 \ 145 arm-sim/-msoft-float' 146</pre> 147 <p>They can be combined as many times as you wish, in arbitrary ways. This 148list: 149 150<pre class="smallexample"> ..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}" 151</pre> 152 <p>will generate four combinations, all involving ‘<samp><span class="samp">-Wextra</span></samp>’. 153 154 <p>The disadvantage to this method is that the testsuites are run in serial, 155which is a waste on multiprocessor systems. For users with GNU Make and 156a shell which performs brace expansion, you can run the testsuites in 157parallel by having the shell perform the combinations and <samp><span class="command">make</span></samp> 158do the parallel runs. Instead of using ‘<samp><span class="samp">--target_board</span></samp>’, use a 159special makefile target: 160 161<pre class="smallexample"> make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/... 162</pre> 163 <p>For example, 164 165<pre class="smallexample"> make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} 166</pre> 167 <p>will run three concurrent “make-gcc” testsuites, eventually testing all 168ten combinations as described above. Note that this is currently only 169supported in the <samp><span class="file">gcc</span></samp> subdirectory. (To see how this works, try 170typing <samp><span class="command">echo</span></samp> before the example given here.) 171 172<h3 class="section"><a name="TOC2"></a>Additional testing for Java Class Libraries</h3> 173 174<p>The Java runtime tests can be executed via ‘<samp><span class="samp">make check</span></samp>’ 175in the <samp><var>target</var><span class="file">/libjava/testsuite</span></samp> directory in 176the build tree. 177 178 <p>The <a href="http://sourceware.org/mauve/">Mauve Project</a> provides 179a suite of tests for the Java Class Libraries. This suite can be run 180as part of libgcj testing by placing the Mauve tree within the libjava 181testsuite at <samp><span class="file">libjava/testsuite/libjava.mauve/mauve</span></samp>, or by 182specifying the location of that tree when invoking ‘<samp><span class="samp">make</span></samp>’, as in 183‘<samp><span class="samp">make MAUVEDIR=~/mauve check</span></samp>’. 184 185<h3 class="section"><a name="TOC3"></a>How to interpret test results</h3> 186 187<p>The result of running the testsuite are various <samp><span class="file">*.sum</span></samp> and <samp><span class="file">*.log</span></samp> 188files in the testsuite subdirectories. The <samp><span class="file">*.log</span></samp> files contain a 189detailed log of the compiler invocations and the corresponding 190results, the <samp><span class="file">*.sum</span></samp> files summarize the results. These summaries 191contain status codes for all tests: 192 193 <ul> 194<li>PASS: the test passed as expected 195<li>XPASS: the test unexpectedly passed 196<li>FAIL: the test unexpectedly failed 197<li>XFAIL: the test failed as expected 198<li>UNSUPPORTED: the test is not supported on this platform 199<li>ERROR: the testsuite detected an error 200<li>WARNING: the testsuite detected a possible problem 201</ul> 202 203 <p>It is normal for some tests to report unexpected failures. At the 204current time the testing harness does not allow fine grained control 205over whether or not a test is expected to fail. This problem should 206be fixed in future releases. 207 208<h3 class="section"><a name="TOC4"></a>Submitting test results</h3> 209 210<p>If you want to report the results to the GCC project, use the 211<samp><span class="file">contrib/test_summary</span></samp> shell script. Start it in the <var>objdir</var> with 212 213<pre class="smallexample"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \ 214 -m gcc-testresults@gcc.gnu.org |sh 215</pre> 216 <p>This script uses the <samp><span class="command">Mail</span></samp> program to send the results, so 217make sure it is in your <samp><span class="env">PATH</span></samp>. The file <samp><span class="file">your_commentary.txt</span></samp> is 218prepended to the testsuite summary and should contain any special 219remarks you have on your results or your build environment. Please 220do not edit the testsuite result block or the subject line, as these 221messages may be automatically processed. 222 223 <p><hr /> 224<p><a href="./index.html">Return to the GCC Installation page</a> 225 226<!-- ***Final install*********************************************************** --> 227<!-- ***Binaries**************************************************************** --> 228<!-- ***Specific**************************************************************** --> 229<!-- ***Old documentation****************************************************** --> 230<!-- ***GFDL******************************************************************** --> 231<!-- *************************************************************************** --> 232<!-- Part 6 The End of the Document --> 233</body></html> 234 235