1240116SmarcelMajor changes between releases Automated Testing Framework 2240116Smarcel=========================================================================== 3240116Smarcel 4240116Smarcel 5262855SjmmvChanges in version 0.20 6262855Sjmmv*********************** 7262855Sjmmv 8262855SjmmvExperimental version released on February 7th, 2014. 9262855Sjmmv 10262855SjmmvThis is the first release without the code for the deprecated tools. If 11262855Sjmmvyou require such code, please fetch a copy of the 0.19 release and extract 12262855Sjmmvthe 'tools' directory for your own consumption. 13262855Sjmmv 14262855Sjmmv* Removed the deprecated tools. This includes atf-config, atf-report, 15262855Sjmmv atf-run and atf-version. 16262855Sjmmv 17272307Srodrigc* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang 18272307Srodrigc that ships with FreeBSD 11.0-CURRENT. 19262855Sjmmv 20272307Srodrigc 21262855SjmmvChanges in version 0.19 22262855Sjmmv*********************** 23262855Sjmmv 24262855SjmmvExperimental version released on February 7th, 2014. 25262855Sjmmv 26262855SjmmvThis is the last release to bundle the code for the deprecated tools. 27262855SjmmvThe next release will drop their code and will stop worrying about 28262855Sjmmvbackwards compatibility between the ATF libraries and what the old tools 29262855Sjmmvmay or may not support. 30262855Sjmmv 31262855SjmmvIf you still require the old tools for some reason, grab a copy of the 32262855Sjmmv'tools' directory now. The code in this directory is standalone and 33262855Sjmmvdoes not depend on any internal details of atf-c++ any longer. 34262855Sjmmv 35262855Sjmmv* Various fixes and improvements to support running as part of the FreeBSD 36262855Sjmmv test suite. 37262855Sjmmv 38262855Sjmmv* Project hosting moved from Google Code (as a subproject of Kyua) to 39262855Sjmmv GitHub (as a first-class project). The main reason for the change is 40262855Sjmmv the suppression of binary downloads in Google Code on Jan 15th, 2014. 41262855Sjmmv See https://github.com/jmmv/atf/ 42262855Sjmmv 43262855Sjmmv* Removed builtin help from atf-sh(1) and atf-check(1) for simplicity 44262855Sjmmv reasons. In other words, their -h option is gone. 45262855Sjmmv 46262855Sjmmv* Moved the code of the deprecated tools into a 'tools' directory and 47262855Sjmmv completely decoupled their code from the internals of atf-c++. The 48262855Sjmmv reason for this is to painlessly allow a third-party to maintain a 49262855Sjmmv copy of these tools after we delete them because upcoming changes to 50262855Sjmmv atf-c++ would break the stale tools. 51262855Sjmmv 52262855Sjmmv 53260029SjmmvChanges in version 0.18 54260029Sjmmv*********************** 55260029Sjmmv 56260029SjmmvExperimental version released on November 16th, 2013. 57260029Sjmmv 58260029Sjmmv* Issue 45: Added require.memory support in atf-run for FreeBSD. 59260029Sjmmv 60260029Sjmmv* Fixed an issue with the handling of cin with libc++. 61260029Sjmmv 62260029Sjmmv* Issue 64: Fixed various mandoc formatting warnings. 63260029Sjmmv 64260029Sjmmv* NetBSD PR bin/48284: Made atf-check flush its progress message to 65260029Sjmmv stdout so that an interrupted test case always shows the last message 66260029Sjmmv being executed. 67260029Sjmmv 68260029Sjmmv* NetBSD PR bin/48285: Fixed atf_check examples in atf-sh-api(3). 69260029Sjmmv 70260029Sjmmv 71260029SjmmvChanges in version 0.17 72260029Sjmmv*********************** 73260029Sjmmv 74260029SjmmvExperimental version released on February 14th, 2013. 75260029Sjmmv 76260029Sjmmv* Added the atf_utils_cat_file, atf_utils_compare_file, 77260029Sjmmv atf_utils_copy_file, atf_utils_create_file, atf_utils_file_exists, 78260029Sjmmv atf_utils_fork, atf_utils_grep_file, atf_utils_grep_string, 79260029Sjmmv atf_utils_readline, atf_utils_redirect and atf_utils_wait utility 80260029Sjmmv functions to atf-c-api. Documented the already-public 81260029Sjmmv atf_utils_free_charpp function. 82260029Sjmmv 83260029Sjmmv* Added the cat_file, compare_file, copy_file, create_file, file_exists, 84260029Sjmmv fork, grep_collection, grep_file, grep_string, redirect and wait 85260029Sjmmv functions to the atf::utils namespace of atf-c++-api. These are 86260029Sjmmv wrappers around the same functions added to the atf-c-api library. 87260029Sjmmv 88260029Sjmmv* Added the ATF_CHECK_MATCH, ATF_CHECK_MATCH_MSG, ATF_REQUIRE_MATCH and 89260029Sjmmv ATF_REQUIRE_MATCH_MSG macros to atf-c to simplify the validation of a 90260029Sjmmv string against a regular expression. 91260029Sjmmv 92260029Sjmmv* Miscellaneous fixes for manpage typos and compilation problems with 93260029Sjmmv clang. 94260029Sjmmv 95260029Sjmmv* Added caching of the results of those configure tests that rely on 96260029Sjmmv executing a test program. This should help crossbuild systems by 97260029Sjmmv providing a mechanism to pre-specify what the results should be. 98260029Sjmmv 99260029Sjmmv* PR bin/45690: Make atf-report convert any non-printable characters to 100260029Sjmmv a plain-text representation (matching their corresponding hexadecimal 101260029Sjmmv entities) in XML output files. This is to prevent the output of test 102260029Sjmmv cases from breaking xsltproc later. 103260029Sjmmv 104260029Sjmmv 105240116SmarcelChanges in version 0.16 106240116Smarcel*********************** 107240116Smarcel 108240116SmarcelExperimental version released on July 10th, 2012. 109240116Smarcel 110240116Smarcel* Added a --enable-tools flag to configure to request the build of the 111240116Smarcel deprecated ATF tools, whose build is now disabled by default. In order 112240116Smarcel to continue running tests, you should migrate to Kyua instead of enabling 113240116Smarcel the build of the deprecated tools. The kyua-atf-compat package provides 114240116Smarcel transitional compatibility versions of atf-run and atf-report built on 115240116Smarcel top of Kyua. 116240116Smarcel 117240116Smarcel* Tweaked the ATF_TEST_CASE macro of atf-c++ so that the compiler can 118240116Smarcel detect defined but unused test cases. 119240116Smarcel 120240116Smarcel* PR bin/45859: Fixed some XSLT bugs that resulted in the tc-time and 121240116Smarcel tp-time XML tags leaking into the generated HTML file. Also improved 122240116Smarcel the CSS file slightly to correct alignment and color issues with the 123240116Smarcel timestamps column. 124240116Smarcel 125240116Smarcel* Optimized atf-c++/macros.hpp so that GNU G++ consumes less memory during 126240116Smarcel compilation with GNU G++. 127240116Smarcel 128240116Smarcel* Flipped the default to building shared libraries for atf-c and atf-c++, 129240116Smarcel and started versioning them. As a side-effect, this removes the 130240116Smarcel --enable-unstable-shared flag from configure that appears to not work any 131240116Smarcel more (under NetBSD). Additionally, some distributions require the use of 132240116Smarcel shared libraries for proper dependency tracking (e.g. Fedora), so it is 133240116Smarcel better if we do the right versioning upstream. 134240116Smarcel 135240116Smarcel* Project hosting moved from an adhoc solution (custom web site and 136240116Smarcel Monotone repository) to Google Code (standard wiki and Git). ATF now 137240116Smarcel lives in a subcomponent of the Kyua project. 138240116Smarcel 139240116Smarcel 140240116SmarcelChanges in version 0.15 141240116Smarcel*********************** 142240116Smarcel 143240116SmarcelExperimental version released on January 16th, 2012. 144240116Smarcel 145240116Smarcel* Respect stdin in atf-check. The previous release silenced stdin for any 146240116Smarcel processes spawned by atf, not only test programs, which caused breakage 147240116Smarcel in tests that pipe data through atf-check. 148240116Smarcel 149240116Smarcel* Performance improvements to atf-sh. 150240116Smarcel 151240116Smarcel* Enabled detection of unused parameters and variables in the code and 152240116Smarcel fixed all warnings. 153240116Smarcel 154240116Smarcel* Changed the behavior of "developer mode". Compiler warnings are now 155240116Smarcel enabled unconditionally regardless of whether we are in developer mode or 156240116Smarcel not; developer mode is now only used to perform strict warning checks and 157240116Smarcel to enable assertions. Additionally, developer mode is now only 158240116Smarcel automatically enabled when building from the repository, not for formal 159240116Smarcel releases. 160240116Smarcel 161240116Smarcel* Added new Autoconf M4 macros (ATF_ARG_WITH, ATF_CHECK_C and 162240116Smarcel ATF_CHECK_CXX) to provide a consistent way of defining a --with-arg flag 163240116Smarcel in configure scripts and detecting the presence of any of the ATF 164240116Smarcel bindings. Note that ATF_CHECK_SH was already introduced in 0.14, but it 165240116Smarcel has now been modified to also honor --with-atf if instantiated. 166240116Smarcel 167240116Smarcel* Added timing support to atf-run / atf-report. 168240116Smarcel 169240116Smarcel* Added support for a 'require.memory' property, to specify the minimum 170240116Smarcel amount of physical memory needed by the test case to yield valid results. 171240116Smarcel 172240116Smarcel* PR bin/45690: Force an ISO-8859-1 encoding in the XML files generated by 173240116Smarcel atf-report so that invalid data in the output of test cases does not 174240116Smarcel mangle our report. 175240116Smarcel 176240116Smarcel 177240116SmarcelChanges in version 0.14 178240116Smarcel*********************** 179240116Smarcel 180240116SmarcelExperimental version released on June 14th, 2011. 181240116Smarcel 182240116Smarcel* Added a pkg-config file for atf-sh and an aclocal file to ease the 183240116Smarcel detection of atf-sh from autoconf scripts. 184240116Smarcel 185240116Smarcel* Made the default test case body defined by atf_sh fail. This is to 186240116Smarcel ensure that test cases are properly defined in test programs and helps 187240116Smarcel in catching typos in the names of the body functions. 188240116Smarcel 189240116Smarcel* PR bin/44882: Made atf-run connect the stdin of test cases to /dev/zero. 190240116Smarcel This provides more consistent results with "normal" execution (in 191240116Smarcel particular, when tests are executed detached from a terminal). 192240116Smarcel 193240116Smarcel* Made atf-run hardcode TZ=UTC for test cases. It used to undefine TZ, but 194240116Smarcel that does not take into account that libc determines the current timezone 195240116Smarcel from a configuration file. 196240116Smarcel 197240116Smarcel* All test programs will now print a warning when they are not run through 198240116Smarcel atf-run(1) stating that this is unsupported and may deliver incorrect 199240116Smarcel results. 200240116Smarcel 201240116Smarcel* Added support for the 'require.files' test-case property. This allows 202240116Smarcel test cases to specify installed files that must be present for the test 203240116Smarcel case to run. 204240116Smarcel 205240116Smarcel 206240116SmarcelChanges in version 0.13 207240116Smarcel*********************** 208240116Smarcel 209240116SmarcelExperimental version released on March 31st, 2011. 210240116Smarcel 211240116SmarcelThis is the first release after the creation of the Kyua project, a more 212240116Smarcelmodular and reliable replacement for ATF. From now on, ATF will change to 213240116Smarcelaccomodate the transition to this new codebase, but ATF will still continue 214240116Smarcelto see development in the short/medium term. Check out the project page at 215240116Smarcelhttp://code.google.com/p/kyua/ for more details. 216240116Smarcel 217240116SmarcelThe changes in this release are: 218240116Smarcel 219240116Smarcel* Added support to run the tests with the Kyua runtime engine (kyua-cli), a 220240116Smarcel new package that aims to replace atf-run and atf-report. The ATF tests 221240116Smarcel can be run with the new system by issuing a 'make installcheck-kyua' from 222240116Smarcel the top-level directory of the project (assuming the 'kyua' binary is 223240116Smarcel available during the configuration stage of ATF). 224240116Smarcel 225240116Smarcel* atf-run and atf-report are now in maintenance mode (but *not* deprecated 226240116Smarcel yet!). Kyua already implements a new, much more reliable runtime engine 227240116Smarcel that provides similar features to these tools. That said, it is not 228240116Smarcel complete yet so all development efforts should go towards it. 229240116Smarcel 230240116Smarcel* If GDB is installed, atf-run dumps the stack trace of crashing test 231240116Smarcel programs in an attempt to aid debugging. Contributed by Antti Kantee. 232240116Smarcel 233240116Smarcel* Reverted default timeout change in previous release and reset its value 234240116Smarcel to 5 minutes. This was causing several issues, specially when running 235240116Smarcel the existing NetBSD test suite in qemu. 236240116Smarcel 237240116Smarcel* Fixed the 'match' output checker in atf-check to properly validate the 238240116Smarcel last line of a file even if it does not have a newline. 239240116Smarcel 240240116Smarcel* Added the ATF_REQUIRE_IN and ATF_REQUIRE_NOT_IN macros to atf-c++ to 241240116Smarcel check for the presence (or lack thereof) of an element in a collection. 242240116Smarcel 243240116Smarcel* PR bin/44176: Fixed a race condition in atf-run that would crash atf-run 244240116Smarcel when the cleanup of a test case triggered asynchronous modifications to 245240116Smarcel its work directory (e.g. killing a daemon process that cleans up a pid 246240116Smarcel file in the work directory). 247240116Smarcel 248240116Smarcel* PR bin/44301: Fixed the sample XSLT file to report bogus test programs 249240116Smarcel instead of just listing them as having 0 test cases. 250240116Smarcel 251240116Smarcel 252240116SmarcelChanges in version 0.12 253240116Smarcel*********************** 254240116Smarcel 255240116SmarcelExperimental version released on November 7th, 2010. 256240116Smarcel 257240116Smarcel* Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as 258240116Smarcel ATF_REQUIRE_THROW but allows checking for the validity of the exception's 259240116Smarcel error message by means of a regular expression. 260240116Smarcel 261240116Smarcel* Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a 262240116Smarcel regular expression match in a string. 263240116Smarcel 264240116Smarcel* Changed the default timeout for test cases from 5 minutes to 30 seconds. 265240116Smarcel 30 seconds is long enough for virtually all tests to complete, and 5 266240116Smarcel minutes is a way too long pause in a test suite where a single test case 267240116Smarcel stalls. 268240116Smarcel 269240116Smarcel* Deprecated the use.fs property. While this seemed like a good idea in 270240116Smarcel the first place to impose more control on what test cases can do, it 271240116Smarcel turns out to be bad. First, use.fs=false prevents bogus test cases 272240116Smarcel from dumping core so after-the-fact debugging is harder. Second, 273240116Smarcel supporting use.fs adds a lot of unnecessary complexity. atf-run will 274240116Smarcel now ignore any value provided to use.fs and will allow test cases to 275240116Smarcel freely access the file system if they wish to. 276240116Smarcel 277240116Smarcel* Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c 278240116Smarcel library. The 'text' module became private in 0.11 but was being used 279240116Smarcel externally to simplify the parsing of configuration variables. 280240116Smarcel 281240116Smarcel* Made atf-run recognize the 'unprivileged-user' configuration variable 282240116Smarcel and automatically drop root privileges when a test case sets 283240116Smarcel require.user=unprivileged. Note that this is, by no means, done for 284240116Smarcel security purposes; this is just for user convenience; tests should, in 285240116Smarcel general, not be blindly run as root in the first place. 286240116Smarcel 287240116Smarcel 288240116SmarcelChanges in version 0.11 289240116Smarcel*********************** 290240116Smarcel 291240116SmarcelExperimental version released on October 20th, 2010. 292240116Smarcel 293240116Smarcel* The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match 294240116Smarcel their counterparts in atf-c. 295240116Smarcel 296240116Smarcel* Clearly separated the modules in atf-c that are supposed to be public 297240116Smarcel from those that are implementation details. The header files for the 298240116Smarcel internal modules are not installed any more. 299240116Smarcel 300240116Smarcel* Made the atf-check tool private. It is only required by atf-sh and being 301240116Smarcel public has the danger of causing confusion. Also, making it private 302240116Smarcel simplifies the public API of atf. 303240116Smarcel 304240116Smarcel* Changed atf-sh to enable per-command error checking (set -e) by default. 305240116Smarcel This catches many cases in which a test case is broken but it is not 306240116Smarcel reported as such because execution continues. 307240116Smarcel 308240116Smarcel* Fixed the XSTL and CSS stylesheets to support expected failures. 309240116Smarcel 310240116Smarcel 311240116SmarcelChanges in version 0.10 312240116Smarcel*********************** 313240116Smarcel 314240116SmarcelExperimental version released on July 2nd, 2010. 315240116Smarcel 316240116SmarcelMiscellaneous features 317240116Smarcel 318240116Smarcel* Added expected failures support to test cases and atf-run. These 319240116Smarcel include, for example, expected clean exits, expected reception of fatal 320240116Smarcel signals, expected timeouts and expected errors in condition checks. 321240116Smarcel These statuses can be used to denote test cases that are known to fail 322240116Smarcel due to a bug in the code they are testing. atf-report reports these 323240116Smarcel tests separately but they do not count towards the failed test cases 324240116Smarcel amount. 325240116Smarcel 326240116Smarcel* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to 327240116Smarcel allow easy checking of call failures that update errno. 328240116Smarcel 329240116Smarcel* Added the has.cleanup meta-data property to test caes that specifies 330240116Smarcel whether the test case has a cleanup routine or not; its value is 331240116Smarcel automatically set. This property is read by atf-run to know if it has to 332240116Smarcel run the cleanup routine; skipping this run for every test case 333240116Smarcel significantly speeds up the run time of test suites. 334240116Smarcel 335240116Smarcel* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to 336240116Smarcel take the expected exception as the first argument and the statement to 337240116Smarcel execute as the second argument. 338240116Smarcel 339240116SmarcelChanges in atf-check 340240116Smarcel 341240116Smarcel* Changed atf-check to support negating the status and output checks by 342240116Smarcel prefixing them with not- and added support to specify multiple checkers 343240116Smarcel for stdout and stderr, not only one. 344240116Smarcel 345240116Smarcel* Added the match output checker to atf-check to look for regular 346240116Smarcel expressions in the stdout and stderr of commands. 347240116Smarcel 348240116Smarcel* Modified the exit checks in atf-check to support checking for the 349240116Smarcel reception of signals. 350240116Smarcel 351240116SmarcelCode simplifications and cleanups 352240116Smarcel 353240116Smarcel* Removed usage messages from test programs to simplify the 354240116Smarcel implementation of every binding by a significant amount. They just now 355240116Smarcel refer the user to the appropriate manual page and do not attempt to wrap 356240116Smarcel lines on terminal boundaries. Test programs are not supposed to be run 357240116Smarcel by users directly so this minor interface regression is not important. 358240116Smarcel 359240116Smarcel* Removed the atf-format internal utility, which is unused after the 360240116Smarcel change documented above. 361240116Smarcel 362240116Smarcel* Removed the atf-cleanup internal utility. It has been unused since the 363240116Smarcel test case isolation was moved to atf-run in 0.8 364240116Smarcel 365240116Smarcel* Splitted the Makefile.am into smaller files for easier maintenance and 366240116Smarcel dropped the use of M4. Only affects users building from the repository 367240116Smarcel sources. 368240116Smarcel 369240116Smarcel* Intermixed tests with the source files in the source tree to provide 370240116Smarcel them more visibility and easier access. The tests directory is gone from 371240116Smarcel the source tree and tests are now suffixed by _test, not prefixed by t_. 372240116Smarcel 373240116Smarcel* Simplifications to the atf-c library: removed the io, tcr and ui 374240116Smarcel modules as they had become unnecessary after all simplifications 375240116Smarcel introduced since the 0.8 release. 376240116Smarcel 377240116Smarcel* Removed the application/X-atf-tcr format introduced in 0.8 release. 378240116Smarcel Tests now print a much simplified format that is easy to parse and nicer 379240116Smarcel to read by end users. As a side effect, the default for test cases is 380240116Smarcel now to print their results to stdout unless otherwise stated by providing 381240116Smarcel the -r flag. 382240116Smarcel 383240116Smarcel* Removed XML distribution documents and replaced them with plain-text 384240116Smarcel documents. They provided little value and introduced a lot of complexity 385240116Smarcel to the build system. 386240116Smarcel 387240116Smarcel* Simplified the output of atf-version by not attempting to print a 388240116Smarcel revision number when building form a distfile. Makes the build system 389240116Smarcel easier to maintain. 390240116Smarcel 391240116Smarcel 392240116SmarcelChanges in version 0.9 393240116Smarcel********************** 394240116Smarcel 395240116SmarcelExperimental version released on June 3rd, 2010. 396240116Smarcel 397240116Smarcel* Added atf-sh, an interpreter to process test programs written using 398240116Smarcel the shell API. This is not really a shell interpreter by itself though: 399240116Smarcel it is just a wrapper around the system shell that eases the loading of 400240116Smarcel the necessary ATF libraries. 401240116Smarcel 402240116Smarcel* Removed atf-compile in favour of atf-sh. 403240116Smarcel 404240116Smarcel* Added the use.fs metadata property to test case, which is used to 405240116Smarcel specify which test cases require file system access. This is to 406240116Smarcel highlight dependencies on external resources more clearly and to speed up 407240116Smarcel the execution of test suites by skipping the creation of many unnecessary 408240116Smarcel work directories. 409240116Smarcel 410240116Smarcel* Fixed test programs to get a sane default value for their source 411240116Smarcel directory. This means that it should not be necessary any more to pass 412240116Smarcel -s when running test programs that do not live in the current directory. 413240116Smarcel 414240116Smarcel* Defining test case headers became optional. This is trivial to achieve 415240116Smarcel in shell-based tests but a bit ugly in C and C++. In C, use the new 416240116Smarcel ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use 417240116Smarcel ATF_TEST_CASE_WITHOUT_HEAD. 418240116Smarcel 419240116Smarcel 420240116SmarcelChanges in version 0.8 421240116Smarcel********************** 422240116Smarcel 423240116SmarcelExperimental version released on May 7th, 2010. 424240116Smarcel 425240116Smarcel* Test programs no longer run several test cases in a row. The execution 426240116Smarcel of a test program now requires a test case name, and that single test 427240116Smarcel case is executed. To execute several test cases, use the atf-run utility 428240116Smarcel as usual. 429240116Smarcel 430240116Smarcel* Test programs no longer fork a subprocess to isolate the execution of 431240116Smarcel test cases. They run the test case code in-process, and a crash of the 432240116Smarcel test case will result in a crash of the test program. This is to ease 433240116Smarcel debugging of faulty test cases. 434240116Smarcel 435240116Smarcel* Test programs no longer isolate their test cases. This means that they 436240116Smarcel will not create temporary directories nor sanitize the environment any 437240116Smarcel more. Yes: running a test case that depends on system state by hand will 438240116Smarcel most likely yield different results depending on where (machine, 439240116Smarcel directory, user environment, etc.) it is run. Isolation has been moved 440240116Smarcel to atf-run. 441240116Smarcel 442240116Smarcel* Test programs no longer print a cryptic format (application/X-atf-tcs) 443240116Smarcel on a special file channel. They can now print whatever they want on the 444240116Smarcel screen. Because test programs can now only run one test case every time, 445240116Smarcel providing controlled output is not necessary any more. 446240116Smarcel 447240116Smarcel* Test programs no longer write their status into a special file 448240116Smarcel descriptor. Instead, they create a file with the results, which is later 449240116Smarcel parsed by atf-run. This changes the semantics of the -r flag. 450240116Smarcel 451240116Smarcel* atf-run has been adjusted to perform the test case isolation. As a 452240116Smarcel result, there is now a single canonical place that implements the 453240116Smarcel isolation of test caes. In previous releases, the three language 454240116Smarcel bindings (C, C++ and shell) had to be kept in sync with each other (read: 455240116Smarcel not a nice thing to do at all). As a side effect of this change, writing 456240116Smarcel bindings for other languages will be much, much easier from now on. 457240116Smarcel 458240116Smarcel* atf-run forks test programs on a test case basis, instead of on a test 459240116Smarcel program basis as it did before. This is to provide the test case 460240116Smarcel isolation that was before implemented by the test programs themselves. 461240116Smarcel 462240116Smarcel* Removed the atf-exec tool. This was used to implement test case 463240116Smarcel isolation in atf-sh, but it is now unnecessary. 464240116Smarcel 465240116Smarcel* It is now optional to define the descr meta-data property. It has been 466240116Smarcel proven to be mostly useless, because test cases often carry a descriptive 467240116Smarcel name of their own. 468240116Smarcel 469240116Smarcel 470240116SmarcelChanges in version 0.7 471240116Smarcel********************** 472240116Smarcel 473240116SmarcelExperimental version released on December 22nd, 2009. 474240116Smarcel 475240116Smarcel* Added build-time checks to atf-c and atf-c++. A binding for atf-sh 476240116Smarcel will come later. 477240116Smarcel 478240116Smarcel* Migrated all build-time checks for header files to proper ATF tests. 479240116Smarcel This demonstrates the use of the new feature described above. 480240116Smarcel 481240116Smarcel* Added an internal API for child process management. 482240116Smarcel 483240116Smarcel* Converted all plain-text distribution documents to a Docbook canonical 484240116Smarcel version, and include pre-generated plain text and HTML copies in the 485240116Smarcel distribution file. 486240116Smarcel 487240116Smarcel* Simplified the contents of the Makefile.am by regenerating it from a 488240116Smarcel canonical Makefile.am.m4 source. As a side-effect, some dependency 489240116Smarcel specifications were fixed. 490240116Smarcel 491240116Smarcel* Migrated all checks from the check target to installcheck, as these 492240116Smarcel require ATF to be installed. 493240116Smarcel 494240116Smarcel* Fixed sign comparison mismatches triggered by the now-enabled 495240116Smarcel -Wsign-compare. 496240116Smarcel 497240116Smarcel* Fixed many memory and object leaks. 498240116Smarcel 499240116Smarcel 500240116SmarcelChanges in version 0.6 501240116Smarcel********************** 502240116Smarcel 503240116SmarcelExperimental version released on January 18th, 2009. 504240116Smarcel 505240116Smarcel* Make atf-exec be able to kill its child process after a certain period 506240116Smarcel of time; this is controlled through the new -t option. 507240116Smarcel 508240116Smarcel* Change atf-sh to use atf-exec's -t option to control the test case's 509240116Smarcel timeouts, instead of doing it internally. Same behavior as before, but 510240116Smarcel noticeably faster. 511240116Smarcel 512240116Smarcel* atf-exec's -g option and atf-killpg are gone due to the previous 513240116Smarcel change. 514240116Smarcel 515240116Smarcel* Added the atf-check(1) tool, a program that executes a given command 516240116Smarcel and checks its exit code against a known value and allows the management 517240116Smarcel of stdout and stderr in multiple ways. This replaces the previous 518240116Smarcel atf_check function in the atf-sh library and exposes this functionality 519240116Smarcel to both atf-c and atf-c++. 520240116Smarcel 521240116Smarcel* Added the ATF_REQUIRE family of macros to the C interface. These help 522240116Smarcel in checking for fatal test conditions. The old ATF_CHECK macros now 523240116Smarcel perform non-fatal checks only. I.e. by using ATF_CHECK, the test case 524240116Smarcel can now continue its execution and the failures will not be reported 525240116Smarcel until the end of the whole run. 526240116Smarcel 527240116Smarcel* Extended the amount of ATF_CHECK_* C macros with new ones to provide 528240116Smarcel more features to the developer. These also have their corresponding 529240116Smarcel counterparts in the ATF_REQUIRE_* family. The new macros (listing the 530240116Smarcel suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and 531240116Smarcel _STREQ_MSG. 532240116Smarcel 533240116Smarcel 534240116SmarcelChanges in version 0.5 535240116Smarcel********************** 536240116Smarcel 537240116SmarcelExperimental version released on May 1st, 2008. 538240116Smarcel 539240116Smarcel* Clauses 3 and 4 of the BSD license used by the project were dropped. 540240116Smarcel All the code is now under a 2-clause BSD license compatible with the GNU 541240116Smarcel General Public License (GPL). 542240116Smarcel 543240116Smarcel* Added a C-only binding so that binary test programs do not need to be 544240116Smarcel tied to C++ at all. This binding is now known as the atf-c library. 545240116Smarcel 546240116Smarcel* Renamed the C++ binding to atf-c++ for consistency with the new atf-c. 547240116Smarcel 548240116Smarcel* Renamed the POSIX shell binding to atf-sh for consistency with the new 549240116Smarcel atf-c and atf-c++. 550240116Smarcel 551240116Smarcel* Added a -w flag to test programs through which it is possible to 552240116Smarcel specify the work directory to be used. This was possible in prior 553240116Smarcel releases by defining the workdir configuration variable (-v workdir=...), 554240116Smarcel but was a conceptually incorrect mechanism. 555240116Smarcel 556240116Smarcel* Test programs now preserve the execution order of test cases when they 557240116Smarcel are given in the command line. Even those mentioned more than once are 558240116Smarcel executed multiple times to comply with the user's requests. 559240116Smarcel 560240116Smarcel 561240116SmarcelChanges in version 0.4 562240116Smarcel********************** 563240116Smarcel 564240116SmarcelExperimental version released on February 4th, 2008. 565240116Smarcel 566240116Smarcel* Added two new manual pages, atf-c++-api and atf-sh-api, describing the 567240116Smarcel C++ and POSIX shell interfaces used to write test programs. 568240116Smarcel 569240116Smarcel* Added a pkg-config file, useful to get the flags to build against the 570240116Smarcel C++ library or to easily detect the presence of ATF. 571240116Smarcel 572240116Smarcel* Added a way for test cases to require a specific architecture and/or 573240116Smarcel machine type through the new 'require.arch' and 'require.machine' 574240116Smarcel meta-data properties, respectively. 575240116Smarcel 576240116Smarcel* Added the 'timeout' property to test cases, useful to set an 577240116Smarcel upper-bound limit for the test's run time and thus prevent global test 578240116Smarcel program stalls due to the test case's misbehavior. 579240116Smarcel 580240116Smarcel* Added the atf-exec(1) internal utility, used to execute a command 581240116Smarcel after changing the process group it belongs to. 582240116Smarcel 583240116Smarcel* Added the atf-killpg(1) internal utility, used to kill process groups. 584240116Smarcel 585240116Smarcel* Multiple portability fixes. Of special interest, full support for 586240116Smarcel SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12 587240116Smarcel C++ compiler. 588240116Smarcel 589240116Smarcel* Fixed a serious bug that prevented atf-run(1) from working at all 590240116Smarcel under Fedora 8 x86_64. Due to the nature of the bug, other platforms 591240116Smarcel were likely affected too. 592240116Smarcel 593240116Smarcel 594240116SmarcelChanges in version 0.3 595240116Smarcel********************** 596240116Smarcel 597240116SmarcelExperimental version released on November 11th, 2007. 598240116Smarcel 599240116Smarcel* Added XML output support to atf-report. This is accompanied by a DTD 600240116Smarcel for the format's structure and sample XSLT/CSS files to post-process this 601240116Smarcel output and convert it to a plain HTML report. 602240116Smarcel 603240116Smarcel* Changed atf-run to add system information to the report it generates. 604240116Smarcel This is currently used by atf-report's XML output only, and is later 605240116Smarcel printed in the HTML reports in a nice and useful summary table. The user 606240116Smarcel and system administrator are allowed to tune this feature by means of 607240116Smarcel hooks. 608240116Smarcel 609240116Smarcel* Removed the test cases' 'isolated' property. This was intended to 610240116Smarcel avoid touching the file system at all when running the related test case, 611240116Smarcel but this has not been true for a long while: some control files are 612240116Smarcel unconditionally required for several purposes, and we cannot easily get 613240116Smarcel rid of them. This way we remove several critical and delicate pieces of 614240116Smarcel code. 615240116Smarcel 616240116Smarcel* Improved atf-report's CSV output format to include information about 617240116Smarcel test programs too. 618240116Smarcel 619240116Smarcel* Fixed the tests that used atf-compile to not require this tool as a 620240116Smarcel helper. Avoids systems without build-time utilities to skip many tests 621240116Smarcel that could otherwise be run. (E.g. NetBSD without the comp.tgz set 622240116Smarcel installed.) 623240116Smarcel 624240116Smarcel* Many general cleanups: Fixed many pieces of code marked as ugly and/or 625240116Smarcel incomplete. 626240116Smarcel 627240116Smarcel 628240116SmarcelChanges in version 0.2 629240116Smarcel********************** 630240116Smarcel 631240116SmarcelExperimental version released on September 20th, 2007. 632240116Smarcel 633240116Smarcel* Test cases now get a known umask on entry. 634240116Smarcel 635240116Smarcel* atf-run now detects many unexpected failures caused by test programs and 636240116Smarcel reports them as bogus tests. atf-report is able to handle these new 637240116Smarcel errors and nicely reports them to the user. 638240116Smarcel 639240116Smarcel* All the data formats read and written by the tools have been 640240116Smarcel documented and cleaned up. These include those grammars that define how 641240116Smarcel the different components communicate with each other as well as the 642240116Smarcel format of files written by the developers and users: the Atffiles and the 643240116Smarcel configuration files. 644240116Smarcel 645240116Smarcel* Added the atf-version tool, a utility that displays information about 646240116Smarcel the currently installed version of ATF. 647240116Smarcel 648240116Smarcel* Test cases can now define an optional cleanup routine to undo their 649240116Smarcel actions regardless of their exit status. 650240116Smarcel 651240116Smarcel* atf-report now summarizes the list of failed (bogus) test programs 652240116Smarcel when using the ticker output format. 653240116Smarcel 654240116Smarcel* Test programs now capture some termination signals and clean up any 655240116Smarcel temporary files before exiting the program. 656240116Smarcel 657240116Smarcel* Multiple bug fixes and improvements all around. 658240116Smarcel 659240116Smarcel 660240116SmarcelChanges in version 0.1 661240116Smarcel********************** 662240116Smarcel 663240116SmarcelExperimental version released on August 20th, 2007. 664240116Smarcel 665240116Smarcel* First public version. This was released coinciding with the end of the 666240116Smarcel Google Summer of Code 2007 program. 667240116Smarcel 668240116Smarcel 669240116Smarcel=========================================================================== 670240116Smarcelvim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2 671