UPDATING revision 1.207
1$NetBSD: UPDATING,v 1.207 2009/12/26 11:00:46 mbalmer Exp $ 2 3This file (UPDATING) is intended to be a brief reference to recent 4changes that might cause problems in the build process, and a guide for 5what to do if something doesn't work. 6 7For a more detailed description of the recommended way to build NetBSD 8using build.sh, see the BUILDING file. 9 10Note that much of the advice in this UPDATING file was written before 11build.sh existed. Nevertheless, the advice here may be useful for 12working around specific problems with build.sh. 13 14See also: BUILDING, build.sh, Makefile. 15 16Recent changes: 17^^^^^^^^^^^^^^^ 18 1920091101: 20 After updating, it may be necessary to make the 'cleandir' 21 target in src/tools/yacc/ and in src/usr.bin/yacc/ before a 22 'build.sh -u tools' or 'build.sh -u distribution'. Ditto 23 src/tools/lex/ and src/usr.bin/lex/. 24 2520091001: 26 On amd64 you must rebuild tools (to get the new binutils) 27 before building a kernel, or the build fails on cpufunc.S. 28 2920091001: 30 An error will create a ./usr/X11R7/lib/X11/xkb/compiled/xkb 31 symlink, failing the build. Delete the link, and the subdir 32 it is in, and retry your build. 33 3420090718: 35 libc build changed so that strchr() provides the extra entry 36 point for index(). Update build of libc.a (and libc_pic.a) 37 may fail because the archive contains the unwanted index.o. 38 (Similarly for strrchr() and rindex().) 39 4020090709: 41 Native Xorg was upgraded again. Builds will probably fail again 42 without a clean objdir, at least for src/external/mit/xorg. 43 4420090616: 45 Native Xorg was upgraded. Builds will need a clean objdir for 46 src/external/mit/xorg. Upgrading a system from sets will not 47 work properly yet as the /usr/X11R7/lib/X11/xkb/symbols/pc 48 subdirectory has been changed into a file, and this needs to 49 be manually "rm -r"'ed before installing xbase.tgz. 50 5120090501: 52 Several new functions were added to string.h/libc, and this 53 can cause autoconf problems during the tool build for people 54 who fail to clean out their tools objects properly. If you 55 note messages about stpcpy, stpncpy or strnlen accompanying a 56 failure during the tool build, clean out all your tools 57 objects and start again. 58 5920090325: 60 The i386 port was switched to i486 default toolchain. This requires 61 cleaning your src/tools directory and $TOOLDIR and rebuilding them. 62 6320090126: 64 The __posix_fadvise50 system call changed assembly stub type. You 65 need to 'rm -f __posix_fadvise50.*' in the libc build directory 66 to avoid using the old assembly stub. 67 6820090202: 69 pkg_install now depends on the pkgdb cache for automatic conflict 70 detection. It is recommented to rebuild the cache with 71 ``pkg_admin rebuild''. 72 7320090110: 74 time_t and dev_t have been bumped to 64 bit quantities. To upgrade: 75 1. Make sure your kernel has COMPAT_50 in it. Build and install. 76 This is needed even in the MODULAR kernel because there is 77 conditionally compiled code in rtsock.c. 78 2. make sure build.sh completes and the binaries in a chroot work 79 before installing. 80 3. If you don't use build.sh and you build directly to root, and 81 your build breaks in the middle, don't despair. Make sure headers 82 are installed properly, and start building libraries first libc 83 and libutil, install them and then continue building all the 84 libraries in src/lib and src/gnu/lib and install them. Once 85 the new libraries are installed, you can restart the build. 86 4. If you compile packages and you notice link time warnings, 87 rebuild the required packages to update their shared libraries. 88 Any package you rebuild will require rebuilding all the packages 89 that depend on it. 90 5. Next time you run pwd_mkdb with the new binary, the file 91 will be upgraded and it will not be backwards compatible. 92 6. The utmpx/wtmpx files (/var/run/utmpx and /var/log/wtmpx, see 93 lastlogx(5)) have been versioned, and there is a heuristic 94 for utmp. You are better off removing the old files after 95 upgrading. The automated clearing of /var/run during 96 boot, and the automated rotating of files in /var/log by 97 newsyslog(8), may mean that you do not have to remove the 98 files manually. 99 7. The optional accounting file (/var/account/acct, see 100 accton(8)) has not been versioned, and will need to be 101 removed. The automatic rotation of the accounting file by 102 /etc/daily limits the bad consequences of failure to remove 103 the file. 104 8. Application software that writes time_t to binary files on 105 disk will break or need attention. Most notably: if you are 106 using PostgreSQL < 8.4, you need to dump your databases, 107 rebuild PostgreSQL with the new time_t, then restore. 108 10920081219: 110 config(1) has been updated, and one of the files it creates - 111 swapnetbsd.o - has changed format. You need to rebuild config 112 (done automatically by build.sh) and then you need to rerun 113 config on all kernel configuration files before rebuilding those 114 kernels. 115 11620081205: 117 If you build with MKX11=no, you should remove /etc/rc.d/xdm and 118 /etc/rc.d/xfs from DESTDIR because those files were moved to the xetc 119 set and will appear as extra files for MKX11=no update builds. 120 12120081122: 122 On i386, various kernel options(4) in GENERIC including 123 file systems have been disabled and moved into kernel modules. 124 Before trying a new GENERIC kernel, you have to prepare the 125 following files as well as a new GENERIC kernel: 126 127 - build and install kernel modules from src/sys/modules 128 129 - install the latest bootloader, which will load a module 130 for the file system from which the kernel is loaded automatically 131 132 If you have to load your kernel from a file system which is not of 133 the same type as the root file system, you have to load the necessary 134 file system module manually on the boot prompt or in the boot.cfg file. 135 13620080827: 137 If you built and installed a libc from sources between 138 2008/08/20 and 2008/08/26 you got a broken strtouq(3) 139 which results in false errors reported by lint(1). 140 Since this breaks the libc build itself, manual help is 141 needed -- lint must be disabled temporarily, e.g.: 142 $ (cd lib/libc && make MKLINT=no dependall install) 143 14420080813: 145 MKDEBUG build was broken because the .depend files did not know 146 about .go files. You need to remove all .depend files and rebuild. 147 14820080802: 149 A regression in binary compatibility for pthread_mutex_t has 150 been fixed. Unfortunately, the price is breaking compatibility 151 for -current. 152 153 Threaded programs (using libpthread) and C++ programs (using 154 libstdc++) compiled after 20070907 and before 20080802 need to 155 be recompiled. 156 157 One way to find affected pkgsrc packages: 158 159 $ grep REQUIRES=/usr/lib/libpthread /var/db/pkg/*/+BUILD_INFO 160 $ grep REQUIRES=/usr/lib/libstdc++ /var/db/pkg/*/+BUILD_INFO 161 16220080731: 163 WAPBL (metadata journaling support) has been added, but at this 164 time isn't backwards compatible with pre-WAPBL aware kernels 165 and userland (fsck_ffs in particular). Please make sure you 166 don't use a journaled filesystem with an older kernel/userland, 167 especially an uncleanly mounted journaled filesystem. WAPBL 168 also requires the super block to be in the UFS2 format. You 169 can use fsck_ffs -c 4 to update the superblock format. 170 17120080721: 172 Assembler warnings are now fatal if $WARNS>0 and $NOGCCERROR 173 isn't defined. 174 17520080531: 176 The ioctl number of DRVSUSPENDDEV command on /dev/drvctl changed 177 from 125 (conflicted with DRVCTLCOMMAND) to 129. The drvctl(8) 178 utility needs to be rebuilt and reinstalled as older binaries 179 won't work correctly. The following sequence of commands: 180 181 $ (cd sys/sys/ && nbmake-$arch includes) 182 $ (cd sbin/drvctl/ && nbmake-$arch clean) 183 $ (cd sbin/drvctl/ && nbmake-$arch all) 184 185 leaves new drvctl utility in sbin/drvctl build directory. 186 18720080503: 188 The <bsd.lib.mk> variable MKPRIVATELIB was renamed to LIBISPRIVATE. 189 19020080521: 191 For a while, unprivileged UPDATE builds would fail to 192 succeed at the checkflist stage, complaining that 193 ${DESTDIR}/stand/<arch>/ did not exist. A fix for this 194 problem was committed to share/mk/bsd.kmodule.mk, revision 1.9. 195 If you already hit this problem, update the .mk file, 196 remove ${DESTDIR}/stand/<arch>, and re-run the build. 197 19820080303: 199 Linker warnings are now fatal if $WARNS>0. 200 20120080126: 202 The posix_fadvise system call has been changed from an assembly 203 stub, to a c file that calls an assembly stub. You need to 204 'rm -f posix_fadvise.* .depend' in the libc build directory to 205 avoid using the old assembly stub. 206 20720071209: 208 The acpiec(4) driver has been split into two attachments. If you 209 get ACPI errors before the attachment, please update your kernel 210 configuration file appropriately or see GENERIC for more details. 211 21220071115: 213 The it(4) driver has been renamed to itesio(4) and the old port 214 argument specified in the kernel configuration file is not valid 215 anymore. The itesio(4) driver now uses the Super I/O address port 216 rather than the EC address port. Please update your kernel 217 configuration file appropriately or see GENERIC for more details. 218 21920071028: 220 The pccons(4) driver has been removed from the NetBSD/shark port. 221 You need to update any custom kernel configuration file you have 222 to remove any references to pccons (which includes removing the 223 now useless XSERVER option) and replace them with the correct 224 entries for the wscons driver. See the GENERIC configuration file 225 for more details. 226 22720070913: 228 A latent bug in dhclient/dhcpd that caused it to be unable to 229 enumerate interfaces was fixed. The bug began to cause 230 problems after 20070911 when the kernel's SIOCGIFCONF 231 implementation was repaired. From 20070529 to 20070911 racoon 232 could not enumerate interfaces. (These are noted because 233 normal kernel/userspace version matching hygiene is not 234 sufficient to avoid this problem.) Ensure that both kernel 235 and userland are from after 20070913. 236 23720070703: 238 nbinstall has been renamed ${MACHINE_GNU_ARCH}-install. It 239 calls the target-specific strip program, and the logic to pass 240 down STRIP from make has been removed. This requires a 241 re-installation of tools. 242 24320070422: 244 The way OS emulations lookup filenames inside the emulation root 245 has been changed. Rather than modify the pathname (and copy back 246 to userspace) namei() and lookup() directly check the emulation 247 root. One side effect is that absolute symlinks inside the emulated 248 root file system will be relative to that file system - unless they 249 start /../ this is useful when the emulated root is a real install 250 that has such links. 251 This might affect symlinks that have been added to reference outside 252 the emulated root. 253 25420070412: 255 The pckbc driver on sgimips IP32 has been removed. Use macekbc 256 instead. See the GENERIC32_IP3x kernel configuration for an 257 example. 258 25920070319: 260 src/lib/libc/Makefile revision 1.129 broke libc and ld.elf_so 261 on many platforms due to incorrect flags settings. If you 262 updated and built after about 20070315, do "nbmake-$arch 263 cleandir" in src/lib/libc and src/libexec/ld.elf_so to force a 264 rebuild of object files that might have been built 265 incorrectly, and ensure that you have at least 266 src/lib/libc/Makefile 1.130. 267 26820070210: 269 src/sys/sys/{sa.h,savar.h} were removed. 270 find ${OBJDIR} \( -name .depend -o -name '*.d' \) -print \ 271 | xargs egrep -l '/sa.h|/savar.h' | xargs rm 272 will allow dependencies on those files to get get rebuilt 273 27420070209: 275 The threading model was changed when the newlock2 branch 276 was merged to NetBSD-current. If you boot with a new 277 kernel (version 4.99.10), then you also need a new pthread 278 library (/usr/lib/libpthread.so.0.7). If you boot with 279 an old kernel, then you need the old pthread library 280 (/usr/lib/libpthread.so.0.6). Provided you keep the kernel and 281 the pthread library in sync, old threaded applications should 282 continue to work with an old or new kernel. Note that named(8) 283 is the only threaded application in the base system. 284 28520061214: 286 Following the move of string_to_flags() and flags_to_string() 287 from the bin/ls/ sources to libutil, users doing UPDATE builds 288 will need to do a "make cleandir" in 289 tools/mtree/, tools/makefs/, tools/binstall/, tools/pax/, 290 bin/pax/, bin/ls/, usr.sbin/mtree/, usr.sbin/makefs/, 291 usr.bin/xinstall/, libexec/ftpd/, rescue/, as well 292 as the installation images in distrib/ 293 in order to excise stale references to the old stat_flags.h header 294 file in the ls sources -- stat_flags.h has been removed. 295 29620061108: 297 The configure script used in the src/tools/gcc compiler has been 298 changed to indicate that our libc has ssp support built-in and 299 does not depend on -lssp and -lssp-nonshared. You'll need to 300 make clean in src/tools/gcc first to rebuild the compiler. 301 30220061009: 303 The sysctl variables net.inet{,6}.tcp{,6}.newreno are no longer 304 available. Use net.inet{,6}.tcp{,6}.congctl.selected instead. 305 30620060814: 307 The vt, vidcconsole, kbd, and rpckbd drivers on acorn32 have been 308 withdrawn. Use vidcvideo and pckbd instead. See the GENERIC 309 kernel configuration for an example. X servers from the last 310 few years should cope. 311 31220060703: 313 MPACPI is no more. We always configure PCI interrupts using ACPI 314 if we have an ACPI kernel. The option MPACPI_SCANPCI has been renamed 315 to ACPI_SCANPCI. Thanks to work from fvdl. 316 31720060627: 318 socket(2) has changed, and its system call has been versioned. 319 For userlands with the old version of socket(2), make sure that 320 your kernel has 'options COMPAT_30' set, or else 'bad system call' 321 errors will result. 322 323Hints for a more successful build: 324^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 325 Use build.sh, but do not use its "expert mode": 326 This will automatically build the tools in the 327 correct order, and it will keep the tools and the 328 new build products from interfering with the running 329 system. This will allow you to ignore most of the 330 other advice in this file. 331 Build a new kernel first: 332 This makes sure that any new system calls or features 333 expected by the new userland will be present. This 334 helps to avoid critical errors when upgrading. 335 Use object directories: 336 This helps to keep stale object 337 files from polluting the build if a Makefile "forgets" 338 about one. It also makes it easier to clean up after 339 a build. It's also necessary if you want to use the 340 same source tree for multiple machines. 341 To use object directories with build.sh: 342 a) invoke build.sh with the "-M" or "-O" options. 343 To use object directories without using build.sh: 344 a) cd /usr/src ; make cleandir 345 b) Add "OBJMACHINE=yes" to /etc/mk.conf 346 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 347 d) cd /usr/src ; make build 348 Note that running "make obj" in a directory will create 349 in obj.$MACHINE directory. 350 Build to a DESTDIR: 351 This helps to keep old installed files (especially libraries) 352 from interfering with the new build. 353 To build to a DESTDIR with build.sh, use the "-D" option. 354 To build to a DESTDIR without using build.sh, set the DESTDIR 355 environment variable before running make build. It should be 356 set to the pathname of an initially empty directory. 357 Problems: if you do not use build.sh, you might need to 358 update critical utilities without using DESTDIR since 359 nothing is executed from what is installed in DESTDIR. 360 (See critical utils, below.) 361 Build often: 362 This keeps critical utilities current enough to not choke 363 on any other part of the source tree that depends on up to 364 date functionality. If you use build.sh, you should not have 365 this problem. 366 367What to do if things don't work: 368^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 369When things don't work there is usually a few things that commonly 370should be done. 371 1) make includes 372 This should be done automatically by make build. 373 2) cd share/mk && make install 374 Again, automatically done by make build. 375 376Failsafe rebuild of a small part of the tree: 377^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 378To make sure you rebuild something correctly you want to do 379something like the following: 380 1) Make sure the includes and .mk files are up to date. 381 2) Make sure any program used to build the particular 382 utility is up to date. (yacc, lex, etc...) 383 3) cd ...path/to/util... 384 make cleandir 385 rm ...all obj directories... 386 make cleandir # yes, again 387 make obj 388 make depend && make 389 390Failsafe rebuild of the entire tree: 391^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 392If you really want to make sure the source tree is clean and 393ready for a build try the following. Note that sourcing /etc/mk.conf 394(a make(1) Makefile) in this manner is not right, and will not work 395for anyone who uses any make(1) features in /etc/mk.conf. 396 397---cut here--- 398#!/bin/sh 399. /etc/mk.conf 400 401if [ -z $NETBSDSRCDIR ] ; then 402 NETBSDSRCDIR=/usr/src 403fi 404if [ \! -d $NETBSDSRCDIR ] ; then 405 echo Unable to find sources 406 exit 1 407fi 408find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 409 410if [ -z $BSDOBJDIR ] ; then 411 BSDOBJDIR=/usr/obj 412fi 413if [ -d $BSDOBJDIR ] ; then 414 rm -rf $BSDOBJDIR 415fi 416 417cd $NETBSDSRCDIR && make cleandir 418 419---cut here--- 420 421Critical utilities: 422^^^^^^^^^^^^^^^^^^^ 423 usr.bin/compile_et 424 usr.bin/make 425 usr.bin/yacc 426 usr.bin/lex 427 usr.bin/xlint 428 usr.bin/config 429 430Other problems and possible solutions: 431^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 432Symptom:Complaints involving a Makefile. 433Fix: Rebuild usr.bin/make: 434 cd usr.bin/make && make && make install 435 Or, a failsafe method if that doesn't work: 436 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 437 438Fix: Make sure .mk files are up to date. 439 cd share/mk && make install 440 441Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 442Fix: Rebuild usr.bin/config 443 444Symptom: 445Fix: Rebuild usr.bin/yacc 446 447Symptom: 448Fix: Rebuild usr.bin/lex 449 450Symptom: 451Fix: rm /usr/lib/libbfd.a 452 453Symptom:Obsolete intermediate files are used during compilation 454Fix: Try the following sequence of commands in the directory in question. 455 make cleandir; rm `make print-objdir`; make cleandir; make obj 456 (If you built the tree without "make obj" in the past, obsolete files 457 may remain. The command tries to clean everything up) 458 459Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 460 pointer type 461Fix: Rebuild and install usr.bin/menuc 462 463Symptom:mklocale not found during build in share/locale/ctype 464Fix: Build and install usr.bin/mklocale 465 466Symptom:undefined reference to `__assert13' or `__unsetenv13' 467Fix: Rebuild and install lib/libc 468 469Symptom:usr.bin/config fails to build. 470Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 471 472Symptom:undefined reference to `getprogname' or `setprogname' 473Fix: Rebuild and install lib/libc 474 475Symptom:lint does not understand the '-X' option 476Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 477