UPDATING revision 1.208
1$NetBSD: UPDATING,v 1.208 2010/02/04 22:25:21 roy Exp $ 2 3This file (UPDATING) is intended to be a brief reference to recent 4changes that might cause problems in the build process, and a guide for 5what to do if something doesn't work. 6 7For a more detailed description of the recommended way to build NetBSD 8using build.sh, see the BUILDING file. 9 10Note that much of the advice in this UPDATING file was written before 11build.sh existed. Nevertheless, the advice here may be useful for 12working around specific problems with build.sh. 13 14See also: BUILDING, build.sh, Makefile. 15 16Recent changes: 17^^^^^^^^^^^^^^^ 18 1920100204: 20 The termcap database has been removed from the sources, 21 but has not been marked obsolete so it is not removed 22 from the system when upgrading. 23 As such, you will need to remove them from your object 24 and destination directories. 25 2620091101: 27 After updating, it may be necessary to make the 'cleandir' 28 target in src/tools/yacc/ and in src/usr.bin/yacc/ before a 29 'build.sh -u tools' or 'build.sh -u distribution'. Ditto 30 src/tools/lex/ and src/usr.bin/lex/. 31 3220091001: 33 On amd64 you must rebuild tools (to get the new binutils) 34 before building a kernel, or the build fails on cpufunc.S. 35 3620091001: 37 An error will create a ./usr/X11R7/lib/X11/xkb/compiled/xkb 38 symlink, failing the build. Delete the link, and the subdir 39 it is in, and retry your build. 40 4120090718: 42 libc build changed so that strchr() provides the extra entry 43 point for index(). Update build of libc.a (and libc_pic.a) 44 may fail because the archive contains the unwanted index.o. 45 (Similarly for strrchr() and rindex().) 46 4720090709: 48 Native Xorg was upgraded again. Builds will probably fail again 49 without a clean objdir, at least for src/external/mit/xorg. 50 5120090616: 52 Native Xorg was upgraded. Builds will need a clean objdir for 53 src/external/mit/xorg. Upgrading a system from sets will not 54 work properly yet as the /usr/X11R7/lib/X11/xkb/symbols/pc 55 subdirectory has been changed into a file, and this needs to 56 be manually "rm -r"'ed before installing xbase.tgz. 57 5820090501: 59 Several new functions were added to string.h/libc, and this 60 can cause autoconf problems during the tool build for people 61 who fail to clean out their tools objects properly. If you 62 note messages about stpcpy, stpncpy or strnlen accompanying a 63 failure during the tool build, clean out all your tools 64 objects and start again. 65 6620090325: 67 The i386 port was switched to i486 default toolchain. This requires 68 cleaning your src/tools directory and $TOOLDIR and rebuilding them. 69 7020090126: 71 The __posix_fadvise50 system call changed assembly stub type. You 72 need to 'rm -f __posix_fadvise50.*' in the libc build directory 73 to avoid using the old assembly stub. 74 7520090202: 76 pkg_install now depends on the pkgdb cache for automatic conflict 77 detection. It is recommented to rebuild the cache with 78 ``pkg_admin rebuild''. 79 8020090110: 81 time_t and dev_t have been bumped to 64 bit quantities. To upgrade: 82 1. Make sure your kernel has COMPAT_50 in it. Build and install. 83 This is needed even in the MODULAR kernel because there is 84 conditionally compiled code in rtsock.c. 85 2. make sure build.sh completes and the binaries in a chroot work 86 before installing. 87 3. If you don't use build.sh and you build directly to root, and 88 your build breaks in the middle, don't despair. Make sure headers 89 are installed properly, and start building libraries first libc 90 and libutil, install them and then continue building all the 91 libraries in src/lib and src/gnu/lib and install them. Once 92 the new libraries are installed, you can restart the build. 93 4. If you compile packages and you notice link time warnings, 94 rebuild the required packages to update their shared libraries. 95 Any package you rebuild will require rebuilding all the packages 96 that depend on it. 97 5. Next time you run pwd_mkdb with the new binary, the file 98 will be upgraded and it will not be backwards compatible. 99 6. The utmpx/wtmpx files (/var/run/utmpx and /var/log/wtmpx, see 100 lastlogx(5)) have been versioned, and there is a heuristic 101 for utmp. You are better off removing the old files after 102 upgrading. The automated clearing of /var/run during 103 boot, and the automated rotating of files in /var/log by 104 newsyslog(8), may mean that you do not have to remove the 105 files manually. 106 7. The optional accounting file (/var/account/acct, see 107 accton(8)) has not been versioned, and will need to be 108 removed. The automatic rotation of the accounting file by 109 /etc/daily limits the bad consequences of failure to remove 110 the file. 111 8. Application software that writes time_t to binary files on 112 disk will break or need attention. Most notably: if you are 113 using PostgreSQL < 8.4, you need to dump your databases, 114 rebuild PostgreSQL with the new time_t, then restore. 115 11620081219: 117 config(1) has been updated, and one of the files it creates - 118 swapnetbsd.o - has changed format. You need to rebuild config 119 (done automatically by build.sh) and then you need to rerun 120 config on all kernel configuration files before rebuilding those 121 kernels. 122 12320081205: 124 If you build with MKX11=no, you should remove /etc/rc.d/xdm and 125 /etc/rc.d/xfs from DESTDIR because those files were moved to the xetc 126 set and will appear as extra files for MKX11=no update builds. 127 12820081122: 129 On i386, various kernel options(4) in GENERIC including 130 file systems have been disabled and moved into kernel modules. 131 Before trying a new GENERIC kernel, you have to prepare the 132 following files as well as a new GENERIC kernel: 133 134 - build and install kernel modules from src/sys/modules 135 136 - install the latest bootloader, which will load a module 137 for the file system from which the kernel is loaded automatically 138 139 If you have to load your kernel from a file system which is not of 140 the same type as the root file system, you have to load the necessary 141 file system module manually on the boot prompt or in the boot.cfg file. 142 14320080827: 144 If you built and installed a libc from sources between 145 2008/08/20 and 2008/08/26 you got a broken strtouq(3) 146 which results in false errors reported by lint(1). 147 Since this breaks the libc build itself, manual help is 148 needed -- lint must be disabled temporarily, e.g.: 149 $ (cd lib/libc && make MKLINT=no dependall install) 150 15120080813: 152 MKDEBUG build was broken because the .depend files did not know 153 about .go files. You need to remove all .depend files and rebuild. 154 15520080802: 156 A regression in binary compatibility for pthread_mutex_t has 157 been fixed. Unfortunately, the price is breaking compatibility 158 for -current. 159 160 Threaded programs (using libpthread) and C++ programs (using 161 libstdc++) compiled after 20070907 and before 20080802 need to 162 be recompiled. 163 164 One way to find affected pkgsrc packages: 165 166 $ grep REQUIRES=/usr/lib/libpthread /var/db/pkg/*/+BUILD_INFO 167 $ grep REQUIRES=/usr/lib/libstdc++ /var/db/pkg/*/+BUILD_INFO 168 16920080731: 170 WAPBL (metadata journaling support) has been added, but at this 171 time isn't backwards compatible with pre-WAPBL aware kernels 172 and userland (fsck_ffs in particular). Please make sure you 173 don't use a journaled filesystem with an older kernel/userland, 174 especially an uncleanly mounted journaled filesystem. WAPBL 175 also requires the super block to be in the UFS2 format. You 176 can use fsck_ffs -c 4 to update the superblock format. 177 17820080721: 179 Assembler warnings are now fatal if $WARNS>0 and $NOGCCERROR 180 isn't defined. 181 18220080531: 183 The ioctl number of DRVSUSPENDDEV command on /dev/drvctl changed 184 from 125 (conflicted with DRVCTLCOMMAND) to 129. The drvctl(8) 185 utility needs to be rebuilt and reinstalled as older binaries 186 won't work correctly. The following sequence of commands: 187 188 $ (cd sys/sys/ && nbmake-$arch includes) 189 $ (cd sbin/drvctl/ && nbmake-$arch clean) 190 $ (cd sbin/drvctl/ && nbmake-$arch all) 191 192 leaves new drvctl utility in sbin/drvctl build directory. 193 19420080503: 195 The <bsd.lib.mk> variable MKPRIVATELIB was renamed to LIBISPRIVATE. 196 19720080521: 198 For a while, unprivileged UPDATE builds would fail to 199 succeed at the checkflist stage, complaining that 200 ${DESTDIR}/stand/<arch>/ did not exist. A fix for this 201 problem was committed to share/mk/bsd.kmodule.mk, revision 1.9. 202 If you already hit this problem, update the .mk file, 203 remove ${DESTDIR}/stand/<arch>, and re-run the build. 204 20520080303: 206 Linker warnings are now fatal if $WARNS>0. 207 20820080126: 209 The posix_fadvise system call has been changed from an assembly 210 stub, to a c file that calls an assembly stub. You need to 211 'rm -f posix_fadvise.* .depend' in the libc build directory to 212 avoid using the old assembly stub. 213 21420071209: 215 The acpiec(4) driver has been split into two attachments. If you 216 get ACPI errors before the attachment, please update your kernel 217 configuration file appropriately or see GENERIC for more details. 218 21920071115: 220 The it(4) driver has been renamed to itesio(4) and the old port 221 argument specified in the kernel configuration file is not valid 222 anymore. The itesio(4) driver now uses the Super I/O address port 223 rather than the EC address port. Please update your kernel 224 configuration file appropriately or see GENERIC for more details. 225 22620071028: 227 The pccons(4) driver has been removed from the NetBSD/shark port. 228 You need to update any custom kernel configuration file you have 229 to remove any references to pccons (which includes removing the 230 now useless XSERVER option) and replace them with the correct 231 entries for the wscons driver. See the GENERIC configuration file 232 for more details. 233 23420070913: 235 A latent bug in dhclient/dhcpd that caused it to be unable to 236 enumerate interfaces was fixed. The bug began to cause 237 problems after 20070911 when the kernel's SIOCGIFCONF 238 implementation was repaired. From 20070529 to 20070911 racoon 239 could not enumerate interfaces. (These are noted because 240 normal kernel/userspace version matching hygiene is not 241 sufficient to avoid this problem.) Ensure that both kernel 242 and userland are from after 20070913. 243 24420070703: 245 nbinstall has been renamed ${MACHINE_GNU_ARCH}-install. It 246 calls the target-specific strip program, and the logic to pass 247 down STRIP from make has been removed. This requires a 248 re-installation of tools. 249 25020070422: 251 The way OS emulations lookup filenames inside the emulation root 252 has been changed. Rather than modify the pathname (and copy back 253 to userspace) namei() and lookup() directly check the emulation 254 root. One side effect is that absolute symlinks inside the emulated 255 root file system will be relative to that file system - unless they 256 start /../ this is useful when the emulated root is a real install 257 that has such links. 258 This might affect symlinks that have been added to reference outside 259 the emulated root. 260 26120070412: 262 The pckbc driver on sgimips IP32 has been removed. Use macekbc 263 instead. See the GENERIC32_IP3x kernel configuration for an 264 example. 265 26620070319: 267 src/lib/libc/Makefile revision 1.129 broke libc and ld.elf_so 268 on many platforms due to incorrect flags settings. If you 269 updated and built after about 20070315, do "nbmake-$arch 270 cleandir" in src/lib/libc and src/libexec/ld.elf_so to force a 271 rebuild of object files that might have been built 272 incorrectly, and ensure that you have at least 273 src/lib/libc/Makefile 1.130. 274 27520070210: 276 src/sys/sys/{sa.h,savar.h} were removed. 277 find ${OBJDIR} \( -name .depend -o -name '*.d' \) -print \ 278 | xargs egrep -l '/sa.h|/savar.h' | xargs rm 279 will allow dependencies on those files to get get rebuilt 280 28120070209: 282 The threading model was changed when the newlock2 branch 283 was merged to NetBSD-current. If you boot with a new 284 kernel (version 4.99.10), then you also need a new pthread 285 library (/usr/lib/libpthread.so.0.7). If you boot with 286 an old kernel, then you need the old pthread library 287 (/usr/lib/libpthread.so.0.6). Provided you keep the kernel and 288 the pthread library in sync, old threaded applications should 289 continue to work with an old or new kernel. Note that named(8) 290 is the only threaded application in the base system. 291 29220061214: 293 Following the move of string_to_flags() and flags_to_string() 294 from the bin/ls/ sources to libutil, users doing UPDATE builds 295 will need to do a "make cleandir" in 296 tools/mtree/, tools/makefs/, tools/binstall/, tools/pax/, 297 bin/pax/, bin/ls/, usr.sbin/mtree/, usr.sbin/makefs/, 298 usr.bin/xinstall/, libexec/ftpd/, rescue/, as well 299 as the installation images in distrib/ 300 in order to excise stale references to the old stat_flags.h header 301 file in the ls sources -- stat_flags.h has been removed. 302 30320061108: 304 The configure script used in the src/tools/gcc compiler has been 305 changed to indicate that our libc has ssp support built-in and 306 does not depend on -lssp and -lssp-nonshared. You'll need to 307 make clean in src/tools/gcc first to rebuild the compiler. 308 30920061009: 310 The sysctl variables net.inet{,6}.tcp{,6}.newreno are no longer 311 available. Use net.inet{,6}.tcp{,6}.congctl.selected instead. 312 31320060814: 314 The vt, vidcconsole, kbd, and rpckbd drivers on acorn32 have been 315 withdrawn. Use vidcvideo and pckbd instead. See the GENERIC 316 kernel configuration for an example. X servers from the last 317 few years should cope. 318 31920060703: 320 MPACPI is no more. We always configure PCI interrupts using ACPI 321 if we have an ACPI kernel. The option MPACPI_SCANPCI has been renamed 322 to ACPI_SCANPCI. Thanks to work from fvdl. 323 32420060627: 325 socket(2) has changed, and its system call has been versioned. 326 For userlands with the old version of socket(2), make sure that 327 your kernel has 'options COMPAT_30' set, or else 'bad system call' 328 errors will result. 329 330Hints for a more successful build: 331^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 332 Use build.sh, but do not use its "expert mode": 333 This will automatically build the tools in the 334 correct order, and it will keep the tools and the 335 new build products from interfering with the running 336 system. This will allow you to ignore most of the 337 other advice in this file. 338 Build a new kernel first: 339 This makes sure that any new system calls or features 340 expected by the new userland will be present. This 341 helps to avoid critical errors when upgrading. 342 Use object directories: 343 This helps to keep stale object 344 files from polluting the build if a Makefile "forgets" 345 about one. It also makes it easier to clean up after 346 a build. It's also necessary if you want to use the 347 same source tree for multiple machines. 348 To use object directories with build.sh: 349 a) invoke build.sh with the "-M" or "-O" options. 350 To use object directories without using build.sh: 351 a) cd /usr/src ; make cleandir 352 b) Add "OBJMACHINE=yes" to /etc/mk.conf 353 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 354 d) cd /usr/src ; make build 355 Note that running "make obj" in a directory will create 356 in obj.$MACHINE directory. 357 Build to a DESTDIR: 358 This helps to keep old installed files (especially libraries) 359 from interfering with the new build. 360 To build to a DESTDIR with build.sh, use the "-D" option. 361 To build to a DESTDIR without using build.sh, set the DESTDIR 362 environment variable before running make build. It should be 363 set to the pathname of an initially empty directory. 364 Problems: if you do not use build.sh, you might need to 365 update critical utilities without using DESTDIR since 366 nothing is executed from what is installed in DESTDIR. 367 (See critical utils, below.) 368 Build often: 369 This keeps critical utilities current enough to not choke 370 on any other part of the source tree that depends on up to 371 date functionality. If you use build.sh, you should not have 372 this problem. 373 374What to do if things don't work: 375^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 376When things don't work there is usually a few things that commonly 377should be done. 378 1) make includes 379 This should be done automatically by make build. 380 2) cd share/mk && make install 381 Again, automatically done by make build. 382 383Failsafe rebuild of a small part of the tree: 384^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 385To make sure you rebuild something correctly you want to do 386something like the following: 387 1) Make sure the includes and .mk files are up to date. 388 2) Make sure any program used to build the particular 389 utility is up to date. (yacc, lex, etc...) 390 3) cd ...path/to/util... 391 make cleandir 392 rm ...all obj directories... 393 make cleandir # yes, again 394 make obj 395 make depend && make 396 397Failsafe rebuild of the entire tree: 398^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 399If you really want to make sure the source tree is clean and 400ready for a build try the following. Note that sourcing /etc/mk.conf 401(a make(1) Makefile) in this manner is not right, and will not work 402for anyone who uses any make(1) features in /etc/mk.conf. 403 404---cut here--- 405#!/bin/sh 406. /etc/mk.conf 407 408if [ -z $NETBSDSRCDIR ] ; then 409 NETBSDSRCDIR=/usr/src 410fi 411if [ \! -d $NETBSDSRCDIR ] ; then 412 echo Unable to find sources 413 exit 1 414fi 415find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 416 417if [ -z $BSDOBJDIR ] ; then 418 BSDOBJDIR=/usr/obj 419fi 420if [ -d $BSDOBJDIR ] ; then 421 rm -rf $BSDOBJDIR 422fi 423 424cd $NETBSDSRCDIR && make cleandir 425 426---cut here--- 427 428Critical utilities: 429^^^^^^^^^^^^^^^^^^^ 430 usr.bin/compile_et 431 usr.bin/make 432 usr.bin/yacc 433 usr.bin/lex 434 usr.bin/xlint 435 usr.bin/config 436 437Other problems and possible solutions: 438^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 439Symptom:Complaints involving a Makefile. 440Fix: Rebuild usr.bin/make: 441 cd usr.bin/make && make && make install 442 Or, a failsafe method if that doesn't work: 443 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 444 445Fix: Make sure .mk files are up to date. 446 cd share/mk && make install 447 448Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 449Fix: Rebuild usr.bin/config 450 451Symptom: 452Fix: Rebuild usr.bin/yacc 453 454Symptom: 455Fix: Rebuild usr.bin/lex 456 457Symptom: 458Fix: rm /usr/lib/libbfd.a 459 460Symptom:Obsolete intermediate files are used during compilation 461Fix: Try the following sequence of commands in the directory in question. 462 make cleandir; rm `make print-objdir`; make cleandir; make obj 463 (If you built the tree without "make obj" in the past, obsolete files 464 may remain. The command tries to clean everything up) 465 466Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 467 pointer type 468Fix: Rebuild and install usr.bin/menuc 469 470Symptom:mklocale not found during build in share/locale/ctype 471Fix: Build and install usr.bin/mklocale 472 473Symptom:undefined reference to `__assert13' or `__unsetenv13' 474Fix: Rebuild and install lib/libc 475 476Symptom:usr.bin/config fails to build. 477Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 478 479Symptom:undefined reference to `getprogname' or `setprogname' 480Fix: Rebuild and install lib/libc 481 482Symptom:lint does not understand the '-X' option 483Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 484