UPDATING revision 1.100
1$NetBSD: UPDATING,v 1.100 2003/12/12 12:36:37 keihan Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120031203: 12 New binutils builds may fail due to old dependencies. 13 It's necessary to "make cleandir" to ensure that 14 the dependencies will be rebuilt correctly. 15 1620031111: 17 A newer mkdep is needed. Error noting that is 18 cc: Ambiguous abbreviation -- 19 2020031008: 21 /usr/include/sys/disklabel_mbr.h was removed. 22 It's necessary to "make cleandir" to ensure that 23 the dependencies will be rebuilt correctly. 24 2520030906: 26 With the addition of siginfo support the old signal trampoline 27 code has been deprecated to COMPAT_16. Make sure that your running 28 kernel has COMPAT_16 enabled before building userland. 29 3020030801: 31 With the new openssl, there is some header and library shuffling. 32 rm -f /usr/include/des.h /usr/include/kerberosIV/* /lib/libdes* \ 33 /usr/lib/libdes* before building. 34 3520030703: 36 Texinfo was updated to 4.6. To avoid failures when trying to 37 build the included texinfo files, do: 38 39 cd src/gnu/usr.bin/texinfo 40 make MKINFO=no dependall install 41 4220030630: 43 Groff was update to 1.19; it's probably necessary to do 44 cd share/mk && make install 45 cd src/gnu/usr.bin/groff 46 make MKMAN=no dependall install 47 (untested). 48 4920030516: 50 Due to bugs in the export handling code, invalid export lines 51 were accepted before and caused the kernel to panic when 52 mountd got restarted because it freed memory that had already 53 been freed. This has been fixed and the kernel checks 54 export addresses very strictly. If you upgrade your kernel, 55 make sure you also upgrade mountd, because if your export 56 file contains lines with an old inet4 address syntax (i.e. 57 a.b.c or a.b or a), they will get rejected by the new kernel. 58 5920030402: 60 The superblock layout for FFS was changed. If you have 1.6 61 fsck binaries, they will signal a fatal superblock mismatch 62 with the first alternate, because they compare too many 63 fields (even ones that aren't useful). If possible, upgrade 64 your fsck_ffs binary before using a new kernel. 65 None of this signals actual filesystem damage. 66 6720030324: 68 sendmail version 8.12.8 was imported. Since sendmail is 69 now setgid to the smmsp group, and runs in "collection" 70 mode for most common activities, there is a new config 71 file called submit.cf that needs to live in /etc/mail. 72 The generic submit.cf sample in /usr/share/sendmail/cf 73 is named netbsd-msp.cf. Upgrading your regular sendmail 74 configuration file is also strongly advised. 75 76 See the section named "MESSAGE SUBMISSION PROGRAM" in 77 the updated /usr/share/sendmail/README file for more 78 information. 79 8020030117: 81 Texinfo was updated to 4.3. To avoid failures when trying to 82 build the included texinfo files, do: 83 84 cd src/gnu/usr.bin/texinfo 85 make MKINFO=no dependall install 86 8720021223: 88 The METALOG format changed slightly, to remove the leading 89 "${DESTDIR}" from path names. 90 This only affects people building with UNPRIVED. 91 For complete safety, remove the DESTDIR entirely and 92 update tools/mtree, before running make build. 93 9420021219: 95 CVS repository layout was changed. See the following for details 96 if you are using (anonymous) cvs to update your tree. 97 98 http://mail-index.NetBSD.org/netbsd-announce/2002/12/19/0000.html 99 10020021219: 101 install(1) had a '-N dbdir' option added, to specify an 102 alternate location to look up users & groups (instead 103 of the host system passwd(5) and group(5) databases). 104 105 The build system was modified to take advantage of 106 this option (using ${NETBSDSRCDIR}/etc), so if you 107 use USETOOLS==no, you may have to rebuild and 108 reinstall usr.bin/xinstall first. 109 11020021130: 111 fparseln(3) moved from libutil to libc. 112 If building to DESTDIR=/, reinstall the includes 113 and rebuild libc: 114 make includes 115 make do-lib-libc 116 If using build.sh, "cd tools/compat && make clean" 117 before rebuilding the tools. 118 11920021126: 120 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 121 including the optional "SYS_INCLUDE=symlinks" support. 122 All header files, including <sys/*.h> are copied into 123 /usr/include. 124 12520021121: 126 The C run-time support files crtbegin.o and crtend.o 127 (and their companions crtbeginS.o and crtendS.o) were 128 split up, with new crti.o and crtn.o files resulting. 129 This means that libtool needs to be rebuilt once the 130 new libraries are installed. The process of rebuilding 131 libtool will cause it to automatically notice the new 132 required files, but it *must* be rebuilt in order to 133 do this. 134 135 An out-of-date libtool will result in shared libraries 136 which lack _init() and _fini() routines, which means that 137 their global contructors/destructors will not be invoked. 138 13920021121: 140 A bug related to how ARM ELF objects were tagged has been 141 corrected. 142 143 NetBSD ARM ELF uses the soft-VFP floating point model by 144 default. However, the assembler lacked support for marking 145 objects as using the VFP floating point format, and the 146 compiler was not properly passing the flag indicating "soft-VFP" 147 to the assembler. 148 149 Unfortunately, this means that the linker will now consider 150 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 151 incompatible with new (properly marked) objects. 152 153 The problem will only manifest itself if you attempt to compile 154 a new program using the fixed toolchain, and link that program 155 against old libraries which do not have the proper "softvfp" 156 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 157 WILL CONTINUE TO WORK PROPERLY. 158 159 The only work-around for the problem is to recompile all of 160 the libraries on the system. The easiest way to do this for 161 system libraries is to install a binary snapshot; they are 162 generally available on releng.NetBSD.org. Any packages you 163 have installed which supply libraries will have to be recompiled 164 if you wish to link new programs against those libraries. 165 166 If you have questions about this matter, please contact 167 port-arm@NetBSD.org. 168 16920021011: 170 Systrace has been improved to support privilege elevation. 171 Updating the kernel requires the userland part of systrace 172 to be rebuilt. 173 17420021010: 175 The config(8) grammar was changed to allow options to register 176 dependencies on attributes, as well as other options. Users 177 must update and reinstall usr.sbin/config before building a new 178 kernel. 179 18020021009: 181 A new attribute dependency syntax was introduced to config(8), 182 which is now used by the SCSI configuration description. Users 183 must update and reinstall usr.sbin/config before building a new 184 kernel. 185 18620021003: 187 Several changes have been made to the autoconfiguration 188 framework. Users must update and reinstall usr.sbin/config 189 before building a new kernel. 190 19120021001: 192 The i386mp branch has been merged. To compile a kernel, users 193 will need to add the option 'cpu* at mainbus?' to their configuration 194 file. Multiprocessor kernels will need 195 ioapic* at mainbus? apid ? 196 options MULTIPROCESSOR 197 options COM_MPLOCK 198 19920020922: 200 MKDYNAMICROOT=yes enabled by default, which means that 201 certain shared libraries are installed into /lib, the shared 202 linker is installed into /libexec, and all programs in /bin 203 and /sbin are dynamically linked. 204 If you do not use "make build", you should ensure that 205 you have the libraries and shared linker in the new locations, 206 with: 207 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 208 20920020917: 210 USE_NEW_TOOLCHAIN has been replaced with: 211 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 212 there is no working in-tree toolchain (hppa, ns32k, sh5, 213 x86_64). 214 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 215 root of an external toolchain (e.g. /usr/local/gnu). This 216 enables the cross-build framework even for TOOLCHAIN_MISSING 217 platforms. 218 21920020906: 220 gehenna-devsw has been merged into the trunk. Need to update and 221 reinstall usr.sbin/config before building the kernel. 222 22320020822: 224 Crunched rescue tools (contents of /bin and /sbin, plus others) 225 are now provided in /rescue. 226 227 To ensure that these are built statically linked (no matter 228 what the setting of LDSTATIC is), use a crunchgen(1) built 229 from sources newer than 20020820 (see the next entry). 230 23120020820: 232 crunchgen(1) changed to ensure that the generated program 233 is statically linked. 234 235 Solution: update and reinstall usr.bin/crunch 236 23720020605: 238 smmsp user/group has been added for sendmail. 239 240 Add the following into /etc/group: 241 242 smmsp:*:17: 243 244 and the following to /etc/master.passwd (via vipw): 245 246 smmsp:*:17:17::0:0:Sendmail Message Submission Program:/nonexistent:/sbin/nologin 247 24820020515: 249 sshd user/group has been added. Need to hand add this in, or sshd 250 will not let you log in (with default, or UsePrivlegeSeparation=yes) 251 252 Add the following into /etc/group: 253 254 sshd:*:16: 255 256 and the following to /etc/master.passwd (via vipw): 257 258 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 259 260 Also /var/chroot/sshd directory needs to be present (digged as part of 261 the build process). 262 26320020426: 264 NBUILDJOBS obsoleted in favor of just using -j. 265 26620020426: 267 etc/postinstall added, which performs various checks for 268 configuration file updates and changes, and can fix most of 269 the problems identified. 270 This should make it much easier to upgrade a system's 271 configuration from earlier systems (as far back as NetBSD 1.5). 272 27320020320: 274 <bsd.lib.mk> needs a new install(1) for its "-a cmd" support. 275 Build and install at usr.bin/xinstall before the build. 276 27720020319: 278 Raw IPv6 socket now makes strict checking for sa_family and sa_len 279 on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold 280 newer than November 2001 when you upgrade the kernel. 281 28220020311: 283 ssh configuration files were moved from /etc to /etc/ssh. Beware 284 if you restart your machine from remote. Note that sshd.conf needs 285 to be changed (due to the use of "/etc" inside). 286 28720020223: 288 Users of the VAX port will need to rebuild and install gas 289 so it deal with the now present register prefix used in all 290 the VAX assembly files. 291 29220020118: 293 ntpd user/group has been added. Need to hand add this in or builds 294 will break as mtree aborts early. 295 296 Add the following into /etc/group: 297 298 ntpd:*:15: 299 300 and the following to /etc/master.passwd (via vipw): 301 302 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 303 30420011207: 305 If you're attempting to build a snapshot on sparc64 and are getting 306 reloc errors from the toolchain groff binary this means your native 307 toolchain has some broken C++ bits. 308 309 To fix: 310 311 Build a new toolchain (i.e. build.sh -t) 312 Use the new toolchain to build and install natively (i.e. /usr/lib) 313 314 gnu/lib/libgcc 315 gnu/lib/libstdc++ 316 317 After this a snapshot will be able to be built. 318 31920011201: 320 In order for a sparc64 build to work you must have a working awk. If 321 you've built and installed a system with the new toolchain up to this 322 point you do not have a working awk as its ability to do floating 323 point is broken. 324 325 To build: 326 327 remake and install gnu/lib/libgcc 328 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 329 against the new libgcc.a) 330 33120011128: 332 Kernel config information was changed to use defflag in 333 the various "files" files. Bug fixes to config(8) are 334 required in order for this to work properly. Make sure 335 to build and install in usr.sbin/config before attempting 336 to build a new kernel. 337 33820011030: 339 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 340 before building libc. 341 34220011029: 343 The new document BUILDING.mdoc (view with nroff | more, or 344 see pre-generated .txt and .html versions) describes the build 345 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 346 build process, are intended in the long run to replace this 347 manual update log. 348 349 Users building a USE_NEW_TOOLCHAIN system should read the 350 BUILDING document for caveats. Generally, BUILDING supersedes 351 UPDATING for these systems, as tool updating is taken care of 352 by the new build system. 353 35420011028: 355 src/etc/Makefile now needs install to be able to handle 356 symlinks that point to nowhere. A bug in install that 357 prevented this was corrected. 358 359 Solution: update and reinstall usr.bin/xinstall 360 Better Solution: Use the new toolchain and it will just work 361 for you. 362 36320011006: 364 /etc/mtree/NetBSD.dist has been updated to take advantage of 365 absolute path support added to mtree(8). Older mtree(8)s don't 366 understand the format. 367 368 Solution: update and reinstall usr.sbin/mtree 369 37020011004: 371 Crunchgen has been updated to work via reach-over makefiles. Updating 372 is suggested before running a snapshot build 373 37420010915: 375 The new "ubcperf" code committed by Chuck Silvers removed 376 a header file, uvm/uvm_vnode.h. There may be stale .depend 377 files that still reference this file. 378 379 Solution: "make cleandir && make dependall" in affected 380 directories. 381 38220010803: 383 grep.info is now built from grep.texi using makeinfo. Since it 384 requires makeinfo v4.0, you need to install new texinfo before 385 building gnu/usr.bin/grep. To install new texinfo, please follow 386 the instruction described in 20010726 entry. 387 38820010803: 389 (i386 only): i386 kernel now uses new instructions like 390 `fxsave' which old gas doesn't understand. To build the 391 kernel successfully, you need to build and install a new toolchain, 392 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 393 from your kernel configuration until you rebuild your userland. 394 See 20011029 above and BUILDING file in this directory for more information. 395 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 396 39720010731: 398 Bootloader update on ELF platforms. DDB in kernels from before 399 this will be unable to read symbol tables provided by newer 400 bootloaders. 401 40220010726: 403 Texinfo was updated to 4.0. To avoid failures when trying to 404 build the included texinfo files, do: 405 406 cd src/gnu/usr.bin/texinfo 407 make MKINFO=no dependall install 408 40920010718: 410 Enabled correct .init/.fini processing in crt0. The way this 411 was done was to change a -I directive to cc(1), which means 412 make(1) will have a stale dependency (it will be checking the 413 timestamp on the wrong "dot_init.h"). 414 415 The symptom you will see is that new programs die with SIGSEGV 416 if you have a stale dependency. 417 418 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 419 before starting your build. 420 42120010628: 422 A construct was added to uvm_page.h that uncovered a bug 423 in lint(1). If you get a warning/error about a non-portable 424 bitfield, update your lint(1) before proceeding. 425 42620010226: 427 Added named user/group to system. Need to hand add this in or builds 428 will break as mtree aborts early. 429 430 To work around add by hand: 431 432 named:*:14: 433 434 to /etc/group and add: 435 436 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 437 438 to master.passwd (use vipw for instance if doing by hand). 439 440 Now a make build should progress. 441 44220010219: 443 get/setprogname() added. Any hostprogs that may use this will need 444 to be bootstrapped manually until the host system is current. 445 446 Known problems: sys/arch/macppc/stand/fixcoff 447 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 448 CFLAGS and rebuilding should work) 449 usr.sbin/mdsetimage - Build a static copy if 450 building a snapshot before fully bootstrapped. 451 45220010204: 453 prepare the code to compile with stricter gcc flags. in 454 particular start eliminating redundant declarations. Yacc 455 needs to be installed before make build. 456 45720010114: 458 introduce .if commands(target) in make(1). You need to 459 bring everything up-to-date first, then without installing 460 anything make and install in usr.bin/make, then proceed 461 with make build. 462 46320010101: 464 bsd.subdir.mk committed 20001230 had a bug which caused 465 afterinstall targets to run too soon; update again. 466 46720001230: 468 New share/mk files needed to support .WAIT in SUBDIR variables. 469 If you get make errors, 470 (cd share/mk; make install) 471 Also, PRINTOBJDIR has changed and is now used more heavily. 472 47320001019: 474 The `ca' device driver has been replaced by `ld'; although the 475 major and minor numbers haven't changed, you should update your /dev 476 directory. 477 47820000929: 479 The following make directives are obsoleted. 480 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 481 By default, RSA is built into libcrypto. IDEA and RC5 will not be 482 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 483 additional library libcrypto_{idea,rc5}. 484 485 486Hints for a more successful build: 487^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 488 Build a new kernel first: 489 This makes sure that any new system calls or features 490 expected by the new userland will be present. This 491 helps to avoid critical errors when upgrading. 492 Use object directories: 493 This helps to keep stale object 494 files from polluting the build if a Makefile "forgets" 495 about one. It also makes it easier to clean up after 496 a build. It's also necessary if you want to use the 497 same source tree for multiple machines. 498 To use object directories: 499 a) cd /usr/src ; make cleandir 500 b) Add "OBJMACHINE=yes" to /etc/mk.conf 501 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 502 d) cd /usr/src ; make build 503 Note that running "make obj" in a directory will create 504 in obj.$MACHINE directory. 505 Build to a DESTDIR: 506 This helps to keep old 507 installed files (especially libraries) from interfering 508 with the new build. 509 To build to a DESTDIR, set the DESTDIR environment 510 variable before running make build. It should be set to 511 the pathname of an initially empty directory. 512 Problems: you might need to update critical utilities 513 without using DESTDIR since nothing is executed 514 from what is installed in DESTDIR. 515 (See critical utils, below) 516 Build often: 517 This keeps critical utilities current enough to not choke 518 on any other part of the source tree that depends on up to 519 date functionality. 520 521What to do if things don't work: 522^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 523When things don't work there is usually a few things that commonly 524should be done. 525 1) make includes 526 This should be done automatically by make build. 527 2) cd share/mk && make install 528 Again, automatically done by make build. 529 530Failsafe rebuild of a small part of the tree: 531^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 532To make sure you rebuild something correctly you want to do 533something like the following: 534 1) Make sure the includes and .mk files are up to date. 535 2) Make sure any program used to build the particular 536 utility is up to date. (yacc, lex, etc...) 537 3) cd ...path/to/util... 538 make cleandir 539 rm ...all obj directories... 540 make cleandir # yes, again 541 make obj 542 make depend && make 543 544Failsafe rebuild of the entire tree: 545^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 546If you really want to make sure the source tree is clean and 547ready for a build try the following. Note that sourcing /etc/mk.conf 548(a make(1) Makefile) in this manner is not right, and will not work 549for anyone who uses any make(1) features in /etc/mk.conf. 550 551---cut here--- 552#!/bin/sh 553. /etc/mk.conf 554 555if [ -z $NETBSDSRCDIR ] ; then 556 NETBSDSRCDIR=/usr/src 557fi 558if [ \! -d $NETBSDSRCDIR ] ; then 559 echo Unable to find sources 560 exit 1 561fi 562find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 563 564if [ -z $BSDOBJDIR ] ; then 565 BSDOBJDIR=/usr/obj 566fi 567if [ -d $BSDOBJDIR ] ; then 568 rm -rf $BSDOBJDIR 569fi 570 571cd $NETBSDSRCDIR && make cleandir 572 573---cut here--- 574 575Critical utilities: 576^^^^^^^^^^^^^^^^^^^ 577 gnu/usr.bin/egcs 578 usr.bin/compile_et 579 usr.bin/make 580 usr.bin/yacc 581 usr.bin/lex 582 usr.bin/xlint 583 usr.sbin/config 584 585Other problems and possible solutions: 586^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 587Symptom:Unreasonable compiler errors. 588Fix: Rebuild gnu/usr.bin/egcs 589 590Symptom:Complaints involving a Makefile. 591Fix: Rebuild usr.bin/make: 592 cd usr.bin/make && make && make install 593 Or, a failsafe method if that doesn't work: 594 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 595 596Fix: Make sure .mk files are up to date. 597 cd share/mk && make install 598 599Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 600Fix: Rebuild usr.sbin/config 601 602Symptom: 603Fix: Rebuild usr.bin/yacc 604 605Symptom: 606Fix: Rebuild usr.bin/lex 607 608Symptom: 609Fix: rm /usr/lib/libbfd.a 610 611Symptom:Obsolete intermediate files are used during compilation 612Fix: Try the following sequence of commands in the directory in question. 613 make cleandir; rm `make print-objdir`; make cleandir; make obj 614 (If you built the tree without "make obj" in the past, obsolete files 615 may remain. The command tries to clean everything up) 616 617Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 618Fix: Rebuild and install usr.bin/menuc 619 620Symptom:mklocale not found during build in share/locale/ctype 621Fix: Build and install usr.bin/mklocale 622 623Symptom:undefined reference to `__assert13' or `__unsetenv13' 624Fix: Rebuild and install lib/libc 625 626Symptom:usr.sbin/config fails to build. 627Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 628 629Symptom:undefined reference to `getprogname' or `setprogname' 630Fix: Rebuild and install lib/libc 631 632Symptom:lint does not understand the '-X' option 633Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 634