UPDATING revision 1.102
1$NetBSD: UPDATING,v 1.102 2004/01/09 19:36:28 dbj Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120040109: 12 Compatibility for old ffs superblock layouts has been 13 added, and the restrictive fsck checks have been reenabled 14 when using those layouts. If you have been using -current 15 since 20030402, you may find that fsck again signals fatal 16 superblock mismatches. To work around, you can use 17 fsck -b 32 to restore an alternate superblock. 18 1920031203: 20 New binutils builds may fail due to old dependencies. 21 It's necessary to "make cleandir" to ensure that 22 the dependencies will be rebuilt correctly. 23 2420031111: 25 A newer mkdep is needed. Error noting that is 26 cc: Ambiguous abbreviation -- 27 2820031008: 29 /usr/include/sys/disklabel_mbr.h was removed. 30 It's necessary to "make cleandir" to ensure that 31 the dependencies will be rebuilt correctly. 32 3320030906: 34 With the addition of siginfo support the old signal trampoline 35 code has been deprecated to COMPAT_16. Make sure that your running 36 kernel has COMPAT_16 enabled before building userland. 37 3820030801: 39 With the new openssl, there is some header and library shuffling. 40 rm -f /usr/include/des.h /usr/include/kerberosIV/* /lib/libdes* \ 41 /usr/lib/libdes* before building. 42 4320030703: 44 Texinfo was updated to 4.6. To avoid failures when trying to 45 build the included texinfo files, do: 46 47 cd src/gnu/usr.bin/texinfo 48 make MKINFO=no dependall install 49 5020030630: 51 Groff was updated to 1.19; it's probably necessary to do 52 cd share/mk && make install 53 cd src/gnu/usr.bin/groff 54 make MKMAN=no dependall install 55 (untested). 56 5720030516: 58 Due to bugs in the export handling code, invalid export lines 59 were accepted before and caused the kernel to panic when 60 mountd got restarted because it freed memory that had already 61 been freed. This has been fixed and the kernel checks 62 export addresses very strictly. If you upgrade your kernel, 63 make sure you also upgrade mountd, because if your export 64 file contains lines with an old inet4 address syntax (i.e. 65 a.b.c or a.b or a), they will get rejected by the new kernel. 66 6720030402: 68 The superblock layout for FFS was changed. If you have 1.6 69 fsck binaries, they will signal a fatal superblock mismatch 70 with the first alternate, because they compare too many 71 fields (even ones that aren't useful). If possible, upgrade 72 your fsck_ffs binary before using a new kernel. 73 None of this signals actual filesystem damage. 74 7520030324: 76 sendmail version 8.12.8 was imported. Since sendmail is 77 now setgid to the smmsp group, and runs in "collection" 78 mode for most common activities, there is a new config 79 file called submit.cf that needs to live in /etc/mail. 80 The generic submit.cf sample in /usr/share/sendmail/cf 81 is named netbsd-msp.cf. Upgrading your regular sendmail 82 configuration file is also strongly advised. 83 84 See the section named "MESSAGE SUBMISSION PROGRAM" in 85 the updated /usr/share/sendmail/README file for more 86 information. 87 8820030117: 89 Texinfo was updated to 4.3. To avoid failures when trying to 90 build the included texinfo files, do: 91 92 cd src/gnu/usr.bin/texinfo 93 make MKINFO=no dependall install 94 9520021223: 96 The METALOG format changed slightly, to remove the leading 97 "${DESTDIR}" from path names. 98 This only affects people building with UNPRIVED. 99 For complete safety, remove the DESTDIR entirely and 100 update tools/mtree, before running make build. 101 10220021219: 103 CVS repository layout was changed. See the following for details 104 if you are using (anonymous) cvs to update your tree. 105 106 http://mail-index.NetBSD.org/netbsd-announce/2002/12/19/0000.html 107 10820021219: 109 install(1) had a '-N dbdir' option added, to specify an 110 alternate location to look up users & groups (instead 111 of the host system passwd(5) and group(5) databases). 112 113 The build system was modified to take advantage of 114 this option (using ${NETBSDSRCDIR}/etc), so if you 115 use USETOOLS==no, you may have to rebuild and 116 reinstall usr.bin/xinstall first. 117 11820021130: 119 fparseln(3) moved from libutil to libc. 120 If building to DESTDIR=/, reinstall the includes 121 and rebuild libc: 122 make includes 123 make do-lib-libc 124 If using build.sh, "cd tools/compat && make clean" 125 before rebuilding the tools. 126 12720021126: 128 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 129 including the optional "SYS_INCLUDE=symlinks" support. 130 All header files, including <sys/*.h> are copied into 131 /usr/include. 132 13320021121: 134 The C run-time support files crtbegin.o and crtend.o 135 (and their companions crtbeginS.o and crtendS.o) were 136 split up, with new crti.o and crtn.o files resulting. 137 This means that libtool needs to be rebuilt once the 138 new libraries are installed. The process of rebuilding 139 libtool will cause it to automatically notice the new 140 required files, but it *must* be rebuilt in order to 141 do this. 142 143 An out-of-date libtool will result in shared libraries 144 which lack _init() and _fini() routines, which means that 145 their global contructors/destructors will not be invoked. 146 14720021121: 148 A bug related to how ARM ELF objects were tagged has been 149 corrected. 150 151 NetBSD ARM ELF uses the soft-VFP floating point model by 152 default. However, the assembler lacked support for marking 153 objects as using the VFP floating point format, and the 154 compiler was not properly passing the flag indicating "soft-VFP" 155 to the assembler. 156 157 Unfortunately, this means that the linker will now consider 158 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 159 incompatible with new (properly marked) objects. 160 161 The problem will only manifest itself if you attempt to compile 162 a new program using the fixed toolchain, and link that program 163 against old libraries which do not have the proper "softvfp" 164 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 165 WILL CONTINUE TO WORK PROPERLY. 166 167 The only work-around for the problem is to recompile all of 168 the libraries on the system. The easiest way to do this for 169 system libraries is to install a binary snapshot; they are 170 generally available on releng.NetBSD.org. Any packages you 171 have installed which supply libraries will have to be recompiled 172 if you wish to link new programs against those libraries. 173 174 If you have questions about this matter, please contact 175 port-arm@NetBSD.org. 176 17720021011: 178 Systrace has been improved to support privilege elevation. 179 Updating the kernel requires the userland part of systrace 180 to be rebuilt. 181 18220021010: 183 The config(8) grammar was changed to allow options to register 184 dependencies on attributes, as well as other options. Users 185 must update and reinstall usr.sbin/config before building a new 186 kernel. 187 18820021009: 189 A new attribute dependency syntax was introduced to config(8), 190 which is now used by the SCSI configuration description. Users 191 must update and reinstall usr.sbin/config before building a new 192 kernel. 193 19420021003: 195 Several changes have been made to the autoconfiguration 196 framework. Users must update and reinstall usr.sbin/config 197 before building a new kernel. 198 19920021001: 200 The i386mp branch has been merged. To compile a kernel, users 201 will need to add the option 'cpu* at mainbus?' to their configuration 202 file. Multiprocessor kernels will need 203 ioapic* at mainbus? apid ? 204 options MULTIPROCESSOR 205 options COM_MPLOCK 206 20720020922: 208 MKDYNAMICROOT=yes enabled by default, which means that 209 certain shared libraries are installed into /lib, the shared 210 linker is installed into /libexec, and all programs in /bin 211 and /sbin are dynamically linked. 212 If you do not use "make build", you should ensure that 213 you have the libraries and shared linker in the new locations, 214 with: 215 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 216 21720020917: 218 USE_NEW_TOOLCHAIN has been replaced with: 219 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 220 there is no working in-tree toolchain (hppa, ns32k, sh5, 221 x86_64). 222 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 223 root of an external toolchain (e.g. /usr/local/gnu). This 224 enables the cross-build framework even for TOOLCHAIN_MISSING 225 platforms. 226 22720020906: 228 gehenna-devsw has been merged into the trunk. Need to update and 229 reinstall usr.sbin/config before building the kernel. 230 23120020822: 232 Crunched rescue tools (contents of /bin and /sbin, plus others) 233 are now provided in /rescue. 234 235 To ensure that these are built statically linked (no matter 236 what the setting of LDSTATIC is), use a crunchgen(1) built 237 from sources newer than 20020820 (see the next entry). 238 23920020820: 240 crunchgen(1) changed to ensure that the generated program 241 is statically linked. 242 243 Solution: update and reinstall usr.bin/crunch 244 24520020605: 246 smmsp user/group has been added for sendmail. 247 248 Add the following into /etc/group: 249 250 smmsp:*:17: 251 252 and the following to /etc/master.passwd (via vipw): 253 254 smmsp:*:17:17::0:0:Sendmail Message Submission Program:/nonexistent:/sbin/nologin 255 25620020515: 257 sshd user/group has been added. Need to hand add this in, or sshd 258 will not let you log in (with default, or UsePrivlegeSeparation=yes) 259 260 Add the following into /etc/group: 261 262 sshd:*:16: 263 264 and the following to /etc/master.passwd (via vipw): 265 266 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 267 268 Also /var/chroot/sshd directory needs to be present (digged as part of 269 the build process). 270 27120020426: 272 NBUILDJOBS obsoleted in favor of just using -j. 273 27420020426: 275 etc/postinstall added, which performs various checks for 276 configuration file updates and changes, and can fix most of 277 the problems identified. 278 This should make it much easier to upgrade a system's 279 configuration from earlier systems (as far back as NetBSD 1.5). 280 28120020320: 282 <bsd.lib.mk> needs a new install(1) for its "-a cmd" support. 283 Build and install at usr.bin/xinstall before the build. 284 28520020319: 286 Raw IPv6 socket now makes strict checking for sa_family and sa_len 287 on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold 288 newer than November 2001 when you upgrade the kernel. 289 29020020311: 291 ssh configuration files were moved from /etc to /etc/ssh. Beware 292 if you restart your machine from remote. Note that sshd.conf needs 293 to be changed (due to the use of "/etc" inside). 294 29520020223: 296 Users of the VAX port will need to rebuild and install gas 297 so it deal with the now present register prefix used in all 298 the VAX assembly files. 299 30020020118: 301 ntpd user/group has been added. Need to hand add this in or builds 302 will break as mtree aborts early. 303 304 Add the following into /etc/group: 305 306 ntpd:*:15: 307 308 and the following to /etc/master.passwd (via vipw): 309 310 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 311 31220011207: 313 If you're attempting to build a snapshot on sparc64 and are getting 314 reloc errors from the toolchain groff binary this means your native 315 toolchain has some broken C++ bits. 316 317 To fix: 318 319 Build a new toolchain (i.e. build.sh -t) 320 Use the new toolchain to build and install natively (i.e. /usr/lib) 321 322 gnu/lib/libgcc 323 gnu/lib/libstdc++ 324 325 After this a snapshot will be able to be built. 326 32720011201: 328 In order for a sparc64 build to work you must have a working awk. If 329 you've built and installed a system with the new toolchain up to this 330 point you do not have a working awk as its ability to do floating 331 point is broken. 332 333 To build: 334 335 remake and install gnu/lib/libgcc 336 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 337 against the new libgcc.a) 338 33920011128: 340 Kernel config information was changed to use defflag in 341 the various "files" files. Bug fixes to config(8) are 342 required in order for this to work properly. Make sure 343 to build and install in usr.sbin/config before attempting 344 to build a new kernel. 345 34620011030: 347 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 348 before building libc. 349 35020011029: 351 The new document BUILDING.mdoc (view with nroff | more, or 352 see pre-generated .txt and .html versions) describes the build 353 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 354 build process, are intended in the long run to replace this 355 manual update log. 356 357 Users building a USE_NEW_TOOLCHAIN system should read the 358 BUILDING document for caveats. Generally, BUILDING supersedes 359 UPDATING for these systems, as tool updating is taken care of 360 by the new build system. 361 36220011028: 363 src/etc/Makefile now needs install to be able to handle 364 symlinks that point to nowhere. A bug in install that 365 prevented this was corrected. 366 367 Solution: update and reinstall usr.bin/xinstall 368 Better Solution: Use the new toolchain and it will just work 369 for you. 370 37120011006: 372 /etc/mtree/NetBSD.dist has been updated to take advantage of 373 absolute path support added to mtree(8). Older mtree(8)s don't 374 understand the format. 375 376 Solution: update and reinstall usr.sbin/mtree 377 37820011004: 379 Crunchgen has been updated to work via reach-over makefiles. Updating 380 is suggested before running a snapshot build 381 38220010915: 383 The new "ubcperf" code committed by Chuck Silvers removed 384 a header file, uvm/uvm_vnode.h. There may be stale .depend 385 files that still reference this file. 386 387 Solution: "make cleandir && make dependall" in affected 388 directories. 389 39020010803: 391 grep.info is now built from grep.texi using makeinfo. Since it 392 requires makeinfo v4.0, you need to install new texinfo before 393 building gnu/usr.bin/grep. To install new texinfo, please follow 394 the instruction described in 20010726 entry. 395 39620010803: 397 (i386 only): i386 kernel now uses new instructions like 398 `fxsave' which old gas doesn't understand. To build the 399 kernel successfully, you need to build and install a new toolchain, 400 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 401 from your kernel configuration until you rebuild your userland. 402 See 20011029 above and BUILDING file in this directory for more information. 403 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 404 40520010731: 406 Bootloader update on ELF platforms. DDB in kernels from before 407 this will be unable to read symbol tables provided by newer 408 bootloaders. 409 41020010726: 411 Texinfo was updated to 4.0. To avoid failures when trying to 412 build the included texinfo files, do: 413 414 cd src/gnu/usr.bin/texinfo 415 make MKINFO=no dependall install 416 41720010718: 418 Enabled correct .init/.fini processing in crt0. The way this 419 was done was to change a -I directive to cc(1), which means 420 make(1) will have a stale dependency (it will be checking the 421 timestamp on the wrong "dot_init.h"). 422 423 The symptom you will see is that new programs die with SIGSEGV 424 if you have a stale dependency. 425 426 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 427 before starting your build. 428 42920010628: 430 A construct was added to uvm_page.h that uncovered a bug 431 in lint(1). If you get a warning/error about a non-portable 432 bitfield, update your lint(1) before proceeding. 433 43420010226: 435 Added named user/group to system. Need to hand add this in or builds 436 will break as mtree aborts early. 437 438 To work around add by hand: 439 440 named:*:14: 441 442 to /etc/group and add: 443 444 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 445 446 to master.passwd (use vipw for instance if doing by hand). 447 448 Now a make build should progress. 449 45020010219: 451 get/setprogname() added. Any hostprogs that may use this will need 452 to be bootstrapped manually until the host system is current. 453 454 Known problems: sys/arch/macppc/stand/fixcoff 455 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 456 CFLAGS and rebuilding should work) 457 usr.sbin/mdsetimage - Build a static copy if 458 building a snapshot before fully bootstrapped. 459 46020010204: 461 prepare the code to compile with stricter gcc flags. in 462 particular start eliminating redundant declarations. Yacc 463 needs to be installed before make build. 464 46520010114: 466 introduce .if commands(target) in make(1). You need to 467 bring everything up-to-date first, then without installing 468 anything make and install in usr.bin/make, then proceed 469 with make build. 470 47120010101: 472 bsd.subdir.mk committed 20001230 had a bug which caused 473 afterinstall targets to run too soon; update again. 474 47520001230: 476 New share/mk files needed to support .WAIT in SUBDIR variables. 477 If you get make errors, 478 (cd share/mk; make install) 479 Also, PRINTOBJDIR has changed and is now used more heavily. 480 48120001019: 482 The `ca' device driver has been replaced by `ld'; although the 483 major and minor numbers haven't changed, you should update your /dev 484 directory. 485 48620000929: 487 The following make directives are obsoleted. 488 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 489 By default, RSA is built into libcrypto. IDEA and RC5 will not be 490 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 491 additional library libcrypto_{idea,rc5}. 492 493 494Hints for a more successful build: 495^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 496 Build a new kernel first: 497 This makes sure that any new system calls or features 498 expected by the new userland will be present. This 499 helps to avoid critical errors when upgrading. 500 Use object directories: 501 This helps to keep stale object 502 files from polluting the build if a Makefile "forgets" 503 about one. It also makes it easier to clean up after 504 a build. It's also necessary if you want to use the 505 same source tree for multiple machines. 506 To use object directories: 507 a) cd /usr/src ; make cleandir 508 b) Add "OBJMACHINE=yes" to /etc/mk.conf 509 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 510 d) cd /usr/src ; make build 511 Note that running "make obj" in a directory will create 512 in obj.$MACHINE directory. 513 Build to a DESTDIR: 514 This helps to keep old 515 installed files (especially libraries) from interfering 516 with the new build. 517 To build to a DESTDIR, set the DESTDIR environment 518 variable before running make build. It should be set to 519 the pathname of an initially empty directory. 520 Problems: you might need to update critical utilities 521 without using DESTDIR since nothing is executed 522 from what is installed in DESTDIR. 523 (See critical utils, below) 524 Build often: 525 This keeps critical utilities current enough to not choke 526 on any other part of the source tree that depends on up to 527 date functionality. 528 529What to do if things don't work: 530^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 531When things don't work there is usually a few things that commonly 532should be done. 533 1) make includes 534 This should be done automatically by make build. 535 2) cd share/mk && make install 536 Again, automatically done by make build. 537 538Failsafe rebuild of a small part of the tree: 539^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 540To make sure you rebuild something correctly you want to do 541something like the following: 542 1) Make sure the includes and .mk files are up to date. 543 2) Make sure any program used to build the particular 544 utility is up to date. (yacc, lex, etc...) 545 3) cd ...path/to/util... 546 make cleandir 547 rm ...all obj directories... 548 make cleandir # yes, again 549 make obj 550 make depend && make 551 552Failsafe rebuild of the entire tree: 553^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 554If you really want to make sure the source tree is clean and 555ready for a build try the following. Note that sourcing /etc/mk.conf 556(a make(1) Makefile) in this manner is not right, and will not work 557for anyone who uses any make(1) features in /etc/mk.conf. 558 559---cut here--- 560#!/bin/sh 561. /etc/mk.conf 562 563if [ -z $NETBSDSRCDIR ] ; then 564 NETBSDSRCDIR=/usr/src 565fi 566if [ \! -d $NETBSDSRCDIR ] ; then 567 echo Unable to find sources 568 exit 1 569fi 570find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 571 572if [ -z $BSDOBJDIR ] ; then 573 BSDOBJDIR=/usr/obj 574fi 575if [ -d $BSDOBJDIR ] ; then 576 rm -rf $BSDOBJDIR 577fi 578 579cd $NETBSDSRCDIR && make cleandir 580 581---cut here--- 582 583Critical utilities: 584^^^^^^^^^^^^^^^^^^^ 585 gnu/usr.bin/egcs 586 usr.bin/compile_et 587 usr.bin/make 588 usr.bin/yacc 589 usr.bin/lex 590 usr.bin/xlint 591 usr.sbin/config 592 593Other problems and possible solutions: 594^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 595Symptom:Unreasonable compiler errors. 596Fix: Rebuild gnu/usr.bin/egcs 597 598Symptom:Complaints involving a Makefile. 599Fix: Rebuild usr.bin/make: 600 cd usr.bin/make && make && make install 601 Or, a failsafe method if that doesn't work: 602 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 603 604Fix: Make sure .mk files are up to date. 605 cd share/mk && make install 606 607Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 608Fix: Rebuild usr.sbin/config 609 610Symptom: 611Fix: Rebuild usr.bin/yacc 612 613Symptom: 614Fix: Rebuild usr.bin/lex 615 616Symptom: 617Fix: rm /usr/lib/libbfd.a 618 619Symptom:Obsolete intermediate files are used during compilation 620Fix: Try the following sequence of commands in the directory in question. 621 make cleandir; rm `make print-objdir`; make cleandir; make obj 622 (If you built the tree without "make obj" in the past, obsolete files 623 may remain. The command tries to clean everything up) 624 625Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 626Fix: Rebuild and install usr.bin/menuc 627 628Symptom:mklocale not found during build in share/locale/ctype 629Fix: Build and install usr.bin/mklocale 630 631Symptom:undefined reference to `__assert13' or `__unsetenv13' 632Fix: Rebuild and install lib/libc 633 634Symptom:usr.sbin/config fails to build. 635Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 636 637Symptom:undefined reference to `getprogname' or `setprogname' 638Fix: Rebuild and install lib/libc 639 640Symptom:lint does not understand the '-X' option 641Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 642