UPDATING revision 1.97
1$NetBSD: UPDATING,v 1.97 2003/10/08 04:25:43 lukem Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120031008: 12 /usr/include/sys/disklabel_mbr.h was removed. 13 It's necessary to "make cleandir" to ensure that 14 the dependencies will be rebuilt correctly. 15 1620030906: 17 With the addition of siginfo support the old signal trampoline 18 code has been deprecated to COMPAT_16. Make sure that your running 19 kernel has COMPAT_16 enabled before building userland. 20 2120030801: 22 With the new openssl, there is some header and library shuffling. 23 rm -f /usr/include/des.h /usr/include/kerberosIV/* /lib/libdes* \ 24 /usr/lib/libdes* before building. 25 2620030703: 27 Texinfo was updated to 4.6. To avoid failures when trying to 28 build the included texinfo files, do: 29 30 cd src/gnu/usr.bin/texinfo 31 make MKINFO=no dependall install 32 3320030630: 34 Groff was update to 1.19; it's probably necessary to do 35 cd share/mk && make install 36 cd src/gnu/usr.bin/groff 37 make MKMAN=no dependall install 38 (untested). 39 4020030516: 41 Due to bugs in the export handling code, invalid export lines 42 were accepted before and caused the kernel to panic when 43 mountd got restarted because it freed memory that had already 44 been freed. This has been fixed and the kernel checks 45 export addresses very strictly. If you upgrade your kernel, 46 make sure you also upgrade mountd, because if your export 47 file contains lines with an old inet4 address syntax (i.e. 48 a.b.c or a.b or a), they will get rejected by the new kernel. 49 5020030402: 51 The superblock layout for FFS was changed. If you have 1.6 52 fsck binaries, they will signal a fatal superblock mismatch 53 with the first alternate, because they compare too many 54 fields (even ones that aren't useful). If possible, upgrade 55 your fsck_ffs binary before using a new kernel. 56 None of this signals actual filesystem damage. 57 5820030324: 59 sendmail version 8.12.8 was imported. Since sendmail is 60 now setgid to the smmsp group, and runs in "collection" 61 mode for most common activities, there is a new config 62 file called submit.cf that needs to live in /etc/mail. 63 The generic submit.cf sample in /usr/share/sendmail/cf 64 is named netbsd-msp.cf. Upgrading your regular sendmail 65 configuration file is also strongly advised. 66 67 See the section named "MESSAGE SUBMISSION PROGRAM" in 68 the updated /usr/share/sendmail/README file for more 69 information. 70 7120030117: 72 Texinfo was updated to 4.3. To avoid failures when trying to 73 build the included texinfo files, do: 74 75 cd src/gnu/usr.bin/texinfo 76 make MKINFO=no dependall install 77 7820021223: 79 The METALOG format changed slightly, to remove the leading 80 "${DESTDIR}" from path names. 81 This only affects people building with UNPRIVED. 82 For complete safety, remove the DESTDIR entirely and 83 update tools/mtree, before running make build. 84 8520021219: 86 CVS repository layout was changed. See the following for details 87 if you are using (anonymous) cvs to update your tree. 88 89 http://mail-index.NetBSD.org/netbsd-announce/2002/12/19/0000.html 90 9120021219: 92 install(1) had a '-N dbdir' option added, to specify an 93 alternate location to look up users & groups (instead 94 of the host system passwd(5) and group(5) databases). 95 96 The build system was modified to take advantage of 97 this option (using ${NETBSDSRCDIR}/etc), so if you 98 use USETOOLS==no, you may have to rebuild and 99 reinstall usr.bin/xinstall first. 100 10120021130: 102 fparseln(3) moved from libutil to libc. 103 If building to DESTDIR=/, reinstall the includes 104 and rebuild libc: 105 make includes 106 make do-lib-libc 107 If using build.sh, "cd tools/compat && make clean" 108 before rebuilding the tools. 109 11020021126: 111 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 112 including the optional "SYS_INCLUDE=symlinks" support. 113 All header files, including <sys/*.h> are copied into 114 /usr/include. 115 11620021121: 117 The C run-time support files crtbegin.o and crtend.o 118 (and their companions crtbeginS.o and crtendS.o) were 119 split up, with new crti.o and crtn.o files resulting. 120 This means that libtool needs to be rebuilt once the 121 new libraries are installed. The process of rebuilding 122 libtool will cause it to automatically notice the new 123 required files, but it *must* be rebuilt in order to 124 do this. 125 126 An out-of-date libtool will result in shared libraries 127 which lack _init() and _fini() routines, which means that 128 their global contructors/destructors will not be invoked. 129 13020021121: 131 A bug related to how ARM ELF objects were tagged has been 132 corrected. 133 134 NetBSD ARM ELF uses the soft-VFP floating point model by 135 default. However, the assembler lacked support for marking 136 objects as using the VFP floating point format, and the 137 compiler was not properly passing the flag indicating "soft-VFP" 138 to the assembler. 139 140 Unfortunately, this means that the linker will now consider 141 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 142 incompatible with new (properly marked) objects. 143 144 The problem will only manifest itself if you attempt to compile 145 a new program using the fixed toolchain, and link that program 146 against old libraries which do not have the proper "softvfp" 147 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 148 WILL CONTINUE TO WORK PROPERLY. 149 150 The only work-around for the problem is to recompile all of 151 the libraries on the system. The easiest way to do this for 152 system libraries is to install a binary snapshot; they are 153 generally available on releng.NetBSD.org. Any packages you 154 have installed which supply libraries will have to be recompiled 155 if you wish to link new programs against those libraries. 156 157 If you have questions about this matter, please contact 158 port-arm@NetBSD.org. 159 16020021011: 161 Systrace has been improved to support privilege elevation. 162 Updating the kernel requires the userland part of systrace 163 to be rebuilt. 164 16520021010: 166 The config(8) grammar was changed to allow options to register 167 dependencies on attributes, as well as other options. Users 168 must update and reinstall usr.sbin/config before building a new 169 kernel. 170 17120021009: 172 A new attribute dependency syntax was introduced to config(8), 173 which is now used by the SCSI configuration description. Users 174 must update and reinstall usr.sbin/config before building a new 175 kernel. 176 17720021003: 178 Several changes have been made to the autoconfiguration 179 framework. Users must update and reinstall usr.sbin/config 180 before building a new kernel. 181 18220021001: 183 The i386mp branch has been merged. To compile a kernel, users 184 will need to add the option 'cpu* at mainbus?' to their configuration 185 file. Multiprocessor kernels will need 186 ioapic* at mainbus? apid ? 187 options MULTIPROCESSOR 188 options COM_MPLOCK 189 19020020922: 191 MKDYNAMICROOT=yes enabled by default, which means that 192 certain shared libraries are installed into /lib, the shared 193 linker is installed into /libexec, and all programs in /bin 194 and /sbin are dynamically linked. 195 If you do not use "make build", you should ensure that 196 you have the libraries and shared linker in the new locations, 197 with: 198 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 199 20020020917: 201 USE_NEW_TOOLCHAIN has been replaced with: 202 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 203 there is no working in-tree toolchain (hppa, ns32k, sh5, 204 x86_64). 205 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 206 root of an external toolchain (e.g. /usr/local/gnu). This 207 enables the cross-build framework even for TOOLCHAIN_MISSING 208 platforms. 209 21020020906: 211 gehenna-devsw has been merged into the trunk. Need to update and 212 reinstall usr.sbin/config before building the kernel. 213 21420020822: 215 Crunched rescue tools (contents of /bin and /sbin, plus others) 216 are now provided in /rescue. 217 218 To ensure that these are built statically linked (no matter 219 what the setting of LDSTATIC is), use a crunchgen(1) built 220 from sources newer than 20020820 (see the next entry). 221 22220020820: 223 crunchgen(1) changed to ensure that the generated program 224 is statically linked. 225 226 Solution: update and reinstall usr.bin/crunch 227 22820020605: 229 smmsp user/group has been added for sendmail. 230 231 Add the following into /etc/group: 232 233 smmsp:*:17: 234 235 and the following to /etc/master.passwd (via vipw): 236 237 smmsp:*:17:17::0:0:Sendmail Message Submission Program:/nonexistent:/sbin/nologin 238 23920020515: 240 sshd user/group has been added. Need to hand add this in, or sshd 241 will not let you log in (with default, or UsePrivlegeSeparation=yes) 242 243 Add the following into /etc/group: 244 245 sshd:*:16: 246 247 and the following to /etc/master.passwd (via vipw): 248 249 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 250 251 Also /var/chroot/sshd directory needs to be present (digged as part of 252 the build process). 253 25420020426: 255 NBUILDJOBS obsoleted in favor of just using -j. 256 25720020426: 258 etc/postinstall added, which performs various checks for 259 configuration file updates and changes, and can fix most of 260 the problems identified. 261 This should make it much easier to upgrade a system's 262 configuration from earlier systems (as far back as NetBSD 1.5). 263 26420020320: 265 <bsd.lib.mk> needs a new install(1) for its "-a cmd" support. 266 Build and install at usr.bin/xinstall before the build. 267 26820020319: 269 Raw IPv6 socket now makes strict checking for sa_family and sa_len 270 on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold 271 newer than November 2001 when you upgrade the kernel. 272 27320020311: 274 ssh configuration files were moved from /etc to /etc/ssh. Beware 275 if you restart your machine from remote. Note that sshd.conf needs 276 to be changed (due to the use of "/etc" inside). 277 27820020223: 279 Users of the VAX port will need to rebuild and install gas 280 so it deal with the now present register prefix used in all 281 the VAX assembly files. 282 28320020118: 284 ntpd user/group has been added. Need to hand add this in or builds 285 will break as mtree aborts early. 286 287 Add the following into /etc/group: 288 289 ntpd:*:15: 290 291 and the following to /etc/master.passwd (via vipw): 292 293 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 294 29520011207: 296 If you're attempting to build a snapshot on sparc64 and are getting 297 reloc errors from the toolchain groff binary this means your native 298 toolchain has some broken C++ bits. 299 300 To fix: 301 302 Build a new toolchain (i.e. build.sh -t) 303 Use the new toolchain to build and install natively (i.e. /usr/lib) 304 305 gnu/lib/libgcc 306 gnu/lib/libstdc++ 307 308 After this a snapshot will be able to be built. 309 31020011201: 311 In order for a sparc64 build to work you must have a working awk. If 312 you've built and installed a system with the new toolchain up to this 313 point you do not have a working awk as its ability to do floating 314 point is broken. 315 316 To build: 317 318 remake and install gnu/lib/libgcc 319 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 320 against the new libgcc.a) 321 32220011128: 323 Kernel config information was changed to use defflag in 324 the various "files" files. Bug fixes to config(8) are 325 required in order for this to work properly. Make sure 326 to build and install in usr.sbin/config before attempting 327 to build a new kernel. 328 32920011030: 330 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 331 before building libc. 332 33320011029: 334 The new document BUILDING.mdoc (view with nroff | more, or 335 see pre-generated .txt and .html versions) describes the build 336 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 337 build process, are intended in the long run to replace this 338 manual update log. 339 340 Users building a USE_NEW_TOOLCHAIN system should read the 341 BUILDING document for caveats. Generally, BUILDING supersedes 342 UPDATING for these systems, as tool updating is taken care of 343 by the new build system. 344 34520011028: 346 src/etc/Makefile now needs install to be able to handle 347 symlinks that point to nowhere. A bug in install that 348 prevented this was corrected. 349 350 Solution: update and reinstall usr.bin/xinstall 351 Better Solution: Use the new toolchain and it will just work 352 for you. 353 35420011006: 355 /etc/mtree/NetBSD.dist has been updated to take advantage of 356 absolute path support added to mtree(8). Older mtree(8)s don't 357 understand the format. 358 359 Solution: update and reinstall usr.sbin/mtree 360 36120011004: 362 Crunchgen has been updated to work via reach-over makefiles. Updating 363 is suggested before running a snapshot build 364 36520010915: 366 The new "ubcperf" code committed by Chuck Silvers removed 367 a header file, uvm/uvm_vnode.h. There may be stale .depend 368 files that still reference this file. 369 370 Solution: "make cleandir && make dependall" in affected 371 directories. 372 37320010803: 374 grep.info is now built from grep.texi using makeinfo. Since it 375 requires makeinfo v4.0, you need to install new texinfo before 376 building gnu/usr.bin/grep. To install new texinfo, please follow 377 the instruction described in 20010726 entry. 378 37920010803: 380 (i386 only): i386 kernel now uses new instructions like 381 `fxsave' which old gas doesn't understand. To build the 382 kernel successfully, you need to build and install a new toolchain, 383 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 384 from your kernel configuration until you rebuild your userland. 385 See 20011029 above and BUILDING file in this directory for more information. 386 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 387 38820010731: 389 Bootloader update on ELF platforms. DDB in kernels from before 390 this will be unable to read symbol tables provided by newer 391 bootloaders. 392 39320010726: 394 Texinfo was updated to 4.0. To avoid failures when trying to 395 build the included texinfo files, do: 396 397 cd src/gnu/usr.bin/texinfo 398 make MKINFO=no dependall install 399 40020010718: 401 Enabled correct .init/.fini processing in crt0. The way this 402 was done was to change a -I directive to cc(1), which means 403 make(1) will have a stale dependency (it will be checking the 404 timestamp on the wrong "dot_init.h"). 405 406 The symptom you will see is that new programs die with SIGSEGV 407 if you have a stale dependency. 408 409 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 410 before starting your build. 411 41220010628: 413 A construct was added to uvm_page.h that uncovered a bug 414 in lint(1). If you get a warning/error about a non-portable 415 bitfield, update your lint(1) before proceeding. 416 41720010226: 418 Added named user/group to system. Need to hand add this in or builds 419 will break as mtree aborts early. 420 421 To work around add by hand: 422 423 named:*:14: 424 425 to /etc/group and add: 426 427 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 428 429 to master.passwd (use vipw for instance if doing by hand). 430 431 Now a make build should progress. 432 43320010219: 434 get/setprogname() added. Any hostprogs that may use this will need 435 to be bootstrapped manually until the host system is current. 436 437 Known problems: sys/arch/macppc/stand/fixcoff 438 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 439 CFLAGS and rebuilding should work) 440 usr.sbin/mdsetimage - Build a static copy if 441 building a snapshot before fully bootstrapped. 442 44320010204: 444 prepare the code to compile with stricter gcc flags. in 445 particular start eliminating redundant declarations. Yacc 446 needs to be installed before make build. 447 44820010114: 449 introduce .if commands(target) in make(1). You need to 450 bring everything up-to-date first, then without installing 451 anything make and install in usr.bin/make, then proceed 452 with make build. 453 45420010101: 455 bsd.subdir.mk committed 20001230 had a bug which caused 456 afterinstall targets to run too soon; update again. 457 45820001230: 459 New share/mk files needed to support .WAIT in SUBDIR variables. 460 If you get make errors, 461 (cd share/mk; make install) 462 Also, PRINTOBJDIR has changed and is now used more heavily. 463 46420001019: 465 The `ca' device driver has been replaced by `ld'; although the 466 major and minor numbers haven't changed, you should update your /dev 467 directory. 468 46920000929: 470 The following make directives are obsoleted. 471 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 472 By default, RSA is built into libcrypto. IDEA and RC5 will not be 473 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 474 additional library libcrypto_{idea,rc5}. 475 476 477Hints for a more successful build: 478^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 479 Build a new kernel first: 480 This makes sure that any new system calls or features 481 expected by the new userland will be present. This 482 helps to avoid critical errors when upgrading. 483 Use object directories: 484 This helps to keep stale object 485 files from polluting the build if a Makefile "forgets" 486 about one. It also makes it easier to clean up after 487 a build. It's also necessary if you want to use the 488 same source tree for multiple machines. 489 To use object directories: 490 a) cd /usr/src ; make cleandir 491 b) Add "OBJMACHINE=yes" to /etc/mk.conf 492 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 493 d) cd /usr/src ; make build 494 Note that running "make obj" in a directory will create 495 in obj.$MACHINE directory. 496 Build to a DESTDIR: 497 This helps to keep old 498 installed files (especially libraries) from interfering 499 with the new build. 500 To build to a DESTDIR, set the DESTDIR environment 501 variable before running make build. It should be set to 502 the pathname of an initially empty directory. 503 Problems: you might need to update critical utilities 504 without using DESTDIR since nothing is executed 505 from what is installed in DESTDIR. 506 (See critical utils, below) 507 Build often: 508 This keeps critical utilities current enough to not choke 509 on any other part of the source tree that depends on up to 510 date functionality. 511 512What to do if things don't work: 513^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 514When things don't work there is usually a few things that commonly 515should be done. 516 1) make includes 517 This should be done automatically by make build. 518 2) cd share/mk && make install 519 Again, automatically done by make build. 520 521Failsafe rebuild of a small part of the tree: 522^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 523To make sure you rebuild something correctly you want to do 524something like the following: 525 1) Make sure the includes and .mk files are up to date. 526 2) Make sure any program used to build the particular 527 utility is up to date. (yacc, lex, etc...) 528 3) cd ...path/to/util... 529 make cleandir 530 rm ...all obj directories... 531 make cleandir # yes, again 532 make obj 533 make depend && make 534 535Failsafe rebuild of the entire tree: 536^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 537If you really want to make sure the source tree is clean and 538ready for a build try the following. Note that sourcing /etc/mk.conf 539(a make(1) Makefile) in this manner is not right, and will not work 540for anyone who uses any make(1) features in /etc/mk.conf. 541 542---cut here--- 543#!/bin/sh 544. /etc/mk.conf 545 546if [ -z $NETBSDSRCDIR ] ; then 547 NETBSDSRCDIR=/usr/src 548fi 549if [ \! -d $NETBSDSRCDIR ] ; then 550 echo Unable to find sources 551 exit 1 552fi 553find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 554 555if [ -z $BSDOBJDIR ] ; then 556 BSDOBJDIR=/usr/obj 557fi 558if [ -d $BSDOBJDIR ] ; then 559 rm -rf $BSDOBJDIR 560fi 561 562cd $NETBSDSRCDIR && make cleandir 563 564---cut here--- 565 566Critical utilities: 567^^^^^^^^^^^^^^^^^^^ 568 gnu/usr.bin/egcs 569 usr.bin/compile_et 570 usr.bin/make 571 usr.bin/yacc 572 usr.bin/lex 573 usr.bin/xlint 574 usr.sbin/config 575 576Other problems and possible solutions: 577^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 578Symptom:Unreasonable compiler errors. 579Fix: Rebuild gnu/usr.bin/egcs 580 581Symptom:Complaints involving a Makefile. 582Fix: Rebuild usr.bin/make: 583 cd usr.bin/make && make && make install 584 Or, a failsafe method if that doesn't work: 585 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 586 587Fix: Make sure .mk files are up to date. 588 cd share/mk && make install 589 590Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 591Fix: Rebuild usr.sbin/config 592 593Symptom: 594Fix: Rebuild usr.bin/yacc 595 596Symptom: 597Fix: Rebuild usr.bin/lex 598 599Symptom: 600Fix: rm /usr/lib/libbfd.a 601 602Symptom:Obsolete intermediate files are used during compilation 603Fix: Try the following sequence of commands in the directory in question. 604 make cleandir; rm `make print-objdir`; make cleandir; make obj 605 (If you built the tree without "make obj" in the past, obsolete files 606 may remain. The command tries to clean everything up) 607 608Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 609Fix: Rebuild and install usr.bin/menuc 610 611Symptom:mklocale not found during build in share/locale/ctype 612Fix: Build and install usr.bin/mklocale 613 614Symptom:undefined reference to `__assert13' or `__unsetenv13' 615Fix: Rebuild and install lib/libc 616 617Symptom:usr.sbin/config fails to build. 618Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 619 620Symptom:undefined reference to `getprogname' or `setprogname' 621Fix: Rebuild and install lib/libc 622 623Symptom:lint does not understand the '-X' option 624Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 625