UPDATING revision 1.91
1$NetBSD: UPDATING,v 1.91 2003/07/01 08:13:41 wiz Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120030630: 12 Groff was update to 1.19; it's probably necessary to do 13 cd share/mk && make install 14 cd src/gnu/usr.bin/groff 15 make MKMAN=no dependall install 16 (untested). 17 1820030516: 19 Due to bugs in the export handling code, invalid export lines 20 were accepted before and caused the kernel to panic when 21 mountd got restarted because it freed memory that had already 22 been freed. This has been fixed and the kernel checks 23 export addresses very strictly. If you upgrade your kernel, 24 make sure you also upgrade mountd, because if your export 25 file contains lines with an old inet4 address syntax (i.e. 26 a.b.c or a.b or a), they will get rejected by the new kernel. 27 2820030402: 29 The superblock layout for FFS was changed. If you have 1.6 30 fsck binaries, they will signal a fatal superblock mismatch 31 with the first alternate, because they compare too many 32 fields (even ones that aren't useful). If possible, upgrade 33 your fsck_ffs binary before using a new kernel. 34 None of this signals actual filesystem damage. 35 3620030324: 37 sendmail version 8.12.8 was imported. Since sendmail is 38 now setgid to the smmsp group, and runs in "collection" 39 mode for most common activities, there is a new config 40 file called submit.cf that needs to live in /etc/mail. 41 The generic submit.cf sample in /usr/share/sendmail/cf 42 is named netbsd-msp.cf. Upgrading your regular sendmail 43 configuration file is also strongly advised. 44 45 See the section named "MESSAGE SUBMISSION PROGRAM" in 46 the updated /usr/share/sendmail/README file for more 47 information. 48 4920030117: 50 Texinfo was updated to 4.3. To avoid failures when trying to 51 build the included texinfo files, do: 52 53 cd src/gnu/usr.bin/texinfo 54 make MKINFO=no dependall install 55 5620021223: 57 The METALOG format changed slightly, to remove the leading 58 "${DESTDIR}" from path names. 59 This only affects people building with UNPRIVED. 60 For complete safety, remove the DESTDIR entirely and 61 update tools/mtree, before running make build. 62 6320021219: 64 CVS repository layout was changed. See the following for details 65 if you are using (anonymous) cvs to update your tree. 66 67 http://mail-index.netbsd.org/netbsd-announce/2002/12/19/0000.html 68 6920021219: 70 install(1) had a '-N dbdir' option added, to specify an 71 alternate location to look up users & groups (instead 72 of the host system passwd(5) and group(5) databases). 73 74 The build system was modified to take advantage of 75 this option (using ${NETBSDSRCDIR}/etc), so if you 76 use USETOOLS==no, you may have to rebuild and 77 reinstall usr.bin/xinstall first. 78 7920021130: 80 fparseln(3) moved from libutil to libc. 81 If building to DESTDIR=/, reinstall the includes 82 and rebuild libc: 83 make includes 84 make do-lib-libc 85 If using build.sh, "cd tools/compat && make clean" 86 before rebuilding the tools. 87 8820021126: 89 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 90 including the optional "SYS_INCLUDE=symlinks" support. 91 All header files, including <sys/*.h> are copied into 92 /usr/include. 93 9420021121: 95 The C run-time support files crtbegin.o and crtend.o 96 (and their companions crtbeginS.o and crtendS.o) were 97 split up, with new crti.o and crtn.o files resulting. 98 This means that libtool needs to be rebuilt once the 99 new libraries are installed. The process of rebuilding 100 libtool will cause it to automatically notice the new 101 required files, but it *must* be rebuilt in order to 102 do this. 103 104 An out-of-date libtool will result in shared libraries 105 which lack _init() and _fini() routines, which means that 106 their global contructors/destructors will not be invoked. 107 10820021121: 109 A bug related to how ARM ELF objects were tagged has been 110 corrected. 111 112 NetBSD ARM ELF uses the soft-VFP floating point model by 113 default. However, the assembler lacked support for marking 114 objects as using the VFP floating point format, and the 115 compiler was not properly passing the flag indicating "soft-VFP" 116 to the assembler. 117 118 Unfortunately, this means that the linker will now consider 119 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 120 incompatible with new (properly marked) objects. 121 122 The problem will only manifest itself if you attempt to compile 123 a new program using the fixed toolchain, and link that program 124 against old libraries which do not have the proper "softvfp" 125 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 126 WILL CONTINUE TO WORK PROPERLY. 127 128 The only work-around for the problem is to recompile all of 129 the libraries on the system. The easiest way to do this for 130 system libraries is to install a binary snapshot; they are 131 generally available on releng.netbsd.org. Any packages you 132 have installed which supply libraries will have to be recompiled 133 if you wish to link new programs against those libraries. 134 135 If you have questions about this matter, please contact 136 port-arm@netbsd.org. 137 13820021011: 139 Systrace has been improved to support privilege elevation. 140 Updating the kernel requires the userland part of systrace 141 to be rebuilt. 142 14320021010: 144 The config(8) grammar was changed to allow options to register 145 dependencies on attributes, as well as other options. Users 146 must update and reinstall usr.sbin/config before building a new 147 kernel. 148 14920021009: 150 A new attribute dependency syntax was introduced to config(8), 151 which is now used by the SCSI configuration description. Users 152 must update and reinstall usr.sbin/config before building a new 153 kernel. 154 15520021003: 156 Several changes have been made to the autoconfiguration 157 framework. Users must update and reinstall usr.sbin/config 158 before building a new kernel. 159 16020021001: 161 The i386mp branch has been merged. To compile a kernel, users 162 will need to add the option 'cpu* at mainbus?' to their configuration 163 file. Multiprocessor kernels will need 164 ioapic* at mainbus? apid ? 165 options MULTIPROCESSOR 166 options COM_MPLOCK 167 16820020922: 169 MKDYNAMICROOT=yes enabled by default, which means that 170 certain shared libraries are installed into /lib, the shared 171 linker is installed into /libexec, and all programs in /bin 172 and /sbin are dynamically linked. 173 If you do not use "make build", you should ensure that 174 you have the libraries and shared linker in the new locations, 175 with: 176 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 177 17820020917: 179 USE_NEW_TOOLCHAIN has been replaced with: 180 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 181 there is no working in-tree toolchain (hppa, ns32k, sh5, 182 x86_64). 183 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 184 root of an external toolchain (e.g. /usr/local/gnu). This 185 enables the cross-build framework even for TOOLCHAIN_MISSING 186 platforms. 187 18820020906: 189 gehenna-devsw has been merged into the trunk. Need to update and 190 reinstall usr.sbin/config before building the kernel. 191 19220020822: 193 Crunched rescue tools (contents of /bin and /sbin, plus others) 194 are now provided in /rescue. 195 196 To ensure that these are built statically linked (no matter 197 what the setting of LDSTATIC is), use a crunchgen(1) built 198 from sources newer than 20020820 (see the next entry). 199 20020020820: 201 crunchgen(1) changed to ensure that the generated program 202 is statically linked. 203 204 Solution: update and reinstall usr.bin/crunch 205 20620020605: 207 smmsp user/group has been added for sendmail. 208 209 Add the following into /etc/group: 210 211 smmsp:*:17: 212 213 and the following to /etc/master.passwd (via vipw): 214 215 smmsp:*:17:17::0:0:Sendmail Message Submission Program:/nonexistent:/sbin/nologin 216 21720020515: 218 sshd user/group has been added. Need to hand add this in, or sshd 219 will not let you log in (with default, or UsePrivlegeSeparation=yes) 220 221 Add the following into /etc/group: 222 223 sshd:*:16: 224 225 and the following to /etc/master.passwd (via vipw): 226 227 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 228 229 Also /var/chroot/sshd directory needs to be present (digged as part of 230 the build process). 231 23220020426: 233 NBUILDJOBS obsoleted in favor of just using -j. 234 23520020426: 236 etc/postinstall added, which performs various checks for 237 configuration file updates and changes, and can fix most of 238 the problems identified. 239 This should make it much easier to upgrade a system's 240 configuration from earlier systems (as far back as NetBSD 1.5). 241 24220020320: 243 <bsd.lib.mk> needs a new install(1) for its "-a cmd" support. 244 Build and install at usr.bin/xinstall before the build. 245 24620020319: 247 Raw IPv6 socket now makes strict checking for sa_family and sa_len 248 on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold 249 newer than November 2001 when you upgrade the kernel. 250 25120020311: 252 ssh configuration files were moved from /etc to /etc/ssh. Beware 253 if you restart your machine from remote. Note that sshd.conf needs 254 to be changed (due to the use of "/etc" inside). 255 25620020223: 257 Users of the VAX port will need to rebuild and install gas 258 so it deal with the now present register prefix used in all 259 the VAX assembly files. 260 26120020118: 262 ntpd user/group has been added. Need to hand add this in or builds 263 will break as mtree aborts early. 264 265 Add the following into /etc/group: 266 267 ntpd:*:15: 268 269 and the following to /etc/master.passwd (via vipw): 270 271 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 272 27320011207: 274 If you're attempting to build a snapshot on sparc64 and are getting 275 reloc errors from the toolchain groff binary this means your native 276 toolchain has some broken C++ bits. 277 278 To fix: 279 280 Build a new toolchain (i.e. build.sh -t) 281 Use the new toolchain to build and install natively (i.e. /usr/lib) 282 283 gnu/lib/libgcc 284 gnu/lib/libstdc++ 285 286 After this a snapshot will be able to be built. 287 28820011201: 289 In order for a sparc64 build to work you must have a working awk. If 290 you've built and installed a system with the new toolchain up to this 291 point you do not have a working awk as its ability to do floating 292 point is broken. 293 294 To build: 295 296 remake and install gnu/lib/libgcc 297 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 298 against the new libgcc.a) 299 30020011128: 301 Kernel config information was changed to use defflag in 302 the various "files" files. Bug fixes to config(8) are 303 required in order for this to work properly. Make sure 304 to build and install in usr.sbin/config before attempting 305 to build a new kernel. 306 30720011030: 308 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 309 before building libc. 310 31120011029: 312 The new document BUILDING.mdoc (view with nroff | more, or 313 see pre-generated .txt and .html versions) describes the build 314 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 315 build process, are intended in the long run to replace this 316 manual update log. 317 318 Users building a USE_NEW_TOOLCHAIN system should read the 319 BUILDING document for caveats. Generally, BUILDING supersedes 320 UPDATING for these systems, as tool updating is taken care of 321 by the new build system. 322 32320011028: 324 src/etc/Makefile now needs install to be able to handle 325 symlinks that point to nowhere. A bug in install that 326 prevented this was corrected. 327 328 Solution: update and reinstall usr.bin/xinstall 329 Better Solution: Use the new toolchain and it will just work 330 for you. 331 33220011006: 333 /etc/mtree/NetBSD.dist has been updated to take advantage of 334 absolute path support added to mtree(8). Older mtree(8)s don't 335 understand the format. 336 337 Solution: update and reinstall usr.sbin/mtree 338 33920011004: 340 Crunchgen has been updated to work via reach-over makefiles. Updating 341 is suggested before running a snapshot build 342 34320010915: 344 The new "ubcperf" code committed by Chuck Silvers removed 345 a header file, uvm/uvm_vnode.h. There may be stale .depend 346 files that still reference this file. 347 348 Solution: "make cleandir && make dependall" in affected 349 directories. 350 35120010803: 352 grep.info is now built from grep.texi using makeinfo. Since it 353 requires makeinfo v4.0, you need to install new texinfo before 354 building gnu/usr.bin/grep. To install new texinfo, please follow 355 the instruction described in 20010726 entry. 356 35720010803: 358 (i386 only): i386 kernel now uses new instructions like 359 `fxsave' which old gas doesn't understand. To build the 360 kernel successfully, you need to build and install a new toolchain, 361 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 362 from your kernel configuration until you rebuild your userland. 363 See 20011029 above and BUILDING file in this directory for more information. 364 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 365 36620010731: 367 Bootloader update on ELF platforms. DDB in kernels from before 368 this will be unable to read symbol tables provided by newer 369 bootloaders. 370 37120010726: 372 Texinfo was updated to 4.0. To avoid failures when trying to 373 build the included texinfo files, do: 374 375 cd src/gnu/usr.bin/texinfo 376 make MKINFO=no dependall install 377 37820010718: 379 Enabled correct .init/.fini processing in crt0. The way this 380 was done was to change a -I directive to cc(1), which means 381 make(1) will have a stale dependency (it will be checking the 382 timestamp on the wrong "dot_init.h"). 383 384 The symptom you will see is that new programs die with SIGSEGV 385 if you have a stale dependency. 386 387 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 388 before starting your build. 389 39020010628: 391 A construct was added to uvm_page.h that uncovered a bug 392 in lint(1). If you get a warning/error about a non-portable 393 bitfield, update your lint(1) before proceeding. 394 39520010226: 396 Added named user/group to system. Need to hand add this in or builds 397 will break as mtree aborts early. 398 399 To work around add by hand: 400 401 named:*:14: 402 403 to /etc/group and add: 404 405 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 406 407 to master.passwd (use vipw for instance if doing by hand). 408 409 Now a make build should progress. 410 41120010219: 412 get/setprogname() added. Any hostprogs that may use this will need 413 to be bootstrapped manually until the host system is current. 414 415 Known problems: sys/arch/macppc/stand/fixcoff 416 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 417 CFLAGS and rebuilding should work) 418 usr.sbin/mdsetimage - Build a static copy if 419 building a snapshot before fully bootstrapped. 420 42120010204: 422 prepare the code to compile with stricter gcc flags. in 423 particular start eliminating redundant declarations. Yacc 424 needs to be installed before make build. 425 42620010114: 427 introduce .if commands(target) in make(1). You need to 428 bring everything up-to-date first, then without installing 429 anything make and install in usr.bin/make, then proceed 430 with make build. 431 43220010101: 433 bsd.subdir.mk committed 20001230 had a bug which caused 434 afterinstall targets to run too soon; update again. 435 43620001230: 437 New share/mk files needed to support .WAIT in SUBDIR variables. 438 If you get make errors, 439 (cd share/mk; make install) 440 Also, PRINTOBJDIR has changed and is now used more heavily. 441 44220001019: 443 The `ca' device driver has been replaced by `ld'; although the 444 major and minor numbers haven't changed, you should update your /dev 445 directory. 446 44720000929: 448 The following make directives are obsoleted. 449 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 450 By default, RSA is built into libcrypto. IDEA and RC5 will not be 451 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 452 additional library libcrypto_{idea,rc5}. 453 454 455Hints for a more successful build: 456^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 457 Build a new kernel first: 458 This makes sure that any new system calls or features 459 expected by the new userland will be present. This 460 helps to avoid critical errors when upgrading. 461 Use object directories: 462 This helps to keep stale object 463 files from polluting the build if a Makefile "forgets" 464 about one. It also makes it easier to clean up after 465 a build. It's also necessary if you want to use the 466 same source tree for multiple machines. 467 To use object directories: 468 a) cd /usr/src ; make cleandir 469 b) Add "OBJMACHINE=yes" to /etc/mk.conf 470 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 471 d) cd /usr/src ; make build 472 Note that running "make obj" in a directory will create 473 in obj.$MACHINE directory. 474 Build to a DESTDIR: 475 This helps to keep old 476 installed files (especially libraries) from interfering 477 with the new build. 478 To build to a DESTDIR, set the DESTDIR environment 479 variable before running make build. It should be set to 480 the pathname of an initially empty directory. 481 Problems: you might need to update critical utilities 482 without using DESTDIR since nothing is executed 483 from what is installed in DESTDIR. 484 (See critical utils, below) 485 Build often: 486 This keeps critical utilities current enough to not choke 487 on any other part of the source tree that depends on up to 488 date functionality. 489 490What to do if things don't work: 491^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 492When things don't work there is usually a few things that commonly 493should be done. 494 1) make includes 495 This should be done automatically by make build. 496 2) cd share/mk && make install 497 Again, automatically done by make build. 498 499Failsafe rebuild of a small part of the tree: 500^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 501To make sure you rebuild something correctly you want to do 502something like the following: 503 1) Make sure the includes and .mk files are up to date. 504 2) Make sure any program used to build the particular 505 utility is up to date. (yacc, lex, etc...) 506 3) cd ...path/to/util... 507 make cleandir 508 rm ...all obj directories... 509 make cleandir # yes, again 510 make obj 511 make depend && make 512 513Failsafe rebuild of the entire tree: 514^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 515If you really want to make sure the source tree is clean and 516ready for a build try the following. Note that sourcing /etc/mk.conf 517(a make(1) Makefile) in this manner is not right, and will not work 518for anyone who uses any make(1) features in /etc/mk.conf. 519 520---cut here--- 521#!/bin/sh 522. /etc/mk.conf 523 524if [ -z $NETBSDSRCDIR ] ; then 525 NETBSDSRCDIR=/usr/src 526fi 527if [ \! -d $NETBSDSRCDIR ] ; then 528 echo Unable to find sources 529 exit 1 530fi 531find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 532 533if [ -z $BSDOBJDIR ] ; then 534 BSDOBJDIR=/usr/obj 535fi 536if [ -d $BSDOBJDIR ] ; then 537 rm -rf $BSDOBJDIR 538fi 539 540cd $NETBSDSRCDIR && make cleandir 541 542---cut here--- 543 544Critical utilities: 545^^^^^^^^^^^^^^^^^^^ 546 gnu/usr.bin/egcs 547 usr.bin/compile_et 548 usr.bin/make 549 usr.bin/yacc 550 usr.bin/lex 551 usr.bin/xlint 552 usr.sbin/config 553 554Other problems and possible solutions: 555^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 556Symptom:Unreasonable compiler errors. 557Fix: Rebuild gnu/usr.bin/egcs 558 559Symptom:Complaints involving a Makefile. 560Fix: Rebuild usr.bin/make: 561 cd usr.bin/make && make && make install 562 Or, a failsafe method if that doesn't work: 563 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 564 565Fix: Make sure .mk files are up to date. 566 cd share/mk && make install 567 568Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 569Fix: Rebuild usr.sbin/config 570 571Symptom: 572Fix: Rebuild usr.bin/yacc 573 574Symptom: 575Fix: Rebuild usr.bin/lex 576 577Symptom: 578Fix: rm /usr/lib/libbfd.a 579 580Symptom:Obsolete intermediate files are used during compilation 581Fix: Try the following sequence of commands in the directory in question. 582 make cleandir; rm `make print-objdir`; make cleandir; make obj 583 (If you built the tree without "make obj" in the past, obsolete files 584 may remain. The command tries to clean everything up) 585 586Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 587Fix: Rebuild and install usr.bin/menuc 588 589Symptom:mklocale not found during build in share/locale/ctype 590Fix: Build and install usr.bin/mklocale 591 592Symptom:undefined reference to `__assert13' or `__unsetenv13' 593Fix: Rebuild and install lib/libc 594 595Symptom:usr.sbin/config fails to build. 596Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 597 598Symptom:undefined reference to `getprogname' or `setprogname' 599Fix: Rebuild and install lib/libc 600 601Symptom:lint does not understand the '-X' option 602Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 603