UPDATING revision 1.80
1$NetBSD: UPDATING,v 1.80 2002/12/19 12:48:42 lukem Exp $ 2 3This file is intended to be a brief introduction to the build 4process and a reference on what to do if something doesn't work. 5 6For a more detailed description see Makefile. 7 8Recent changes: 9^^^^^^^^^^^^^^^ 10 1120021219: 12 install(1) had a '-N dbdir' option added, to specify an 13 alternate location to look up users & groups (instead 14 of the host system passwd(5) and group(5) databases). 15 16 The build system was modified to take advantage of 17 this option (using ${NETBSDSRCDIR}/etc), so if you 18 use USE_TOOLS==no, you may have to rebuild and 19 reinstall usr.bin/xinstall first. 20 2120021130: 22 fparseln(3) moved from libutil to libc. 23 If building to DESTDIR=/, reinstall the includes 24 and rebuild libc: 25 make includes 26 make do-lib-libc 27 If using build.sh, "cd tools/compat && make clean" 28 before rebuilding the tools. 29 3020021126: 31 The mk.conf(5) variable SYS_INCLUDE has been deprecated, 32 including the optional "SYS_INCLUDE=symlinks" support. 33 All header files, including <sys/*.h> are copied into 34 /usr/include. 35 3620021121: 37 The C run-time support files crtbegin.o and crtend.o 38 (and their companions crtbeginS.o and crtendS.o) were 39 split up, with new crti.o and crtn.o files resulting. 40 This means that libtool needs to be rebuilt once the 41 new libraries are installed. The process of rebuilding 42 libtool will cause it to automatically notice the new 43 required files, but it *must* be rebuilt in order to 44 do this. 45 46 An out-of-date libtool will result in shared libraries 47 which lack _init() and _fini() routines, which means that 48 their global contructors/destructors will not be invoked. 49 5020021121: 51 A bug related to how ARM ELF objects were tagged has been 52 corrected. 53 54 NetBSD ARM ELF uses the soft-VFP floating point model by 55 default. However, the assembler lacked support for marking 56 objects as using the VFP floating point format, and the 57 compiler was not properly passing the flag indicating "soft-VFP" 58 to the assembler. 59 60 Unfortunately, this means that the linker will now consider 61 old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be 62 incompatible with new (properly marked) objects. 63 64 The problem will only manifest itself if you attempt to compile 65 a new program using the fixed toolchain, and link that program 66 against old libraries which do not have the proper "softvfp" 67 markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES 68 WILL CONTINUE TO WORK PROPERLY. 69 70 The only work-around for the problem is to recompile all of 71 the libraries on the system. The easiest way to do this for 72 system libraries is to install a binary snapshot; they are 73 generally available on releng.netbsd.org. Any packages you 74 have installed which supply libraries will have to be recompiled 75 if you wish to link new programs against those libraries. 76 77 If you have questions about this matter, please contact 78 port-arm@netbsd.org. 79 8020021011: 81 Systrace has been improved to support privilege elevation. 82 Updating the kernel requires the userland part of systrace 83 to be rebuilt. 84 8520021010: 86 The config(8) grammar was changed to allow options to register 87 dependencies on attributes, as well as other options. Users 88 must update and reinstall usr.sbin/config before building a new 89 kernel. 90 9120021009: 92 A new attribute dependency syntax was introduced to config(8), 93 which is now used by the SCSI configuration description. Users 94 must update and reinstall usr.sbin/config before building a new 95 kernel. 96 9720021003: 98 Several changes have been made to the autoconfiguration 99 framework. Users must update and reinstall usr.sbin/config 100 before building a new kernel. 101 10220021001: 103 The i386mp branch has been merged. To compile a kernel, users 104 will need to add the option 'cpu* at mainbus?' to their configuration 105 file. Multiprocessor kernels will need 106 ioapic* at mainbus? apid ? 107 options MULTIPROCESSOR 108 options COM_MPLOCK 109 11020020922: 111 MKDYNAMICROOT=yes enabled by default, which means that 112 certain shared libraries are installed into /lib, the shared 113 linker is installed into /libexec, and all programs in /bin 114 and /sbin are dynamically linked. 115 If you do not use "make build", you should ensure that 116 you have the libraries and shared linker in the new locations, 117 with: 118 make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 119 12020020917: 121 USE_NEW_TOOLCHAIN has been replaced with: 122 - TOOLCHAIN_MISSING -- set to "yes" on platforms for which 123 there is no working in-tree toolchain (hppa, ns32k, sh5, 124 x86_64). 125 - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the 126 root of an external toolchain (e.g. /usr/local/gnu). This 127 enables the cross-build framework even for TOOLCHAIN_MISSING 128 platforms. 129 13020020906: 131 gehenna-devsw has been merged into the trunk. Need to update and 132 reinstall usr.sbin/config before building the kernel. 133 13420020822: 135 Crunched rescue tools (contents of /bin and /sbin, plus others) 136 are now provided in /rescue. 137 138 To ensure that these are built statically linked (no matter 139 what the setting of LDSTATIC is), use a crunchgen(1) built 140 from sources newer than 20020820 (see the next entry). 141 14220020820: 143 crunchgen(1) changed to ensure that the generated program 144 is statically linked. 145 146 Solution: update and reinstall usr.bin/crunch 147 14820020515: 149 sshd user/group has been added. Need to hand add this in, or sshd 150 will not let you log in (with default, or UsePrivlegeSeparation=yes) 151 152 Add the following into /etc/group: 153 154 sshd:*:16: 155 156 and the following to /etc/master.passwd (via vipw): 157 158 sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin 159 160 Also /var/chroot/sshd directory needs to be present (digged as part of 161 the build process). 162 16320020426: 164 NBUILDJOBS obsoleted in favor of just using -j. 165 16620020426: 167 etc/postinstall added, which performs various checks for 168 configuration file updates and changes, and can fix most of 169 the problems identified. 170 This should make it much easier to upgrade a system's 171 configuration from earlier systems (as far back as NetBSD 1.5). 172 17320020320: 174 <bsd.lib.mk> needs a new install(1) for its "-a cmd" support. 175 Build and install at usr.bin/xinstall before the build. 176 17720020319: 178 Raw IPv6 socket now makes strict checking for sa_family and sa_len 179 on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold 180 newer than November 2001 when you upgrade the kernel. 181 18220020311: 183 ssh configuration files were moved from /etc to /etc/ssh. Beware 184 if you restart your machine from remote. Note that sshd.conf needs 185 to be changed (due to the use of "/etc" inside). 186 18720020223: 188 Users of the VAX port will need to rebuild and install gas 189 so it deal with the now present register prefix used in all 190 the VAX assembly files. 191 19220020118: 193 ntpd user/group has been added. Need to hand add this in or builds 194 will break as mtree aborts early. 195 196 Add the following into /etc/group: 197 198 ntpd:*:15: 199 200 and the following to /etc/master.passwd (via vipw): 201 202 ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 203 20420011207: 205 If you're attempting to build a snapshot on sparc64 and are getting 206 reloc errors from the toolchain groff binary this means your native 207 toolchain has some broken C++ bits. 208 209 To fix: 210 211 Build a new toolchain (i.e. build.sh -t) 212 Use the new toolchain to build and install natively (i.e. /usr/lib) 213 214 gnu/lib/libgcc 215 gnu/lib/libstdc++ 216 217 After this a snapshot will be able to be built. 218 21920011201: 220 In order for a sparc64 build to work you must have a working awk. If 221 you've built and installed a system with the new toolchain up to this 222 point you do not have a working awk as its ability to do floating 223 point is broken. 224 225 To build: 226 227 remake and install gnu/lib/libgcc 228 remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links 229 against the new libgcc.a) 230 23120011128: 232 Kernel config information was changed to use defflag in 233 the various "files" files. Bug fixes to config(8) are 234 required in order for this to work properly. Make sure 235 to build and install in usr.sbin/config before attempting 236 to build a new kernel. 237 23820011030: 239 libc/locale/wcstod.c now needs new lint(1). Update lint(1) 240 before building libc. 241 24220011029: 243 The new document BUILDING.mdoc (view with nroff | more, or 244 see pre-generated .txt and .html versions) describes the build 245 procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN 246 build process, are intended in the long run to replace this 247 manual update log. 248 249 Users building a USE_NEW_TOOLCHAIN system should read the 250 BUILDING document for caveats. Generally, BUILDING supersedes 251 UPDATING for these systems, as tool updating is taken care of 252 by the new build system. 253 25420011028: 255 src/etc/Makefile now needs install to be able to handle 256 symlinks that point to nowhere. A bug in install that 257 prevented this was corrected. 258 259 Solution: update and reinstall usr.bin/xinstall 260 Better Solution: Use the new toolchain and it will just work 261 for you. 262 26320011006: 264 /etc/mtree/NetBSD.dist has been updated to take advantage of 265 absolute path support added to mtree(8). Older mtree(8)s don't 266 understand the format. 267 268 Solution: update and reinstall usr.sbin/mtree 269 27020011004: 271 Crunchgen has been updated to work via reach-over makefiles. Updating 272 is suggested before running a snapshot build 273 27420010915: 275 The new "ubcperf" code committed by Chuck Silvers removed 276 a header file, uvm/uvm_vnode.h. There may be stale .depend 277 files that still reference this file. 278 279 Solution: "make cleandir && make dependall" in affected 280 directories. 281 28220010803: 283 grep.info is now built from grep.texi using makeinfo. Since it 284 requires makeinfo v4.0, you need to install new texinfo before 285 building gnu/usr.bin/grep. To install new texinfo, please follow 286 the instruction described in 20010726 entry. 287 28820010803: 289 (i386 only): i386 kernel now uses new instructions like 290 `fxsave' which old gas doesn't understand. To build the 291 kernel successfully, you need to build and install a new toolchain, 292 (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" 293 from your kernel configuration until you rebuild your userland. 294 See 20011029 above and BUILDING file in this directory for more information. 295 [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 296 29720010731: 298 Bootloader update on ELF platforms. DDB in kernels from before 299 this will be unable to read symbol tables provided by newer 300 bootloaders. 301 30220010726: 303 Texinfo was updated to 4.0. To avoid failures when trying to 304 build the included texinfo files, do: 305 306 cd src/gnu/usr.bin/texinfo 307 make MKINFO=no dependall install 308 30920010718: 310 Enabled correct .init/.fini processing in crt0. The way this 311 was done was to change a -I directive to cc(1), which means 312 make(1) will have a stale dependency (it will be checking the 313 timestamp on the wrong "dot_init.h"). 314 315 The symptom you will see is that new programs die with SIGSEGV 316 if you have a stale dependency. 317 318 Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so 319 before starting your build. 320 32120010628: 322 A construct was added to uvm_page.h that uncovered a bug 323 in lint(1). If you get a warning/error about a non-portable 324 bitfield, update your lint(1) before proceeding. 325 32620010226: 327 Added named user/group to system. Need to hand add this in or builds 328 will break as mtree aborts early. 329 330 To work around add by hand: 331 332 named:*:14: 333 334 to /etc/group and add: 335 336 named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin 337 338 to master.passwd (use vipw for instance if doing by hand). 339 340 Now a make build should progress. 341 34220010219: 343 get/setprogname() added. Any hostprogs that may use this will need 344 to be bootstrapped manually until the host system is current. 345 346 Known problems: sys/arch/macppc/stand/fixcoff 347 usr.sbin/config (adding -DMAKE_BOOTSTRAP to 348 CFLAGS and rebuilding should work) 349 usr.sbin/mdsetimage - Build a static copy if 350 building a snapshot before fully bootstrapped. 351 35220010204: 353 prepare the code to compile with stricter gcc flags. in 354 particular start eliminating redundant declarations. Yacc 355 needs to be installed before make build. 356 35720010114: 358 introduce .if commands(target) in make(1). You need to 359 bring everything up-to-date first, then without installing 360 anything make and install in usr.bin/make, then proceed 361 with make build. 362 36320010101: 364 bsd.subdir.mk committed 20001230 had a bug which caused 365 afterinstall targets to run too soon; update again. 366 36720001230: 368 New share/mk files needed to support .WAIT in SUBDIR variables. 369 If you get make errors, 370 (cd share/mk; make install) 371 Also, PRINTOBJDIR has changed and is now used more heavily. 372 37320001019: 374 The `ca' device driver has been replaced by `ld'; although the 375 major and minor numbers haven't changed, you should update your /dev 376 directory. 377 37820000929: 379 The following make directives are obsoleted. 380 MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA 381 By default, RSA is built into libcrypto. IDEA and RC5 will not be 382 built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build 383 additional library libcrypto_{idea,rc5}. 384 385 386Hints for a more successful build: 387^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 388 Build a new kernel first: 389 This makes sure that any new system calls or features 390 expected by the new userland will be present. This 391 helps to avoid critical errors when upgrading. 392 Use object directories: 393 This helps to keep stale object 394 files from polluting the build if a Makefile "forgets" 395 about one. It also makes it easier to clean up after 396 a build. It's also necessary if you want to use the 397 same source tree for multiple machines. 398 To use object directories: 399 a) cd /usr/src ; make cleandir 400 b) Add "OBJMACHINE=yes" to /etc/mk.conf 401 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 402 d) cd /usr/src ; make build 403 Note that running "make obj" in a directory will create 404 in obj.$MACHINE directory. 405 Build to a DESTDIR: 406 This helps to keep old 407 installed files (especially libraries) from interfering 408 with the new build. 409 To build to a DESTDIR, set the DESTDIR environment 410 variable before running make build. It should be set to 411 the pathname of an initially empty directory. 412 Problems: you might need to update critical utilities 413 without using DESTDIR since nothing is executed 414 from what is installed in DESTDIR. 415 (See critical utils, below) 416 Build often: 417 This keeps critical utilities current enough to not choke 418 on any other part of the source tree that depends on up to 419 date functionality. 420 421What to do if things don't work: 422^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 423When things don't work there is usually a few things that commonly 424should be done. 425 1) make includes 426 This should be done automatically by make build. 427 2) cd share/mk && make install 428 Again, automatically done by make build. 429 430Failsafe rebuild of a small part of the tree: 431^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 432To make sure you rebuild something correctly you want to do 433something like the following: 434 1) Make sure the includes and .mk files are up to date. 435 2) Make sure any program used to build the particular 436 utility is up to date. (yacc, lex, etc...) 437 3) cd ...path/to/util... 438 make cleandir 439 rm ...all obj directories... 440 make cleandir # yes, again 441 make obj 442 make depend && make 443 444Failsafe rebuild of the entire tree: 445^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 446If you really want to make sure the source tree is clean and 447ready for a build try the following. Note that sourcing /etc/mk.conf 448(a make(1) Makefile) in this manner is not right, and will not work 449for anyone who uses any make(1) features in /etc/mk.conf. 450 451---cut here--- 452#!/bin/sh 453. /etc/mk.conf 454 455if [ -z $NETBSDSRCDIR ] ; then 456 NETBSDSRCDIR=/usr/src 457fi 458if [ \! -d $NETBSDSRCDIR ] ; then 459 echo Unable to find sources 460 exit 1 461fi 462find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 463 464if [ -z $BSDOBJDIR ] ; then 465 BSDOBJDIR=/usr/obj 466fi 467if [ -d $BSDOBJDIR ] ; then 468 rm -rf $BSDOBJDIR 469fi 470 471cd $NETBSDSRCDIR && make cleandir 472 473---cut here--- 474 475Critical utilities: 476^^^^^^^^^^^^^^^^^^^ 477 gnu/usr.bin/egcs 478 usr.bin/compile_et 479 usr.bin/make 480 usr.bin/yacc 481 usr.bin/lex 482 usr.bin/xlint 483 usr.sbin/config 484 485Other problems and possible solutions: 486^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 487Symptom:Unreasonable compiler errors. 488Fix: Rebuild gnu/usr.bin/egcs 489 490Symptom:Complaints involving a Makefile. 491Fix: Rebuild usr.bin/make: 492 cd usr.bin/make && make && make install 493 Or, a failsafe method if that doesn't work: 494 cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin 495 496Fix: Make sure .mk files are up to date. 497 cd share/mk && make install 498 499Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 500Fix: Rebuild usr.sbin/config 501 502Symptom: 503Fix: Rebuild usr.bin/yacc 504 505Symptom: 506Fix: Rebuild usr.bin/lex 507 508Symptom: 509Fix: rm /usr/lib/libbfd.a 510 511Symptom:Obsolete intermediate files are used during compilation 512Fix: Try the following sequence of commands in the directory in question. 513 make cleandir; rm `make print-objdir`; make cleandir; make obj 514 (If you built the tree without "make obj" in the past, obsolete files 515 may remain. The command tries to clean everything up) 516 517Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type 518Fix: Rebuild and install usr.bin/menuc 519 520Symptom:mklocale not found during build in share/locale/ctype 521Fix: Build and install usr.bin/mklocale 522 523Symptom:undefined reference to `__assert13' 524Fix: Rebuild and install lib/libc 525 526Symptom:usr.sbin/config fails to build. 527Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 528 529Symptom:undefined reference to `getprogname' or `setprogname' 530Fix: Rebuild and install lib/libc 531 532Symptom:lint does not understand the '-X' option 533Fix: May need to build & install libs with NOLINT=1 before rebuilding lint 534