GC old code that was originally implemented to facilitate adoption by
other BSDs, which never happened, so make things simpler for further
development.

Ditches config.h entirely since it gets reduced to 2 defines.

okay tb@

Remove unused variables; ok millert@

spelling fixes; from paul tagliamonte
amendments to his diff are noted on tech

introduce a "Buf_Reinit" function for handling static buffers that can
be reused throughout running make. Instead of recreating buffers of
dubious appropriate size, have one single buffer for various stages of
parsing.

rename a few variable/functions to have better names.
adjust comments to be more meaningful
reorder predecessors/successors fields in an order that
makes more sense to me.

okay millert@

GC old cruft.
noticed and patch Michael W. Bombardieri

fix two nasty out-of-bound accesses that killed (sporadically mk38 and mk38bis.
prodded by bluhm@, okay bluhm@

small obvious cleanups:
- remove a lot of unnecessary casts
- zap extra param that's no longer needed
- add proper prototype and make function static

okay natano@

Use %zu to print size_t's rather than casting to u_long.

ok espie@

switch to <limits.h>; ok millert

If some mythical system lacks PATH_MAX, do not set it to MAXPATHLEN+1
ok millert

adjust to ohash being in libutil now, and to the interface changes.
fix potential integer overflows in memory allocation (mostly for pedagogical
purposes, these are unlikely to overflow in practice)
move the rest of lst.lib stuff into its own directory.

wrappers for is*/to* that make damn sure their arguments are
cast to unsigned chars.

okay deraadt@

remove TIMESTAMP abstraction layer, prodded by theo.
while there, clean up includes.
use strtoll for ar timestamps (pretty much unused in reality, more
standard conforming than anything)

use idea from Todd to adapt to time_t being 32 bits OR 64 bits
(pedantically correct: INT_MIN would work just fine up to 1910 or so...)

okay millert@, gone thru a make build.

more changes, discussed and tested by various people.
- put back some job control, turns out it's necessary when we don't run a
shell.
- zap old #ifdef CLEANUP code... probably doesn't even compile.
- kill most of the OP_LIB code. Just keep a wee little bit for compatibility
(deprecated .LIBS and .INCLUDES, warns for weird dependencies instead of
erroring out).
- much improved debugging and -p output: sort variables, targets, rules,
output stuff in a nicer format mimicing input.
- better error message when no command is found, explain where the target comes from.
- sort final error list by file.
- show system files in errors as <bsd.prog.mk>
- reincorporate random delay, that was dropped
- optimize siginfo output by not regenerating the whole string each time.
- finish zapping old LocationInfo field that's no longer used.

Correct $OpenBSD$ stuff

two small changes:
- allow variables in SysV modifiers, as requested by matthieu@
(since recursive variables are an extension, this just extends the
extension)
- variation on :Q called :QL (quote list), which does quote every character
EXCEPT for whitespace. e.g.,

toto:
@for i in ${VAR:QL} ...

Add a new default variable in Makefile context, MACHINE_CPU. Its value is
decided at compile-time and is either MACHINE_CPU from <machine/param.h> if
it is defined, or the same value as MACHINE_ARCH otherwise.
This will be used to allow ports with suffixes to their canonical MACHINE_ARCH
to provide this canonical name as MACHINE_CPU, and in turn to let Makefiles
do TRT.
ok kettenis@

modify timestamp handling to grab current time instead of relying on a
stale timestamp.

all those diffs sent to people ages ago, who didn't answer, except for
Theo, who said he ran it with no issues, so let's get this in...

simplify dynamic variable handling a great deal:
first remove all usage of Varq_Append by building the string directly.
then replace `common' handling with specialized handling for dynamic
strings (since they no longer need a buffer). Finally, identify the place
where the variable value needs to be copied because it's going to be free'd
or erased soon, and finally, use simple char* pointers.

Shaves about 80 bytes off every gnode structure, and kills quite a few
unnecessary malloc()s as well.

rename make -> must_make, made -> built_status
to make them easier to find in source files.

#ifdef stuff that is only used under #ifdef; ok espie

Work done at p2k7.


This is a really big step towards getting parallel make to work.


Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).



Tested by lots of people, thanks go to miod@, and robert@ among other people.

Quick summary of what this does:

- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.

always define the svr4namelist. Doesn't hurt, even if we don't use it.

fix the arch code mostly.
- stop using sprintf for code that can use our buffers.
- stop modifying parsed line, rely on Var_Substi instead.

to do that, we need to go through an intermediate routine that uses
one single buffer to build archive names.
also introduce an add_archive_node function that makes things way simpler.
also rename a few variables.

kill extra spaces at end of line

rename dirSearchPath -> defaultPath, and openDirectories -> knownDirectories

minor reindent issues

rename Targ_FmtTime into time_to_string and move it, as it's not related
to target nodes at all (reduces modules inter-dependencies)

reuse free_hash

reindent arch.c

cut up dir.c into dir.c/direxpand.c as there is very little interface between
two modules that do different things.

more dead code: we always define LIBRARIES and INCLUDES

remove dead code: #define RECHECK is always on

everywhere except in Var_Parse, we always add/append variables to the
VAR_GLOBAL context, so make it the common case: rename the basic functions
to Var_Set_with_ctxt/Var_Append_with_ctxt, define Var_Set and Var_Append
as macros that specify VAR_GLOBAL, and use these.

okay miod@

simplify computations in the var module: instead of advancing a char *
and keeping track of a length, we just advance the char *, and ditch
the length. We can still get the length at the end of the top-level
functions to satisfy existing interfaces.

Much simpler code, less error-prone.

Okay millert@

move the code that grabs a value in Var_Parse in its own function,
get_expanded_value.

Extend the code a bit to be much more thorough in case of a recursive
expansion: shows exactly the cycle of variable names involved.

okay millert@

change Var_ParseSkip API to increment the position instead of returning a
length, simplifies code.

(warns a bit, symptom of some further issues to fix).

okay millert@

remove some bogus *p tests from charles longeau
ok deraadt millert

use stdint.h where appropriate. okay millert@

ISO function declarations, trim a few comments, rename a few variables to
more explicit/more consistent names.

okay otto@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

get rid of some strcpy/sprintf.
ok krw@, matthieu@, deraadt@

use defined(__ELF__) instead of a list of ELF architectures from which
sparc64 was missing.
This fixes a bug with ELF static libraries on sparc64

occured->occurred

idea from deraadt@ via NetBSD
millert@ ok

p.s. Next commit will fix a typo in the sys/

Better warning fix, in line with the rest of the code
(__attribute((__unused__)) denotes parameters that MAY be unused in the
code, e.g., it's mostly a `shut up warnings' device).

Use Str_concat instead of fixed buffers and snprintf in building paths.
Replace MAXPATHLEN with PATH_MAX (synch with op-make).
ok naddy@

-Wall cleanup, mostly useless otherwise

Take includes out of lst.h, re-add what's needed to separate files.
Removes remaining lint stuff from lst.lib.

Mostly clean-up:
- cut up those huge include files into separate interfaces for all modules.
Put the interface documentation there, and not with the implementation.
- light-weight includes for needed concrete types (lst_t.h, timestamp_t.h).
- cut out some more logically separate parts: cmd_exec, varname, parsevar,
timestamp.
- put all error handling functions together, so that we will be able to
clean them up.
- more systematic naming: functioni to handle interval, function to handle
string.
- put the init/end code apart to minimize coupling.
- kill weird types like ReturnStatus and Boolean. Use standard bool (with a
fallback for non-iso systems)
- better interface documentation for lots of subsystems.

As a result, make compilation goes somewhat faster (5%, even considering
the largish BSD copyrights to read). The corresponding preprocessed
source goes down from 1,5M to 1M.

A few minor code changes as well: Parse_DoVar is no longer destructive.
Parse_IsVar functionality is folded into Parse_DoVar (as it knows what an
assignment is), a few more interval handling functions. Avoid calling
XXX_End when they do nothing, just #define XXX_End to nothing.

Parse_DoVar is slightly more general: it will handle compound assignments
as long as they make sense, e.g., VAR +!= cmd
will work. As a side effect, VAR++=value now triggers an error
(two + in assignment).
- this stuff doesn't occur in portable Makefiles.
- writing VAR++ = value or VAR+ +=value disambiguates it.
- this is a good thing, it uncovered a bug in bsd.port.mk.

Tested by naddy@. Okayed millert@. I'll handle the fallback if there is
any. This went through a full make build anyways, including isakmpd
(without mickey's custom binutils, as he didn't see fit to share it with me).

Synch with my current work.
Numerous changes:
- generate can build several tables
- style cleanup
- statistics code
- use variable names throughout (struct Name)
- recursive variables everywhere
- faster parser (pass buffer along instead of allocating multiple copies)
- correct parser. Handles comments everywhere, and ; correctly
- more string intervals
- simplified dir.c, less recursion.
- extended for loops
- sinclude()
- finished removing extra junk from Lst_*
- handles ${@D} and friends in a simpler way
- cleaned up and modular VarModifiers handling.
- recognizes some gnu Makefile usages and errors out about them.

Additionally, some extra functionality is defined by FEATURES. The set of
functionalities is currently hardcoded to OpenBSD defaults, but this may
include support for some NetBSD extensions, like ODE modifiers.

Backed by miod@ and millert@, who finally got sick of my endless patches...

Use the ohash_* that's now in libc.

This patch fixes ArchFindMember so that it handles SVR4 members
similarly to read_archive.

Note we no longer bother seeking back to the start of the header, as only
ArchTouch accesses that header, and can do the seek itself.

With this, arch handling should be working, more or less.
thanks to Todd, Miod, Naddy for reviewing those patches.

This does fix some nasty issues: ar field members are NOT null-terminated.

Hence, read_archive must be very careful to parse stuff correctly: don't
use str* when mem* are appropriate, copy numeric fields and ensure they're
terminated...

We take of the SVR4 archive handling data structure from our archive,
and put it into a specific structure (it is wasteful to keep lugging
SVR4 structures once an archive is parsed).

By tweaking read_archive slightly, we can achieve a nicer interface
to ArchSVR4Entry.

Note a bug in make: ArchFindMember does (currently) not use the SVR4 code,
hence some archive members won't be found in non-caching mode...

Factorise duplicated code involved in touching archives.

As it stands now, arch.c does not work.
Its main failing is that ar headers are NOT null terminated.
This code is atrocious. This change starts cleaning that up.

Replace the list of cached archives with a hash,
streamline the logic of ArchMTimeMember,
by taking out the actual function that does the reading (read_archive).

More fixes to come.

Change the time stamp interface to use an abstract datatype.
Define two possible interfaces: the classic one,
and the new one (used where available) that depends on timespec.

Better granularity, make is now able to distinguish between files that
were built during the same second.

Fix Arch_MemMTime (previous change I did was bogus)

This kills the last old hashing table, in arch.c

Slight optimizations: instead of storing archive members, just keep
the modification time, as we don't care for the rest of the archive
information. Lazily compute mtime, stash ascii date instead, and convert
to mtime when needed (storing an out_of_date value to mark the unconverted
values).

Archive handling is atrocious and need some clean-up.

Thanks to miod@ who took the time to review those patches.

Introduce a few macros to handle timestamps in an abstract way.

Replace the time stamp hash in dir.c with an open hashing structure.

In doing so, remove some nasty casts, simplify code a bit:
Dir_MTime can return a modification time, since make does not make
a distinction between out-of-date and non-existent files.

Replace the old hash used to hold file names within a directory with
open hashing.

An interesting optimization is that the open hashing interface is more
fine-grained, hence we can compute the correct hash value at the start
of Dir_FindFile, and reuse it for each hash structure into which we look
(the effect is measurable on large directories along with objdir/VPATH).

Remove a few unnecessary Lst_Open/Lst_Close that serve no purpose except
obfuscating the code.

The interface to dir.h changes slightly, hence tedious includes changes...

Some systematic clean-up.
- UNUSED macro that expands to __attribute__((unused)) for gcc
- move rcsid around so that they can be tagged UNUSED.
- activate -Wunused.
- use UNUSED instead of kludgy junk for function arguments.
- add extern to all extern prototypes.
- update comments in lst.h.
- clean up var.c a little bit, constifying arguments, updating comments...

Once those special variable are taken care of, other Var functions can take
the GNode's context directly. We rename that special Lst to `SymTable *'
in prevision of things to come.

Along the line, we lose the special GNodes affected to VAR_CMD, VAR_GLOBAL,
VAR_ENV, which become simple Lsts... This is not a problem, except when
getting to a context's name for debugging (handled very nicely by
offsetof).

Again, this is a preparatory patch, which does not gain anything except
for cleaning up issues...

Reviewed by millert@ and miod@, like the previous patch

Start of variable fixes and speed-ups.

This patch may seem a bit non-sensical at first. It simply introduces some
new interface. Specifically, recognizes that some variable names
(.TARGET/$@, .OODATE/$?, .ALLSRC/$>, .IMPSRC/$<, .PREFIX/$*, .ARCHIVE/$!,
.MEMBER/$%) are `special' (the actual variables which are local to a
target, e.g. GNode).

Currently, The Varq functions (for Varquick access) are only stubs to the
normal functions.

This fixes a very important detail before proceeding to turn variable lists
into hash tables: if every GNode holds a hash table, initialization times
for those will be very costly. But generic GNodes only hold those seven
special variables... which can be stored directly into a small array;
the only general cases are the environment, the command line and
global variables.

Trivial consequences of the previous list changes:

- audit code for Lst_Datum, it's never applied to an empty pointer,
so check can be removed -> turn into a macro,
- Lst_First, Lst_Last can become macro as well
- specialized version of Lst_Succ (Lst_Adv) to use in loops where it cannot
fail,
- Lst_Open can no longer fail. Trim down corresponding code.

Reviewed millert@, miod@

This patch introduces a distinction between
Lst_Init (constructor) and Lst_New (allocation + construction)
Lst_Destroy (destructor) and Lst_Delete (deallocation + destruction),
and uses that to turn most dynamic allocation of lists (Lst pointers)
into static structures (LIST).

Most of this is mundane, except for allGNs in targ.c, where the code must
be checked to verify that Targ_Init is called soon enough.

Lst_New is a temporary addition. All lists will soon be static.

Reviewed by millert@, like the previous patch.

Clean-up patch: use `void *' instead of old-fashioned ClientData/Address.

Remove idiotic, braindead casts T* -> void*
They serve no purpose, except hiding potential bugs.

In particular, remove (ClientData) cast from macro, showing potentially
troublesome use of Hashes to store time_t.

Bug-fix: make should behave sensibly when presented with negative times...

- let *_MTime return booleans, as that's what they're used for, the time_t
is set as a side effect.

- use OUT_OF_DATE for a date starting point, set it at the origin of time.

Finish changing all Var_Parse arguments to size_t. Thanks to millert@
for reminding me.

Rearrange Lst_Find interface to conform better with other functions.

Nothing ever checks ReturnStatus on Lst_Insert, Lst_Append, Lst_AtFront,
Lst_AtEnd, Lst_Concat, Lst_Remove, Lst_Replace.

Don't bother returning one.

Lst_DeQueue already checks for empty lists.

NIL, NILGNODE, etc, are only glorified NULL.
Get rid of them.

Get rid of list.h, nothing uses it anyway.

make does not use circular lists, get rid of the extra weight.

Var_Subst is actually two distinct functions folded into one:
split the function specific to for.c out, and give them more sensible
arguments at the same time.

This makes .for loop handling more efficient, as we have some heuristic
to evaluate the size of the buffer needed...

Extra parameter no longer needed, ditch.

Efficiency patch from NetBSD:
make spends time freeing data structures right before exiting.
So don't bother.
Keep the code inside #ifdef, so that it's still there if someone
ever wants to use make code inside a library.

Fix some mips -> __mips__ stuff.

Modifications from netbsd:
- don't interfere with MACHINE/MACHINE_ARCH defines for bootstrap
- type clean-up, time_t, and printing `unknown' ints
- fix TARGET/MEMBER bug in archive rules
- memmove...
- cleaner Error handler.
- reentrant brk_string
- .MAKE env variable
- preliminary scaffolding for .NOPATH

Other improvements:
- efree
- shellneed streamlined
- display Stop in .CURDIR after an error.
- document most features and misfeatures.
- add a few OpenBSD notes to the tutorial.

Use SEEK_CUR as "whence" in fseek, not 1.

Fix a bug where make gets confused by targets beginning with a period (``.'')
and tried to do a suffix conversion, following a NULL pointer in the
proccess. Also add some sanity checks so we don't blindly assume
strchr returns non-NULL.

powerpc ELF

(foo *)NULL -> NULL

Sync with NetBSD:
- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
- Fix NetBSD PR #2930: declare missing variable.

Do not consider non-existent archives as up-to-date on non-a.out systems

Do not look for a.out t.o.c sections in the mips-based ports

Sync up with NetBSD:
(christos)
Fix bug reported by Greg Hudson where leaf (source only) nodes were
referenced only by their basename and not by their full pathname. This
breaks when .PATH or MAKEOBJDIR are used. There might be Makefiles
around that try to work around this bug by prepending ${.CURDIR} to
the sources, and they should be found and fixed. Also a lot of the gunk
in suff.c that was attempting to work around the same problem could be
removed.
(christos)
- Move -D flags from Makefile to config.h and explain what they do. Add
-Wall -Wno-unused to CFLAGS. Add new define SYSVVARSUB to enable SysV
style variable substitutions and enable them.
- Add SunOS style command substitutions via SUNSHCMD
- Fix core dump with '{variable = value'
(christos)
Fix bug where make will always exit with 0, even when one or more
parallel jobs failed. (Only affects parallel make code)
(christos)
Protect __P from being multiply defined (for systems that already
define it)
(christos) Add strdup() since ultrix is missing it.
From Larry Schwimmer <rosebud@cyclone.Stanford.EDU>
(christos) Add estrdup(), a checked version of strdup and use it.
(christos) Recognize SVR4 style long filename entries in archives.
(thorpej) Tidy up some RCS ids a bit.

RANLIBMAG should not be set on alphas, as make doesn't know ECOFF archives

From NetBSD: merge of 960317

From NetBSD:
Support SVR4 style archives.
Fix pr/1421 (from Matthew Green) and pr/1997 (from Jeff Thieleke).
In ParseDoInclude(), make a temporary copy of the current file name
while searching for ""-type include files, since the current file name
might not be a writeable string.

from christos@netbsd:
Minor:
- ${.PREFIX} should never contain a full pathname
- Fixed gcc -Wall warnings
Major:
- compatMake is now FALSE. This means that we are now running in
full pmake mode:
* rules on dependency lines can be executed in parallel and or
out of sequence:

foo: bar baz

can fire the rule for baz before the rule for bar is fired.
To enforce bar to be fired before baz, another rule needs to be
added. [bar: baz]
* adjacent shell commands in a target are now executed by a single
invocation of the shell, not one invocation of the shell per line
(compatMake can be turned off using the -B flag)
- The -j flag now works... I.e. make -j 4 will fork up to four jobs in
parallel when it can. The target name is printed before each burst
of output caused by the target execution as '--- target ---', when j > 1
- I have changed all the Makefiles so that they work with make -j N, and
I have tested the whole netbsd by:
'make -j 4 cleandir; make -j 4 depend; make -j 4; make -j 4 install'
- I have not compiled or tested this version of make with -DREMOTE.
- Turn compat mode on by default. It gets turned off when the -j without
the -B flag is specified. [Thus you can use -j 1 to turn it off].
- Fix malloc -> emalloc as Gordon noted.
Updates for POSIX/SVR4 compiling:
arch.c: Don't require ranlib stuff. Not everybody has it.
dir.c: SunOS-4 != Solaris; change #ifdef sun to #if sun && !__svr4__
job.c, compat.c: Don't use 'union wait', use int and the W*() macros.
main.c: Check for uname() == -1; some unames return > 0...
util.c, job.c: Add signal() with BSD semantics for svr4, don't use bsd
sigmask and friends.

from cgd@netbsd:
pull in make.h. (PAlloc() now uses emalloc(), which is prototyped in
make.h. If the prototype is not in scope on the Alpha, I see lots
of "cast to pointer from integer of different size" warnings.)

branches: 1.1.1;
Initial revision

Remove unused variables; ok millert@

spelling fixes; from paul tagliamonte
amendments to his diff are noted on tech

introduce a "Buf_Reinit" function for handling static buffers that can
be reused throughout running make. Instead of recreating buffers of
dubious appropriate size, have one single buffer for various stages of
parsing.

rename a few variable/functions to have better names.
adjust comments to be more meaningful
reorder predecessors/successors fields in an order that
makes more sense to me.

okay millert@

GC old cruft.
noticed and patch Michael W. Bombardieri

fix two nasty out-of-bound accesses that killed (sporadically mk38 and mk38bis.
prodded by bluhm@, okay bluhm@

small obvious cleanups:
- remove a lot of unnecessary casts
- zap extra param that's no longer needed
- add proper prototype and make function static

okay natano@

Use %zu to print size_t's rather than casting to u_long.

ok espie@

switch to <limits.h>; ok millert

If some mythical system lacks PATH_MAX, do not set it to MAXPATHLEN+1
ok millert

adjust to ohash being in libutil now, and to the interface changes.
fix potential integer overflows in memory allocation (mostly for pedagogical
purposes, these are unlikely to overflow in practice)
move the rest of lst.lib stuff into its own directory.

wrappers for is*/to* that make damn sure their arguments are
cast to unsigned chars.

okay deraadt@

remove TIMESTAMP abstraction layer, prodded by theo.
while there, clean up includes.
use strtoll for ar timestamps (pretty much unused in reality, more
standard conforming than anything)

use idea from Todd to adapt to time_t being 32 bits OR 64 bits
(pedantically correct: INT_MIN would work just fine up to 1910 or so...)

okay millert@, gone thru a make build.

more changes, discussed and tested by various people.
- put back some job control, turns out it's necessary when we don't run a
shell.
- zap old #ifdef CLEANUP code... probably doesn't even compile.
- kill most of the OP_LIB code. Just keep a wee little bit for compatibility
(deprecated .LIBS and .INCLUDES, warns for weird dependencies instead of
erroring out).
- much improved debugging and -p output: sort variables, targets, rules,
output stuff in a nicer format mimicing input.
- better error message when no command is found, explain where the target comes from.
- sort final error list by file.
- show system files in errors as <bsd.prog.mk>
- reincorporate random delay, that was dropped
- optimize siginfo output by not regenerating the whole string each time.
- finish zapping old LocationInfo field that's no longer used.

Correct $OpenBSD$ stuff

two small changes:
- allow variables in SysV modifiers, as requested by matthieu@
(since recursive variables are an extension, this just extends the
extension)
- variation on :Q called :QL (quote list), which does quote every character
EXCEPT for whitespace. e.g.,

toto:
@for i in ${VAR:QL} ...

Add a new default variable in Makefile context, MACHINE_CPU. Its value is
decided at compile-time and is either MACHINE_CPU from <machine/param.h> if
it is defined, or the same value as MACHINE_ARCH otherwise.
This will be used to allow ports with suffixes to their canonical MACHINE_ARCH
to provide this canonical name as MACHINE_CPU, and in turn to let Makefiles
do TRT.
ok kettenis@

modify timestamp handling to grab current time instead of relying on a
stale timestamp.

all those diffs sent to people ages ago, who didn't answer, except for
Theo, who said he ran it with no issues, so let's get this in...

simplify dynamic variable handling a great deal:
first remove all usage of Varq_Append by building the string directly.
then replace `common' handling with specialized handling for dynamic
strings (since they no longer need a buffer). Finally, identify the place
where the variable value needs to be copied because it's going to be free'd
or erased soon, and finally, use simple char* pointers.

Shaves about 80 bytes off every gnode structure, and kills quite a few
unnecessary malloc()s as well.

rename make -> must_make, made -> built_status
to make them easier to find in source files.

#ifdef stuff that is only used under #ifdef; ok espie

Work done at p2k7.


This is a really big step towards getting parallel make to work.


Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).



Tested by lots of people, thanks go to miod@, and robert@ among other people.

Quick summary of what this does:

- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.

always define the svr4namelist. Doesn't hurt, even if we don't use it.

fix the arch code mostly.
- stop using sprintf for code that can use our buffers.
- stop modifying parsed line, rely on Var_Substi instead.

to do that, we need to go through an intermediate routine that uses
one single buffer to build archive names.
also introduce an add_archive_node function that makes things way simpler.
also rename a few variables.

kill extra spaces at end of line

rename dirSearchPath -> defaultPath, and openDirectories -> knownDirectories

minor reindent issues

rename Targ_FmtTime into time_to_string and move it, as it's not related
to target nodes at all (reduces modules inter-dependencies)

reuse free_hash

reindent arch.c

cut up dir.c into dir.c/direxpand.c as there is very little interface between
two modules that do different things.

more dead code: we always define LIBRARIES and INCLUDES

remove dead code: #define RECHECK is always on

everywhere except in Var_Parse, we always add/append variables to the
VAR_GLOBAL context, so make it the common case: rename the basic functions
to Var_Set_with_ctxt/Var_Append_with_ctxt, define Var_Set and Var_Append
as macros that specify VAR_GLOBAL, and use these.

okay miod@

simplify computations in the var module: instead of advancing a char *
and keeping track of a length, we just advance the char *, and ditch
the length. We can still get the length at the end of the top-level
functions to satisfy existing interfaces.

Much simpler code, less error-prone.

Okay millert@

move the code that grabs a value in Var_Parse in its own function,
get_expanded_value.

Extend the code a bit to be much more thorough in case of a recursive
expansion: shows exactly the cycle of variable names involved.

okay millert@

change Var_ParseSkip API to increment the position instead of returning a
length, simplifies code.

(warns a bit, symptom of some further issues to fix).

okay millert@

remove some bogus *p tests from charles longeau
ok deraadt millert

use stdint.h where appropriate. okay millert@

ISO function declarations, trim a few comments, rename a few variables to
more explicit/more consistent names.

okay otto@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

get rid of some strcpy/sprintf.
ok krw@, matthieu@, deraadt@

use defined(__ELF__) instead of a list of ELF architectures from which
sparc64 was missing.
This fixes a bug with ELF static libraries on sparc64

occured->occurred

idea from deraadt@ via NetBSD
millert@ ok

p.s. Next commit will fix a typo in the sys/

Better warning fix, in line with the rest of the code
(__attribute((__unused__)) denotes parameters that MAY be unused in the
code, e.g., it's mostly a `shut up warnings' device).

Use Str_concat instead of fixed buffers and snprintf in building paths.
Replace MAXPATHLEN with PATH_MAX (synch with op-make).
ok naddy@

-Wall cleanup, mostly useless otherwise

Take includes out of lst.h, re-add what's needed to separate files.
Removes remaining lint stuff from lst.lib.

Mostly clean-up:
- cut up those huge include files into separate interfaces for all modules.
Put the interface documentation there, and not with the implementation.
- light-weight includes for needed concrete types (lst_t.h, timestamp_t.h).
- cut out some more logically separate parts: cmd_exec, varname, parsevar,
timestamp.
- put all error handling functions together, so that we will be able to
clean them up.
- more systematic naming: functioni to handle interval, function to handle
string.
- put the init/end code apart to minimize coupling.
- kill weird types like ReturnStatus and Boolean. Use standard bool (with a
fallback for non-iso systems)
- better interface documentation for lots of subsystems.

As a result, make compilation goes somewhat faster (5%, even considering
the largish BSD copyrights to read). The corresponding preprocessed
source goes down from 1,5M to 1M.

A few minor code changes as well: Parse_DoVar is no longer destructive.
Parse_IsVar functionality is folded into Parse_DoVar (as it knows what an
assignment is), a few more interval handling functions. Avoid calling
XXX_End when they do nothing, just #define XXX_End to nothing.

Parse_DoVar is slightly more general: it will handle compound assignments
as long as they make sense, e.g., VAR +!= cmd
will work. As a side effect, VAR++=value now triggers an error
(two + in assignment).
- this stuff doesn't occur in portable Makefiles.
- writing VAR++ = value or VAR+ +=value disambiguates it.
- this is a good thing, it uncovered a bug in bsd.port.mk.

Tested by naddy@. Okayed millert@. I'll handle the fallback if there is
any. This went through a full make build anyways, including isakmpd
(without mickey's custom binutils, as he didn't see fit to share it with me).

Synch with my current work.
Numerous changes:
- generate can build several tables
- style cleanup
- statistics code
- use variable names throughout (struct Name)
- recursive variables everywhere
- faster parser (pass buffer along instead of allocating multiple copies)
- correct parser. Handles comments everywhere, and ; correctly
- more string intervals
- simplified dir.c, less recursion.
- extended for loops
- sinclude()
- finished removing extra junk from Lst_*
- handles ${@D} and friends in a simpler way
- cleaned up and modular VarModifiers handling.
- recognizes some gnu Makefile usages and errors out about them.

Additionally, some extra functionality is defined by FEATURES. The set of
functionalities is currently hardcoded to OpenBSD defaults, but this may
include support for some NetBSD extensions, like ODE modifiers.

Backed by miod@ and millert@, who finally got sick of my endless patches...

Use the ohash_* that's now in libc.

This patch fixes ArchFindMember so that it handles SVR4 members
similarly to read_archive.

Note we no longer bother seeking back to the start of the header, as only
ArchTouch accesses that header, and can do the seek itself.

With this, arch handling should be working, more or less.
thanks to Todd, Miod, Naddy for reviewing those patches.

This does fix some nasty issues: ar field members are NOT null-terminated.

Hence, read_archive must be very careful to parse stuff correctly: don't
use str* when mem* are appropriate, copy numeric fields and ensure they're
terminated...

We take of the SVR4 archive handling data structure from our archive,
and put it into a specific structure (it is wasteful to keep lugging
SVR4 structures once an archive is parsed).

By tweaking read_archive slightly, we can achieve a nicer interface
to ArchSVR4Entry.

Note a bug in make: ArchFindMember does (currently) not use the SVR4 code,
hence some archive members won't be found in non-caching mode...

Factorise duplicated code involved in touching archives.

As it stands now, arch.c does not work.
Its main failing is that ar headers are NOT null terminated.
This code is atrocious. This change starts cleaning that up.

Replace the list of cached archives with a hash,
streamline the logic of ArchMTimeMember,
by taking out the actual function that does the reading (read_archive).

More fixes to come.

Change the time stamp interface to use an abstract datatype.
Define two possible interfaces: the classic one,
and the new one (used where available) that depends on timespec.

Better granularity, make is now able to distinguish between files that
were built during the same second.

Fix Arch_MemMTime (previous change I did was bogus)

This kills the last old hashing table, in arch.c

Slight optimizations: instead of storing archive members, just keep
the modification time, as we don't care for the rest of the archive
information. Lazily compute mtime, stash ascii date instead, and convert
to mtime when needed (storing an out_of_date value to mark the unconverted
values).

Archive handling is atrocious and need some clean-up.

Thanks to miod@ who took the time to review those patches.

Introduce a few macros to handle timestamps in an abstract way.

Replace the time stamp hash in dir.c with an open hashing structure.

In doing so, remove some nasty casts, simplify code a bit:
Dir_MTime can return a modification time, since make does not make
a distinction between out-of-date and non-existent files.

Replace the old hash used to hold file names within a directory with
open hashing.

An interesting optimization is that the open hashing interface is more
fine-grained, hence we can compute the correct hash value at the start
of Dir_FindFile, and reuse it for each hash structure into which we look
(the effect is measurable on large directories along with objdir/VPATH).

Remove a few unnecessary Lst_Open/Lst_Close that serve no purpose except
obfuscating the code.

The interface to dir.h changes slightly, hence tedious includes changes...

Some systematic clean-up.
- UNUSED macro that expands to __attribute__((unused)) for gcc
- move rcsid around so that they can be tagged UNUSED.
- activate -Wunused.
- use UNUSED instead of kludgy junk for function arguments.
- add extern to all extern prototypes.
- update comments in lst.h.
- clean up var.c a little bit, constifying arguments, updating comments...

Once those special variable are taken care of, other Var functions can take
the GNode's context directly. We rename that special Lst to `SymTable *'
in prevision of things to come.

Along the line, we lose the special GNodes affected to VAR_CMD, VAR_GLOBAL,
VAR_ENV, which become simple Lsts... This is not a problem, except when
getting to a context's name for debugging (handled very nicely by
offsetof).

Again, this is a preparatory patch, which does not gain anything except
for cleaning up issues...

Reviewed by millert@ and miod@, like the previous patch

Start of variable fixes and speed-ups.

This patch may seem a bit non-sensical at first. It simply introduces some
new interface. Specifically, recognizes that some variable names
(.TARGET/$@, .OODATE/$?, .ALLSRC/$>, .IMPSRC/$<, .PREFIX/$*, .ARCHIVE/$!,
.MEMBER/$%) are `special' (the actual variables which are local to a
target, e.g. GNode).

Currently, The Varq functions (for Varquick access) are only stubs to the
normal functions.

This fixes a very important detail before proceeding to turn variable lists
into hash tables: if every GNode holds a hash table, initialization times
for those will be very costly. But generic GNodes only hold those seven
special variables... which can be stored directly into a small array;
the only general cases are the environment, the command line and
global variables.

Trivial consequences of the previous list changes:

- audit code for Lst_Datum, it's never applied to an empty pointer,
so check can be removed -> turn into a macro,
- Lst_First, Lst_Last can become macro as well
- specialized version of Lst_Succ (Lst_Adv) to use in loops where it cannot
fail,
- Lst_Open can no longer fail. Trim down corresponding code.

Reviewed millert@, miod@

This patch introduces a distinction between
Lst_Init (constructor) and Lst_New (allocation + construction)
Lst_Destroy (destructor) and Lst_Delete (deallocation + destruction),
and uses that to turn most dynamic allocation of lists (Lst pointers)
into static structures (LIST).

Most of this is mundane, except for allGNs in targ.c, where the code must
be checked to verify that Targ_Init is called soon enough.

Lst_New is a temporary addition. All lists will soon be static.

Reviewed by millert@, like the previous patch.

Clean-up patch: use `void *' instead of old-fashioned ClientData/Address.

Remove idiotic, braindead casts T* -> void*
They serve no purpose, except hiding potential bugs.

In particular, remove (ClientData) cast from macro, showing potentially
troublesome use of Hashes to store time_t.

Bug-fix: make should behave sensibly when presented with negative times...

- let *_MTime return booleans, as that's what they're used for, the time_t
is set as a side effect.

- use OUT_OF_DATE for a date starting point, set it at the origin of time.

Finish changing all Var_Parse arguments to size_t. Thanks to millert@
for reminding me.

Rearrange Lst_Find interface to conform better with other functions.

Nothing ever checks ReturnStatus on Lst_Insert, Lst_Append, Lst_AtFront,
Lst_AtEnd, Lst_Concat, Lst_Remove, Lst_Replace.

Don't bother returning one.

Lst_DeQueue already checks for empty lists.

NIL, NILGNODE, etc, are only glorified NULL.
Get rid of them.

Get rid of list.h, nothing uses it anyway.

make does not use circular lists, get rid of the extra weight.

Var_Subst is actually two distinct functions folded into one:
split the function specific to for.c out, and give them more sensible
arguments at the same time.

This makes .for loop handling more efficient, as we have some heuristic
to evaluate the size of the buffer needed...

Extra parameter no longer needed, ditch.

Efficiency patch from NetBSD:
make spends time freeing data structures right before exiting.
So don't bother.
Keep the code inside #ifdef, so that it's still there if someone
ever wants to use make code inside a library.

Fix some mips -> __mips__ stuff.

Modifications from netbsd:
- don't interfere with MACHINE/MACHINE_ARCH defines for bootstrap
- type clean-up, time_t, and printing `unknown' ints
- fix TARGET/MEMBER bug in archive rules
- memmove...
- cleaner Error handler.
- reentrant brk_string
- .MAKE env variable
- preliminary scaffolding for .NOPATH

Other improvements:
- efree
- shellneed streamlined
- display Stop in .CURDIR after an error.
- document most features and misfeatures.
- add a few OpenBSD notes to the tutorial.

Use SEEK_CUR as "whence" in fseek, not 1.

Fix a bug where make gets confused by targets beginning with a period (``.'')
and tried to do a suffix conversion, following a NULL pointer in the
proccess. Also add some sanity checks so we don't blindly assume
strchr returns non-NULL.

powerpc ELF

(foo *)NULL -> NULL

Sync with NetBSD:
- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
- Fix NetBSD PR #2930: declare missing variable.

Do not consider non-existent archives as up-to-date on non-a.out systems

Do not look for a.out t.o.c sections in the mips-based ports

Sync up with NetBSD:
(christos)
Fix bug reported by Greg Hudson where leaf (source only) nodes were
referenced only by their basename and not by their full pathname. This
breaks when .PATH or MAKEOBJDIR are used. There might be Makefiles
around that try to work around this bug by prepending ${.CURDIR} to
the sources, and they should be found and fixed. Also a lot of the gunk
in suff.c that was attempting to work around the same problem could be
removed.
(christos)
- Move -D flags from Makefile to config.h and explain what they do. Add
-Wall -Wno-unused to CFLAGS. Add new define SYSVVARSUB to enable SysV
style variable substitutions and enable them.
- Add SunOS style command substitutions via SUNSHCMD
- Fix core dump with '{variable = value'
(christos)
Fix bug where make will always exit with 0, even when one or more
parallel jobs failed. (Only affects parallel make code)
(christos)
Protect __P from being multiply defined (for systems that already
define it)
(christos) Add strdup() since ultrix is missing it.
From Larry Schwimmer <rosebud@cyclone.Stanford.EDU>
(christos) Add estrdup(), a checked version of strdup and use it.
(christos) Recognize SVR4 style long filename entries in archives.
(thorpej) Tidy up some RCS ids a bit.

RANLIBMAG should not be set on alphas, as make doesn't know ECOFF archives

From NetBSD: merge of 960317

From NetBSD:
Support SVR4 style archives.
Fix pr/1421 (from Matthew Green) and pr/1997 (from Jeff Thieleke).
In ParseDoInclude(), make a temporary copy of the current file name
while searching for ""-type include files, since the current file name
might not be a writeable string.

from christos@netbsd:
Minor:
- ${.PREFIX} should never contain a full pathname
- Fixed gcc -Wall warnings
Major:
- compatMake is now FALSE. This means that we are now running in
full pmake mode:
* rules on dependency lines can be executed in parallel and or
out of sequence:

foo: bar baz

can fire the rule for baz before the rule for bar is fired.
To enforce bar to be fired before baz, another rule needs to be
added. [bar: baz]
* adjacent shell commands in a target are now executed by a single
invocation of the shell, not one invocation of the shell per line
(compatMake can be turned off using the -B flag)
- The -j flag now works... I.e. make -j 4 will fork up to four jobs in
parallel when it can. The target name is printed before each burst
of output caused by the target execution as '--- target ---', when j > 1
- I have changed all the Makefiles so that they work with make -j N, and
I have tested the whole netbsd by:
'make -j 4 cleandir; make -j 4 depend; make -j 4; make -j 4 install'
- I have not compiled or tested this version of make with -DREMOTE.
- Turn compat mode on by default. It gets turned off when the -j without
the -B flag is specified. [Thus you can use -j 1 to turn it off].
- Fix malloc -> emalloc as Gordon noted.
Updates for POSIX/SVR4 compiling:
arch.c: Don't require ranlib stuff. Not everybody has it.
dir.c: SunOS-4 != Solaris; change #ifdef sun to #if sun && !__svr4__
job.c, compat.c: Don't use 'union wait', use int and the W*() macros.
main.c: Check for uname() == -1; some unames return > 0...
util.c, job.c: Add signal() with BSD semantics for svr4, don't use bsd
sigmask and friends.

from cgd@netbsd:
pull in make.h. (PAlloc() now uses emalloc(), which is prototyped in
make.h. If the prototype is not in scope on the Alpha, I see lots
of "cast to pointer from integer of different size" warnings.)

branches: 1.1.1;
Initial revision

spelling fixes; from paul tagliamonte
amendments to his diff are noted on tech

introduce a "Buf_Reinit" function for handling static buffers that can
be reused throughout running make. Instead of recreating buffers of
dubious appropriate size, have one single buffer for various stages of
parsing.

rename a few variable/functions to have better names.
adjust comments to be more meaningful
reorder predecessors/successors fields in an order that
makes more sense to me.

okay millert@

GC old cruft.
noticed and patch Michael W. Bombardieri

fix two nasty out-of-bound accesses that killed (sporadically mk38 and mk38bis.
prodded by bluhm@, okay bluhm@

small obvious cleanups:
- remove a lot of unnecessary casts
- zap extra param that's no longer needed
- add proper prototype and make function static

okay natano@

Use %zu to print size_t's rather than casting to u_long.

ok espie@

switch to <limits.h>; ok millert

If some mythical system lacks PATH_MAX, do not set it to MAXPATHLEN+1
ok millert

adjust to ohash being in libutil now, and to the interface changes.
fix potential integer overflows in memory allocation (mostly for pedagogical
purposes, these are unlikely to overflow in practice)
move the rest of lst.lib stuff into its own directory.

wrappers for is*/to* that make damn sure their arguments are
cast to unsigned chars.

okay deraadt@

remove TIMESTAMP abstraction layer, prodded by theo.
while there, clean up includes.
use strtoll for ar timestamps (pretty much unused in reality, more
standard conforming than anything)

use idea from Todd to adapt to time_t being 32 bits OR 64 bits
(pedantically correct: INT_MIN would work just fine up to 1910 or so...)

okay millert@, gone thru a make build.

more changes, discussed and tested by various people.
- put back some job control, turns out it's necessary when we don't run a
shell.
- zap old #ifdef CLEANUP code... probably doesn't even compile.
- kill most of the OP_LIB code. Just keep a wee little bit for compatibility
(deprecated .LIBS and .INCLUDES, warns for weird dependencies instead of
erroring out).
- much improved debugging and -p output: sort variables, targets, rules,
output stuff in a nicer format mimicing input.
- better error message when no command is found, explain where the target comes from.
- sort final error list by file.
- show system files in errors as <bsd.prog.mk>
- reincorporate random delay, that was dropped
- optimize siginfo output by not regenerating the whole string each time.
- finish zapping old LocationInfo field that's no longer used.

Correct $OpenBSD$ stuff

two small changes:
- allow variables in SysV modifiers, as requested by matthieu@
(since recursive variables are an extension, this just extends the
extension)
- variation on :Q called :QL (quote list), which does quote every character
EXCEPT for whitespace. e.g.,

toto:
@for i in ${VAR:QL} ...

Add a new default variable in Makefile context, MACHINE_CPU. Its value is
decided at compile-time and is either MACHINE_CPU from <machine/param.h> if
it is defined, or the same value as MACHINE_ARCH otherwise.
This will be used to allow ports with suffixes to their canonical MACHINE_ARCH
to provide this canonical name as MACHINE_CPU, and in turn to let Makefiles
do TRT.
ok kettenis@

modify timestamp handling to grab current time instead of relying on a
stale timestamp.

all those diffs sent to people ages ago, who didn't answer, except for
Theo, who said he ran it with no issues, so let's get this in...

simplify dynamic variable handling a great deal:
first remove all usage of Varq_Append by building the string directly.
then replace `common' handling with specialized handling for dynamic
strings (since they no longer need a buffer). Finally, identify the place
where the variable value needs to be copied because it's going to be free'd
or erased soon, and finally, use simple char* pointers.

Shaves about 80 bytes off every gnode structure, and kills quite a few
unnecessary malloc()s as well.

rename make -> must_make, made -> built_status
to make them easier to find in source files.

#ifdef stuff that is only used under #ifdef; ok espie

Work done at p2k7.


This is a really big step towards getting parallel make to work.


Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).



Tested by lots of people, thanks go to miod@, and robert@ among other people.

Quick summary of what this does:

- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.

always define the svr4namelist. Doesn't hurt, even if we don't use it.

fix the arch code mostly.
- stop using sprintf for code that can use our buffers.
- stop modifying parsed line, rely on Var_Substi instead.

to do that, we need to go through an intermediate routine that uses
one single buffer to build archive names.
also introduce an add_archive_node function that makes things way simpler.
also rename a few variables.

kill extra spaces at end of line

rename dirSearchPath -> defaultPath, and openDirectories -> knownDirectories

minor reindent issues

rename Targ_FmtTime into time_to_string and move it, as it's not related
to target nodes at all (reduces modules inter-dependencies)

reuse free_hash

reindent arch.c

cut up dir.c into dir.c/direxpand.c as there is very little interface between
two modules that do different things.

more dead code: we always define LIBRARIES and INCLUDES

remove dead code: #define RECHECK is always on

everywhere except in Var_Parse, we always add/append variables to the
VAR_GLOBAL context, so make it the common case: rename the basic functions
to Var_Set_with_ctxt/Var_Append_with_ctxt, define Var_Set and Var_Append
as macros that specify VAR_GLOBAL, and use these.

okay miod@

simplify computations in the var module: instead of advancing a char *
and keeping track of a length, we just advance the char *, and ditch
the length. We can still get the length at the end of the top-level
functions to satisfy existing interfaces.

Much simpler code, less error-prone.

Okay millert@

move the code that grabs a value in Var_Parse in its own function,
get_expanded_value.

Extend the code a bit to be much more thorough in case of a recursive
expansion: shows exactly the cycle of variable names involved.

okay millert@

change Var_ParseSkip API to increment the position instead of returning a
length, simplifies code.

(warns a bit, symptom of some further issues to fix).

okay millert@

remove some bogus *p tests from charles longeau
ok deraadt millert

use stdint.h where appropriate. okay millert@

ISO function declarations, trim a few comments, rename a few variables to
more explicit/more consistent names.

okay otto@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

get rid of some strcpy/sprintf.
ok krw@, matthieu@, deraadt@

use defined(__ELF__) instead of a list of ELF architectures from which
sparc64 was missing.
This fixes a bug with ELF static libraries on sparc64

occured->occurred

idea from deraadt@ via NetBSD
millert@ ok

p.s. Next commit will fix a typo in the sys/

Better warning fix, in line with the rest of the code
(__attribute((__unused__)) denotes parameters that MAY be unused in the
code, e.g., it's mostly a `shut up warnings' device).

Use Str_concat instead of fixed buffers and snprintf in building paths.
Replace MAXPATHLEN with PATH_MAX (synch with op-make).
ok naddy@

-Wall cleanup, mostly useless otherwise

Take includes out of lst.h, re-add what's needed to separate files.
Removes remaining lint stuff from lst.lib.

Mostly clean-up:
- cut up those huge include files into separate interfaces for all modules.
Put the interface documentation there, and not with the implementation.
- light-weight includes for needed concrete types (lst_t.h, timestamp_t.h).
- cut out some more logically separate parts: cmd_exec, varname, parsevar,
timestamp.
- put all error handling functions together, so that we will be able to
clean them up.
- more systematic naming: functioni to handle interval, function to handle
string.
- put the init/end code apart to minimize coupling.
- kill weird types like ReturnStatus and Boolean. Use standard bool (with a
fallback for non-iso systems)
- better interface documentation for lots of subsystems.

As a result, make compilation goes somewhat faster (5%, even considering
the largish BSD copyrights to read). The corresponding preprocessed
source goes down from 1,5M to 1M.

A few minor code changes as well: Parse_DoVar is no longer destructive.
Parse_IsVar functionality is folded into Parse_DoVar (as it knows what an
assignment is), a few more interval handling functions. Avoid calling
XXX_End when they do nothing, just #define XXX_End to nothing.

Parse_DoVar is slightly more general: it will handle compound assignments
as long as they make sense, e.g., VAR +!= cmd
will work. As a side effect, VAR++=value now triggers an error
(two + in assignment).
- this stuff doesn't occur in portable Makefiles.
- writing VAR++ = value or VAR+ +=value disambiguates it.
- this is a good thing, it uncovered a bug in bsd.port.mk.

Tested by naddy@. Okayed millert@. I'll handle the fallback if there is
any. This went through a full make build anyways, including isakmpd
(without mickey's custom binutils, as he didn't see fit to share it with me).

Synch with my current work.
Numerous changes:
- generate can build several tables
- style cleanup
- statistics code
- use variable names throughout (struct Name)
- recursive variables everywhere
- faster parser (pass buffer along instead of allocating multiple copies)
- correct parser. Handles comments everywhere, and ; correctly
- more string intervals
- simplified dir.c, less recursion.
- extended for loops
- sinclude()
- finished removing extra junk from Lst_*
- handles ${@D} and friends in a simpler way
- cleaned up and modular VarModifiers handling.
- recognizes some gnu Makefile usages and errors out about them.

Additionally, some extra functionality is defined by FEATURES. The set of
functionalities is currently hardcoded to OpenBSD defaults, but this may
include support for some NetBSD extensions, like ODE modifiers.

Backed by miod@ and millert@, who finally got sick of my endless patches...

Use the ohash_* that's now in libc.

This patch fixes ArchFindMember so that it handles SVR4 members
similarly to read_archive.

Note we no longer bother seeking back to the start of the header, as only
ArchTouch accesses that header, and can do the seek itself.

With this, arch handling should be working, more or less.
thanks to Todd, Miod, Naddy for reviewing those patches.

This does fix some nasty issues: ar field members are NOT null-terminated.

Hence, read_archive must be very careful to parse stuff correctly: don't
use str* when mem* are appropriate, copy numeric fields and ensure they're
terminated...

We take of the SVR4 archive handling data structure from our archive,
and put it into a specific structure (it is wasteful to keep lugging
SVR4 structures once an archive is parsed).

By tweaking read_archive slightly, we can achieve a nicer interface
to ArchSVR4Entry.

Note a bug in make: ArchFindMember does (currently) not use the SVR4 code,
hence some archive members won't be found in non-caching mode...

Factorise duplicated code involved in touching archives.

As it stands now, arch.c does not work.
Its main failing is that ar headers are NOT null terminated.
This code is atrocious. This change starts cleaning that up.

Replace the list of cached archives with a hash,
streamline the logic of ArchMTimeMember,
by taking out the actual function that does the reading (read_archive).

More fixes to come.

Change the time stamp interface to use an abstract datatype.
Define two possible interfaces: the classic one,
and the new one (used where available) that depends on timespec.

Better granularity, make is now able to distinguish between files that
were built during the same second.

Fix Arch_MemMTime (previous change I did was bogus)

This kills the last old hashing table, in arch.c

Slight optimizations: instead of storing archive members, just keep
the modification time, as we don't care for the rest of the archive
information. Lazily compute mtime, stash ascii date instead, and convert
to mtime when needed (storing an out_of_date value to mark the unconverted
values).

Archive handling is atrocious and need some clean-up.

Thanks to miod@ who took the time to review those patches.

Introduce a few macros to handle timestamps in an abstract way.

Replace the time stamp hash in dir.c with an open hashing structure.

In doing so, remove some nasty casts, simplify code a bit:
Dir_MTime can return a modification time, since make does not make
a distinction between out-of-date and non-existent files.

Replace the old hash used to hold file names within a directory with
open hashing.

An interesting optimization is that the open hashing interface is more
fine-grained, hence we can compute the correct hash value at the start
of Dir_FindFile, and reuse it for each hash structure into which we look
(the effect is measurable on large directories along with objdir/VPATH).

Remove a few unnecessary Lst_Open/Lst_Close that serve no purpose except
obfuscating the code.

The interface to dir.h changes slightly, hence tedious includes changes...

Some systematic clean-up.
- UNUSED macro that expands to __attribute__((unused)) for gcc
- move rcsid around so that they can be tagged UNUSED.
- activate -Wunused.
- use UNUSED instead of kludgy junk for function arguments.
- add extern to all extern prototypes.
- update comments in lst.h.
- clean up var.c a little bit, constifying arguments, updating comments...

Once those special variable are taken care of, other Var functions can take
the GNode's context directly. We rename that special Lst to `SymTable *'
in prevision of things to come.

Along the line, we lose the special GNodes affected to VAR_CMD, VAR_GLOBAL,
VAR_ENV, which become simple Lsts... This is not a problem, except when
getting to a context's name for debugging (handled very nicely by
offsetof).

Again, this is a preparatory patch, which does not gain anything except
for cleaning up issues...

Reviewed by millert@ and miod@, like the previous patch

Start of variable fixes and speed-ups.

This patch may seem a bit non-sensical at first. It simply introduces some
new interface. Specifically, recognizes that some variable names
(.TARGET/$@, .OODATE/$?, .ALLSRC/$>, .IMPSRC/$<, .PREFIX/$*, .ARCHIVE/$!,
.MEMBER/$%) are `special' (the actual variables which are local to a
target, e.g. GNode).

Currently, The Varq functions (for Varquick access) are only stubs to the
normal functions.

This fixes a very important detail before proceeding to turn variable lists
into hash tables: if every GNode holds a hash table, initialization times
for those will be very costly. But generic GNodes only hold those seven
special variables... which can be stored directly into a small array;
the only general cases are the environment, the command line and
global variables.

Trivial consequences of the previous list changes:

- audit code for Lst_Datum, it's never applied to an empty pointer,
so check can be removed -> turn into a macro,
- Lst_First, Lst_Last can become macro as well
- specialized version of Lst_Succ (Lst_Adv) to use in loops where it cannot
fail,
- Lst_Open can no longer fail. Trim down corresponding code.

Reviewed millert@, miod@

This patch introduces a distinction between
Lst_Init (constructor) and Lst_New (allocation + construction)
Lst_Destroy (destructor) and Lst_Delete (deallocation + destruction),
and uses that to turn most dynamic allocation of lists (Lst pointers)
into static structures (LIST).

Most of this is mundane, except for allGNs in targ.c, where the code must
be checked to verify that Targ_Init is called soon enough.

Lst_New is a temporary addition. All lists will soon be static.

Reviewed by millert@, like the previous patch.

Clean-up patch: use `void *' instead of old-fashioned ClientData/Address.

Remove idiotic, braindead casts T* -> void*
They serve no purpose, except hiding potential bugs.

In particular, remove (ClientData) cast from macro, showing potentially
troublesome use of Hashes to store time_t.

Bug-fix: make should behave sensibly when presented with negative times...

- let *_MTime return booleans, as that's what they're used for, the time_t
is set as a side effect.

- use OUT_OF_DATE for a date starting point, set it at the origin of time.

Finish changing all Var_Parse arguments to size_t. Thanks to millert@
for reminding me.

Rearrange Lst_Find interface to conform better with other functions.

Nothing ever checks ReturnStatus on Lst_Insert, Lst_Append, Lst_AtFront,
Lst_AtEnd, Lst_Concat, Lst_Remove, Lst_Replace.

Don't bother returning one.

Lst_DeQueue already checks for empty lists.

NIL, NILGNODE, etc, are only glorified NULL.
Get rid of them.

Get rid of list.h, nothing uses it anyway.

make does not use circular lists, get rid of the extra weight.

Var_Subst is actually two distinct functions folded into one:
split the function specific to for.c out, and give them more sensible
arguments at the same time.

This makes .for loop handling more efficient, as we have some heuristic
to evaluate the size of the buffer needed...

Extra parameter no longer needed, ditch.

Efficiency patch from NetBSD:
make spends time freeing data structures right before exiting.
So don't bother.
Keep the code inside #ifdef, so that it's still there if someone
ever wants to use make code inside a library.

Fix some mips -> __mips__ stuff.

Modifications from netbsd:
- don't interfere with MACHINE/MACHINE_ARCH defines for bootstrap
- type clean-up, time_t, and printing `unknown' ints
- fix TARGET/MEMBER bug in archive rules
- memmove...
- cleaner Error handler.
- reentrant brk_string
- .MAKE env variable
- preliminary scaffolding for .NOPATH

Other improvements:
- efree
- shellneed streamlined
- display Stop in .CURDIR after an error.
- document most features and misfeatures.
- add a few OpenBSD notes to the tutorial.

Use SEEK_CUR as "whence" in fseek, not 1.

Fix a bug where make gets confused by targets beginning with a period (``.'')
and tried to do a suffix conversion, following a NULL pointer in the
proccess. Also add some sanity checks so we don't blindly assume
strchr returns non-NULL.

powerpc ELF

(foo *)NULL -> NULL

Sync with NetBSD:
- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
- Fix NetBSD PR #2930: declare missing variable.

Do not consider non-existent archives as up-to-date on non-a.out systems

Do not look for a.out t.o.c sections in the mips-based ports

Sync up with NetBSD:
(christos)
Fix bug reported by Greg Hudson where leaf (source only) nodes were
referenced only by their basename and not by their full pathname. This
breaks when .PATH or MAKEOBJDIR are used. There might be Makefiles
around that try to work around this bug by prepending ${.CURDIR} to
the sources, and they should be found and fixed. Also a lot of the gunk
in suff.c that was attempting to work around the same problem could be
removed.
(christos)
- Move -D flags from Makefile to config.h and explain what they do. Add
-Wall -Wno-unused to CFLAGS. Add new define SYSVVARSUB to enable SysV
style variable substitutions and enable them.
- Add SunOS style command substitutions via SUNSHCMD
- Fix core dump with '{variable = value'
(christos)
Fix bug where make will always exit with 0, even when one or more
parallel jobs failed. (Only affects parallel make code)
(christos)
Protect __P from being multiply defined (for systems that already
define it)
(christos) Add strdup() since ultrix is missing it.
From Larry Schwimmer <rosebud@cyclone.Stanford.EDU>
(christos) Add estrdup(), a checked version of strdup and use it.
(christos) Recognize SVR4 style long filename entries in archives.
(thorpej) Tidy up some RCS ids a bit.

RANLIBMAG should not be set on alphas, as make doesn't know ECOFF archives

From NetBSD: merge of 960317

From NetBSD:
Support SVR4 style archives.
Fix pr/1421 (from Matthew Green) and pr/1997 (from Jeff Thieleke).
In ParseDoInclude(), make a temporary copy of the current file name
while searching for ""-type include files, since the current file name
might not be a writeable string.

from christos@netbsd:
Minor:
- ${.PREFIX} should never contain a full pathname
- Fixed gcc -Wall warnings
Major:
- compatMake is now FALSE. This means that we are now running in
full pmake mode:
* rules on dependency lines can be executed in parallel and or
out of sequence:

foo: bar baz

can fire the rule for baz before the rule for bar is fired.
To enforce bar to be fired before baz, another rule needs to be
added. [bar: baz]
* adjacent shell commands in a target are now executed by a single
invocation of the shell, not one invocation of the shell per line
(compatMake can be turned off using the -B flag)
- The -j flag now works... I.e. make -j 4 will fork up to four jobs in
parallel when it can. The target name is printed before each burst
of output caused by the target execution as '--- target ---', when j > 1
- I have changed all the Makefiles so that they work with make -j N, and
I have tested the whole netbsd by:
'make -j 4 cleandir; make -j 4 depend; make -j 4; make -j 4 install'
- I have not compiled or tested this version of make with -DREMOTE.
- Turn compat mode on by default. It gets turned off when the -j without
the -B flag is specified. [Thus you can use -j 1 to turn it off].
- Fix malloc -> emalloc as Gordon noted.
Updates for POSIX/SVR4 compiling:
arch.c: Don't require ranlib stuff. Not everybody has it.
dir.c: SunOS-4 != Solaris; change #ifdef sun to #if sun && !__svr4__
job.c, compat.c: Don't use 'union wait', use int and the W*() macros.
main.c: Check for uname() == -1; some unames return > 0...
util.c, job.c: Add signal() with BSD semantics for svr4, don't use bsd
sigmask and friends.

from cgd@netbsd:
pull in make.h. (PAlloc() now uses emalloc(), which is prototyped in
make.h. If the prototype is not in scope on the Alpha, I see lots
of "cast to pointer from integer of different size" warnings.)

branches: 1.1.1;
Initial revision

introduce a "Buf_Reinit" function for handling static buffers that can
be reused throughout running make. Instead of recreating buffers of
dubious appropriate size, have one single buffer for various stages of
parsing.

rename a few variable/functions to have better names.
adjust comments to be more meaningful
reorder predecessors/successors fields in an order that
makes more sense to me.

okay millert@

GC old cruft.
noticed and patch Michael W. Bombardieri

fix two nasty out-of-bound accesses that killed (sporadically mk38 and mk38bis.
prodded by bluhm@, okay bluhm@

small obvious cleanups:
- remove a lot of unnecessary casts
- zap extra param that's no longer needed
- add proper prototype and make function static

okay natano@

Use %zu to print size_t's rather than casting to u_long.

ok espie@

switch to <limits.h>; ok millert

If some mythical system lacks PATH_MAX, do not set it to MAXPATHLEN+1
ok millert

adjust to ohash being in libutil now, and to the interface changes.
fix potential integer overflows in memory allocation (mostly for pedagogical
purposes, these are unlikely to overflow in practice)
move the rest of lst.lib stuff into its own directory.

wrappers for is*/to* that make damn sure their arguments are
cast to unsigned chars.

okay deraadt@

remove TIMESTAMP abstraction layer, prodded by theo.
while there, clean up includes.
use strtoll for ar timestamps (pretty much unused in reality, more
standard conforming than anything)

use idea from Todd to adapt to time_t being 32 bits OR 64 bits
(pedantically correct: INT_MIN would work just fine up to 1910 or so...)

okay millert@, gone thru a make build.

more changes, discussed and tested by various people.
- put back some job control, turns out it's necessary when we don't run a
shell.
- zap old #ifdef CLEANUP code... probably doesn't even compile.
- kill most of the OP_LIB code. Just keep a wee little bit for compatibility
(deprecated .LIBS and .INCLUDES, warns for weird dependencies instead of
erroring out).
- much improved debugging and -p output: sort variables, targets, rules,
output stuff in a nicer format mimicing input.
- better error message when no command is found, explain where the target comes from.
- sort final error list by file.
- show system files in errors as <bsd.prog.mk>
- reincorporate random delay, that was dropped
- optimize siginfo output by not regenerating the whole string each time.
- finish zapping old LocationInfo field that's no longer used.

Correct $OpenBSD$ stuff

two small changes:
- allow variables in SysV modifiers, as requested by matthieu@
(since recursive variables are an extension, this just extends the
extension)
- variation on :Q called :QL (quote list), which does quote every character
EXCEPT for whitespace. e.g.,

toto:
@for i in ${VAR:QL} ...

Add a new default variable in Makefile context, MACHINE_CPU. Its value is
decided at compile-time and is either MACHINE_CPU from <machine/param.h> if
it is defined, or the same value as MACHINE_ARCH otherwise.
This will be used to allow ports with suffixes to their canonical MACHINE_ARCH
to provide this canonical name as MACHINE_CPU, and in turn to let Makefiles
do TRT.
ok kettenis@

modify timestamp handling to grab current time instead of relying on a
stale timestamp.

all those diffs sent to people ages ago, who didn't answer, except for
Theo, who said he ran it with no issues, so let's get this in...

simplify dynamic variable handling a great deal:
first remove all usage of Varq_Append by building the string directly.
then replace `common' handling with specialized handling for dynamic
strings (since they no longer need a buffer). Finally, identify the place
where the variable value needs to be copied because it's going to be free'd
or erased soon, and finally, use simple char* pointers.

Shaves about 80 bytes off every gnode structure, and kills quite a few
unnecessary malloc()s as well.

rename make -> must_make, made -> built_status
to make them easier to find in source files.

#ifdef stuff that is only used under #ifdef; ok espie

Work done at p2k7.


This is a really big step towards getting parallel make to work.


Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).



Tested by lots of people, thanks go to miod@, and robert@ among other people.

Quick summary of what this does:

- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.

always define the svr4namelist. Doesn't hurt, even if we don't use it.

fix the arch code mostly.
- stop using sprintf for code that can use our buffers.
- stop modifying parsed line, rely on Var_Substi instead.

to do that, we need to go through an intermediate routine that uses
one single buffer to build archive names.
also introduce an add_archive_node function that makes things way simpler.
also rename a few variables.

kill extra spaces at end of line

rename dirSearchPath -> defaultPath, and openDirectories -> knownDirectories

minor reindent issues

rename Targ_FmtTime into time_to_string and move it, as it's not related
to target nodes at all (reduces modules inter-dependencies)

reuse free_hash

reindent arch.c

cut up dir.c into dir.c/direxpand.c as there is very little interface between
two modules that do different things.

more dead code: we always define LIBRARIES and INCLUDES

remove dead code: #define RECHECK is always on

everywhere except in Var_Parse, we always add/append variables to the
VAR_GLOBAL context, so make it the common case: rename the basic functions
to Var_Set_with_ctxt/Var_Append_with_ctxt, define Var_Set and Var_Append
as macros that specify VAR_GLOBAL, and use these.

okay miod@

simplify computations in the var module: instead of advancing a char *
and keeping track of a length, we just advance the char *, and ditch
the length. We can still get the length at the end of the top-level
functions to satisfy existing interfaces.

Much simpler code, less error-prone.

Okay millert@

move the code that grabs a value in Var_Parse in its own function,
get_expanded_value.

Extend the code a bit to be much more thorough in case of a recursive
expansion: shows exactly the cycle of variable names involved.

okay millert@

change Var_ParseSkip API to increment the position instead of returning a
length, simplifies code.

(warns a bit, symptom of some further issues to fix).

okay millert@

remove some bogus *p tests from charles longeau
ok deraadt millert

use stdint.h where appropriate. okay millert@

ISO function declarations, trim a few comments, rename a few variables to
more explicit/more consistent names.

okay otto@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

get rid of some strcpy/sprintf.
ok krw@, matthieu@, deraadt@

use defined(__ELF__) instead of a list of ELF architectures from which
sparc64 was missing.
This fixes a bug with ELF static libraries on sparc64

occured->occurred

idea from deraadt@ via NetBSD
millert@ ok

p.s. Next commit will fix a typo in the sys/

Better warning fix, in line with the rest of the code
(__attribute((__unused__)) denotes parameters that MAY be unused in the
code, e.g., it's mostly a `shut up warnings' device).

Use Str_concat instead of fixed buffers and snprintf in building paths.
Replace MAXPATHLEN with PATH_MAX (synch with op-make).
ok naddy@

-Wall cleanup, mostly useless otherwise

Take includes out of lst.h, re-add what's needed to separate files.
Removes remaining lint stuff from lst.lib.

Mostly clean-up:
- cut up those huge include files into separate interfaces for all modules.
Put the interface documentation there, and not with the implementation.
- light-weight includes for needed concrete types (lst_t.h, timestamp_t.h).
- cut out some more logically separate parts: cmd_exec, varname, parsevar,
timestamp.
- put all error handling functions together, so that we will be able to
clean them up.
- more systematic naming: functioni to handle interval, function to handle
string.
- put the init/end code apart to minimize coupling.
- kill weird types like ReturnStatus and Boolean. Use standard bool (with a
fallback for non-iso systems)
- better interface documentation for lots of subsystems.

As a result, make compilation goes somewhat faster (5%, even considering
the largish BSD copyrights to read). The corresponding preprocessed
source goes down from 1,5M to 1M.

A few minor code changes as well: Parse_DoVar is no longer destructive.
Parse_IsVar functionality is folded into Parse_DoVar (as it knows what an
assignment is), a few more interval handling functions. Avoid calling
XXX_End when they do nothing, just #define XXX_End to nothing.

Parse_DoVar is slightly more general: it will handle compound assignments
as long as they make sense, e.g., VAR +!= cmd
will work. As a side effect, VAR++=value now triggers an error
(two + in assignment).
- this stuff doesn't occur in portable Makefiles.
- writing VAR++ = value or VAR+ +=value disambiguates it.
- this is a good thing, it uncovered a bug in bsd.port.mk.

Tested by naddy@. Okayed millert@. I'll handle the fallback if there is
any. This went through a full make build anyways, including isakmpd
(without mickey's custom binutils, as he didn't see fit to share it with me).

Synch with my current work.
Numerous changes:
- generate can build several tables
- style cleanup
- statistics code
- use variable names throughout (struct Name)
- recursive variables everywhere
- faster parser (pass buffer along instead of allocating multiple copies)
- correct parser. Handles comments everywhere, and ; correctly
- more string intervals
- simplified dir.c, less recursion.
- extended for loops
- sinclude()
- finished removing extra junk from Lst_*
- handles ${@D} and friends in a simpler way
- cleaned up and modular VarModifiers handling.
- recognizes some gnu Makefile usages and errors out about them.

Additionally, some extra functionality is defined by FEATURES. The set of
functionalities is currently hardcoded to OpenBSD defaults, but this may
include support for some NetBSD extensions, like ODE modifiers.

Backed by miod@ and millert@, who finally got sick of my endless patches...

Use the ohash_* that's now in libc.

This patch fixes ArchFindMember so that it handles SVR4 members
similarly to read_archive.

Note we no longer bother seeking back to the start of the header, as only
ArchTouch accesses that header, and can do the seek itself.

With this, arch handling should be working, more or less.
thanks to Todd, Miod, Naddy for reviewing those patches.

This does fix some nasty issues: ar field members are NOT null-terminated.

Hence, read_archive must be very careful to parse stuff correctly: don't
use str* when mem* are appropriate, copy numeric fields and ensure they're
terminated...

We take of the SVR4 archive handling data structure from our archive,
and put it into a specific structure (it is wasteful to keep lugging
SVR4 structures once an archive is parsed).

By tweaking read_archive slightly, we can achieve a nicer interface
to ArchSVR4Entry.

Note a bug in make: ArchFindMember does (currently) not use the SVR4 code,
hence some archive members won't be found in non-caching mode...

Factorise duplicated code involved in touching archives.

As it stands now, arch.c does not work.
Its main failing is that ar headers are NOT null terminated.
This code is atrocious. This change starts cleaning that up.

Replace the list of cached archives with a hash,
streamline the logic of ArchMTimeMember,
by taking out the actual function that does the reading (read_archive).

More fixes to come.

Change the time stamp interface to use an abstract datatype.
Define two possible interfaces: the classic one,
and the new one (used where available) that depends on timespec.

Better granularity, make is now able to distinguish between files that
were built during the same second.

Fix Arch_MemMTime (previous change I did was bogus)

This kills the last old hashing table, in arch.c

Slight optimizations: instead of storing archive members, just keep
the modification time, as we don't care for the rest of the archive
information. Lazily compute mtime, stash ascii date instead, and convert
to mtime when needed (storing an out_of_date value to mark the unconverted
values).

Archive handling is atrocious and need some clean-up.

Thanks to miod@ who took the time to review those patches.

Introduce a few macros to handle timestamps in an abstract way.

Replace the time stamp hash in dir.c with an open hashing structure.

In doing so, remove some nasty casts, simplify code a bit:
Dir_MTime can return a modification time, since make does not make
a distinction between out-of-date and non-existent files.

Replace the old hash used to hold file names within a directory with
open hashing.

An interesting optimization is that the open hashing interface is more
fine-grained, hence we can compute the correct hash value at the start
of Dir_FindFile, and reuse it for each hash structure into which we look
(the effect is measurable on large directories along with objdir/VPATH).

Remove a few unnecessary Lst_Open/Lst_Close that serve no purpose except
obfuscating the code.

The interface to dir.h changes slightly, hence tedious includes changes...

Some systematic clean-up.
- UNUSED macro that expands to __attribute__((unused)) for gcc
- move rcsid around so that they can be tagged UNUSED.
- activate -Wunused.
- use UNUSED instead of kludgy junk for function arguments.
- add extern to all extern prototypes.
- update comments in lst.h.
- clean up var.c a little bit, constifying arguments, updating comments...

Once those special variable are taken care of, other Var functions can take
the GNode's context directly. We rename that special Lst to `SymTable *'
in prevision of things to come.

Along the line, we lose the special GNodes affected to VAR_CMD, VAR_GLOBAL,
VAR_ENV, which become simple Lsts... This is not a problem, except when
getting to a context's name for debugging (handled very nicely by
offsetof).

Again, this is a preparatory patch, which does not gain anything except
for cleaning up issues...

Reviewed by millert@ and miod@, like the previous patch

Start of variable fixes and speed-ups.

This patch may seem a bit non-sensical at first. It simply introduces some
new interface. Specifically, recognizes that some variable names
(.TARGET/$@, .OODATE/$?, .ALLSRC/$>, .IMPSRC/$<, .PREFIX/$*, .ARCHIVE/$!,
.MEMBER/$%) are `special' (the actual variables which are local to a
target, e.g. GNode).

Currently, The Varq functions (for Varquick access) are only stubs to the
normal functions.

This fixes a very important detail before proceeding to turn variable lists
into hash tables: if every GNode holds a hash table, initialization times
for those will be very costly. But generic GNodes only hold those seven
special variables... which can be stored directly into a small array;
the only general cases are the environment, the command line and
global variables.

Trivial consequences of the previous list changes:

- audit code for Lst_Datum, it's never applied to an empty pointer,
so check can be removed -> turn into a macro,
- Lst_First, Lst_Last can become macro as well
- specialized version of Lst_Succ (Lst_Adv) to use in loops where it cannot
fail,
- Lst_Open can no longer fail. Trim down corresponding code.

Reviewed millert@, miod@

This patch introduces a distinction between
Lst_Init (constructor) and Lst_New (allocation + construction)
Lst_Destroy (destructor) and Lst_Delete (deallocation + destruction),
and uses that to turn most dynamic allocation of lists (Lst pointers)
into static structures (LIST).

Most of this is mundane, except for allGNs in targ.c, where the code must
be checked to verify that Targ_Init is called soon enough.

Lst_New is a temporary addition. All lists will soon be static.

Reviewed by millert@, like the previous patch.

Clean-up patch: use `void *' instead of old-fashioned ClientData/Address.

Remove idiotic, braindead casts T* -> void*
They serve no purpose, except hiding potential bugs.

In particular, remove (ClientData) cast from macro, showing potentially
troublesome use of Hashes to store time_t.

Bug-fix: make should behave sensibly when presented with negative times...

- let *_MTime return booleans, as that's what they're used for, the time_t
is set as a side effect.

- use OUT_OF_DATE for a date starting point, set it at the origin of time.

Finish changing all Var_Parse arguments to size_t. Thanks to millert@
for reminding me.

Rearrange Lst_Find interface to conform better with other functions.

Nothing ever checks ReturnStatus on Lst_Insert, Lst_Append, Lst_AtFront,
Lst_AtEnd, Lst_Concat, Lst_Remove, Lst_Replace.

Don't bother returning one.

Lst_DeQueue already checks for empty lists.

NIL, NILGNODE, etc, are only glorified NULL.
Get rid of them.

Get rid of list.h, nothing uses it anyway.

make does not use circular lists, get rid of the extra weight.

Var_Subst is actually two distinct functions folded into one:
split the function specific to for.c out, and give them more sensible
arguments at the same time.

This makes .for loop handling more efficient, as we have some heuristic
to evaluate the size of the buffer needed...

Extra parameter no longer needed, ditch.

Efficiency patch from NetBSD:
make spends time freeing data structures right before exiting.
So don't bother.
Keep the code inside #ifdef, so that it's still there if someone
ever wants to use make code inside a library.

Fix some mips -> __mips__ stuff.

Modifications from netbsd:
- don't interfere with MACHINE/MACHINE_ARCH defines for bootstrap
- type clean-up, time_t, and printing `unknown' ints
- fix TARGET/MEMBER bug in archive rules
- memmove...
- cleaner Error handler.
- reentrant brk_string
- .MAKE env variable
- preliminary scaffolding for .NOPATH

Other improvements:
- efree
- shellneed streamlined
- display Stop in .CURDIR after an error.
- document most features and misfeatures.
- add a few OpenBSD notes to the tutorial.

Use SEEK_CUR as "whence" in fseek, not 1.

Fix a bug where make gets confused by targets beginning with a period (``.'')
and tried to do a suffix conversion, following a NULL pointer in the
proccess. Also add some sanity checks so we don't blindly assume
strchr returns non-NULL.

powerpc ELF

(foo *)NULL -> NULL

Sync with NetBSD:
- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
- Fix NetBSD PR #2930: declare missing variable.

Do not consider non-existent archives as up-to-date on non-a.out systems

Do not look for a.out t.o.c sections in the mips-based ports

Sync up with NetBSD:
(christos)
Fix bug reported by Greg Hudson where leaf (source only) nodes were
referenced only by their basename and not by their full pathname. This
breaks when .PATH or MAKEOBJDIR are used. There might be Makefiles
around that try to work around this bug by prepending ${.CURDIR} to
the sources, and they should be found and fixed. Also a lot of the gunk
in suff.c that was attempting to work around the same problem could be
removed.
(christos)
- Move -D flags from Makefile to config.h and explain what they do. Add
-Wall -Wno-unused to CFLAGS. Add new define SYSVVARSUB to enable SysV
style variable substitutions and enable them.
- Add SunOS style command substitutions via SUNSHCMD
- Fix core dump with '{variable = value'
(christos)
Fix bug where make will always exit with 0, even when one or more
parallel jobs failed. (Only affects parallel make code)
(christos)
Protect __P from being multiply defined (for systems that already
define it)
(christos) Add strdup() since ultrix is missing it.
From Larry Schwimmer <rosebud@cyclone.Stanford.EDU>
(christos) Add estrdup(), a checked version of strdup and use it.
(christos) Recognize SVR4 style long filename entries in archives.
(thorpej) Tidy up some RCS ids a bit.

RANLIBMAG should not be set on alphas, as make doesn't know ECOFF archives

From NetBSD: merge of 960317

From NetBSD:
Support SVR4 style archives.
Fix pr/1421 (from Matthew Green) and pr/1997 (from Jeff Thieleke).
In ParseDoInclude(), make a temporary copy of the current file name
while searching for ""-type include files, since the current file name
might not be a writeable string.

from christos@netbsd:
Minor:
- ${.PREFIX} should never contain a full pathname
- Fixed gcc -Wall warnings
Major:
- compatMake is now FALSE. This means that we are now running in
full pmake mode:
* rules on dependency lines can be executed in parallel and or
out of sequence:

foo: bar baz

can fire the rule for baz before the rule for bar is fired.
To enforce bar to be fired before baz, another rule needs to be
added. [bar: baz]
* adjacent shell commands in a target are now executed by a single
invocation of the shell, not one invocation of the shell per line
(compatMake can be turned off using the -B flag)
- The -j flag now works... I.e. make -j 4 will fork up to four jobs in
parallel when it can. The target name is printed before each burst
of output caused by the target execution as '--- target ---', when j > 1
- I have changed all the Makefiles so that they work with make -j N, and
I have tested the whole netbsd by:
'make -j 4 cleandir; make -j 4 depend; make -j 4; make -j 4 install'
- I have not compiled or tested this version of make with -DREMOTE.
- Turn compat mode on by default. It gets turned off when the -j without
the -B flag is specified. [Thus you can use -j 1 to turn it off].
- Fix malloc -> emalloc as Gordon noted.
Updates for POSIX/SVR4 compiling:
arch.c: Don't require ranlib stuff. Not everybody has it.
dir.c: SunOS-4 != Solaris; change #ifdef sun to #if sun && !__svr4__
job.c, compat.c: Don't use 'union wait', use int and the W*() macros.
main.c: Check for uname() == -1; some unames return > 0...
util.c, job.c: Add signal() with BSD semantics for svr4, don't use bsd
sigmask and friends.

from cgd@netbsd:
pull in make.h. (PAlloc() now uses emalloc(), which is prototyped in
make.h. If the prototype is not in scope on the Alpha, I see lots
of "cast to pointer from integer of different size" warnings.)

branches: 1.1.1;
Initial revision

rename a few variable/functions to have better names.
adjust comments to be more meaningful
reorder predecessors/successors fields in an order that
makes more sense to me.

okay millert@

GC old cruft.
noticed and patch Michael W. Bombardieri

fix two nasty out-of-bound accesses that killed (sporadically mk38 and mk38bis.
prodded by bluhm@, okay bluhm@

small obvious cleanups:
- remove a lot of unnecessary casts
- zap extra param that's no longer needed
- add proper prototype and make function static

okay natano@

Use %zu to print size_t's rather than casting to u_long.

ok espie@

switch to <limits.h>; ok millert