source: trunk/minix/commands/ash/TOUR@ 10

Last change on this file since 10 was 9, checked in by Mattia Monga, 13 years ago

Minix 3.1.2a

File size: 16.4 KB
Line 
1# @(#)TOUR 5.1 (Berkeley) 3/7/91
2
3 A Tour through Ash
4
5 Copyright 1989 by Kenneth Almquist.
6
7
8DIRECTORIES: The subdirectory bltin contains commands which can
9be compiled stand-alone. The rest of the source is in the main
10ash directory.
11
12SOURCE CODE GENERATORS: Files whose names begin with "mk" are
13programs that generate source code. A complete list of these
14programs is:
15
16 program intput files generates
17 ------- ------------ ---------
18 mkbuiltins builtins builtins.h builtins.c
19 mkinit *.c init.c
20 mknodes nodetypes nodes.h nodes.c
21 mksignames - signames.h signames.c
22 mksyntax - syntax.h syntax.c
23 mktokens - token.def
24 bltin/mkexpr unary_op binary_op operators.h operators.c
25
26There are undoubtedly too many of these. Mkinit searches all the
27C source files for entries looking like:
28
29 INIT {
30 x = 1; /* executed during initialization */
31 }
32
33 RESET {
34 x = 2; /* executed when the shell does a longjmp
35 back to the main command loop */
36 }
37
38 SHELLPROC {
39 x = 3; /* executed when the shell runs a shell procedure */
40 }
41
42It pulls this code out into routines which are when particular
43events occur. The intent is to improve modularity by isolating
44the information about which modules need to be explicitly
45initialized/reset within the modules themselves.
46
47Mkinit recognizes several constructs for placing declarations in
48the init.c file.
49 INCLUDE "file.h"
50includes a file. The storage class MKINIT makes a declaration
51available in the init.c file, for example:
52 MKINIT int funcnest; /* depth of function calls */
53MKINIT alone on a line introduces a structure or union declara-
54tion:
55 MKINIT
56 struct redirtab {
57 short renamed[10];
58 };
59Preprocessor #define statements are copied to init.c without any
60special action to request this.
61
62INDENTATION: The ash source is indented in multiples of six
63spaces. The only study that I have heard of on the subject con-
64cluded that the optimal amount to indent is in the range of four
65to six spaces. I use six spaces since it is not too big a jump
66from the widely used eight spaces. If you really hate six space
67indentation, use the adjind (source included) program to change
68it to something else.
69
70EXCEPTIONS: Code for dealing with exceptions appears in
71exceptions.c. The C language doesn't include exception handling,
72so I implement it using setjmp and longjmp. The global variable
73exception contains the type of exception. EXERROR is raised by
74calling error. EXINT is an interrupt. EXSHELLPROC is an excep-
75tion which is raised when a shell procedure is invoked. The pur-
76pose of EXSHELLPROC is to perform the cleanup actions associated
77with other exceptions. After these cleanup actions, the shell
78can interpret a shell procedure itself without exec'ing a new
79copy of the shell.
80
81INTERRUPTS: In an interactive shell, an interrupt will cause an
82EXINT exception to return to the main command loop. (Exception:
83EXINT is not raised if the user traps interrupts using the trap
84command.) The INTOFF and INTON macros (defined in exception.h)
85provide uninterruptable critical sections. Between the execution
86of INTOFF and the execution of INTON, interrupt signals will be
87held for later delivery. INTOFF and INTON can be nested.
88
89MEMALLOC.C: Memalloc.c defines versions of malloc and realloc
90which call error when there is no memory left. It also defines a
91stack oriented memory allocation scheme. Allocating off a stack
92is probably more efficient than allocation using malloc, but the
93big advantage is that when an exception occurs all we have to do
94to free up the memory in use at the time of the exception is to
95restore the stack pointer. The stack is implemented using a
96linked list of blocks.
97
98STPUTC: If the stack were contiguous, it would be easy to store
99strings on the stack without knowing in advance how long the
100string was going to be:
101 p = stackptr;
102 *p++ = c; /* repeated as many times as needed */
103 stackptr = p;
104The folloing three macros (defined in memalloc.h) perform these
105operations, but grow the stack if you run off the end:
106 STARTSTACKSTR(p);
107 STPUTC(c, p); /* repeated as many times as needed */
108 grabstackstr(p);
109
110We now start a top-down look at the code:
111
112MAIN.C: The main routine performs some initialization, executes
113the user's profile if necessary, and calls cmdloop. Cmdloop is
114repeatedly parses and executes commands.
115
116OPTIONS.C: This file contains the option processing code. It is
117called from main to parse the shell arguments when the shell is
118invoked, and it also contains the set builtin. The -i and -j op-
119tions (the latter turns on job control) require changes in signal
120handling. The routines setjobctl (in jobs.c) and setinteractive
121(in trap.c) are called to handle changes to these options.
122
123PARSING: The parser code is all in parser.c. A recursive des-
124cent parser is used. Syntax tables (generated by mksyntax) are
125used to classify characters during lexical analysis. There are
126three tables: one for normal use, one for use when inside single
127quotes, and one for use when inside double quotes. The tables
128are machine dependent because they are indexed by character vari-
129ables and the range of a char varies from machine to machine.
130
131PARSE OUTPUT: The output of the parser consists of a tree of
132nodes. The various types of nodes are defined in the file node-
133types.
134
135Nodes of type NARG are used to represent both words and the con-
136tents of here documents. An early version of ash kept the con-
137tents of here documents in temporary files, but keeping here do-
138cuments in memory typically results in significantly better per-
139formance. It would have been nice to make it an option to use
140temporary files for here documents, for the benefit of small
141machines, but the code to keep track of when to delete the tem-
142porary files was complex and I never fixed all the bugs in it.
143(AT&T has been maintaining the Bourne shell for more than ten
144years, and to the best of my knowledge they still haven't gotten
145it to handle temporary files correctly in obscure cases.)
146
147The text field of a NARG structure points to the text of the
148word. The text consists of ordinary characters and a number of
149special codes defined in parser.h. The special codes are:
150
151 CTLVAR Variable substitution
152 CTLENDVAR End of variable substitution
153 CTLBACKQ Command substitution
154 CTLBACKQ|CTLQUOTE Command substitution inside double quotes
155 CTLESC Escape next character
156
157A variable substitution contains the following elements:
158
159 CTLVAR type name '=' [ alternative-text CTLENDVAR ]
160
161The type field is a single character specifying the type of sub-
162stitution. The possible types are:
163
164 VSNORMAL $var
165 VSMINUS ${var-text}
166 VSMINUS|VSNUL ${var:-text}
167 VSPLUS ${var+text}
168 VSPLUS|VSNUL ${var:+text}
169 VSQUESTION ${var?text}
170 VSQUESTION|VSNUL ${var:?text}
171 VSASSIGN ${var=text}
172 VSASSIGN|VSNUL ${var=text}
173
174In addition, the type field will have the VSQUOTE flag set if the
175variable is enclosed in double quotes. The name of the variable
176comes next, terminated by an equals sign. If the type is not
177VSNORMAL, then the text field in the substitution follows, ter-
178minated by a CTLENDVAR byte.
179
180Commands in back quotes are parsed and stored in a linked list.
181The locations of these commands in the string are indicated by
182CTLBACKQ and CTLBACKQ+CTLQUOTE characters, depending upon whether
183the back quotes were enclosed in double quotes.
184
185The character CTLESC escapes the next character, so that in case
186any of the CTL characters mentioned above appear in the input,
187they can be passed through transparently. CTLESC is also used to
188escape '*', '?', '[', and '!' characters which were quoted by the
189user and thus should not be used for file name generation.
190
191CTLESC characters have proved to be particularly tricky to get
192right. In the case of here documents which are not subject to
193variable and command substitution, the parser doesn't insert any
194CTLESC characters to begin with (so the contents of the text
195field can be written without any processing). Other here docu-
196ments, and words which are not subject to splitting and file name
197generation, have the CTLESC characters removed during the vari-
198able and command substitution phase. Words which are subject
199splitting and file name generation have the CTLESC characters re-
200moved as part of the file name phase.
201
202EXECUTION: Command execution is handled by the following files:
203 eval.c The top level routines.
204 redir.c Code to handle redirection of input and output.
205 jobs.c Code to handle forking, waiting, and job control.
206 exec.c Code to to path searches and the actual exec sys call.
207 expand.c Code to evaluate arguments.
208 var.c Maintains the variable symbol table. Called from expand.c.
209
210EVAL.C: Evaltree recursively executes a parse tree. The exit
211status is returned in the global variable exitstatus. The alter-
212native entry evalbackcmd is called to evaluate commands in back
213quotes. It saves the result in memory if the command is a buil-
214tin; otherwise it forks off a child to execute the command and
215connects the standard output of the child to a pipe.
216
217JOBS.C: To create a process, you call makejob to return a job
218structure, and then call forkshell (passing the job structure as
219an argument) to create the process. Waitforjob waits for a job
220to complete. These routines take care of process groups if job
221control is defined.
222
223REDIR.C: Ash allows file descriptors to be redirected and then
224restored without forking off a child process. This is accom-
225plished by duplicating the original file descriptors. The redir-
226tab structure records where the file descriptors have be dupli-
227cated to.
228
229EXEC.C: The routine find_command locates a command, and enters
230the command in the hash table if it is not already there. The
231third argument specifies whether it is to print an error message
232if the command is not found. (When a pipeline is set up,
233find_command is called for all the commands in the pipeline be-
234fore any forking is done, so to get the commands into the hash
235table of the parent process. But to make command hashing as
236transparent as possible, we silently ignore errors at that point
237and only print error messages if the command cannot be found
238later.)
239
240The routine shellexec is the interface to the exec system call.
241
242EXPAND.C: Arguments are processed in three passes. The first
243(performed by the routine argstr) performs variable and command
244substitution. The second (ifsbreakup) performs word splitting
245and the third (expandmeta) performs file name generation. If the
246"/u" directory is simulated, then when "/u/username" is replaced
247by the user's home directory, the flag "didudir" is set. This
248tells the cd command that it should print out the directory name,
249just as it would if the "/u" directory were implemented using
250symbolic links.
251
252VAR.C: Variables are stored in a hash table. Probably we should
253switch to extensible hashing. The variable name is stored in the
254same string as the value (using the format "name=value") so that
255no string copying is needed to create the environment of a com-
256mand. Variables which the shell references internally are preal-
257located so that the shell can reference the values of these vari-
258ables without doing a lookup.
259
260When a program is run, the code in eval.c sticks any environment
261variables which precede the command (as in "PATH=xxx command") in
262the variable table as the simplest way to strip duplicates, and
263then calls "environment" to get the value of the environment.
264There are two consequences of this. First, if an assignment to
265PATH precedes the command, the value of PATH before the assign-
266ment must be remembered and passed to shellexec. Second, if the
267program turns out to be a shell procedure, the strings from the
268environment variables which preceded the command must be pulled
269out of the table and replaced with strings obtained from malloc,
270since the former will automatically be freed when the stack (see
271the entry on memalloc.c) is emptied.
272
273BUILTIN COMMANDS: The procedures for handling these are scat-
274tered throughout the code, depending on which location appears
275most appropriate. They can be recognized because their names al-
276ways end in "cmd". The mapping from names to procedures is
277specified in the file builtins, which is processed by the mkbuil-
278tins command.
279
280A builtin command is invoked with argc and argv set up like a
281normal program. A builtin command is allowed to overwrite its
282arguments. Builtin routines can call nextopt to do option pars-
283ing. This is kind of like getopt, but you don't pass argc and
284argv to it. Builtin routines can also call error. This routine
285normally terminates the shell (or returns to the main command
286loop if the shell is interactive), but when called from a builtin
287command it causes the builtin command to terminate with an exit
288status of 2.
289
290The directory bltins contains commands which can be compiled in-
291dependently but can also be built into the shell for efficiency
292reasons. The makefile in this directory compiles these programs
293in the normal fashion (so that they can be run regardless of
294whether the invoker is ash), but also creates a library named
295bltinlib.a which can be linked with ash. The header file bltin.h
296takes care of most of the differences between the ash and the
297stand-alone environment. The user should call the main routine
298"main", and #define main to be the name of the routine to use
299when the program is linked into ash. This #define should appear
300before bltin.h is included; bltin.h will #undef main if the pro-
301gram is to be compiled stand-alone.
302
303CD.C: This file defines the cd and pwd builtins. The pwd com-
304mand runs /bin/pwd the first time it is invoked (unless the user
305has already done a cd to an absolute pathname), but then
306remembers the current directory and updates it when the cd com-
307mand is run, so subsequent pwd commands run very fast. The main
308complication in the cd command is in the docd command, which
309resolves symbolic links into actual names and informs the user
310where the user ended up if he crossed a symbolic link.
311
312SIGNALS: Trap.c implements the trap command. The routine set-
313signal figures out what action should be taken when a signal is
314received and invokes the signal system call to set the signal ac-
315tion appropriately. When a signal that a user has set a trap for
316is caught, the routine "onsig" sets a flag. The routine dotrap
317is called at appropriate points to actually handle the signal.
318When an interrupt is caught and no trap has been set for that
319signal, the routine "onint" in error.c is called.
320
321OUTPUT: Ash uses it's own output routines. There are three out-
322put structures allocated. "Output" represents the standard out-
323put, "errout" the standard error, and "memout" contains output
324which is to be stored in memory. This last is used when a buil-
325tin command appears in backquotes, to allow its output to be col-
326lected without doing any I/O through the UNIX operating system.
327The variables out1 and out2 normally point to output and errout,
328respectively, but they are set to point to memout when appropri-
329ate inside backquotes.
330
331INPUT: The basic input routine is pgetc, which reads from the
332current input file. There is a stack of input files; the current
333input file is the top file on this stack. The code allows the
334input to come from a string rather than a file. (This is for the
335-c option and the "." and eval builtin commands.) The global
336variable plinno is saved and restored when files are pushed and
337popped from the stack. The parser routines store the number of
338the current line in this variable.
339
340DEBUGGING: If DEBUG is defined in shell.h, then the shell will
341write debugging information to the file $HOME/trace. Most of
342this is done using the TRACE macro, which takes a set of printf
343arguments inside two sets of parenthesis. Example:
344"TRACE(("n=%d0, n))". The double parenthesis are necessary be-
345cause the preprocessor can't handle functions with a variable
346number of arguments. Defining DEBUG also causes the shell to
347generate a core dump if it is sent a quit signal. The tracing
348code is in show.c.
Note: See TracBrowser for help on using the repository browser.