-SBCL is -- ahem! -- not particularly well documented at this point.
+SBCL is -- ahem! -- not completely documented at this point.
What can I say? Help with documentation might not be refused.:-)
-The old CMUCL documentation, in the cmucl/ subdirectory, is still
-somewhat useful. The old user's manual is very useful. Most of the
-CMUCL extensions to Common Lisp have gone away, but the general
-information about how to use the Python compiler is still very
-relevant.
+There is a user manual in DocBook format, in user-manual.sgml.
+It's based on the CMU CL user manual, and some of its chapters
+aren't done, just notes that "this is similar to chapter such-and-such
+of the CMU CL user manual".
+
+The old CMU CL documentation can still be useful both for
+missing chapters of the user manual and for documentation of
+the internals of the system. It's available from SourceForge
+by anonymous ftp.
+++ /dev/null
-'BAR
-VARREF
-'TEST
-UPCASE
-ENDLISP
-SUBSEQ
-ENDDEFUN
-FUNARGS
-GENSYM
-VARS
-UNINTERNED
-VAR
-VSOURCE
-CLISP
-COND
-MYSTUFF
-TRADEOFFS
-PATHNAME
-LLISP
-CMUCL
-REF
-YETMOREKEYS
-CLEANUP
-ARGS
-DEFUN
-ZOQ
-FOO
-'S
-CLTL
-MACROEXPANDS
-MACROEXPANSION
-PROXY
-ERRORFUL
-EQ
-ECASE
-PYTHON
-DEFMACRO
-PROMISCUOUS
-FLAMAGE
-DEBUGGABILITY
-FEATUREFULNESS
-DEBUGGABLE
-ENDDEFVAR
-MACROEXPANDED
-DEFVAR
-ENDDEFMAC
-KWD
-MGROUP
-MSTAR
-DEFMAC
-OFFS
-NOTINLINE
-TRADEOFF
-FUNCALL
-SOMEVAL
-SOMEFUN
-CM
-DEFTYPE
-CONSING
-FIXNUMS
-BIGNUMS
-FROB
-'FOO
-RECOMPILES
-FTYPE
-TYPECASE
-TYPEP
-UNTYPED
-UNIONED
-GLOBALS
-MODICUM
-MACREF
-SLEAZING
-ES
-STEELE
-ETYPECASE
-'EQL
-'IDENTITY
-'FUN
-LOCALFUN
-ISQRT
-ODDP
-MYFUN
-POS
-ZOW
-YOW
-'YOW
-CADR
-ZEROP
-RES
-EXPT
-PARED
-PUSHING
-'ING
-RPLACD
-IOTA
-NTHCDR
-NTH
-CADDDR
-RPLACA
-CADDR
-FIENDS
-SQRT
-'SQRT
-LISPY
-BLANKSPACE
-MYCHAPTER
-UNENCAPSULATED
-ENCAPSULATIONS
-UNENCAPSULATE
-UNTRACED
-UNTRACE
-EVALED
-SPEC
-PUSHES
-TRUENAME
-MYMAC
-UNINFORMATIVE
-FOOBAR
-BAZ
-BACKQUOTE
-MALFORMED
-MOREKEYS
-FUNREF
-QUIRKS
-UNDILUTED
-DISASSEMBLY
-NAN
-DENORMALIZED
-ENDDEFCONST
-DEFCONST
-HASHTABLES
-EFF
-OBFUSCATING
-SNOC
-GRUE
-GORP
-FLO
-NUM
-VEC
-MULTBY
-SOMEOTHERFUN
-'CHAR
-NOTP
-TESTP
-FUNVAR
-RAZ
-ZUG
-XFF
-IO
-GC'ING
-EXT
-MEGABYTE
-SYS
-UX
-ED
-MATCHMAKER
-DIRED
-PCL
-CLOS
-CONFORMANCE
-ENDDEFCON
-DEFCON
-DECLAIM
-DEFSTRUCT
-ENUM
-EXTERN
-LOWERCASING
-DEREFERENCED
-MOPT
-STRUCT
-DEFTP
-ENDDEFTP
-MALLOC
-CSH
-PXLREF
-ATYPE
-CONSTRUCTUED
-ANAME
-PXREF
-ENV
-ONECOLUMN
-TP
-VR
-FN
-PRINTINDEX
-UNNUMBERED
-TWOCOLUMN
-TLF
-UNCOMPILED
-DEACTIVATE
-CALLABLE
-UNREFERENCED
-SUPPLIEDP
-INTERNING
-UNHANDLED
-BACKTRACING
-TEX
-OOB
-OBJ
-PRIN
-OBJS
-GP
-LINKERS
-CC
-AR
-CFUN
-INTS
-SIZEOF
-PRINTF
-CFOO
-SUBFORM
-SVREF
-STASH
-FOOS
-LC
-LD
-'N
-'X
-ERRNO
-UPPERCASING
-EXPR
-ADDR
-'STR
-STR
-DEREF
-PTR
-SWINDOW
-IWINDOW
-'SLIDER
-DRAWABLE
-'KEY
-'EXT
-TIMEOUTS
-'MY
-ID
-PIXMAPS
-'EQ
-FUNCALLED
-XWINDOW
-'IH
-SIGSTOP
-GETPID
-SIGTSTP
-SCP
-SIGINT
-IH
-CNT
-GENERALRETURN
-DEFMACX
-'NUKEGARBAGE
-GR
-HASSLE
-PREPENDS
-TIMEOUT
-FD
-MSG
-SYSCALL
-UNHELPFUL
-PREPENDED
-VM
-PAGEREF
-INT
-PORTSID
-PORTSNAME
-SERVPORT
-KERN
-DATATYPES
-TTY
-STDERR
-STDOUT
-STDIN
-CMD
-AUX
-PS
-UNACCOUNTED
-RUNTIMES
-PROFILER
-UNPROFILE
-REPROFILED
-UNPROFILED
-CF
-ELT
-VOPS
-MAPCAR
-OPTIONALS
-CONSES
-CONTORTIONS
-ALISTS
-ALIST
-ASSOC
-EXP
-MYEXP
-DEFCONSTANT
-INCF
-MEMQ
-COERCIONS
-EQL
-LOGAND
-AREF
-CONSP
-TYPEN
-LOGIOR
-EQUIV
-SUPERTYPE
-DEFMETHOD
-SUBFORMS
-CERROR
-PSETQ
-TAGBODY
-DOTIMES
-PLOQ
-ROQ
-SPECS
-MPLUS
-STEPPER
-FDEFINITION
-FUNCALLABLE
-ST
-BR
-DB
-LB
-LL
-HFILL
-PP
-VPRINT
-TH
-ARGLISTS
-SETQ
-NAMESPACE
-SUBFUNCTION
-BACKTRACE
-'B
-FLET
-ARG
-'A
-CPSUBINDEX
-PROGN
-CONTRIB
-WEEKDAYS
-GREENWICH
-TIMEZONE
-DEST
-WEEKDAY
-JAN
-CINDEX
-NAMESTRING
-PATHNAMES
-FASL
-SIGSEGV
-PLIST
-'ABLE
-SETF
-PID
-EXECVE
-DEV
-SUBPROCESS
-PTY
-'TH
-UNSUPPLIED
-DEFVARX
-GCS
-CONSED
-GC'ED
-GC
-TRASHING
-XLIB
-CL
-HI
-COMMONLOOPS
-CTRL
-XLREF
-DEFUNX
-DEFCONSTX
-SUBSUBSECTION
-VINDEXED
-TINDEXED
-RESEARCHCREDIT
-EM
-WHOLEY
-SKEF
-KAUFMANN
-TODD
-KOLOJEJCHICK
-BUSDIECKER
-''
-NOINDENT
-MOORE
-TIM
-LOTT
-LEINEN
-HALLGREN
-GLEICHAUF
-DUNNING
-TED
-BADER
-MYLISP
-NOINIT
-FINDEXED
-INIT
-EVAL
-SUBDIRECTORIES
-COPYRIGHTED
-FTP
-LANG
-COMP
-MEG
-MEGABYTES
-UNCOMPRESS
-CD
-OS
-USERNAME
-SLISP
-RT
-LIB
-SETENV
-SAMP
-SETPATH
-LOGIN
-MISC
-USR
-MODMISC
-TXT
-DOC
-EXECUTABLES
-PERQ
-UNTAGGED
-BENCHMARKING
-WINDOWING
-INTRO
-DOCS
-EDU
-AFS
-VSPACE
-IFINFO
-DIR
-SETFILENAME
-TABLEOFCONTENTS
-PAGENUMBERING
-CLEARPAGE
-MAKETITLE
-ARPASUPPORT
-CITATIONINFO
-TRNUMBER
-IFTEX
-SUNOS
-SPARC
-DECSTATIONS
-THEABSTRACT
-DEF
-KY
-CP
-NEWINDEX
-ALWAYSREFILL
-PAGESTYLE
-CMULISP
-TITLEPAGE
-ELISP
-LATEXINFO
-DOCUMENTSTYLE
+++ /dev/null
-%% CMU Common Lisp User's Manual.
-%%
-%% Aug 97 Raymond Toy
-%% This is a modified version of the original CMUCL User's Manual.
-%% The key changes are modification of this file to use standard
-%% LaTeX2e. This means latexinfo isn't going to work anymore.
-%% However, Latex2html support has been added.
-%%
-%% Jan 1998 Paul Werkowski
-%% A few of the packages below are not part of the standard LaTeX2e
-%% distribution, and must be obtained from a repository. At this time
-%% I was able to fetch from
-%% ftp.cdrom.com:pub/tex/ctan/macros/latex/contrib/supported/
-%% camel/index.ins
-%% camel/index.dtx
-%% calc/calc.ins
-%% calc/calc.dtx
-%% changebar/changebar.ins
-%% changebar/changebar.dtx
-%% One runs latex on the .ins file to produce .tex and/or .sty
-%% files that must be put in a path searched by latex.
-%%
-\documentclass{report}
-\usepackage{changebar}
-\usepackage{xspace}
-\usepackage{alltt}
-\usepackage{index}
-\usepackage{verbatim}
-\usepackage{ifthen}
-\usepackage{calc}
-%\usepackage{html2e}
-\usepackage{html,color}
-\usepackage{varioref}
-
-%% Define the indices. We need one for Types, Variables, Functions,
-%% and a general concept index.
-\makeindex
-\newindex{types}{tdx}{tnd}{Type Index}
-\newindex{vars}{vdx}{vnd}{Variable Index}
-\newindex{funs}{fdx}{fnd}{Function Index}
-\newindex{concept}{cdx}{cnd}{Concept Index}
-
-\newcommand{\tindexed}[1]{\index[types]{#1}\textsf{#1}}
-\newcommand{\findexed}[1]{\index[funs]{#1}\textsf{#1}}
-\newcommand{\vindexed}[1]{\index[vars]{#1}\textsf{*#1*}}
-\newcommand{\cindex}[1]{\index[concept]{#1}}
-\newcommand{\cpsubindex}[2]{\index[concept]{#1!#2}}
-
-%% This code taken from the LaTeX companion. It's meant as a
-%% replacement for the description environment. We want one that
-%% prints description items in a fixed size box and puts the
-%% description itself on the same line or the next depending on the
-%% size of the item.
-\newcommand{\entrylabel}[1]{\mbox{#1}\hfil}
-\newenvironment{entry}{%
- \begin{list}{}%
- {\renewcommand{\makelabel}{\entrylabel}%
- \setlength{\labelwidth}{45pt}%
- \setlength{\leftmargin}{\labelwidth+\labelsep}}}%
- {\end{list}}
-
-\newlength{\Mylen}
-\newcommand{\Lentrylabel}[1]{%
- \settowidth{\Mylen}{#1}%
- \ifthenelse{\lengthtest{\Mylen > \labelwidth}}%
- {\parbox[b]{\labelwidth}% term > labelwidth
- {\makebox[0pt][l]{#1}\\}}%
- {#1}%
- \hfil\relax}
-\newenvironment{Lentry}{%
- \renewcommand{\entrylabel}{\Lentrylabel}
- \begin{entry}}%
- {\end{entry}}
-
-\newcommand{\fcntype}[1]{\textit{#1}}
-\newcommand{\argtype}[1]{\textit{#1}}
-\newcommand{\fcnname}[1]{\textsf{#1}}
-
-\newlength{\formnamelen} % length of a name of a form
-\newlength{\pboxargslen} % length of parbox for arguments
-\newlength{\typelen} % length of the type label for the form
-
-\newcommand{\args}[1]{#1}
-\newcommand{\keys}[1]{\textsf{\&key} \= #1}
-\newcommand{\morekeys}[1]{\\ \> #1}
-\newcommand{\yetmorekeys}[1]{\\ \> #1}
-
-\newcommand{\defunvspace}{\ifhmode\unskip \par\fi\addvspace{18pt plus 12pt minus 6pt}}
-
-
-%% \layout[pkg]{name}{param list}{type}
-%%
-%% This lays out a entry like so:
-%%
-%% pkg:name arg1 arg2 [Function]
-%%
-%% where [Function] is flush right.
-%%
-\newcommand{\layout}[4][\mbox{}]{%
- \par\noindent
- \fcnname{#1#2\hspace{1em}}%
- \settowidth{\formnamelen}{\fcnname{#1#2\hspace{1em}}}%
- \settowidth{\typelen}{[\argtype{#4}]}%
- \setlength{\pboxargslen}{\linewidth}%
- \addtolength{\pboxargslen}{-1\formnamelen}%
- \addtolength{\pboxargslen}{-1\typelen}%
- \begin{minipage}[t]{\pboxargslen}
- \begin{tabbing}
- #3
- \end{tabbing}
- \end{minipage}
- \hfill[\fcntype{#4}]%
- \par\addvspace{2pt plus 2pt minus 2pt}}
-
-\newcommand{\vrindexbold}[1]{\index[vars]{#1|textbf}}
-\newcommand{\fnindexbold}[1]{\index[funs]{#1|textbf}}
-
-%% Define a new type
-%%
-%% \begin{deftp}{typeclass}{typename}{args}
-%% some description
-%% \end{deftp}
-\newenvironment{deftp}[3]{%
- \par\bigskip\index[types]{#2|textbf}%
- \layout{#2}{\var{#3}}{#1}
- }{}
-
-%% Define a function
-%%
-%% \begin{defun}{pkg}{name}{params}
-%% \defunx[pkg]{name}{params}
-%% description of function
-%% \end{defun}
-\newenvironment{defun}[3]{%
- \par\defunvspace\fnindexbold{#2}\label{FN:#2}%
- \layout[#1]{#2}{#3}{Function}
- }{}
-\newcommand{\defunx}[3][\mbox{}]{%
- \par\fnindexbold{#2}\label{FN:#2}%
- \layout[#1]{#2}{#3}{Function}}
-
-%% Define a macro
-%%
-%% \begin{defmac}{pkg}{name}{params}
-%% \defmacx[pkg]{name}{params}
-%% description of macro
-%% \end{defmac}
-\newenvironment{defmac}[3]{%
- \par\defunvspace\fnindexbold{#2}\label{FN:#2}%
- \layout[#1]{#2}{#3}{Macro}}{}
-\newcommand{\defmacx}[3][\mbox{}]{%
- \par\fnindexbold{#2}\label{FN:#2}%
- \layout[#1]{#2}{#3}{Function}}
-
-%% Define a variable
-%%
-%% \begin{defvar}{pkg}{name}
-%% \defvarx[pkg]{name}
-%% description of defvar
-%% \end{defvar}
-\newenvironment{defvar}[2]{%
- \par\defunvspace\vrindexbold{#2}\label{VR:#2}
- \layout[#1]{*#2*}{}{Variable}}{}
-\newcommand{\defvarx}[2][\mbox{}]{%
- \par\vrindexbold{#2}\label{VR:#2}
- \layout[#1]{*#2*}{}{Variable}}
-
-%% Define a constant
-%%
-%% \begin{defconst}{pkg}{name}
-%% \ddefconstx[pkg]{name}
-%% description of defconst
-%% \end{defconst}
-\newcommand{\defconstx}[2][\mbox{}]{%
- \layout[#1]{#2}{}{Constant}}
-\newenvironment{defconst}[2]{%
- \defunvspace\defconstx[#1]{#2}}
-
-\newenvironment{example}{\begin{quote}\begin{alltt}}{\end{alltt}\end{quote}}
-\newenvironment{lisp}{\begin{example}}{\end{example}}
-\newenvironment{display}{\begin{quote}\begin{alltt}}{\end{alltt}\end{quote}}
-
-\newcommand{\hide}[1]{}
-\newcommand{\trnumber}[1]{#1}
-\newcommand{\citationinfo}[1]{#1}
-\newcommand{\var}[1]{{\textsf{\textsl{#1}}\xspace}}
-\newcommand{\code}[1]{\textnormal{{\sffamily #1}}}
-\newcommand{\file}[1]{`\texttt{#1}'}
-\newcommand{\samp}[1]{`\texttt{#1}'}
-\newcommand{\kwd}[1]{\code{:#1}}
-\newcommand{\F}[1]{\code{#1}}
-\newcommand{\w}[1]{\hbox{#1}}
-\renewcommand{\b}[1]{\textrm{\textbf{#1}}}
-\renewcommand{\i}[1]{\textit{#1}}
-\newcommand{\ctrl}[1]{$\uparrow$\textsf{#1}}
-\newcommand{\result}{$\Rightarrow$}
-\newcommand{\myequiv}{$\equiv$}
-\newcommand{\back}[1]{\(\backslash\)#1}
-\newcommand{\pxlref}[1]{see section~\ref{#1}, page~\pageref{#1}}
-\newcommand{\xlref}[1]{See section~\ref{#1}, page~\pageref{#1}}
-
-\newcommand{\false}{\textsf{nil}}
-\newcommand{\true}{\textsf{t}}
-\newcommand{\nil}{\textsf{nil}}
-\newcommand{\FALSE}{\textsf{nil}}
-\newcommand{\TRUE}{\textsf{t}}
-\newcommand{\NIL}{\textsf{nil}}
-
-\newcommand{\ampoptional}{\textsf{\&optional}}
-\newcommand{\amprest}{\textsf{\&rest}}
-\newcommand{\ampbody}{\textsf{\&body}}
-\newcommand{\mopt}[1]{{$\,\{$}\textnormal{\textsf{\textsl{#1\/}}}{$\}\,$}}
-\newcommand{\mstar}[1]{{$\,\{$}\textnormal{\textsf{\textsl{#1\/}}}{$\}^*\,$}}
-\newcommand{\mplus}[1]{{$\,\{$}\textnormal{\textsf{\textsl{#1\/}}}{$\}^+\,$}}
-\newcommand{\mgroup}[1]{{$\,\{$}\textnormal{\textsf{\textsl{#1\/}}}{$\}\,$}}
-\newcommand{\mor}{$|$}
-
-\newcommand{\funref}[1]{\findexed{#1} (page~\pageref{FN:#1})}
-\newcommand{\specref}[1]{\findexed{#1} (page~\pageref{FN:#1})}
-\newcommand{\macref}[1]{\findexed{#1} (page~\pageref{FN:#1})}
-\newcommand{\varref}[1]{\vindexed{#1} (page~\pageref{VR:#1})}
-\newcommand{\conref}[1]{\conindexed{#1} (page~\pageref{VR:#1})}
-
-%% Some common abbreviations
-\newcommand{\clisp}{Common Lisp}
-\newcommand{\dash}{---}
-\newcommand{\alien}{Alien}
-\newcommand{\aliens}{Aliens}
-\newcommand{\Aliens}{Aliens}
-\newcommand{\Alien}{Alien}
-\newcommand{\Hemlock}{Hemlock}
-\newcommand{\hemlock}{Hemlock}
-\newcommand{\python}{Python}
-\newcommand{\Python}{Python}
-\newcommand{\cmucl}{CMU Common Lisp}
-\newcommand{\llisp}{Common Lisp}
-\newcommand{\Llisp}{Common Lisp}
-\newcommand{\cltl}{\emph{Common Lisp: The Language}}
-\newcommand{\cltltwo}{\emph{Common Lisp: The Language 2}}
-
-%% Replacement commands when we run latex2html. This should be last
-%% so that latex2html uses these commands instead of the LaTeX
-%% commands above.
-\begin{htmlonly}
- \usepackage{makeidx}
-
- \newcommand{\var}[1]{\textnormal{\textit{#1}}}
- \newcommand{\code}[1]{\textnormal{\texttt{#1}}}
- %%\newcommand{\printindex}[1][\mbox{}]{}
-
- %% We need the quote environment because the alltt is broken. The
- %% quote environment helps us in postprocessing to result to get
- %% what we want.
- \newenvironment{example}{\begin{quote}\begin{alltt}}{\end{alltt}\end{quote}}
- \newenvironment{display}{\begin{quote}\begin{alltt}}{\end{alltt}\end{quote}}
-
- \newcommand{\textnormal}[1]{\rm #1}
- \newcommand{\hbox}[1]{\mbox{#1}}
- \newcommand{\xspace}{}
- \newcommand{newindex}[4]{}
-
- \newcommand{\pxlref}[1]{see section~\ref{#1}}
- \newcommand{\xlref}[1]{See section~\ref{#1}}
-
- \newcommand{\tindexed}[1]{\index{#1}\texttt{#1}}
- \newcommand{\findexed}[1]{\index{#1}\texttt{#1}}
- \newcommand{\vindexed}[1]{\index{#1}\texttt{*#1*}}
- \newcommand{\cindex}[1]{\index{#1}}
- \newcommand{\cpsubindex}[2]{\index{#1!#2}}
-
- \newcommand{\keys}[1]{\texttt{\&key} #1}
- \newcommand{\morekeys}[1]{#1}
- \newcommand{\yetmorekeys}[1]{#1}
-
- \newenvironment{defun}[3]{%
- \textbf{[Function]}\\
- \texttt{#1#2} \emph{#3}\\}{}
- \newcommand{\defunx}[3][\mbox{}]{%
- \texttt{#1#2} {\em #3}\\}
- \newenvironment{defmac}[3]{%
- \textbf{[Macro]}\\
- \texttt{#1#2} \emph{#3}\\}{}
- \newcommand{\defmacx}[3][\mbox{}]{%
- \texttt{#1#2} {\em #3}\\}
- \newenvironment{defvar}[2]{%
- \textbf{[Variable]}\\
- \texttt{#1*#2*}\\ \\}{}
- \newcommand{\defvarx}[2][\mbox{}]{%
- \texttt{#1*#2*}\\}
- \newenvironment{defconst}[2]{%
- \textbf{[Constant]}\\
- \texttt{#1#2}\\}{}
- \newcommand{\defconstx}[2][\mbox{}]{\texttt{#1#2}\\}
- \newenvironment{deftp}[3]{%
- \textbf{[#1]}\\
- \texttt{#2} \textit{#3}\\}{}
- \newenvironment{Lentry}{\begin{description}}{\end{description}}
-\end{htmlonly}
-
-%% Set up margins
-\setlength{\oddsidemargin}{-10pt}
-\setlength{\evensidemargin}{-10pt}
-\setlength{\topmargin}{-40pt}
-\setlength{\headheight}{12pt}
-\setlength{\headsep}{25pt}
-\setlength{\footskip}{30pt}
-\setlength{\textheight}{9.25in}
-\setlength{\textwidth}{6.75in}
-\setlength{\columnsep}{0.375in}
-\setlength{\columnseprule}{0pt}
-
-
-\setcounter{tocdepth}{2}
-\setcounter{secnumdepth}{3}
-\def\textfraction{.1}
-\def\bottomfraction{.9} % was .3
-\def\topfraction{.9}
-
-\pagestyle{headings}
-
-\begin{document}
-%%\alwaysrefill
-\relax
-%%\newindex{cp}
-%%\newindex{ky}
-
-\newcommand{\theabstract}{%
-
- CMU Common Lisp is an implementation of that Common Lisp runs on
- various Unix workstations. See the README file in the distribution
- for current platforms. The largest single part of this document
- describes the Python compiler and the programming styles and
- techniques that the compiler encourages. The rest of the document
- describes extensions and the implementation dependent choices made
- in developing this implementation of Common Lisp. We have added
- several extensions, including a source level debugger, an interface
- to Unix system calls, a foreign function call interface, support for
- interprocess communication and remote procedure call, and other
- features that provide a good environment for developing Lisp code.
- }
-
-\newcommand{\researchcredit}{%
- This research was sponsored by the Defense Advanced Research
- Projects Agency, Information Science and Technology Office, under
- the title \emph{Research on Parallel Computing} issued by DARPA/CMO
- under Contract MDA972-90-C-0035 ARPA Order No. 7330.
-
- The views and conclusions contained in this document are those of
- the authors and should not be interpreted as representing the
- official policies, either expressed or implied, of the Defense
- Advanced Research Projects Agency or the U.S. government.}
-
-\pagestyle{empty}
-\title{CMU Common Lisp User's Manual}
-
-%%\author{Robert A. MacLachlan, \var{Editor}}
-%%\date{July 1992}
-%%\trnumber{CMU-CS-92-161}
-%%\citationinfo{
-%%\begin{center}
-%%Supersedes Technical Reports CMU-CS-87-156 and CMU-CS-91-108.
-%%\end{center}
-%%}
-%%%%\arpasupport{strategic}
-%%\abstract{\theabstract}
-%%%%\keywords{lisp, Common Lisp, manual, compiler,
-%%%% programming language implementation, programming environment}
-
-%%\maketitle
-\begin{latexonly}
-
-%% \title{CMU Common Lisp User's Manual}
-
- \author{Robert A. MacLachlan,
- \emph{Editor}%
- \thanks{\small This research was sponsored by the Defense Advanced
- Research Projects Agency, Information Science and Technology
- Office, under the title \emph{Research on Parallel Computing}
- issued by DARPA/CMO under Contract MDA972-90-C-0035 ARPA Order No.
- 7330. The views and conclusions contained in this document are
- those of the authors and should not be interpreted as representing
- the official policies, either expressed or implied, of the Defense
- Advanced Research Projects Agency or the U.S. government.}}
-
-
-
-\date{\bigskip
- July 1992 \\ CMU-CS-92-161 \\
- \vspace{0.25in}
- October 31, 1997 \\
- Net Version \\
- \vspace{0.75in} {\small
- School of Computer Science \\
- Carnegie Mellon University \\
- Pittsburgh, PA 15213} \\
- \vspace{0.5in} \small Supersedes Technical Reports CMU-CS-87-156 and
- CMU-CS-91-108.\\
- \vspace{0.5in} \textbf{Abstract} \medskip
- \begin{quote}
- \theabstract
- \end{quote}
- }
-
-\maketitle
-\end{latexonly}
-
-%% Nice HTML version of the title page
-\begin{rawhtml}
-
- <h1 align=center>CMU Common Lisp User's Manual</h1>
-
- <p align=center>Robert A. MacLachlan, <EM>Editor</EM>
- </p>
- <p align=center>
- July 1992 <BR>
- CMU-CS-92-161 <BR>
- </p>
- <br>
- <p align=center>
- July 1997 <BR>
- Net Version <BR>
- </p>
-
- <p align=center>
- School of Computer Science <BR>
- Carnegie Mellon University <BR>
- Pittsburgh, PA 15213 <BR>
- </p>
- <br>
- <p>
- Supersedes Technical Reports CMU-CS-87-156 and
- CMU-CS-91-108.<BR>
- </p>
-
- <p align=center>
- <b>Abstract</b>
- <blockquote>
- CMU Common Lisp is an implementation of that Common Lisp runs on
- various Unix workstations. See the README file in the
- distribution for current platforms. The largest single part of
- this document describes the Python compiler and the programming
- styles and techniques that the compiler encourages. The rest of
- the document describes extensions and the implementation
- dependent choices made in developing this implementation of
- Common Lisp. We have added several extensions, including a
- source level debugger, an interface to Unix system calls, a
- foreign function call interface, support for interprocess
- communication and remote procedure call, and other features that
- provide a good environment for developing Lisp code.
- </blockquote>
- </p>
- <blockquote><font size=-1>
- This research was sponsored by the Defense Advanced Research
- Projects Agency, Information Science and Technology Office, under
- the title <em>Research on Parallel Computing</em> issued by DARPA/CMO
- under Contract MDA972-90-C-0035 ARPA Order No. 7330.
- <p>
- The views and conclusions contained in this document are those of
- the authors and should not be interpreted as representing the
- official policies, either expressed or implied, of the Defense
- Advanced Research Projects Agency or the U.S. government.
- </p></font>
- </blockquote>
- </p>
-\end{rawhtml}
-\clearpage
-\vspace*{\fill}
-\textbf{Keywords:} lisp, Common Lisp, manual, compiler,
-programming language implementation, programming environment
-\clearpage
-\pagestyle{headings}
-\pagenumbering{roman}
-\tableofcontents
-
-\clearpage
-\pagenumbering{arabic}
-%%\end{iftex}
-
-%%\setfilename{cmu-user.info}
-%%\node Top, Introduction, (dir), (dir)
-
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/intro.ms}
-
-
-
-\hide{ -*- Dictionary: cmu-user -*- }
-\begin{comment}
-* Introduction::
-* Design Choices and Extensions::
-* The Debugger::
-* The Compiler::
-* Advanced Compiler Use and Efficiency Hints::
-* UNIX Interface::
-* Event Dispatching with SERVE-EVENT::
-* Alien Objects::
-* Interprocess Communication under LISP::
-* Debugger Programmer's Interface::
-* Function Index::
-* Variable Index::
-* Type Index::
-* Concept Index::
-
- --- The Detailed Node Listing ---
-
-Introduction
-
-* Support::
-* Local Distribution of CMU Common Lisp::
-* Net Distribution of CMU Common Lisp::
-* Source Availability::
-* Command Line Options::
-* Credits::
-
-Design Choices and Extensions
-
-* Data Types::
-* Default Interrupts for Lisp::
-* Packages::
-* The Editor::
-* Garbage Collection::
-* Describe::
-* The Inspector::
-* Load::
-* The Reader::
-* Running Programs from Lisp::
-* Saving a Core Image::
-* Pathnames::
-* Filesystem Operations::
-* Time Parsing and Formatting::
-* Lisp Library::
-
-Data Types
-
-* Symbols::
-* Integers::
-* Floats::
-* Characters::
-* Array Initialization::
-
-Floats
-
-* IEEE Special Values::
-* Negative Zero::
-* Denormalized Floats::
-* Floating Point Exceptions::
-* Floating Point Rounding Mode::
-* Accessing the Floating Point Modes::
-
-The Inspector
-
-* The Graphical Interface::
-* The TTY Inspector::
-
-Running Programs from Lisp
-
-* Process Accessors::
-
-Pathnames
-
-* Unix Pathnames::
-* Wildcard Pathnames::
-* Logical Pathnames::
-* Search Lists::
-* Predefined Search-Lists::
-* Search-List Operations::
-* Search List Example::
-
-Logical Pathnames
-
-* Search Lists::
-* Search List Example::
-
-Search-List Operations
-
-* Search List Example::
-
-Filesystem Operations
-
-* Wildcard Matching::
-* File Name Completion::
-* Miscellaneous Filesystem Operations::
-
-The Debugger
-
-* Debugger Introduction::
-* The Command Loop::
-* Stack Frames::
-* Variable Access::
-* Source Location Printing::
-* Compiler Policy Control::
-* Exiting Commands::
-* Information Commands::
-* Breakpoint Commands::
-* Function Tracing::
-* Specials::
-
-Stack Frames
-
-* Stack Motion::
-* How Arguments are Printed::
-* Function Names::
-* Funny Frames::
-* Debug Tail Recursion::
-* Unknown Locations and Interrupts::
-
-Variable Access
-
-* Variable Value Availability::
-* Note On Lexical Variable Access::
-
-Source Location Printing
-
-* How the Source is Found::
-* Source Location Availability::
-
-Breakpoint Commands
-
-* Breakpoint Example::
-
-Function Tracing
-
-* Encapsulation Functions::
-
-The Compiler
-
-* Compiler Introduction::
-* Calling the Compiler::
-* Compilation Units::
-* Interpreting Error Messages::
-* Types in Python::
-* Getting Existing Programs to Run::
-* Compiler Policy::
-* Open Coding and Inline Expansion::
-
-Compilation Units
-
-* Undefined Warnings::
-
-Interpreting Error Messages
-
-* The Parts of the Error Message::
-* The Original and Actual Source::
-* The Processing Path::
-* Error Severity::
-* Errors During Macroexpansion::
-* Read Errors::
-* Error Message Parameterization::
-
-Types in Python
-
-* Compile Time Type Errors::
-* Precise Type Checking::
-* Weakened Type Checking::
-
-Compiler Policy
-
-* The Optimize Declaration::
-* The Optimize-Interface Declaration::
-
-Advanced Compiler Use and Efficiency Hints
-
-* Advanced Compiler Introduction::
-* More About Types in Python::
-* Type Inference::
-* Source Optimization::
-* Tail Recursion::
-* Local Call::
-* Block Compilation::
-* Inline Expansion::
-* Byte Coded Compilation::
-* Object Representation::
-* Numbers::
-* General Efficiency Hints::
-* Efficiency Notes::
-* Profiling::
-
-Advanced Compiler Introduction
-
-* Types::
-* Optimization::
-* Function Call::
-* Representation of Objects::
-* Writing Efficient Code::
-
-More About Types in Python
-
-* More Types Meaningful::
-* Canonicalization::
-* Member Types::
-* Union Types::
-* The Empty Type::
-* Function Types::
-* The Values Declaration::
-* Structure Types::
-* The Freeze-Type Declaration::
-* Type Restrictions::
-* Type Style Recommendations::
-
-Type Inference
-
-* Variable Type Inference::
-* Local Function Type Inference::
-* Global Function Type Inference::
-* Operation Specific Type Inference::
-* Dynamic Type Inference::
-* Type Check Optimization::
-
-Source Optimization
-
-* Let Optimization::
-* Constant Folding::
-* Unused Expression Elimination::
-* Control Optimization::
-* Unreachable Code Deletion::
-* Multiple Values Optimization::
-* Source to Source Transformation::
-* Style Recommendations::
-
-Tail Recursion
-
-* Tail Recursion Exceptions::
-
-Local Call
-
-* Self-Recursive Calls::
-* Let Calls::
-* Closures::
-* Local Tail Recursion::
-* Return Values::
-
-Block Compilation
-
-* Block Compilation Semantics::
-* Block Compilation Declarations::
-* Compiler Arguments::
-* Practical Difficulties::
-* Context Declarations::
-* Context Declaration Example::
-
-Inline Expansion
-
-* Inline Expansion Recording::
-* Semi-Inline Expansion::
-* The Maybe-Inline Declaration::
-
-Object Representation
-
-* Think Before You Use a List::
-* Structure Representation::
-* Arrays::
-* Vectors::
-* Bit-Vectors::
-* Hashtables::
-
-Numbers
-
-* Descriptors::
-* Non-Descriptor Representations::
-* Variables::
-* Generic Arithmetic::
-* Fixnums::
-* Word Integers::
-* Floating Point Efficiency::
-* Specialized Arrays::
-* Specialized Structure Slots::
-* Interactions With Local Call::
-* Representation of Characters::
-
-General Efficiency Hints
-
-* Compile Your Code::
-* Avoid Unnecessary Consing::
-* Complex Argument Syntax::
-* Mapping and Iteration::
-* Trace Files and Disassembly::
-
-Efficiency Notes
-
-* Type Uncertainty::
-* Efficiency Notes and Type Checking::
-* Representation Efficiency Notes::
-* Verbosity Control::
-
-Profiling
-
-* Profile Interface::
-* Profiling Techniques::
-* Nested or Recursive Calls::
-* Clock resolution::
-* Profiling overhead::
-* Additional Timing Utilities::
-* A Note on Timing::
-* Benchmarking Techniques::
-
-UNIX Interface
-
-* Reading the Command Line::
-* Lisp Equivalents for C Routines::
-* Type Translations::
-* System Area Pointers::
-* Unix System Calls::
-* File Descriptor Streams::
-* Making Sense of Mach Return Codes::
-* Unix Interrupts::
-
-Unix Interrupts
-
-* Changing Interrupt Handlers::
-* Examples of Signal Handlers::
-
-Event Dispatching with SERVE-EVENT
-
-* Object Sets::
-* The SERVE-EVENT Function::
-* Using SERVE-EVENT with Unix File Descriptors::
-* Using SERVE-EVENT with the CLX Interface to X::
-* A SERVE-EVENT Example::
-
-Using SERVE-EVENT with the CLX Interface to X
-
-* Without Object Sets::
-* With Object Sets::
-
-A SERVE-EVENT Example
-
-* Without Object Sets Example::
-* With Object Sets Example::
-
-Alien Objects
-
-* Introduction to Aliens::
-* Alien Types::
-* Alien Operations::
-* Alien Variables::
-* Alien Data Structure Example::
-* Loading Unix Object Files::
-* Alien Function Calls::
-* Step-by-Step Alien Example::
-
-Alien Types
-
-* Defining Alien Types::
-* Alien Types and Lisp Types::
-* Alien Type Specifiers::
-* The C-Call Package::
-
-Alien Operations
-
-* Alien Access Operations::
-* Alien Coercion Operations::
-* Alien Dynamic Allocation::
-
-Alien Variables
-
-* Local Alien Variables::
-* External Alien Variables::
-
-Alien Function Calls
-
-* alien-funcall:: The alien-funcall Primitive
-* def-alien-routine:: The def-alien-routine Macro
-* def-alien-routine Example::
-* Calling Lisp from C::
-
-Interprocess Communication under LISP
-
-* The REMOTE Package::
-* The WIRE Package::
-* Out-Of-Band Data::
-
-The REMOTE Package
-
-* Connecting Servers and Clients::
-* Remote Evaluations::
-* Remote Objects::
-* Host Addresses::
-
-The WIRE Package
-
-* Untagged Data::
-* Tagged Data::
-* Making Your Own Wires::
-
-Debugger Programmer's Interface
-
-* DI Exceptional Conditions::
-* Debug-variables::
-* Frames::
-* Debug-functions::
-* Debug-blocks::
-* Breakpoints::
-* Code-locations::
-* Debug-sources::
-* Source Translation Utilities::
-
-DI Exceptional Conditions
-
-* Debug-conditions::
-* Debug-errors::
-\end{comment}
-
-%%\node Introduction, Design Choices and Extensions, Top, Top
-\chapter{Introduction}
-
-CMU Common Lisp is a public-domain implementation of Common Lisp developed in
-the Computer Science Department of Carnegie Mellon University. \cmucl{} runs
-on various Unix workstations---see the README file in the distribution for
-current platforms. This document describes the implementation based on the
-Python compiler. Previous versions of CMU Common Lisp ran on the IBM RT PC
-and (when known as Spice Lisp) on the Perq workstation. See \code{man cmucl}
-(\file{man/man1/cmucl.1}) for other general information.
-
-\cmucl{} sources and executables are freely available via anonymous FTP; this
-software is ``as is'', and has no warranty of any kind. CMU and the
-authors assume no responsibility for the consequences of any use of this
-software. See \file{doc/release-notes.txt} for a description of the
-state of the release you have.
-
-\begin{comment}
-* Support::
-* Local Distribution of CMU Common Lisp::
-* Net Distribution of CMU Common Lisp::
-* Source Availability::
-* Command Line Options::
-* Credits::
-\end{comment}
-
-%%\node Support, Local Distribution of CMU Common Lisp, Introduction, Introduction
-\section{Support}
-
-The CMU Common Lisp project is no longer funded, so only minimal support is
-being done at CMU. There is a net community of \cmucl{} users and maintainers
-who communicate via comp.lang.lisp and the cmucl-bugs@cs.cmu.edu
-\begin{changebar}
- cmucl-imp@cons.org
-\end{changebar}
-mailing lists.
-
-This manual contains only implementation-specific information about
-\cmucl. Users will also need a separate manual describing the
-\clisp{} standard. \clisp{} was initially defined in \i{Common Lisp:
- The Language}, by Guy L. Steele Jr. \clisp{} is now undergoing
-standardization by the X3J13 committee of ANSI. The X3J13 spec is not
-yet completed, but a number of clarifications and modification have
-been approved. We intend that \cmucl{} will eventually adhere to the
-X3J13 spec, and we have already implemented many of the changes
-approved by X3J13.
-
-Until the X3J13 standard is completed, the second edition of
-\cltltwo{} is probably the best available manual for the language and
-for our implementation of it. This book has no official role in the
-standardization process, but it does include many of the changes
-adopted since the first edition was completed.
-
-In addition to the language itself, this document describes a number
-of useful library modules that run in \cmucl. \hemlock, an Emacs-like
-text editor, is included as an integral part of the \cmucl{}
-environment. Two documents describe \hemlock{}: the \i{Hemlock User's
- Manual}, and the \i{Hemlock Command Implementor's Manual}.
-
-%%\node Local Distribution of CMU Common Lisp, Net Distribution of CMU Common Lisp, Support, Introduction
-\section{Local Distribution of CMU Common Lisp}
-
-In CMU CS, \cmucl{} should be runnable as \file{/usr/local/bin/cmucl}.
-The full binary distribution should appear under
-\file{/usr/local/lib/cmucl/}. Note that the first time you run Lisp,
-it will take AFS several minutes to copy the image into its local
-cache. Subsequent starts will be much faster.
-
-Or, you can run directly out of the AFS release area (which may be
-necessary on SunOS machines). Put this in your \file{.login} shell
-script:
-\begin{example}
-setenv CMUCLLIB "/afs/cs/misc/cmucl/@sys/beta/lib"
-setenv PATH \${PATH}:/afs/cs/misc/cmucl/@sys/beta/bin
-\end{example}
-
-If you also set \code{MANPATH} or \code{MPATH} (depending on the Unix)
-to point to \file{/usr/local/lib/cmucl/man/}, then `\code{man cmucl}'
-will give an introduction to CMU CL and \samp{man lisp} will describe
-command line options. For installation notes, see the \file{README}
-file in the release area.
-
-See \file{/usr/local/lib/cmucl/doc} for release notes and
-documentation. Hardcopy documentation is available in the document
-room. Documentation supplements may be available for recent
-additions: see the \file{README} file.
-
-Send bug reports and questions to \samp{cmucl-bugs@cs.cmu.edu}. If
-you send a bug report to \samp{gripe} or \samp{help}, they will just
-forward it to this mailing list.
-
-%%\node Net Distribution of CMU Common Lisp, Source Availability, Local Distribution of CMU Common Lisp, Introduction
-\section{Net Distribution of CMU Common Lisp}
-
-\subsection{CMU Distribution}
-Externally, CMU Common Lisp is only available via anonymous FTP. We
-don't have the manpower to make tapes. These are our distribution
-machines:
-\begin{example}
-lisp-rt1.slisp.cs.cmu.edu (128.2.217.9)
-lisp-rt2.slisp.cs.cmu.edu (128.2.217.10)
-\end{example}
-
-Log in with the user \samp{anonymous} and \samp{username@host} as
-password (i.e. your EMAIL address.) When you log in, the current
-directory should be set to the \cmucl{} release area. If you have any
-trouble with FTP access, please send mail to \samp{slisp@cs.cmu.edu}.
-
-The release area holds compressed tar files with names of the form:
-\begin{example}
-\var{version}-\var{machine}_\var{os}.tar.Z
-\end{example}
-FTP compressed tar archives in binary mode. To extract, \samp{cd} to
-the directory that is to be the root of the tree, then type:
-\begin{example}
-uncompress <file.tar.Z | tar xf - .
-\end{example}
-The resulting tree is about 23 megabytes. For installation
-directions, see the section ``site initialization'' in README file at
-the root of the tree.
-
-If poor network connections make it difficult to transfer a 10 meg
-file, the release is also available split into five parts, with the
-suffix \file{.0} to \file{.4}. To extract from multiple files, use:
-\begin{example}
-cat file.tar.Z.* | uncompress | tar xf - .
-\end{example}
-
-The release area also contains source distributions and other binary
-distributions. A listing of the current contents of the release area
-is in \file{FILES}. Major release announcements will be made to
-\code{comp.lang.lisp} until there is enough volume to warrant a
-\code{comp.lang.lisp.cmu}.
-
-\begin{changebar}
-\subsection{Net Distribution}
-Although the CMU Common Lisp project is no longer actively developed
-by CMU, development has continued. You can obtain this version from
-either
-\begin{example}
- ftp://ftp2.cons.org/pub/languages/lisp/cmucl
- http://www2.cons.org:8000/ftp-area/cmucl/
-\end{example}
-Further information can be found via the World Wide Web at
-\begin{example}
- http://www.cons.org/cmucl
-\end{example}
-\end{changebar}
-%%\node Source Availability, Command Line Options, Net Distribution of CMU Common Lisp, Introduction
-\section{Source Availability}
-
-Lisp and documentation sources are available via anonymous FTP ftp to
-any CMU CS machine. All CMU written code is public domain, but CMU CL
-also makes use of two imported packages: PCL and CLX. Although these
-packages are copyrighted, they may be freely distributed without any
-licensing agreement or fee. See the \file{README} file in the binary
-distribution for up-to-date source pointers.
-
-The release area contains a source distribution, which is an image of
-all the \file{.lisp} source files used to build a particular system
-\var{version}:
-\begin{example}
-\var{version}-source.tar.Z (3.6 meg)
-\end{example}
-
-All of our files (including the release area) are actually in the AFS
-file system. On the release machines, the FTP server's home is the
-release directory: \file{/afs/cs.cmu.edu/project/clisp/release}. The
-actual working source areas are in other subdirectories of
-\file{clisp}, and you can directly ``cd'' to those directories if you
-know the name. Due to the way anonymous FTP access control is done,
-it is important to ``cd'' to the source directory with a single
-command, and then do a ``get'' operation.
-
-\begin{changebar}
- Alternatively, you can obtain the current sources via WWW at
- \begin{example}
- http://www.cons.org/cmucl
- \end{example}
- which contains pointers on how to get a \code{tar} file of the
- current sources or how to get an individual file from the sources.
- Binary versions for selected platforms are also available as well.
-\end{changebar}
-
-%%\node Command Line Options, Credits, Source Availability, Introduction
-\section{Command Line Options}
-
-The command line syntax and environment is described in the lisp(1)
-man page in the man/man1 directory of the distribution. See also
-cmucl(1). Currently Lisp accepts the following switches:
-\begin{Lentry}
- \begin{changebar}
- \item[\code{-batch}] specifies batch mode, where all input is
- directed from standard-input. An error code of 0 is returned upon
- encountering an EOF and 1 otherwise.
- \end{changebar}
-\item[\code{-core}] requires an argument that should be the name of a
- core file. Rather than using the default core file
- (\file{lib/lisp.core}), the specified core file is loaded.
-
-\item[\code{-edit}] specifies to enter Hemlock. A file to edit may be
- specified by placing the name of the file between the program name
- (usually \file{lisp}) and the first switch.
-
-\item[\code{-eval}] accepts one argument which should be a Lisp form
- to evaluate during the start up sequence. The value of the form
- will not be printed unless it is wrapped in a form that does output.
-
-\item[\code{-hinit}] accepts an argument that should be the name of
- the hemlock init file to load the first time the function
- \findexed{ed} is invoked. The default is to load
- \file{hemlock-init.\var{object-type}}, or if that does not exist,
- \file{hemlock-init.lisp} from the user's home directory. If the
- file is not in the user's home directory, the full path must be
- specified.
-
-\item[\code{-init}] accepts an argument that should be the name of an
- init file to load during the normal start up sequence. The default
- is to load \file{init.\var{object-type}} or, if that does not exist,
- \file{init.lisp} from the user's home directory. If the file is not
- in the user's home directory, the full path must be specified.
-
-\item[\code{-noinit}] accepts no arguments and specifies that an init
- file should not be loaded during the normal start up sequence.
- Also, this switch suppresses the loading of a hemlock init file when
- Hemlock is started up with the \code{-edit} switch.
-
-\item[\code{-load}] accepts an argument which should be the name of a
- file to load into Lisp before entering Lisp's read-eval-print loop.
-
-\item[\code{-slave}] specifies that Lisp should start up as a
- \i{slave} Lisp and try to connect to an editor Lisp. The name of
- the editor to connect to must be specified\dash{}to find the
- editor's name, use the \hemlock{} ``\code{Accept Slave
- Connections}'' command. The name for the editor Lisp is of the
- form:
- \begin{example}
- \var{machine-name}\code{:}\var{socket}
- \end{example}
- where \var{machine-name} is the internet host name for the machine
- and \var{socket} is the decimal number of the socket to connect to.
-\end{Lentry}
-For more details on the use of the \code{-edit} and \code{-slave}
-switches, see the \i{Hemlock User's Manual}.
-
-Arguments to the above switches can be specified in one of two ways:
-\w{\var{switch}\code{=}\var{value}} or
-\w{\var{switch}<\var{space}>\var{value}}. For example, to start up
-the saved core file mylisp.core use either of the following two
-commands:
-\begin{example}
-\code{lisp -core=mylisp.core
-lisp -core mylisp.core}
-\end{example}
-
-%%\node Credits, , Command Line Options, Introduction
-\section{Credits}
-
-Since 1981 many people have contributed to the development of CMU
-Common Lisp. The currently active members are:
-\begin{display}
-Marco Antoniotti
-David Axmark
-Miles Bader
-Casper Dik
-Scott Fahlman * (fearless leader)
-Paul Gleichauf *
-Richard Harris
-Joerg-Cyril Hoehl
-Chris Hoover
-Simon Leinen
-Sandra Loosemore
-William Lott *
-Robert A. Maclachlan *
-\end{display}
-\noindent
-Many people are voluntarily working on improving CMU Common Lisp. ``*''
-means a full-time CMU employee, and ``+'' means a part-time student
-employee. A partial listing of significant past contributors follows:
-\begin{display}
-Tim Moore
-Sean Hallgren +
-Mike Garland +
-Ted Dunning
-Rick Busdiecker
-Bill Chiles *
-John Kolojejchick
-Todd Kaufmann +
-Dave McDonald *
-Skef Wholey *
-\end{display}
-
-
-\vspace{2 em}
-\researchcredit
-
-\begin{changebar}
- From 1995, development of CMU Common Lisp has been continued by a
- group of volunteers. A partial list of volunteers includes the
- following
- \begin{table}[h]
- \begin{center}
- \begin{tabular}{ll}
- Paul Werkowski & pw@snoopy.mv.com \\
- Peter VanEynde & s950045@uia.ua.ac.be \\
- Marco Antoniotti & marcoxa@PATH.Berkeley.EDU\\
- Martin Cracauer & cracauer@cons.org\\
- Douglas Thomas Crosher & dtc@scrooge.ee.swin.oz.au\\
- Simon Leinen & simon@switch.ch\\
- Rob MacLachlan & ram+@CS.cmu.edu\\
- Raymond Toy & toy@rtp.ericsson.se
- \end{tabular}
- \end{center}
- \end{table}
-
- In particular Paul Werkowski completed the port for the x86
- architecture for FreeBSD. Peter VanEnyde took the FreeBSD port and
- created a Linux version.
-\end{changebar}
-
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/design.ms}
-
-\hide{ -*- Dictionary: cmu-user -*- }
-%%\node Design Choices and Extensions, The Debugger, Introduction, Top
-\chapter{Design Choices and Extensions}
-
-Several design choices in Common Lisp are left to the individual
-implementation, and some essential parts of the programming environment
-are left undefined. This chapter discusses the most important design
-choices and extensions.
-
-\begin{comment}
-* Data Types::
-* Default Interrupts for Lisp::
-* Packages::
-* The Editor::
-* Garbage Collection::
-* Describe::
-* The Inspector::
-* Load::
-* The Reader::
-* Running Programs from Lisp::
-* Saving a Core Image::
-* Pathnames::
-* Filesystem Operations::
-* Time Parsing and Formatting::
-* Lisp Library::
-\end{comment}
-
-%%\node Data Types, Default Interrupts for Lisp, Design Choices and Extensions, Design Choices and Extensions
-\section{Data Types}
-
-\begin{comment}
-* Symbols::
-* Integers::
-* Floats::
-* Characters::
-* Array Initialization::
-\end{comment}
-
-%%\node Symbols, Integers, Data Types, Data Types
-\subsection{Symbols}
-
-As in \cltl, all symbols and package names are printed in lower case, as
-a user is likely to type them. Internally, they are normally stored
-upper case only.
-
-%%\node Integers, Floats, Symbols, Data Types
-\subsection{Integers}
-
-The \tindexed{fixnum} type is equivalent to \code{(signed-byte 30)}.
-Integers outside this range are represented as a \tindexed{bignum} or
-a word integer (\pxlref{word-integers}.) Almost all integers that
-appear in programs can be represented as a \code{fixnum}, so integer
-number consing is rare.
-
-%%\node Floats, Characters, Integers, Data Types
-\subsection{Floats}
-\label{ieee-float}
-
-\cmucl{} supports two floating point formats: \tindexed{single-float}
-and \tindexed{double-float}. These are implemented with IEEE single
-and double float arithmetic, respectively. \code{short-float} is a
-synonym for \code{single-float}, and \code{long-float} is a synonym
-for \code{double-float}. The initial value of
-\vindexed{read-default-float-format} is \code{single-float}.
-
-Both \code{single-float} and \code{double-float} are represented with
-a pointer descriptor, so float operations can cause number consing.
-Number consing is greatly reduced if programs are written to allow the
-use of non-descriptor representations (\pxlref{numeric-types}.)
-
-
-\begin{comment}
-* IEEE Special Values::
-* Negative Zero::
-* Denormalized Floats::
-* Floating Point Exceptions::
-* Floating Point Rounding Mode::
-* Accessing the Floating Point Modes::
-\end{comment}
-
-%%\node IEEE Special Values, Negative Zero, Floats, Floats
-\subsubsection{IEEE Special Values}
-
-\cmucl{} supports the IEEE infinity and NaN special values. These
-non-numeric values will only be generated when trapping is disabled
-for some floating point exception (\pxlref{float-traps}), so users of
-the default configuration need not concern themselves with special
-values.
-
-\begin{defconst}{extensions:}{short-float-positive-infinity}
- \defconstx[extensions:]{short-float-negative-infinity}
- \defconstx[extensions:]{single-float-positive-infinity}
- \defconstx[extensions:]{single-float-negative-infinity}
- \defconstx[extensions:]{double-float-positive-infinity}
- \defconstx[extensions:]{double-float-negative-infinity}
- \defconstx[extensions:]{long-float-positive-infinity}
- \defconstx[extensions:]{long-float-negative-infinity}
-
- The values of these constants are the IEEE positive and negative
- infinity objects for each float format.
-\end{defconst}
-
-\begin{defun}{extensions:}{float-infinity-p}{\args{\var{x}}}
-
- This function returns true if \var{x} is an IEEE float infinity (of
- either sign.) \var{x} must be a float.
-\end{defun}
-
-\begin{defun}{extensions:}{float-nan-p}{\args{\var{x}}}
- \defunx[extensions:]{float-trapping-nan-p}{\args{\var{x}}}
-
- \code{float-nan-p} returns true if \var{x} is an IEEE NaN (Not A
- Number) object. \code{float-trapping-nan-p} returns true only if
- \var{x} is a trapping NaN. With either function, \var{x} must be a
- float.
-\end{defun}
-
-%%\node Negative Zero, Denormalized Floats, IEEE Special Values, Floats
-\subsubsection{Negative Zero}
-
-The IEEE float format provides for distinct positive and negative
-zeros. To test the sign on zero (or any other float), use the
-\clisp{} \findexed{float-sign} function. Negative zero prints as
-\code{-0.0f0} or \code{-0.0d0}.
-
-%%\node Denormalized Floats, Floating Point Exceptions, Negative Zero, Floats
-\subsubsection{Denormalized Floats}
-
-\cmucl{} supports IEEE denormalized floats. Denormalized floats
-provide a mechanism for gradual underflow. The \clisp{}
-\findexed{float-precision} function returns the actual precision of a
-denormalized float, which will be less than \findexed{float-digits}.
-Note that in order to generate (or even print) denormalized floats,
-trapping must be disabled for the underflow exception
-(\pxlref{float-traps}.) The \clisp{}
-\w{\code{least-positive-}\var{format}-\code{float}} constants are
-denormalized.
-
-\begin{defun}{extensions:}{float-normalized-p}{\args{\var{x}}}
-
- This function returns true if \var{x} is a denormalized float.
- \var{x} must be a float.
-\end{defun}
-
-%%\node Floating Point Exceptions, Floating Point Rounding Mode, Denormalized Floats, Floats
-\subsubsection{Floating Point Exceptions}
-\label{float-traps}
-
-The IEEE floating point standard defines several exceptions that occur
-when the result of a floating point operation is unclear or
-undesirable. Exceptions can be ignored, in which case some default
-action is taken, such as returning a special value. When trapping is
-enabled for an exception, a error is signalled whenever that exception
-occurs. These are the possible floating point exceptions:
-\begin{Lentry}
-
-\item[\kwd{underflow}] This exception occurs when the result of an
- operation is too small to be represented as a normalized float in
- its format. If trapping is enabled, the
- \tindexed{floating-point-underflow} condition is signalled.
- Otherwise, the operation results in a denormalized float or zero.
-
-\item[\kwd{overflow}] This exception occurs when the result of an
- operation is too large to be represented as a float in its format.
- If trapping is enabled, the \tindexed{floating-point-overflow}
- exception is signalled. Otherwise, the operation results in the
- appropriate infinity.
-
-\item[\kwd{inexact}] This exception occurs when the result of a
- floating point operation is not exact, i.e. the result was rounded.
- If trapping is enabled, the \code{extensions:floating-point-inexact}
- condition is signalled. Otherwise, the rounded result is returned.
-
-\item[\kwd{invalid}] This exception occurs when the result of an
- operation is ill-defined, such as \code{\w{(/ 0.0 0.0)}}. If
- trapping is enabled, the \code{extensions:floating-point-invalid}
- condition is signalled. Otherwise, a quiet NaN is returned.
-
-\item[\kwd{divide-by-zero}] This exception occurs when a float is
- divided by zero. If trapping is enabled, the
- \tindexed{divide-by-zero} condition is signalled. Otherwise, the
- appropriate infinity is returned.
-\end{Lentry}
-
-%%\node Floating Point Rounding Mode, Accessing the Floating Point Modes, Floating Point Exceptions, Floats
-\subsubsection{Floating Point Rounding Mode}
-\label{float-rounding-modes}
-
-IEEE floating point specifies four possible rounding modes:
-\begin{Lentry}
-
-\item[\kwd{nearest}] In this mode, the inexact results are rounded to
- the nearer of the two possible result values. If the neither
- possibility is nearer, then the even alternative is chosen. This
- form of rounding is also called ``round to even'', and is the form
- of rounding specified for the \clisp{} \findexed{round} function.
-
-\item[\kwd{positive-infinity}] This mode rounds inexact results to the
- possible value closer to positive infinity. This is analogous to
- the \clisp{} \findexed{ceiling} function.
-
-\item[\kwd{negative-infinity}] This mode rounds inexact results to the
- possible value closer to negative infinity. This is analogous to
- the \clisp{} \findexed{floor} function.
-
-\item[\kwd{zero}] This mode rounds inexact results to the possible
- value closer to zero. This is analogous to the \clisp{}
- \findexed{truncate} function.
-\end{Lentry}
-
-\paragraph{Warning:}
-
-Although the rounding mode can be changed with
-\code{set-floating-point-modes}, use of any value other than the
-default (\kwd{nearest}) can cause unusual behavior, since it will
-affect rounding done by \llisp{} system code as well as rounding in
-user code. In particular, the unary \code{round} function will stop
-doing round-to-nearest on floats, and instead do the selected form of
-rounding.
-
-%%\node Accessing the Floating Point Modes, , Floating Point Rounding Mode, Floats
-\subsubsection{Accessing the Floating Point Modes}
-
-These functions can be used to modify or read the floating point modes:
-
-\begin{defun}{extensions:}{set-floating-point-modes}{%
- \keys{\kwd{traps} \kwd{rounding-mode}}
- \morekeys{\kwd{fast-mode} \kwd{accrued-exceptions}}
- \yetmorekeys{\kwd{current-exceptions}}}
- \defunx[extensions:]{get-floating-point-modes}{}
-
- The keyword arguments to \code{set-floating-point-modes} set various
- modes controlling how floating point arithmetic is done:
- \begin{Lentry}
-
- \item[\kwd{traps}] A list of the exception conditions that should
- cause traps. Possible exceptions are \kwd{underflow},
- \kwd{overflow}, \kwd{inexact}, \kwd{invalid} and
- \kwd{divide-by-zero}. Initially all traps except \kwd{inexact}
- are enabled. \xlref{float-traps}.
-
- \item[\kwd{rounding-mode}] The rounding mode to use when the result
- is not exact. Possible values are \kwd{nearest},
- \latex{\kwd{positive\-infinity}}\html{\kwd{positive-infinity}},
- \kwd{negative-infinity} and \kwd{zero}. Initially, the rounding
- mode is \kwd{nearest}. See the warning in section
- \ref{float-rounding-modes} about use of other rounding modes.
-
- \item[\kwd{current-exceptions}, \kwd{accrued-exceptions}] Lists of
- exception keywords used to set the exception flags. The
- \var{current-exceptions} are the exceptions for the previous
- operation, so setting it is not very useful. The
- \var{accrued-exceptions} are a cumulative record of the exceptions
- that occurred since the last time these flags were cleared.
- Specifying \code{()} will clear any accrued exceptions.
-
- \item[\kwd{fast-mode}] Set the hardware's ``fast mode'' flag, if
- any. When set, IEEE conformance or debuggability may be impaired.
- Some machines may not have this feature, in which case the value
- is always \false. No currently supported machines have a fast
- mode.
- \end{Lentry}
- If a keyword argument is not supplied, then the associated state is
- not changed.
-
- \code{get-floating-point-modes} returns a list representing the
- state of the floating point modes. The list is in the same format
- as the keyword arguments to \code{set-floating-point-modes}, so
- \code{apply} could be used with \code{set-floating-point-modes} to
- restore the modes in effect at the time of the call to
- \code{get-floating-point-modes}.
-\end{defun}
-
-\begin{changebar}
-To make handling control of floating-point exceptions, the following
-macro is useful.
-
-\begin{defmac}{ext:}{with-float-traps-masked}{traps \ampbody\ body}
- \code{body} is executed with the selected floating-point exceptions
- given by \code{traps} masked out (disabled). \code{traps} should be
- a list of possible floating-point exceptions that should be ignored.
- Possible values are \kwd{underflow}, \kwd{overflow}, \kwd{inexact},
- \kwd{invalid} and \kwd{divide-by-zero}.
-
- This is equivalent to saving the current traps from
- \code{get-floating-point-modes}, setting the floating-point modes to
- the desired exceptions, running the \code{body}, and restoring the
- saved floating-point modes. The advantage of this macro is that it
- causes less consing to occur.
-
- Some points about the with-float-traps-masked:
-
- \begin{itemize}
- \item Two approaches are available for detecting FP exceptions:
- \begin{enumerate}
- \item enabling the traps and handling the exceptions
- \item disabling the traps and either handling the return values or
- checking the accrued exceptions.
- \end{enumerate}
- Of these the latter is the most portable because on the alpha port
- it is not possible to enable some traps at run-time.
-
- \item To assist the checking of the exceptions within the body any
- accrued exceptions matching the given traps are cleared at the
- start of the body when the traps are masked.
-
- \item To allow the macros to be nested these accrued exceptions are
- restored at the end of the body to their values at the start of
- the body. Thus any exceptions that occurred within the body will
- not affect the accrued exceptions outside the macro.
-
- \item Note that only the given exceptions are restored at the end of
- the body so other exception will be visible in the accrued
- exceptions outside the body.
-
- \item On the x86, setting the accrued exceptions of an unmasked
- exception would cause a FP trap. The macro behaviour of restoring
- the accrued exceptions ensures than if an accrued exception is
- initially not flagged and occurs within the body it will be
- restored/cleared at the exit of the body and thus not cause a
- trap.
-
- \item On the x86, and, perhaps, the hppa, the FP exceptions may be
- delivered at the next FP instruction which requires a FP
- \code{wait} instruction (\code{%vm::float-wait}) if using the lisp
- conditions to catch trap within a \code{handler-bind}. The
- \code{handler-bind} macro does the right thing and inserts a
- float-wait (at the end of its body on the x86). The masking and
- noting of exceptions is also safe here.
-
- \item The setting of the FP flags uses the
- \code{(floating-point-modes)} and the \code{(set
- (floating-point-modes)\ldots)} VOPs. These VOPs blindly update
- the flags which may include other state. We assume this state
- hasn't changed in between getting and setting the state. For
- example, if you used the FP unit between the above calls, the
- state may be incorrectly restored! The
- \code{with-float-traps-masked} macro keeps the intervening code to
- a minimum and uses only integer operations.
- %% Safe byte-compiled?
- %% Perhaps the VOPs (x86) should be smarter and only update some of
- %% the flags, the trap masks and exceptions?
- \end{itemize}
-
-\end{defmac}
-\end{changebar}
-
-%%\node Characters, Array Initialization, Floats, Data Types
-\subsection{Characters}
-
-\cmucl{} implements characters according to \i{Common Lisp: the
- Language II}. The main difference from the first version is that
-character bits and font have been eliminated, and the names of the
-types have been changed. \tindexed{base-character} is the new
-equivalent of the old \tindexed{string-char}. In this implementation,
-all characters are base characters (there are no extended characters.)
-Character codes range between \code{0} and \code{255}, using the ASCII
-encoding.
-\begin{changebar}
- Table~\ref{tbl:chars}~\vpageref{tbl:chars} shows characters
- recognized by \cmucl.
-\end{changebar}
-
-\begin{changebar}
-\begin{table}[tbhp]
- \begin{center}
- \begin{tabular}{|c|c|l|l|l|l|}
- \hline
- \multicolumn{2}{|c|}{ASCII} & \multicolumn{1}{|c}{Lisp} &
- \multicolumn{3}{|c|}{} \\
- \cline{1-2}
- Name & Code & \multicolumn{1}{|c|}{Name} & \multicolumn{3}{|c|}{\raisebox{1.5ex}{Alternatives}}\\
- \hline
- \hline
- \code{nul} & 0 & \code{\#\back{NULL}} & \code{\#\back{NUL}} & &\\
- \code{bel} & 7 & \code{\#\back{BELL}} & & &\\
- \code{bs} & 8 & \code{\#\back{BACKSPACE}} & \code{\#\back{BS}} & &\\
- \code{tab} & 9 & \code{\#\back{TAB}} & & &\\
- \code{lf} & 10 & \code{\#\back{NEWLINE}} & \code{\#\back{NL}} & \code{\#\back{LINEFEED}} & \code{\#\back{LF}}\\
- \code{ff} & 11 & \code{\#\back{VT}} & \code{\#\back{PAGE}} & \code{\#\back{FORM}} &\\
- \code{cr} & 13 & \code{\#\back{RETURN}} & \code{\#\back{CR}} & &\\
- \code{esc} & 27 & \code{\#\back{ESCAPE}} & \code{\#\back{ESC}} & \code{\#\back{ALTMODE}} & \code{\#\back{ALT}}\\
- \code{sp} & 32 & \code{\#\back{SPACE}} & \code{\#\back{SP}} & &\\
- \code{del} & 127 & \code{\#\back{DELETE}} & \code{\#\back{RUBOUT}} & &\\
- \hline
- \end{tabular}
- \caption{Characters recognized by \cmucl}
- \label{tbl:chars}
- \end{center}
-\end{table}
-\end{changebar}
-
-%%\node Array Initialization, , Characters, Data Types
-\subsection{Array Initialization}
-
-If no \kwd{initial-value} is specified, arrays are initialized to zero.
-
-
-%%\node Default Interrupts for Lisp, Packages, Data Types, Design Choices and Extensions
-\section{Default Interrupts for Lisp}
-
-CMU Common Lisp has several interrupt handlers defined when it starts up,
-as follows:
-\begin{Lentry}
-
-\item[\code{SIGINT} (\ctrl{c})] causes Lisp to enter a break loop.
- This puts you into the debugger which allows you to look at the
- current state of the computation. If you proceed from the break
- loop, the computation will proceed from where it was interrupted.
-
-\item[\code{SIGQUIT} (\ctrl{L})] causes Lisp to do a throw to the
- top-level. This causes the current computation to be aborted, and
- control returned to the top-level read-eval-print loop.
-
-\item[\code{SIGTSTP} (\ctrl{z})] causes Lisp to suspend execution and
- return to the Unix shell. If control is returned to Lisp, the
- computation will proceed from where it was interrupted.
-
-\item[\code{SIGILL}, \code{SIGBUS}, \code{SIGSEGV}, and \code{SIGFPE}]
- cause Lisp to signal an error.
-\end{Lentry}
-For keyboard interrupt signals, the standard interrupt character is in
-parentheses. Your \file{.login} may set up different interrupt
-characters. When a signal is generated, there may be some delay before
-it is processed since Lisp cannot be interrupted safely in an arbitrary
-place. The computation will continue until a safe point is reached and
-then the interrupt will be processed. \xlref{signal-handlers} to define
-your own signal handlers.
-
-%%\node Packages, The Editor, Default Interrupts for Lisp, Design Choices and Extensions
-\section{Packages}
-
-When CMU Common Lisp is first started up, the default package is the
-\code{user} package. The \code{user} package uses the
-\code{common-lisp}, \code{extensions}, and \code{pcl} packages. The
-symbols exported from these three packages can be referenced without
-package qualifiers. This section describes packages which have
-exported interfaces that may concern users. The numerous internal
-packages which implement parts of the system are not described here.
-Package nicknames are in parenthesis after the full name.
-\begin{Lentry}
-\item[\code{alien}, \code{c-call}] Export the features of the Alien
- foreign data structure facility (\pxlref{aliens}.)
-
-\item[\code{pcl}] This package contains PCL (Portable CommonLoops),
- which is a portable implementation of CLOS (the Common Lisp Object
- System.) This implements most (but not all) of the features in the
- CLOS chapter of \cltltwo.
-
-\item[\code{debug}] The \code{debug} package contains the command-line
- oriented debugger. It exports utility various functions and
- switches.
-
-\item[\code{debug-internals}] The \code{debug-internals} package
- exports the primitives used to write debuggers.
- \xlref{debug-internals}.
-
-\item[\code{extensions (ext)}] The \code{extensions} packages exports
- local extensions to Common Lisp that are documented in this manual.
- Examples include the \code{save-lisp} function and time parsing.
-
-\item[\code{hemlock (ed)}] The \code{hemlock} package contains all the
- code to implement Hemlock commands. The \code{hemlock} package
- currently exports no symbols.
-
-\item[\code{hemlock-internals (hi)}] The \code{hemlock-internals}
- package contains code that implements low level primitives and
- exports those symbols used to write Hemlock commands.
-
-\item[\code{keyword}] The \code{keyword} package contains keywords
- (e.g., \kwd{start}). All symbols in the \code{keyword} package are
- exported and evaluate to themselves (i.e., the value of the symbol
- is the symbol itself).
-
-\item[\code{profile}] The \code{profile} package exports a simple
- run-time profiling facility (\pxlref{profiling}).
-
-\item[\code{common-lisp (cl lisp)}] The \code{common-lisp} package
- exports all the symbols defined by \i{Common Lisp: the Language} and
- only those symbols. Strictly portable Lisp code will depend only on
- the symbols exported from the \code{lisp} package.
-
-\item[\code{unix}, \code{mach}] These packages export system call
- interfaces to generic BSD Unix and Mach (\pxlref{unix-interface}).
-
-\item[\code{system (sys)}] The \code{system} package contains
- functions and information necessary for system interfacing. This
- package is used by the \code{lisp} package and exports several
- symbols that are necessary to interface to system code.
-
-\item[\code{common-lisp-user (user cl-user)}] The
- \code{common-lisp-user} package is the default package and is where
- a user's code and data is placed unless otherwise specified. This
- package exports no symbols.
-
-\item[\code{xlib}] The \code{xlib} package contains the Common Lisp X
- interface (CLX) to the X11 protocol. This is mostly Lisp code with
- a couple of functions that are defined in C to connect to the
- server.
-
-\item[\code{wire}] The \code{wire} package exports a remote procedure
- call facility (\pxlref{remote}).
-\end{Lentry}
-
-
-%%\node The Editor, Garbage Collection, Packages, Design Choices and Extensions
-\section{The Editor}
-
-The \code{ed} function invokes the Hemlock editor which is described
-in \i{Hemlock User's Manual} and \i{Hemlock Command Implementor's
- Manual}. Most users at CMU prefer to use Hemlock's slave \Llisp{}
-mechanism which provides an interactive buffer for the
-\code{read-eval-print} loop and editor commands for evaluating and
-compiling text from a buffer into the slave \Llisp. Since the editor
-runs in the \Llisp, using slaves keeps users from trashing their
-editor by developing in the same \Llisp{} with \Hemlock.
-
-
-%%\node Garbage Collection, Describe, The Editor, Design Choices and Extensions
-\section{Garbage Collection}
-
-CMU Common Lisp uses a stop-and-copy garbage collector that compacts
-the items in dynamic space every time it runs. Most users cause the
-system to garbage collect (GC) frequently, long before space is
-exhausted. With 16 or 24 megabytes of memory, causing GC's more
-frequently on less garbage allows the system to GC without much (if
-any) paging.
-
-\hide{
-With the default value for the following variable, you can expect a GC to take
-about one minute of elapsed time on a 6 megabyte machine running X as well as
-Lisp. On machines with 8 megabytes or more of memory a GC should run without
-much (if any) paging. GC's run more frequently but tend to take only about 5
-seconds.
-}
-
-The following functions invoke the garbage collector or control whether
-automatic garbage collection is in effect:
-
-\begin{defun}{extensions:}{gc}{}
-
- This function runs the garbage collector. If
- \code{ext:*gc-verbose*} is non-\nil, then it invokes
- \code{ext:*gc-notify-before*} before GC'ing and
- \code{ext:*gc-notify-after*} afterwards.
-\end{defun}
-
-\begin{defun}{extensions:}{gc-off}{}
-
- This function inhibits automatic garbage collection. After calling
- it, the system will not GC unless you call \code{ext:gc} or
- \code{ext:gc-on}.
-\end{defun}
-
-\begin{defun}{extensions:}{gc-on}{}
-
- This function reinstates automatic garbage collection. If the
- system would have GC'ed while automatic GC was inhibited, then this
- will call \code{ext:gc}.
-\end{defun}
-
-%%\node
-\subsection{GC Parameters}
-The following variables control the behavior of the garbage collector:
-
-\begin{defvar}{extensions:}{bytes-consed-between-gcs}
-
- CMU Common Lisp automatically GC's whenever the amount of memory
- allocated to dynamic objects exceeds the value of an internal
- variable. After each GC, the system sets this internal variable to
- the amount of dynamic space in use at that point plus the value of
- the variable \code{ext:*bytes-consed-between-gcs*}. The default
- value is 2000000.
-\end{defvar}
-
-\begin{defvar}{extensions:}{gc-verbose}
-
- This variable controls whether \code{ext:gc} invokes the functions
- in \code{ext:*gc-notify-before*} and
- \code{ext:*gc-notify-after*}. If \code{*gc-verbose*} is \nil,
- \code{ext:gc} foregoes printing any messages. The default value is
- \code{T}.
-\end{defvar}
-
-\begin{defvar}{extensions:}{gc-notify-before}
-
- This variable's value is a function that should notify the user that
- the system is about to GC. It takes one argument, the amount of
- dynamic space in use before the GC measured in bytes. The default
- value of this variable is a function that prints a message similar
- to the following:
-\begin{display}
- \b{[GC threshold exceeded with 2,107,124 bytes in use. Commencing GC.]}
-\end{display}
-\end{defvar}
-
-\begin{defvar}{extensions:}{gc-notify-after}
-
- This variable's value is a function that should notify the user when
- a GC finishes. The function must take three arguments, the amount
- of dynamic spaced retained by the GC, the amount of dynamic space
- freed, and the new threshold which is the minimum amount of space in
- use before the next GC will occur. All values are byte quantities.
- The default value of this variable is a function that prints a
- message similar to the following:
- \begin{display}
- \b{[GC completed with 25,680 bytes retained and 2,096,808 bytes freed.]}
- \b{[GC will next occur when at least 2,025,680 bytes are in use.]}
- \end{display}
-\end{defvar}
-
-Note that a garbage collection will not happen at exactly the new
-threshold printed by the default \code{ext:*gc-notify-after*}
-function. The system periodically checks whether this threshold has
-been exceeded, and only then does a garbage collection.
-
-\begin{defvar}{extensions:}{gc-inhibit-hook}
-
- This variable's value is either a function of one argument or \nil.
- When the system has triggered an automatic GC, if this variable is a
- function, then the system calls the function with the amount of
- dynamic space currently in use (measured in bytes). If the function
- returns \nil, then the GC occurs; otherwise, the system inhibits
- automatic GC as if you had called \code{ext:gc-off}. The writer of
- this hook is responsible for knowing when automatic GC has been
- turned off and for calling or providing a way to call
- \code{ext:gc-on}. The default value of this variable is \nil.
-\end{defvar}
-
-\begin{defvar}{extensions:}{before-gc-hooks}
- \defvarx[extensions:]{after-gc-hooks}
-
- These variables' values are lists of functions to call before or
- after any GC occurs. The system provides these purely for
- side-effect, and the functions take no arguments.
-\end{defvar}
-
-%%\node
-\subsection{Weak Pointers}
-
-A weak pointer provides a way to maintain a reference to an object
-without preventing an object from being garbage collected. If the
-garbage collector discovers that the only pointers to an object are
-weak pointers, then it breaks the weak pointers and deallocates the
-object.
-
-\begin{defun}{extensions:}{make-weak-pointer}{\args{\var{object}}}
- \defunx[extensions:]{weak-pointer-value}{\args{\var{weak-pointer}}}
-
- \code{make-weak-pointer} returns a weak pointer to an object.
- \code{weak-pointer-value} follows a weak pointer, returning the two
- values: the object pointed to (or \false{} if broken) and a boolean
- value which is true if the pointer has been broken.
-\end{defun}
-
-%%\node
-\subsection{Finalization}
-
-Finalization provides a ``hook'' that is triggered when the garbage
-collector reclaims an object. It is usually used to recover non-Lisp
-resources that were allocated to implement the finalized Lisp object.
-For example, when a unix file-descriptor stream is collected,
-finalization is used to close the underlying file descriptor.
-
-\begin{defun}{extensions:}{finalize}{\args{\var{object} \var{function}}}
-
- This function registers \var{object} for finalization.
- \var{function} is called with no arguments when \var{object} is
- reclaimed. Normally \var{function} will be a closure over the
- underlying state that needs to be freed, e.g. the unix file
- descriptor in the fd-stream case. Note that \var{function} must not
- close over \var{object} itself, as this prevents the object from
- ever becoming garbage.
-\end{defun}
-
-\begin{defun}{extensions:}{cancel-finalization}{\args{\var{object}}}
-
- This function cancel any finalization request for \var{object}.
-\end{defun}
-
-%%\node Describe, The Inspector, Garbage Collection, Design Choices and Extensions
-\section{Describe}
-
-In addition to the basic function described below, there are a number of
-switches and other things that can be used to control \code{describe}'s
-behavior.
-
-\begin{defun}{}{describe}{ \args{\var{object} \&optional{} \var{stream}}}
-
- The \code{describe} function prints useful information about
- \var{object} on \var{stream}, which defaults to
- \code{*standard-output*}. For any object, \code{describe} will
- print out the type. Then it prints other information based on the
- type of \var{object}. The types which are presently handled are:
-
- \begin{Lentry}
-
- \item[\tindexed{hash-table}] \code{describe} prints the number of
- entries currently in the hash table and the number of buckets
- currently allocated.
-
- \item[\tindexed{function}] \code{describe} prints a list of the
- function's name (if any) and its formal parameters. If the name
- has function documentation, then it will be printed. If the
- function is compiled, then the file where it is defined will be
- printed as well.
-
- \item[\tindexed{fixnum}] \code{describe} prints whether the integer
- is prime or not.
-
- \item[\tindexed{symbol}] The symbol's value, properties, and
- documentation are printed. If the symbol has a function
- definition, then the function is described.
- \end{Lentry}
- If there is anything interesting to be said about some component of
- the object, describe will invoke itself recursively to describe that
- object. The level of recursion is indicated by indenting output.
-\end{defun}
-
-\begin{defvar}{extensions:}{describe-level}
-
- The maximum level of recursive description allowed. Initially two.
-\end{defvar}
-
-\begin{defvar}{extensions:}{describe-indentation}
-
-The number of spaces to indent for each level of recursive
-description, initially three.
-\end{defvar}
-
-\begin{defvar}{extensions:}{describe-print-level}
- \defvarx[extensions:]{describe-print-length}
-
- The values of \code{*print-level*} and \code{*print-length*} during
- description. Initially two and five.
-\end{defvar}
-
-%%\node The Inspector, Load, Describe, Design Choices and Extensions
-\section{The Inspector}
-
-\cmucl{} has both a graphical inspector that uses X windows and a simple
-terminal-based inspector.
-
-\begin{defun}{}{inspect}{ \args{\ampoptional{} \var{object}}}
-
- \code{inspect} calls the inspector on the optional argument
- \var{object}. If \var{object} is unsupplied, \code{inspect}
- immediately returns \false. Otherwise, the behavior of inspect
- depends on whether Lisp is running under X. When \code{inspect} is
- eventually exited, it returns some selected Lisp object.
-\end{defun}
-
-\begin{comment}
-* The Graphical Interface::
-* The TTY Inspector::
-\end{comment}
-
-%%\node The Graphical Interface, The TTY Inspector, The Inspector, The Inspector
-\subsection{The Graphical Interface}
-\label{motif-interface}
-
-CMU Common Lisp has an interface to Motif which is functionally similar to
-CLM, but works better in CMU CL. See:
-\begin{example}
-\file{doc/motif-toolkit.doc}
-\file{doc/motif-internals.doc}
-\end{example}
-
-This motif interface has been used to write the inspector and graphical
-debugger. There is also a Lisp control panel with a simple file management
-facility, apropos and inspector dialogs, and controls for setting global
-options. See the \code{interface} and \code{toolkit} packages.
-
-\begin{defun}{interface:}{lisp-control-panel}{}
-
- This function creates a control panel for the Lisp process.
-\end{defun}
-
-\begin{defvar}{interface:}{interface-style}
-
- When the graphical interface is loaded, this variable controls
- whether it is used by \code{inspect} and the error system. If the
- value is \kwd{graphics} (the default) and the \code{DISPLAY}
- environment variable is defined, the graphical inspector and
- debugger will be invoked by \findexed{inspect} or when an error is
- signalled. Possible values are \kwd{graphics} and {tty}. If the
- value is \kwd{graphics}, but there is no X display, then we quietly
- use the TTY interface.
-\end{defvar}
-
-%%\node The TTY Inspector, , The Graphical Interface, The Inspector
-\subsection{The TTY Inspector}
-
-If X is unavailable, a terminal inspector is invoked. The TTY inspector
-is a crude interface to \code{describe} which allows objects to be
-traversed and maintains a history. This inspector prints information
-about and object and a numbered list of the components of the object.
-The command-line based interface is a normal
-\code{read}--\code{eval}--\code{print} loop, but an integer \var{n}
-descends into the \var{n}'th component of the current object, and
-symbols with these special names are interpreted as commands:
-\begin{Lentry}
-\item[U] Move back to the enclosing object. As you descend into the
-components of an object, a stack of all the objects previously seen is
-kept. This command pops you up one level of this stack.
-
-\item[Q, E] Return the current object from \code{inspect}.
-
-\item[R] Recompute object display, and print again. Useful if the
-object may have changed.
-
-\item[D] Display again without recomputing.
-
-\item[H, ?] Show help message.
-\end{Lentry}
-
-%%\node Load, The Reader, The Inspector, Design Choices and Extensions
-\section{Load}
-
-\begin{defun}{}{load}{%
- \args{\var{filename}
- \keys{\kwd{verbose} \kwd{print} \kwd{if-does-not-exist}}
- \morekeys{\kwd{if-source-newer} \kwd{contents}}}}
-
- As in standard Common Lisp, this function loads a file containing
- source or object code into the running Lisp. Several CMU extensions
- have been made to \code{load} to conveniently support a variety of
- program file organizations. \var{filename} may be a wildcard
- pathname such as \file{*.lisp}, in which case all matching files are
- loaded.
-
- If \var{filename} has a \code{pathname-type} (or extension), then
- that exact file is loaded. If the file has no extension, then this
- tells \code{load} to use a heuristic to load the ``right'' file.
- The \code{*load-source-types*} and \code{*load-object-types*}
- variables below are used to determine the default source and object
- file types. If only the source or the object file exists (but not
- both), then that file is quietly loaded. Similarly, if both the
- source and object file exist, and the object file is newer than the
- source file, then the object file is loaded. The value of the
- \var{if-source-newer} argument is used to determine what action to
- take when both the source and object files exist, but the object
- file is out of date:
- \begin{Lentry}
- \item[\kwd{load-object}] The object file is loaded even though the
- source file is newer.
-
- \item[\kwd{load-source}] The source file is loaded instead of the
- older object file.
-
- \item[\kwd{compile}] The source file is compiled and then the new
- object file is loaded.
-
- \item[\kwd{query}] The user is asked a yes or no question to
- determine whether the source or object file is loaded.
- \end{Lentry}
- This argument defaults to the value of
- \code{ext:*load-if-source-newer*} (initially \kwd{load-object}.)
-
- The \var{contents} argument can be used to override the heuristic
- (based on the file extension) that normally determines whether to
- load the file as a source file or an object file. If non-null, this
- argument must be either \kwd{source} or \kwd{binary}, which forces
- loading in source and binary mode, respectively. You really
- shouldn't ever need to use this argument.
-\end{defun}
-
-\begin{defvar}{extensions:}{load-source-types}
- \defvarx[extensions:]{load-object-types}
-
- These variables are lists of possible \code{pathname-type} values
- for source and object files to be passed to \code{load}. These
- variables are only used when the file passed to \code{load} has no
- type; in this case, the possible source and object types are used to
- default the type in order to determine the names of the source and
- object files.
-\end{defvar}
-
-\begin{defvar}{extensions:}{load-if-source-newer}
-
- This variable determines the default value of the
- \var{if-source-newer} argument to \code{load}. Its initial value is
- \kwd{load-object}.
-\end{defvar}
-
-%%\node The Reader, Stream Extensions, Load, Design Choices and Extensions
-\section{The Reader}
-
-\begin{defvar}{extensions:}{ignore-extra-close-parentheses}
-
- If this variable is \true{} (the default), then the reader merely
- prints a warning when an extra close parenthesis is detected
- (instead of signalling an error.)
-\end{defvar}
-
-%%\node Stream Extensions, Running Programs from Lisp, The Reader, Design Choices and Extensions
-\section{Stream Extensions}
-\begin{defun}{extensions:}{read-n-bytes}{%
- \args{\var{stream buffer start numbytes}
- \ampoptional{} \var{eof-error-p}}}
-
- On streams that support it, this function reads multiple bytes of
- data into a buffer. The buffer must be a \code{simple-string} or
- \code{(simple-array (unsigned-byte 8) (*))}. The argument
- \var{nbytes} specifies the desired number of bytes, and the return
- value is the number of bytes actually read.
- \begin{itemize}
- \item If \var{eof-error-p} is true, an \tindexed{end-of-file}
- condition is signalled if end-of-file is encountered before
- \var{count} bytes have been read.
-
- \item If \var{eof-error-p} is false, \code{read-n-bytes reads} as
- much data is currently available (up to count bytes.) On pipes or
- similar devices, this function returns as soon as any data is
- available, even if the amount read is less than \var{count} and
- eof has not been hit. See also \funref{make-fd-stream}.
- \end{itemize}
-\end{defun}
-%%\node Running Programs from Lisp, Saving a Core Image, The Reader, Design Choices and Extensions
-\section{Running Programs from Lisp}
-
-It is possible to run programs from Lisp by using the following function.
-
-\begin{defun}{extensions:}{run-program}{%
- \args{\var{program} \var{args}
- \keys{\kwd{env} \kwd{wait} \kwd{pty} \kwd{input}}
- \morekeys{\kwd{if-input-does-not-exist}}
- \yetmorekeys{\kwd{output} \kwd{if-output-exists}}
- \yetmorekeys{\kwd{error} \kwd{if-error-exists}}
- \yetmorekeys{\kwd{status-hook} \kwd{before-execve}}}}
-
- \code{run-program} runs \var{program} in a child process.
- \var{Program} should be a pathname or string naming the program.
- \var{Args} should be a list of strings which this passes to
- \var{program} as normal Unix parameters. For no arguments, specify
- \var{args} as \nil. The value returned is either a process
- structure or \nil. The process interface follows the description of
- \code{run-program}. If \code{run-program} fails to fork the child
- process, it returns \nil.
-
- Except for sharing file descriptors as explained in keyword argument
- descriptions, \code{run-program} closes all file descriptors in the
- child process before running the program. When you are done using a
- process, call \code{process-close} to reclaim system resources. You
- only need to do this when you supply \kwd{stream} for one of
- \kwd{input}, \kwd{output}, or \kwd{error}, or you supply \kwd{pty}
- non-\nil. You can call \code{process-close} regardless of whether
- you must to reclaim resources without penalty if you feel safer.
-
- \code{run-program} accepts the following keyword arguments:
- \begin{Lentry}
-
- \item[\kwd{env}] This is an a-list mapping keywords and
- simple-strings. The default is \code{ext:*environment-list*}. If
- \kwd{env} is specified, \code{run-program} uses the value given
- and does not combine the environment passed to Lisp with the one
- specified.
-
- \item[\kwd{wait}] If non-\nil{} (the default), wait until the child
- process terminates. If \nil, continue running Lisp while the
- child process runs.
-
- \item[\kwd{pty}] This should be one of \true, \nil, or a stream. If
- specified non-\nil, the subprocess executes under a Unix \i{PTY}.
- If specified as a stream, the system collects all output to this
- pty and writes it to this stream. If specified as \true, the
- \code{process-pty} slot contains a stream from which you can read
- the program's output and to which you can write input for the
- program. The default is \nil.
-
- \item[\kwd{input}] This specifies how the program gets its input.
- If specified as a string, it is the name of a file that contains
- input for the child process. \code{run-program} opens the file as
- standard input. If specified as \nil{} (the default), then
- standard input is the file \file{/dev/null}. If specified as
- \true, the program uses the current standard input. This may
- cause some confusion if \kwd{wait} is \nil{} since two processes
- may use the terminal at the same time. If specified as
- \kwd{stream}, then the \code{process-input} slot contains an
- output stream. Anything written to this stream goes to the
- program as input. \kwd{input} may also be an input stream that
- already contains all the input for the process. In this case
- \code{run-program} reads all the input from this stream before
- returning, so this cannot be used to interact with the process.
-
- \item[\kwd{if-input-does-not-exist}] This specifies what to do if
- the input file does not exist. The following values are valid:
- \nil{} (the default) causes \code{run-program} to return \nil{}
- without doing anything; \kwd{create} creates the named file; and
- \kwd{error} signals an error.
-
- \item[\kwd{output}] This specifies what happens with the program's
- output. If specified as a pathname, it is the name of a file that
- contains output the program writes to its standard output. If
- specified as \nil{} (the default), all output goes to
- \file{/dev/null}. If specified as \true, the program writes to
- the Lisp process's standard output. This may cause confusion if
- \kwd{wait} is \nil{} since two processes may write to the terminal
- at the same time. If specified as \kwd{stream}, then the
- \code{process-output} slot contains an input stream from which you
- can read the program's output.
-
- \item[\kwd{if-output-exists}] This specifies what to do if the
- output file already exists. The following values are valid:
- \nil{} causes \code{run-program} to return \nil{} without doing
- anything; \kwd{error} (the default) signals an error;
- \kwd{supersede} overwrites the current file; and \kwd{append}
- appends all output to the file.
-
- \item[\kwd{error}] This is similar to \kwd{output}, except the file
- becomes the program's standard error. Additionally, \kwd{error}
- can be \kwd{output} in which case the program's error output is
- routed to the same place specified for \kwd{output}. If specified
- as \kwd{stream}, the \code{process-error} contains a stream
- similar to the \code{process-output} slot when specifying the
- \kwd{output} argument.
-
- \item[\kwd{if-error-exists}] This specifies what to do if the error
- output file already exists. It accepts the same values as
- \kwd{if-output-exists}.
-
- \item[\kwd{status-hook}] This specifies a function to call whenever
- the process changes status. This is especially useful when
- specifying \kwd{wait} as \nil. The function takes the process as
- a required argument.
-
- \item[\kwd{before-execve}] This specifies a function to run in the
- child process before it becomes the program to run. This is
- useful for actions such as authenticating the child process
- without modifying the parent Lisp process.
- \end{Lentry}
-\end{defun}
-
-
-\begin{comment}
-* Process Accessors::
-\end{comment}
-
-%%\node Process Accessors, , Running Programs from Lisp, Running Programs from Lisp
-\subsection{Process Accessors}
-
-The following functions interface the process returned by \code{run-program}:
-
-\begin{defun}{extensions:}{process-p}{\args{\var{thing}}}
-
- This function returns \true{} if \var{thing} is a process.
- Otherwise it returns \nil{}
-\end{defun}
-
-\begin{defun}{extensions:}{process-pid}{\args{\var{process}}}
-
- This function returns the process ID, an integer, for the
- \var{process}.
-\end{defun}
-
-\begin{defun}{extensions:}{process-status}{\args{\var{process}}}
-
- This function returns the current status of \var{process}, which is
- one of \kwd{running}, \kwd{stopped}, \kwd{exited}, or
- \kwd{signaled}.
-\end{defun}
-
-\begin{defun}{extensions:}{process-exit-code}{\args{\var{process}}}
-
- This function returns either the exit code for \var{process}, if it
- is \kwd{exited}, or the termination signal \var{process} if it is
- \kwd{signaled}. The result is undefined for processes that are
- still alive.
-\end{defun}
-
-\begin{defun}{extensions:}{process-core-dumped}{\args{\var{process}}}
-
- This function returns \true{} if someone used a Unix signal to
- terminate the \var{process} and caused it to dump a Unix core image.
-\end{defun}
-
-\begin{defun}{extensions:}{process-pty}{\args{\var{process}}}
-
- This function returns either the two-way stream connected to
- \var{process}'s Unix \i{PTY} connection or \nil{} if there is none.
-\end{defun}
-
-\begin{defun}{extensions:}{process-input}{\args{\var{process}}}
- \defunx[extensions:]{process-output}{\args{\var{process}}}
- \defunx[extensions:]{process-error}{\args{\var{process}}}
-
- If the corresponding stream was created, these functions return the
- input, output or error file descriptor. \nil{} is returned if there
- is no stream.
-\end{defun}
-
-\begin{defun}{extensions:}{process-status-hook}{\args{\var{process}}}
-
- This function returns the current function to call whenever
- \var{process}'s status changes. This function takes the
- \var{process} as a required argument. \code{process-status-hook} is
- \code{setf}'able.
-\end{defun}
-
-\begin{defun}{extensions:}{process-plist}{\args{\var{process}}}
-
- This function returns annotations supplied by users, and it is
- \code{setf}'able. This is available solely for users to associate
- information with \var{process} without having to build a-lists or
- hash tables of process structures.
-\end{defun}
-
-\begin{defun}{extensions:}{process-wait}{
- \args{\var{process} \ampoptional{} \var{check-for-stopped}}}
-
- This function waits for \var{process} to finish. If
- \var{check-for-stopped} is non-\nil, this also returns when
- \var{process} stops.
-\end{defun}
-
-\begin{defun}{extensions:}{process-kill}{%
- \args{\var{process} \var{signal} \ampoptional{} \var{whom}}}
-
- This function sends the Unix \var{signal} to \var{process}.
- \var{Signal} should be the number of the signal or a keyword with
- the Unix name (for example, \kwd{sigsegv}). \var{Whom} should be
- one of the following:
- \begin{Lentry}
-
- \item[\kwd{pid}] This is the default, and it indicates sending the
- signal to \var{process} only.
-
- \item[\kwd{process-group}] This indicates sending the signal to
- \var{process}'s group.
-
- \item[\kwd{pty-process-group}] This indicates sending the signal to
- the process group currently in the foreground on the Unix \i{PTY}
- connected to \var{process}. This last option is useful if the
- running program is a shell, and you wish to signal the program
- running under the shell, not the shell itself. If
- \code{process-pty} of \var{process} is \nil, using this option is
- an error.
- \end{Lentry}
-\end{defun}
-
-\begin{defun}{extensions:}{process-alive-p}{\args{\var{process}}}
-
- This function returns \true{} if \var{process}'s status is either
- \kwd{running} or \kwd{stopped}.
-\end{defun}
-
-\begin{defun}{extensions:}{process-close}{\args{\var{process}}}
-
- This function closes all the streams associated with \var{process}.
- When you are done using a process, call this to reclaim system
- resources.
-\end{defun}
-
-
-%%\node Saving a Core Image, Pathnames, Running Programs from Lisp, Design Choices and Extensions
-\section{Saving a Core Image}
-
-A mechanism has been provided to save a running Lisp core image and to
-later restore it. This is convenient if you don't want to load several files
-into a Lisp when you first start it up. The main problem is the large
-size of each saved Lisp image, typically at least 20 megabytes.
-
-\begin{defun}{extensions:}{save-lisp}{%
- \args{\var{file}
- \keys{\kwd{purify} \kwd{root-structures} \kwd{init-function}}
- \morekeys{\kwd{load-init-file} \kwd{print-herald} \kwd{site-init}}
- \yetmorekeys{\kwd{process-command-line}}}}
-
- The \code{save-lisp} function saves the state of the currently
- running Lisp core image in \var{file}. The keyword arguments have
- the following meaning:
- \begin{Lentry}
-
- \item[\kwd{purify}] If non-NIL (the default), the core image is
- purified before it is saved (see \funref{purify}.) This reduces
- the amount of work the garbage collector must do when the
- resulting core image is being run. Also, if more than one Lisp is
- running on the same machine, this maximizes the amount of memory
- that can be shared between the two processes.
-
- \item[\kwd{root-structures}]
- \begin{changebar}
- This should be a list of the main entry points in any newly
- loaded systems. This need not be supplied, but locality and/or
- GC performance will be better if they are. Meaningless if
- \kwd{purify} is \nil. See \funref{purify}.
- \end{changebar}
-
- \item[\kwd{init-function}] This is the function that starts running
- when the created core file is resumed. The default function
- simply invokes the top level read-eval-print loop. If the
- function returns the lisp will exit.
-
- \item[\kwd{load-init-file}] If non-NIL, then load an init file;
- either the one specified on the command line or
- ``\w{\file{init.}\var{fasl-type}}'', or, if
- ``\w{\file{init.}\var{fasl-type}}'' does not exist,
- \code{init.lisp} from the user's home directory. If the init file
- is found, it is loaded into the resumed core file before the
- read-eval-print loop is entered.
-
- \item[\kwd{site-init}] If non-NIL, the name of the site init file to
- quietly load. The default is \file{library:site-init}. No error
- is signalled if the file does not exist.
-
- \item[\kwd{print-herald}] If non-NIL (the default), then print out
- the standard Lisp herald when starting.
-
- \item[\kwd{process-command-line}] If non-NIL (the default),
- processes the command line switches and performs the appropriate
- actions.
- \end{Lentry}
-\end{defun}
-
-To resume a saved file, type:
-\begin{example}
-lisp -core file
-\end{example}
-
-\begin{defun}{extensions:}{purify}{
- \args{\var{file}
- \keys{\kwd{root-structures} \kwd{environment-name}}}}
-
- This function optimizes garbage collection by moving all currently
- live objects into non-collected storage. Once statically allocated,
- the objects can never be reclaimed, even if all pointers to them are
- dropped. This function should generally be called after a large
- system has been loaded and initialized.
-
- \begin{Lentry}
- \item[\kwd{root-structures}] is an optional list of objects which
- should be copied first to maximize locality. This should be a
- list of the main entry points for the resulting core image. The
- purification process tries to localize symbols, functions, etc.,
- in the core image so that paging performance is improved. The
- default value is NIL which means that Lisp objects will still be
- localized but probably not as optimally as they could be.
-
- \var{defstruct} structures defined with the \code{(:pure t)}
- option are moved into read-only storage, further reducing GC cost.
- List and vector slots of pure structures are also moved into
- read-only storage.
-
- \item[\kwd{environment-name}] is gratuitous documentation for the
- compacted version of the current global environment (as seen in
- \code{c::*info-environment*}.) If \false{} is supplied, then
- environment compaction is inhibited.
- \end{Lentry}
-\end{defun}
-
-%%\node Pathnames, Filesystem Operations, Saving a Core Image, Design Choices and Extensions
-\section{Pathnames}
-
-In \clisp{} quite a few aspects of \tindexed{pathname} semantics are left to
-the implementation.
-
-\begin{comment}
-* Unix Pathnames::
-* Wildcard Pathnames::
-* Logical Pathnames::
-* Search Lists::
-* Predefined Search-Lists::
-* Search-List Operations::
-* Search List Example::
-\end{comment}
-
-%%\node Unix Pathnames, Wildcard Pathnames, Pathnames, Pathnames
-\subsection{Unix Pathnames}
-\cpsubindex{unix}{pathnames}
-
-Unix pathnames are always parsed with a \code{unix-host} object as the host and
-\code{nil} as the device. The last two dots (\code{.}) in the namestring mark
-the type and version, however if the first character is a dot, it is considered
-part of the name. If the last character is a dot, then the pathname has the
-empty-string as its type. The type defaults to \code{nil} and the version
-defaults to \kwd{newest}.
-\begin{example}
-(defun parse (x)
- (values (pathname-name x) (pathname-type x) (pathname-version x)))
-
-(parse "foo") \result "foo", NIL, :NEWEST
-(parse "foo.bar") \result "foo", "bar", :NEWEST
-(parse ".foo") \result ".foo", NIL, :NEWEST
-(parse ".foo.bar") \result ".foo", "bar", :NEWEST
-(parse "..") \result ".", "", :NEWEST
-(parse "foo.") \result "foo", "", :NEWEST
-(parse "foo.bar.1") \result "foo", "bar", 1
-(parse "foo.bar.baz") \result "foo.bar", "baz", :NEWEST
-\end{example}
-
-The directory of pathnames beginning with a slash (or a search-list,
-\pxlref{search-lists}) is starts \kwd{absolute}, others start with
-\kwd{relative}. The \code{..} directory is parsed as \kwd{up}; there is no
-namestring for \kwd{back}:
-\begin{example}
-(pathname-directory "/usr/foo/bar.baz") \result (:ABSOLUTE "usr" "foo")
-(pathname-directory "../foo/bar.baz") \result (:RELATIVE :UP "foo")
-\end{example}
-
-%%\node Wildcard Pathnames, Logical Pathnames, Unix Pathnames, Pathnames
-\subsection{Wildcard Pathnames}
-
-Wildcards are supported in Unix pathnames. If `\code{*}' is specified for a
-part of a pathname, that is parsed as \kwd{wild}. `\code{**}' can be used as a
-directory name to indicate \kwd{wild-inferiors}. Filesystem operations
-treat \kwd{wild-inferiors} the same as\ \kwd{wild}, but pathname pattern
-matching (e.g. for logical pathname translation, \pxlref{logical-pathnames})
-matches any number of directory parts with `\code{**}' (see
-\pxlref{wildcard-matching}.)
-
-
-`\code{*}' embedded in a pathname part matches any number of characters.
-Similarly, `\code{?}' matches exactly one character, and `\code{[a,b]}'
-matches the characters `\code{a}' or `\code{b}'. These pathname parts are
-parsed as \code{pattern} objects.
-
-Backslash can be used as an escape character in namestring
-parsing to prevent the next character from being treated as a wildcard. Note
-that if typed in a string constant, the backslash must be doubled, since the
-string reader also uses backslash as a quote:
-\begin{example}
-(pathname-name "foo\(\backslash\backslash\)*bar") => "foo*bar"
-\end{example}
-
-%%\node Logical Pathnames, Search Lists, Wildcard Pathnames, Pathnames
-\subsection{Logical Pathnames}
-\cindex{logical pathnames}
-\label{logical-pathnames}
-
-If a namestring begins with the name of a defined logical pathname
-host followed by a colon, then it will be parsed as a logical
-pathname. Both `\code{*}' and `\code{**}' wildcards are implemented.
-\findexed{load-logical-pathname-defaults} on \var{name} looks for a
-logical host definition file in
-\w{\file{library:\var{name}.translations}}. Note that \file{library:}
-designates the search list (\pxlref{search-lists}) initialized to the
-\cmucl{} \file{lib/} directory, not a logical pathname. The format of
-the file is a single list of two-lists of the from and to patterns:
-\begin{example}
-(("foo;*.text" "/usr/ram/foo/*.txt")
- ("foo;*.lisp" "/usr/ram/foo/*.l"))
-\end{example}
-
-\begin{comment}
-* Search Lists::
-* Search List Example::
-\end{comment}
-
-%%\node Search Lists, Predefined Search-Lists, Logical Pathnames, Pathnames
-\subsection{Search Lists}
-\cindex{search lists}
-\label{search-lists}
-
-Search lists are an extension to Common Lisp pathnames. They serve a function
-somewhat similar to Common Lisp logical pathnames, but work more like Unix PATH
-variables. Search lists are used for two purposes:
-\begin{itemize}
-\item They provide a convenient shorthand for commonly used directory names,
-and
-
-\item They allow the abstract (directory structure independent) specification
-of file locations in program pathname constants (similar to logical pathnames.)
-\end{itemize}
-Each search list has an associated list of directories (represented as
-pathnames with no name or type component.) The namestring for any relative
-pathname may be prefixed with ``\var{slist}\code{:}'', indicating that the
-pathname is relative to the search list \var{slist} (instead of to the current
-working directory.) Once qualified with a search list, the pathname is no
-longer considered to be relative.
-
-When a search list qualified pathname is passed to a file-system operation such
-as \code{open}, \code{load} or \code{truename}, each directory in the search
-list is successively used as the root of the pathname until the file is
-located. When a file is written to a search list directory, the file is always
-written to the first directory in the list.
-
-%%\node Predefined Search-Lists, Search-List Operations, Search Lists, Pathnames
-\subsection{Predefined Search-Lists}
-
-These search-lists are initialized from the Unix environment or when Lisp was
-built:
-\begin{Lentry}
-\item[\code{default:}] The current directory at startup.
-
-\item[\code{home:}] The user's home directory.
-
-\item[\code{library:}] The \cmucl{} \file{lib/} directory (\code{CMUCLLIB} environment
-variable.)
-
-\item[\code{path:}] The Unix command path (\code{PATH} environment variable.)
-
-\item[\code{target:}] The root of the tree where \cmucl{} was compiled.
-\end{Lentry}
-It can be useful to redefine these search-lists, for example, \file{library:}
-can be augmented to allow logical pathname translations to be located, and
-\file{target:} can be redefined to point to where \cmucl{} system sources are
-locally installed.
-
-%%\node Search-List Operations, Search List Example, Predefined Search-Lists, Pathnames
-\subsection{Search-List Operations}
-
-These operations define and access search-list definitions. A search-list name
-may be parsed into a pathname before the search-list is actually defined, but
-the search-list must be defined before it can actually be used in a filesystem
-operation.
-
-\begin{defun}{extensions:}{search-list}{\var{name}}
-
- This function returns the list of directories associated with the
- search list \var{name}. If \var{name} is not a defined search list,
- then an error is signaled. When set with \code{setf}, the list of
- directories is changed to the new value. If the new value is just a
- namestring or pathname, then it is interpreted as a one-element
- list. Note that (unlike Unix pathnames), search list names are
- case-insensitive.
-\end{defun}
-
-\begin{defun}{extensions:}{search-list-defined-p}{\var{name}}
- \defunx[extensions:]{clear-search-list}{\var{name}}
-
- \code{search-list-defined-p} returns \true{} if \var{name} is a
- defined search list name, \false{} otherwise.
- \code{clear-search-list} make the search list \var{name} undefined.
-\end{defun}
-
-\begin{defmac}{extensions:}{enumerate-search-list}{%
- \args{(\var{var} \var{pathname} \mopt{result}) \mstar{form}}}
-
- This macro provides an interface to search list resolution. The
- body \var{forms} are executed with \var{var} bound to each
- successive possible expansion for \var{name}. If \var{name} does
- not contain a search-list, then the body is executed exactly once.
- Everything is wrapped in a block named \nil, so \code{return} can be
- used to terminate early. The \var{result} form (default \nil) is
- evaluated to determine the result of the iteration.
-\end{defmac}
-
-\begin{comment}
-* Search List Example::
-\end{comment}
-
-%%\node Search List Example, , Search-List Operations, Pathnames
-\subsection{Search List Example}
-
-The search list \code{code:} can be defined as follows:
-\begin{example}
-(setf (ext:search-list "code:") '("/usr/lisp/code/"))
-\end{example}
-It is now possible to use \code{code:} as an abbreviation for the directory
-\file{/usr/lisp/code/} in all file operations. For example, you can now specify
-\code{code:eval.lisp} to refer to the file \file{/usr/lisp/code/eval.lisp}.
-
-To obtain the value of a search-list name, use the function search-list
-as follows:
-\begin{example}
-(ext:search-list \var{name})
-\end{example}
-Where \var{name} is the name of a search list as described above. For example,
-calling \code{ext:search-list} on \code{code:} as follows:
-\begin{example}
-(ext:search-list "code:")
-\end{example}
-returns the list \code{("/usr/lisp/code/")}.
-
-%%\node Filesystem Operations, Time Parsing and Formatting, Pathnames, Design Choices and Extensions
-\section{Filesystem Operations}
-
-\cmucl{} provides a number of extensions and optional features beyond those
-require by \clisp.
-
-\begin{comment}
-* Wildcard Matching::
-* File Name Completion::
-* Miscellaneous Filesystem Operations::
-\end{comment}
-
-%%\node Wildcard Matching, File Name Completion, Filesystem Operations, Filesystem Operations
-\subsection{Wildcard Matching}
-\label{wildcard-matching}
-
-Unix filesystem operations such as \code{open} will accept wildcard pathnames
-that match a single file (of course, \code{directory} allows any number of
-matches.) Filesystem operations treat \kwd{wild-inferiors} the same as\
-\kwd{wild}.
-
-\begin{defun}{}{directory}{\var{wildname} \keys{\kwd{all} \kwd{check-for-subdirs}}
- \morekeys{\kwd{follow-links}}}
-
- The keyword arguments to this \clisp{} function are a CMU extension.
- The arguments (all default to \code{t}) have the following
- functions:
- \begin{Lentry}
- \item[\kwd{all}] Include files beginning with dot such as
- \file{.login}, similar to ``\code{ls -a}''.
-
- \item[\kwd{check-for-subdirs}] Test whether files are directories,
- similar to ``\code{ls -F}''.
-
- \item[\kwd{follow-links}] Call \code{truename} on each file, which
- expands out all symbolic links. Note that this option can easily
- result in pathnames being returned which have a different
- directory from the one in the \var{wildname} argument.
- \end{Lentry}
-\end{defun}
-
-\begin{defun}{extensions:}{print-directory}{%
- \args{\var{wildname}
- \ampoptional{} \var{stream}
- \keys{\kwd{all} \kwd{verbose}}
- \morekeys{\kwd{return-list}}}}
-
- Print a directory of \var{wildname} listing to \var{stream} (default
- \code{*standard-output*}.) \kwd{all} and \kwd{verbose} both default
- to \false{} and correspond to the ``\code{-a}'' and ``\code{-l}''
- options of \file{ls}. Normally this function returns \false{}, but
- if \kwd{return-list} is true, a list of the matched pathnames are
- returned.
-\end{defun}
-
-%%\node File Name Completion, Miscellaneous Filesystem Operations, Wildcard Matching, Filesystem Operations
-\subsection{File Name Completion}
-
-\begin{defun}{extensions:}{complete-file}{%
- \args{\var{pathname}
- \keys{\kwd{defaults} \kwd{ignore-types}}}}
-
- Attempt to complete a file name to the longest unambiguous prefix.
- If supplied, directory from \kwd{defaults} is used as the ``working
- directory'' when doing completion. \kwd{ignore-types} is a list of
- strings of the pathname types (a.k.a. extensions) that should be
- disregarded as possible matches (binary file names, etc.)
-\end{defun}
-
-\begin{defun}{extensions:}{ambiguous-files}{%
- \args{\var{pathname}
- \ampoptional{} \var{defaults}}}
-
- Return a list of pathnames for all the possible completions of
- \var{pathname} with respect to \var{defaults}.
-\end{defun}
-
-%%\node Miscellaneous Filesystem Operations, , File Name Completion, Filesystem Operations
-\subsection{Miscellaneous Filesystem Operations}
-
-\begin{defun}{extensions:}{default-directory}{}
-
- Return the current working directory as a pathname. If set with
- \code{setf}, set the working directory.
-\end{defun}
-
-\begin{defun}{extensions:}{file-writable}{\var{name}}
-
- This function accepts a pathname and returns \true{} if the current
- process can write it, and \false{} otherwise.
-\end{defun}
-
-\begin{defun}{extensions:}{unix-namestring}{%
- \args{\var{pathname}
- \ampoptional{} \var{for-input}}}
-
- This function converts \var{pathname} into a string that can be used
- with UNIX system calls. Search-lists and wildcards are expanded.
- \var{for-input} controls the treatment of search-lists: when true
- (the default) and the file exists anywhere on the search-list, then
- that absolute pathname is returned; otherwise the first element of
- the search-list is used as the directory.
-\end{defun}
-
-%%\node Time Parsing and Formatting, Lisp Library, Filesystem Operations, Design Choices and Extensions
-\section{Time Parsing and Formatting}
-
-\cindex{time parsing} \cindex{time formatting}
-Functions are provided to allow parsing strings containing time information
-and printing time in various formats are available.
-
-\begin{defun}{extensions:}{parse-time}{%
- \args{\var{time-string}
- \keys{\kwd{error-on-mismatch} \kwd{default-seconds}}
- \morekeys{\kwd{default-minutes} \kwd{default-hours}}
- \yetmorekeys{\kwd{default-day} \kwd{default-month}}
- \yetmorekeys{\kwd{default-year} \kwd{default-zone}}
- \yetmorekeys{\kwd{default-weekday}}}}
-
- \code{parse-time} accepts a string containing a time (e.g.,
- \w{"\code{Jan 12, 1952}"}) and returns the universal time if it is
- successful. If it is unsuccessful and the keyword argument
- \kwd{error-on-mismatch} is non-\FALSE, it signals an error.
- Otherwise it returns \FALSE. The other keyword arguments have the
- following meaning:
- \begin{Lentry}
-
- \item[\kwd{default-seconds}] specifies the default value for the
- seconds value if one is not provided by \var{time-string}. The
- default value is 0.
-
- \item[\kwd{default-minutes}] specifies the default value for the
- minutes value if one is not provided by \var{time-string}. The
- default value is 0.
-
- \item[\kwd{default-hours}] specifies the default value for the hours
- value if one is not provided by \var{time-string}. The default
- value is 0.
-
- \item[\kwd{default-day}] specifies the default value for the day
- value if one is not provided by \var{time-string}. The default
- value is the current day.
-
- \item[\kwd{default-month}] specifies the default value for the month
- value if one is not provided by \var{time-string}. The default
- value is the current month.
-
- \item[\kwd{default-year}] specifies the default value for the year
- value if one is not provided by \var{time-string}. The default
- value is the current year.
-
- \item[\kwd{default-zone}] specifies the default value for the time
- zone value if one is not provided by \var{time-string}. The
- default value is the current time zone.
-
- \item[\kwd{default-weekday}] specifies the default value for the day
- of the week if one is not provided by \var{time-string}. The
- default value is the current day of the week.
- \end{Lentry}
- Any of the above keywords can be given the value \kwd{current} which
- means to use the current value as determined by a call to the
- operating system.
-\end{defun}
-
-\begin{defun}{extensions:}{format-universal-time}{
- \args{\var{dest} \var{universal-time}
- \\
- \keys{\kwd{timezone}}
- \morekeys{\kwd{style} \kwd{date-first}}
- \yetmorekeys{\kwd{print-seconds} \kwd{print-meridian}}
- \yetmorekeys{\kwd{print-timezone} \kwd{print-weekday}}}}
- \defunx[extensions:]{format-decoded-time}{
- \args{\var{dest} \var{seconds} \var{minutes} \var{hours} \var{day} \var{month} \var{year}
- \\
- \keys{\kwd{timezone}}
- \morekeys{\kwd{style} \kwd{date-first}}
- \yetmorekeys{\kwd{print-seconds} \kwd{print-meridian}}
- \yetmorekeys{\kwd{print-timezone} \kwd{print-weekday}}}}
-
- \code{format-universal-time} formats the time specified by
- \var{universal-time}. \code{format-decoded-time} formats the time
- specified by \var{seconds}, \var{minutes}, \var{hours}, \var{day},
- \var{month}, and \var{year}. \var{Dest} is any destination
- accepted by the \code{format} function. The keyword arguments have
- the following meaning:
- \begin{Lentry}
-
- \item[\kwd{timezone}] is an integer specifying the hours west of
- Greenwich. \kwd{timezone} defaults to the current time zone.
-
- \item[\kwd{style}] specifies the style to use in formatting the
- time. The legal values are:
- \begin{Lentry}
-
- \item[\kwd{short}] specifies to use a numeric date.
-
- \item[\kwd{long}] specifies to format months and weekdays as
- words instead of numbers.
-
- \item[\kwd{abbreviated}] is similar to long except the words are
- abbreviated.
-
- \item[\kwd{government}] is similar to abbreviated, except the
- date is of the form ``day month year'' instead of ``month day,
- year''.
- \end{Lentry}
-
- \item[\kwd{date-first}] if non-\false{} (default) will place the
- date first. Otherwise, the time is placed first.
-
- \item[\kwd{print-seconds}] if non-\false{} (default) will format
- the seconds as part of the time. Otherwise, the seconds will be
- omitted.
-
- \item[\kwd{print-meridian}] if non-\false{} (default) will format
- ``AM'' or ``PM'' as part of the time. Otherwise, the ``AM'' or
- ``PM'' will be omitted.
-
- \item[\kwd{print-timezone}] if non-\false{} (default) will format
- the time zone as part of the time. Otherwise, the time zone will
- be omitted.
-
- %%\item[\kwd{print-seconds}]
- %%if non-\false{} (default) will format the seconds as part of
- %%the time. Otherwise, the seconds will be omitted.
-
- \item[\kwd{print-weekday}] if non-\false{} (default) will format
- the weekday as part of date. Otherwise, the weekday will be
- omitted.
- \end{Lentry}
-\end{defun}
-
-%% New stuff
-\begin{changebar}
-\section{Random Number Generation}
-\cindex{random number generation}
-
-\clisp{} includes a random number generator as a standard part of the
-language; however, the implementation of the generator is not
-specified. Two random number generators are available in \cmucl{},
-depending on the version.
-
-\subsection{Original Generator}
-\cpsubindex{random number generation}{original generator}
-The default random number generator uses a lagged Fibonacci generator
-given by
-\begin{displaymath}
- z[i] = z[i - 24] - z[i - 55] \bmod 536870908
-\end{displaymath}
-where $z[i]$ is the $i$'th random number. This generator produces
-small integer-valued numbers. For larger integer, the small random
-integers are concatenated to produce larger integers. For
-floating-point numbers, the bits from this generator are used as the
-bits of the floating-point significand.
-
-\subsection{New Generator}
-\cpsubindex{random number generation}{new generator}
-
-In some versions of \cmucl{}, the original generator above has been
-replaced with a subtract-with-borrow generator
-combined with a Weyl generator.\footnote{The generator described here
- is available if the feature \kwd{new-random} is available.} The
-reason for the change was to use a documented generator which has
-passed tests for randomness.
-
-The subtract-with-borrow generator is described by the following
-equation
-\begin{displaymath}
- z[i] = z[i + 20] - z[i + 5] - b
-\end{displaymath}
-where $z[i]$ is the $i$'th random number, which is a
-\code{double-float}. All of the indices in this equation are
-interpreted modulo 32. The quantity $b$ is carried over from the
-previous iteration and is either 0 or \code{double-float-epsilon}. If
-$z[i]$ is positive, $b$ is set to zero. Otherwise, $b$ is set to
-\code{double-float-epsilon}.
-
-To increase the randomness of this generator, this generator is
-combined with a Weyl generator defined by
-\begin{displaymath}
- x[i] = x[i - 1] - y \bmod 1,
-\end{displaymath}
-where $y = 7097293079245107 \times 2^{-53}$. Thus, the resulting
-random number $r[i]$ is
-\begin{displaymath}
- r[i] = (z[i] - x[i]) \bmod 1
-\end{displaymath}
-
-This generator has been tested by Peter VanEynde using Marsaglia's
-diehard test suite for random number generators; this generator
-passes the test suite.
-
-This generator is designed for generating floating-point random
-numbers. To obtain integers, the bits from the significand of the
-floating-point number are used as the bits of the integer. As many
-floating-point numbers as needed are generated to obtain the desired
-number of bits in the random integer.
-
-For floating-point numbers, this generator can by significantly faster
-than the original generator.
-\end{changebar}
-
-%%\node Lisp Library, , Time Parsing and Formatting, Design Choices and Extensions
-\section{Lisp Library}
-\label{lisp-lib}
-
-The CMU Common Lisp project maintains a collection of useful or interesting
-programs written by users of our system. The library is in
-\file{lib/contrib/}. Two files there that users should read are:
-\begin{Lentry}
-
-\item[CATALOG.TXT]
-This file contains a page for each entry in the library. It
-contains information such as the author, portability or dependency issues, how
-to load the entry, etc.
-
-\item[READ-ME.TXT]
-This file describes the library's organization and all the
-possible pieces of information an entry's catalog description could contain.
-\end{Lentry}
-
-Hemlock has a command \F{Library Entry} that displays a list of the current
-library entries in an editor buffer. There are mode specific commands that
-display catalog descriptions and load entries. This is a simple and convenient
-way to browse the library.
-
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/debug.ms}
-
-
-
-%%\node The Debugger, The Compiler, Design Choices and Extensions, Top
-\chapter{The Debugger} \hide{-*- Dictionary: cmu-user -*-}
-\begin{center}
-\b{By Robert MacLachlan}
-\end{center}
-\cindex{debugger}
-\label{debugger}
-
-\begin{comment}
-* Debugger Introduction::
-* The Command Loop::
-* Stack Frames::
-* Variable Access::
-* Source Location Printing::
-* Compiler Policy Control::
-* Exiting Commands::
-* Information Commands::
-* Breakpoint Commands::
-* Function Tracing::
-* Specials::
-\end{comment}
-
-%%\node Debugger Introduction, The Command Loop, The Debugger, The Debugger
-\section{Debugger Introduction}
-
-The \cmucl{} debugger is unique in its level of support for source-level
-debugging of compiled code. Although some other debuggers allow access of
-variables by name, this seems to be the first \llisp{} debugger that:
-\begin{itemize}
-
-\item
-Tells you when a variable doesn't have a value because it hasn't been
-initialized yet or has already been deallocated, or
-
-\item
-Can display the precise source location corresponding to a code
-location in the debugged program.
-\end{itemize}
-These features allow the debugging of compiled code to be made almost
-indistinguishable from interpreted code debugging.
-
-The debugger is an interactive command loop that allows a user to examine
-the function call stack. The debugger is invoked when:
-\begin{itemize}
-
-\item
-A \tindexed{serious-condition} is signaled, and it is not handled, or
-
-\item
-\findexed{error} is called, and the condition it signals is not handled, or
-
-\item
-The debugger is explicitly invoked with the \clisp{} \findexed{break}
-or \findexed{debug} functions.
-\end{itemize}
-
-{\it Note: there are two debugger interfaces in CMU CL: the TTY debugger
-(described below) and the Motif debugger. Since the difference is only in the
-user interface, much of this chapter also applies to the Motif version.
-\xlref{motif-interface} for a very brief discussion of the graphical
-interface.}
-
-When you enter the TTY debugger, it looks something like this:
-\begin{example}
-Error in function CAR.
-Wrong type argument, 3, should have been of type LIST.
-
-Restarts:
- 0: Return to Top-Level.
-
-Debug (type H for help)
-
-(CAR 3)
-0]
-\end{example}
-The first group of lines describe what the error was that put us in the
-debugger. In this case \code{car} was called on \code{3}. After \code{Restarts:}
-is a list of all the ways that we can restart execution after this error. In
-this case, the only option is to return to top-level. After printing its
-banner, the debugger prints the current frame and the debugger prompt.
-
-%%\f
-%%\node The Command Loop, Stack Frames, Debugger Introduction, The Debugger
-\section{The Command Loop}
-
-The debugger is an interactive read-eval-print loop much like the normal
-top-level, but some symbols are interpreted as debugger commands instead
-of being evaluated. A debugger command starts with the symbol name of
-the command, possibly followed by some arguments on the same line. Some
-commands prompt for additional input. Debugger commands can be
-abbreviated by any unambiguous prefix: \code{help} can be typed as
-\code{h}, \code{he}, etc. For convenience, some commands have
-ambiguous one-letter abbreviations: \code{f} for \code{frame}.
-
-The package is not significant in debugger commands; any symbol with the
-name of a debugger command will work. If you want to show the value of
-a variable that happens also to be the name of a debugger command, you
-can use the \code{list-locals} command or the \code{debug:var}
-function, or you can wrap the variable in a \code{progn} to hide it from
-the command loop.
-
-The debugger prompt is ``\var{frame}\code{]}'', where \var{frame} is the number
-of the current frame. Frames are numbered starting from zero at the top (most
-recent call), increasing down to the bottom. The current frame is the frame
-that commands refer to. The current frame also provides the lexical
-environment for evaluation of non-command forms.
-
-\cpsubindex{evaluation}{debugger} The debugger evaluates forms in the lexical
-environment of the functions being debugged. The debugger can only
-access variables. You can't \code{go} or \code{return-from} into a
-function, and you can't call local functions. Special variable
-references are evaluated with their current value (the innermost binding
-around the debugger invocation)\dash{}you don't get the value that the
-special had in the current frame. \xlref{debug-vars} for more
-information on debugger variable access.
-
-%%\f
-%%\node Stack Frames, Variable Access, The Command Loop, The Debugger
-\section{Stack Frames}
-\cindex{stack frames} \cpsubindex{frames}{stack}
-
-A stack frame is the run-time representation of a call to a function;
-the frame stores the state that a function needs to remember what it is
-doing. Frames have:
-\begin{itemize}
-
-\item
-Variables (\pxlref{debug-vars}), which are the values being operated
-on, and
-
-\item
-Arguments to the call (which are really just particularly interesting
-variables), and
-
-\item
-A current location (\pxlref{source-locations}), which is the place in
-the program where the function was running when it stopped to call another
-function, or because of an interrupt or error.
-\end{itemize}
-
-
-%%\f
-\begin{comment}
-* Stack Motion::
-* How Arguments are Printed::
-* Function Names::
-* Funny Frames::
-* Debug Tail Recursion::
-* Unknown Locations and Interrupts::
-\end{comment}
-
-%%\node Stack Motion, How Arguments are Printed, Stack Frames, Stack Frames
-\subsection{Stack Motion}
-
-These commands move to a new stack frame and print the name of the function
-and the values of its arguments in the style of a Lisp function call:
-\begin{Lentry}
-
-\item[\code{up}]
-Move up to the next higher frame. More recent function calls are considered
-to be higher on the stack.
-
-\item[\code{down}]
-Move down to the next lower frame.
-
-\item[\code{top}]
-Move to the highest frame.
-
-\item[\code{bottom}]
-Move to the lowest frame.
-
-\item[\code{frame} [\textit{n}]]
-Move to the frame with the specified number. Prompts for the number if not
-supplied.
-
-\begin{comment}
-\key{S} [\var{function-name} [\var{n}]]
-
-\item
-Search down the stack for function. Prompts for the function name if not
-supplied. Searches an optional number of times, but doesn't prompt for
-this number; enter it following the function.
-
-\item[\key{R} [\var{function-name} [\var{n}]]]
-Search up the stack for function. Prompts for the function name if not
-supplied. Searches an optional number of times, but doesn't prompt for
-this number; enter it following the function.
-\end{comment}
-\end{Lentry}
-%%\f
-%%\node How Arguments are Printed, Function Names, Stack Motion, Stack Frames
-\subsection{How Arguments are Printed}
-
-A frame is printed to look like a function call, but with the actual argument
-values in the argument positions. So the frame for this call in the source:
-\begin{lisp}
-(myfun (+ 3 4) 'a)
-\end{lisp}
-would look like this:
-\begin{example}
-(MYFUN 7 A)
-\end{example}
-All keyword and optional arguments are displayed with their actual
-values; if the corresponding argument was not supplied, the value will
-be the default. So this call:
-\begin{lisp}
-(subseq "foo" 1)
-\end{lisp}
-would look like this:
-\begin{example}
-(SUBSEQ "foo" 1 3)
-\end{example}
-And this call:
-\begin{lisp}
-(string-upcase "test case")
-\end{lisp}
-would look like this:
-\begin{example}
-(STRING-UPCASE "test case" :START 0 :END NIL)
-\end{example}
-
-The arguments to a function call are displayed by accessing the argument
-variables. Although those variables are initialized to the actual argument
-values, they can be set inside the function; in this case the new value will be
-displayed.
-
-\code{\amprest} arguments are handled somewhat differently. The value of
-the rest argument variable is displayed as the spread-out arguments to
-the call, so:
-\begin{lisp}
-(format t "~A is a ~A." "This" 'test)
-\end{lisp}
-would look like this:
-\begin{example}
-(FORMAT T "~A is a ~A." "This" 'TEST)
-\end{example}
-Rest arguments cause an exception to the normal display of keyword
-arguments in functions that have both \code{\amprest} and \code{\&key}
-arguments. In this case, the keyword argument variables are not
-displayed at all; the rest arg is displayed instead. So for these
-functions, only the keywords actually supplied will be shown, and the
-values displayed will be the argument values, not values of the
-(possibly modified) variables.
-
-If the variable for an argument is never referenced by the function, it will be
-deleted. The variable value is then unavailable, so the debugger prints
-\code{<unused-arg>} instead of the value. Similarly, if for any of a number of
-reasons (described in more detail in section \ref{debug-vars}) the value of the
-variable is unavailable or not known to be available, then
-\code{<unavailable-arg>} will be printed instead of the argument value.
-
-Printing of argument values is controlled by \code{*debug-print-level*} and
-\varref{debug-print-length}.
-
-%%\f
-%%\node Function Names, Funny Frames, How Arguments are Printed, Stack Frames
-\subsection{Function Names}
-\cpsubindex{function}{names}
-\cpsubindex{names}{function}
-
-If a function is defined by \code{defun}, \code{labels}, or \code{flet}, then the
-debugger will print the actual function name after the open parenthesis, like:
-\begin{example}
-(STRING-UPCASE "test case" :START 0 :END NIL)
-((SETF AREF) \#\back{a} "for" 1)
-\end{example}
-Otherwise, the function name is a string, and will be printed in quotes:
-\begin{example}
-("DEFUN MYFUN" BAR)
-("DEFMACRO DO" (DO ((I 0 (1+ I))) ((= I 13))) NIL)
-("SETQ *GC-NOTIFY-BEFORE*")
-\end{example}
-This string name is derived from the \w{\code{def}\var{mumble}} form that encloses
-or expanded into the lambda, or the outermost enclosing form if there is no
-\w{\code{def}\var{mumble}}.
-
-%%\f
-%%\node Funny Frames, Debug Tail Recursion, Function Names, Stack Frames
-\subsection{Funny Frames}
-\cindex{external entry points}
-\cpsubindex{entry points}{external}
-\cpsubindex{block compilation}{debugger implications}
-\cpsubindex{external}{stack frame kind}
-\cpsubindex{optional}{stack frame kind}
-\cpsubindex{cleanup}{stack frame kind}
-
-Sometimes the evaluator introduces new functions that are used to implement a
-user function, but are not directly specified in the source. The main place
-this is done is for checking argument type and syntax. Usually these functions
-do their thing and then go away, and thus are not seen on the stack in the
-debugger. But when you get some sort of error during lambda-list processing,
-you end up in the debugger on one of these funny frames.
-
-These funny frames are flagged by printing ``\code{[}\var{keyword}\code{]}'' after the
-parentheses. For example, this call:
-\begin{lisp}
-(car 'a 'b)
-\end{lisp}
-will look like this:
-\begin{example}
-(CAR 2 A) [:EXTERNAL]
-\end{example}
-And this call:
-\begin{lisp}
-(string-upcase "test case" :end)
-\end{lisp}
-would look like this:
-\begin{example}
-("DEFUN STRING-UPCASE" "test case" 335544424 1) [:OPTIONAL]
-\end{example}
-
-As you can see, these frames have only a vague resemblance to the original
-call. Fortunately, the error message displayed when you enter the debugger
-will usually tell you what problem is (in these cases, too many arguments
-and odd keyword arguments.) Also, if you go down the stack to the frame for
-the calling function, you can display the original source (\pxlref{source-locations}.)
-
-With recursive or block compiled functions (\pxlref{block-compilation}), an \kwd{EXTERNAL} frame may appear before the frame
-representing the first call to the recursive function or entry to the compiled
-block. This is a consequence of the way the compiler does block compilation:
-there is nothing odd with your program. You will also see \kwd{CLEANUP} frames
-during the execution of \code{unwind-protect} cleanup code. Note that inline
-expansion and open-coding affect what frames are present in the debugger, see
-sections \ref{debugger-policy} and \ref{open-coding}.
-
-%%\f
-%%\node Debug Tail Recursion, Unknown Locations and Interrupts, Funny Frames, Stack Frames
-\subsection{Debug Tail Recursion}
-\label{debug-tail-recursion}
-\cindex{tail recursion}
-\cpsubindex{recursion}{tail}
-
-Both the compiler and the interpreter are ``properly tail recursive.'' If a
-function call is in a tail-recursive position, the stack frame will be
-deallocated \i{at the time of the call}, rather than after the call returns.
-Consider this backtrace:
-\begin{example}
-(BAR ...)
-(FOO ...)
-\end{example}
-Because of tail recursion, it is not necessarily the case that
-\code{FOO} directly called \code{BAR}. It may be that \code{FOO} called
-some other function \code{FOO2} which then called \code{BAR}
-tail-recursively, as in this example:
-\begin{example}
-(defun foo ()
- ...
- (foo2 ...)
- ...)
-
-(defun foo2 (...)
- ...
- (bar ...))
-
-(defun bar (...)
- ...)
-\end{example}
-
-Usually the elimination of tail-recursive frames makes debugging more
-pleasant, since these frames are mostly uninformative. If there is any
-doubt about how one function called another, it can usually be
-eliminated by finding the source location in the calling frame (section
-\ref{source-locations}.)
-
-For a more thorough discussion of tail recursion, \pxlref{tail-recursion}.
-
-%%\f
-%%\node Unknown Locations and Interrupts, , Debug Tail Recursion, Stack Frames
-\subsection{Unknown Locations and Interrupts}
-\label{unknown-locations}
-\cindex{unknown code locations}
-\cpsubindex{locations}{unknown}
-\cindex{interrupts}
-\cpsubindex{errors}{run-time}
-
-The debugger operates using special debugging information attached to
-the compiled code. This debug information tells the debugger what it
-needs to know about the locations in the code where the debugger can be
-invoked. If the debugger somehow encounters a location not described in
-the debug information, then it is said to be \var{unknown}. If the code
-location for a frame is unknown, then some variables may be
-inaccessible, and the source location cannot be precisely displayed.
-
-There are three reasons why a code location could be unknown:
-\begin{itemize}
-
-\item
-There is inadequate debug information due to the value of the \code{debug}
-optimization quality. \xlref{debugger-policy}.
-
-\item
-The debugger was entered because of an interrupt such as \code{$\hat{ }C$}.
-
-\item
-A hardware error such as ``\code{bus error}'' occurred in code that was
-compiled unsafely due to the value of the \code{safety} optimization
-quality. \xlref{optimize-declaration}.
-\end{itemize}
-
-In the last two cases, the values of argument variables are accessible,
-but may be incorrect. \xlref{debug-var-validity} for more details on
-when variable values are accessible.
-
-It is possible for an interrupt to happen when a function call or return is in
-progress. The debugger may then flame out with some obscure error or insist
-that the bottom of the stack has been reached, when the real problem is that
-the current stack frame can't be located. If this happens, return from the
-interrupt and try again.
-
-When running interpreted code, all locations should be known. However,
-an interrupt might catch some subfunction of the interpreter at an
-unknown location. In this case, you should be able to go up the stack a
-frame or two and reach an interpreted frame which can be debugged.
-
-%%\f
-%%\node Variable Access, Source Location Printing, Stack Frames, The Debugger
-\section{Variable Access}
-\label{debug-vars}
-\cpsubindex{variables}{debugger access}
-\cindex{debug variables}
-
-There are three ways to access the current frame's local variables in the
-debugger. The simplest is to type the variable's name into the debugger's
-read-eval-print loop. The debugger will evaluate the variable reference as
-though it had appeared inside that frame.
-
-The debugger doesn't really understand lexical scoping; it has just one
-namespace for all the variables in a function. If a symbol is the name of
-multiple variables in the same function, then the reference appears ambiguous,
-even though lexical scoping specifies which value is visible at any given
-source location. If the scopes of the two variables are not nested, then the
-debugger can resolve the ambiguity by observing that only one variable is
-accessible.
-
-When there are ambiguous variables, the evaluator assigns each one a
-small integer identifier. The \code{debug:var} function and the
-\code{list-locals} command use this identifier to distinguish between
-ambiguous variables:
-\begin{Lentry}
-
-\item[\code{list-locals} \mopt{\var{prefix}}]%%\hfill\\
-This command prints the name and value of all variables in the current
-frame whose name has the specified \var{prefix}. \var{prefix} may be a
-string or a symbol. If no \var{prefix} is given, then all available
-variables are printed. If a variable has a potentially ambiguous name,
-then the name is printed with a ``\code{\#}\var{identifier}'' suffix, where
-\var{identifier} is the small integer used to make the name unique.
-\end{Lentry}
-
-\begin{defun}{debug:}{var}{\args{\var{name} \ampoptional{} \var{identifier}}}
-
- This function returns the value of the variable in the current frame
- with the specified \var{name}. If supplied, \var{identifier}
- determines which value to return when there are ambiguous variables.
-
- When \var{name} is a symbol, it is interpreted as the symbol name of
- the variable, i.e. the package is significant. If \var{name} is an
- uninterned symbol (gensym), then return the value of the uninterned
- variable with the same name. If \var{name} is a string,
- \code{debug:var} interprets it as the prefix of a variable name, and
- must unambiguously complete to the name of a valid variable.
-
- This function is useful mainly for accessing the value of uninterned
- or ambiguous variables, since most variables can be evaluated
- directly.
-\end{defun}
-
-%%\f
-\begin{comment}
-* Variable Value Availability::
-* Note On Lexical Variable Access::
-\end{comment}
-
-%%\node Variable Value Availability, Note On Lexical Variable Access, Variable Access, Variable Access
-\subsection{Variable Value Availability}
-\label{debug-var-validity}
-\cindex{availability of debug variables}
-\cindex{validity of debug variables}
-\cindex{debug optimization quality}
-
-The value of a variable may be unavailable to the debugger in portions of the
-program where \clisp{} says that the variable is defined. If a variable value is
-not available, the debugger will not let you read or write that variable. With
-one exception, the debugger will never display an incorrect value for a
-variable. Rather than displaying incorrect values, the debugger tells you the
-value is unavailable.
-
-The one exception is this: if you interrupt (e.g., with \code{$\hat{ }C$}) or if there is
-an unexpected hardware error such as ``\code{bus error}'' (which should only happen
-in unsafe code), then the values displayed for arguments to the interrupted
-frame might be incorrect.\footnote{Since the location of an interrupt or hardware
-error will always be an unknown location (\pxlref{unknown-locations}),
-non-argument variable values will never be available in the interrupted frame.}
-This exception applies only to the interrupted frame: any frame farther down
-the stack will be fine.
-
-The value of a variable may be unavailable for these reasons:
-\begin{itemize}
-
-\item
-The value of the \code{debug} optimization quality may have omitted debug
-information needed to determine whether the variable is available.
-Unless a variable is an argument, its value will only be available when
-\code{debug} is at least \code{2}.
-
-\item
-The compiler did lifetime analysis and determined that the value was no longer
-needed, even though its scope had not been exited. Lifetime analysis is
-inhibited when the \code{debug} optimization quality is \code{3}.
-
-\item
-The variable's name is an uninterned symbol (gensym). To save space, the
-compiler only dumps debug information about uninterned variables when the
-\code{debug} optimization quality is \code{3}.
-
-\item
-The frame's location is unknown (\pxlref{unknown-locations}) because
-the debugger was entered due to an interrupt or unexpected hardware error.
-Under these conditions the values of arguments will be available, but might be
-incorrect. This is the exception above.
-
-\item
-The variable was optimized out of existence. Variables with no reads are
-always optimized away, even in the interpreter. The degree to which the
-compiler deletes variables will depend on the value of the \code{compile-speed}
-optimization quality, but most source-level optimizations are done under all
-compilation policies.
-\end{itemize}
-
-
-Since it is especially useful to be able to get the arguments to a function,
-argument variables are treated specially when the \code{speed} optimization
-quality is less than \code{3} and the \code{debug} quality is at least \code{1}.
-With this compilation policy, the values of argument variables are almost
-always available everywhere in the function, even at unknown locations. For
-non-argument variables, \code{debug} must be at least \code{2} for values to be
-available, and even then, values are only available at known locations.
-
-%%\f
-%%\node Note On Lexical Variable Access, , Variable Value Availability, Variable Access
-\subsection{Note On Lexical Variable Access}
-\cpsubindex{evaluation}{debugger}
-
-When the debugger command loop establishes variable bindings for available
-variables, these variable bindings have lexical scope and dynamic
-extent.\footnote{The variable bindings are actually created using the \clisp{}
-\code{symbol-macro-let} special form.} You can close over them, but such closures
-can't be used as upward funargs.
-
-You can also set local variables using \code{setq}, but if the variable was closed
-over in the original source and never set, then setting the variable in the
-debugger may not change the value in all the functions the variable is defined
-in. Another risk of setting variables is that you may assign a value of a type
-that the compiler proved the variable could never take on. This may result in
-bad things happening.
-
-%%\f
-%%\node Source Location Printing, Compiler Policy Control, Variable Access, The Debugger
-\section{Source Location Printing}
-\label{source-locations}
-\cpsubindex{source location printing}{debugger}
-
-One of CMU \clisp{}'s unique capabilities is source level debugging of compiled
-code. These commands display the source location for the current frame:
-\begin{Lentry}
-
-\item[\code{source} \mopt{\var{context}}]%%\hfill\\
-This command displays the file that the current frame's function was defined
-from (if it was defined from a file), and then the source form responsible for
-generating the code that the current frame was executing. If \var{context} is
-specified, then it is an integer specifying the number of enclosing levels of
-list structure to print.
-
-\item[\code{vsource} \mopt{\var{context}}]%%\hfill\\
-This command is identical to \code{source}, except that it uses the
-global values of \code{*print-level*} and \code{*print-length*} instead
-of the debugger printing control variables \code{*debug-print-level*}
-and \code{*debug-print-length*}.
-\end{Lentry}
-
-The source form for a location in the code is the innermost list present
-in the original source that encloses the form responsible for generating
-that code. If the actual source form is not a list, then some enclosing
-list will be printed. For example, if the source form was a reference
-to the variable \code{*some-random-special*}, then the innermost
-enclosing evaluated form will be printed. Here are some possible
-enclosing forms:
-\begin{example}
-(let ((a *some-random-special*))
- ...)
-
-(+ *some-random-special* ...)
-\end{example}
-
-If the code at a location was generated from the expansion of a macro or a
-source-level compiler optimization, then the form in the original source that
-expanded into that code will be printed. Suppose the file
-\file{/usr/me/mystuff.lisp} looked like this:
-\begin{example}
-(defmacro mymac ()
- '(myfun))
-
-(defun foo ()
- (mymac)
- ...)
-\end{example}
-If \code{foo} has called \code{myfun}, and is waiting for it to return, then the
-\code{source} command would print:
-\begin{example}
-; File: /usr/me/mystuff.lisp
-
-(MYMAC)
-\end{example}
-Note that the macro use was printed, not the actual function call form,
-\code{(myfun)}.
-
-If enclosing source is printed by giving an argument to \code{source} or
-\code{vsource}, then the actual source form is marked by wrapping it in a list
-whose first element is \code{\#:***HERE***}. In the previous example,
-\w{\code{source 1}} would print:
-\begin{example}
-; File: /usr/me/mystuff.lisp
-
-(DEFUN FOO ()
- (#:***HERE***
- (MYMAC))
- ...)
-\end{example}
-
-%%\f
-\begin{comment}
-* How the Source is Found::
-* Source Location Availability::
-\end{comment}
-
-%%\node How the Source is Found, Source Location Availability, Source Location Printing, Source Location Printing
-\subsection{How the Source is Found}
-
-If the code was defined from \llisp{} by \code{compile} or
-\code{eval}, then the source can always be reliably located. If the
-code was defined from a \code{fasl} file created by
-\findexed{compile-file}, then the debugger gets the source forms it
-prints by reading them from the original source file. This is a
-potential problem, since the source file might have moved or changed
-since the time it was compiled.
-
-The source file is opened using the \code{truename} of the source file
-pathname originally given to the compiler. This is an absolute pathname
-with all logical names and symbolic links expanded. If the file can't
-be located using this name, then the debugger gives up and signals an
-error.
-
-If the source file can be found, but has been modified since the time it was
-compiled, the debugger prints this warning:
-\begin{example}
-; File has been modified since compilation:
-; \var{filename}
-; Using form offset instead of character position.
-\end{example}
-where \var{filename} is the name of the source file. It then proceeds using a
-robust but not foolproof heuristic for locating the source. This heuristic
-works if:
-\begin{itemize}
-
-\item
-No top-level forms before the top-level form containing the source have been
-added or deleted, and
-
-\item
-The top-level form containing the source has not been modified much. (More
-precisely, none of the list forms beginning before the source form have been
-added or deleted.)
-\end{itemize}
-
-If the heuristic doesn't work, the displayed source will be wrong, but will
-probably be near the actual source. If the ``shape'' of the top-level form in
-the source file is too different from the original form, then an error will be
-signaled. When the heuristic is used, the the source location commands are
-noticeably slowed.
-
-Source location printing can also be confused if (after the source was
-compiled) a read-macro you used in the code was redefined to expand into
-something different, or if a read-macro ever returns the same \code{eq}
-list twice. If you don't define read macros and don't use \code{\#\#} in
-perverted ways, you don't need to worry about this.
-
-%%\f
-%%\node Source Location Availability, , How the Source is Found, Source Location Printing
-\subsection{Source Location Availability}
-
-\cindex{debug optimization quality}
-Source location information is only available when the \code{debug}
-optimization quality is at least \code{2}. If source location information is
-unavailable, the source commands will give an error message.
-
-If source location information is available, but the source location is
-unknown because of an interrupt or unexpected hardware error
-(\pxlref{unknown-locations}), then the command will print:
-\begin{example}
-Unknown location: using block start.
-\end{example}
-and then proceed to print the source location for the start of the \i{basic
-block} enclosing the code location. \cpsubindex{block}{basic}
-\cpsubindex{block}{start location}
-It's a bit complicated to explain exactly what a basic block is, but
-here are some properties of the block start location:
-\begin{itemize}
-
-\item The block start location may be the same as the true location.
-
-\item The block start location will never be later in the the
- program's flow of control than the true location.
-
-\item No conditional control structures (such as \code{if},
- \code{cond}, \code{or}) will intervene between the block start and
- the true location (but note that some conditionals present in the
- original source could be optimized away.) Function calls \i{do not}
- end basic blocks.
-
-\item The head of a loop will be the start of a block.
-
-\item The programming language concept of ``block structure'' and the
- \clisp{} \code{block} special form are totally unrelated to the
- compiler's basic block.
-\end{itemize}
-
-In other words, the true location lies between the printed location and the
-next conditional (but watch out because the compiler may have changed the
-program on you.)
-
-%%\f
-%%\node Compiler Policy Control, Exiting Commands, Source Location Printing, The Debugger
-\section{Compiler Policy Control}
-\label{debugger-policy}
-\cpsubindex{policy}{debugger}
-\cindex{debug optimization quality}
-\cindex{optimize declaration}
-
-The compilation policy specified by \code{optimize} declarations affects the
-behavior seen in the debugger. The \code{debug} quality directly affects the
-debugger by controlling the amount of debugger information dumped. Other
-optimization qualities have indirect but observable effects due to changes in
-the way compilation is done.
-
-Unlike the other optimization qualities (which are compared in relative value
-to evaluate tradeoffs), the \code{debug} optimization quality is directly
-translated to a level of debug information. This absolute interpretation
-allows the user to count on a particular amount of debug information being
-available even when the values of the other qualities are changed during
-compilation. These are the levels of debug information that correspond to the
-values of the \code{debug} quality:
-\begin{Lentry}
-
-\item[\code{0}]
-Only the function name and enough information to allow the stack to
-be parsed.
-
-\item[\code{\w{$>$ 0}}]
-Any level greater than \code{0} gives level \code{0} plus all
-argument variables. Values will only be accessible if the argument
-variable is never set and
-\code{speed} is not \code{3}. \cmucl{} allows any real value for optimization
-qualities. It may be useful to specify \code{0.5} to get backtrace argument
-display without argument documentation.
-
-\item[\code{1}] Level \code{1} provides argument documentation
-(printed arglists) and derived argument/result type information.
-This makes \findexed{describe} more informative, and allows the
-compiler to do compile-time argument count and type checking for any
-calls compiled at run-time.
-
-\item[\code{2}]
-Level \code{1} plus all interned local variables, source location
-information, and lifetime information that tells the debugger when arguments
-are available (even when \code{speed} is \code{3} or the argument is set.) This is
-the default.
-
-\item[\code{3}]
-Level \code{2} plus all uninterned variables. In addition, lifetime
-analysis is disabled (even when \code{speed} is \code{3}), ensuring that all variable
-values are available at any known location within the scope of the binding.
-This has a speed penalty in addition to the obvious space penalty.
-\end{Lentry}
-
-As you can see, if the \code{speed} quality is \code{3}, debugger performance is
-degraded. This effect comes from the elimination of argument variable
-special-casing (\pxlref{debug-var-validity}.) Some degree of
-speed/debuggability tradeoff is unavoidable, but the effect is not too drastic
-when \code{debug} is at least \code{2}.
-
-\cindex{inline expansion}
-\cindex{semi-inline expansion}
-In addition to \code{inline} and \code{notinline} declarations, the relative values
-of the \code{speed} and \code{space} qualities also change whether functions are
-inline expanded (\pxlref{inline-expansion}.) If a function is inline
-expanded, then there will be no frame to represent the call, and the arguments
-will be treated like any other local variable. Functions may also be
-``semi-inline'', in which case there is a frame to represent the call, but the
-call is to an optimized local version of the function, not to the original
-function.
-
-%%\f
-%%\node Exiting Commands, Information Commands, Compiler Policy Control, The Debugger
-\section{Exiting Commands}
-
-These commands get you out of the debugger.
-
-\begin{Lentry}
-
-\item[\code{quit}]
-Throw to top level.
-
-\item[\code{restart} \mopt{\var{n}}]%%\hfill\\
-Invokes the \var{n}th restart case as displayed by the \code{error}
-command. If \var{n} is not specified, the available restart cases are
-reported.
-
-\item[\code{go}]
-Calls \code{continue} on the condition given to \code{debug}. If there is no
-restart case named \var{continue}, then an error is signaled.
-
-\item[\code{abort}]
-Calls \code{abort} on the condition given to \code{debug}. This is
-useful for popping debug command loop levels or aborting to top level,
-as the case may be.
-
-\begin{comment}
-(\code{debug:debug-return} \var{expression} \mopt{\var{frame}})
-
-\item
-From the current or specified frame, return the result of evaluating
-expression. If multiple values are expected, then this function should be
-called for multiple values.
-\end{comment}
-\end{Lentry}
-
-%%\f
-%%\node Information Commands, Breakpoint Commands, Exiting Commands, The Debugger
-\section{Information Commands}
-
-Most of these commands print information about the current frame or
-function, but a few show general information.
-
-\begin{Lentry}
-
-\item[\code{help}, \code{?}]
-Displays a synopsis of debugger commands.
-
-\item[\code{describe}]
-Calls \code{describe} on the current function, displays number of local
-variables, and indicates whether the function is compiled or interpreted.
-
-\item[\code{print}]
-Displays the current function call as it would be displayed by moving to
-this frame.
-
-\item[\code{vprint} (or \code{pp}) \mopt{\var{verbosity}}]%%\hfill\\
-Displays the current function call using \code{*print-level*} and
-\code{*print-length*} instead of \code{*debug-print-level*} and
-\code{*debug-print-length*}. \var{verbosity} is a small integer
-(default 2) that controls other dimensions of verbosity.
-
-\item[\code{error}]
-Prints the condition given to \code{invoke-debugger} and the active
-proceed cases.
-
-\item[\code{backtrace} \mopt{\var{n}}]\hfill\\
-Displays all the frames from the current to the bottom. Only shows
-\var{n} frames if specified. The printing is controlled by
-\code{*debug-print-level*} and \code{*debug-print-length*}.
-
-\begin{comment}
-(\code{debug:debug-function} \mopt{\var{n}})
-
-\item
-Returns the function from the current or specified frame.
-
-\item[(\code{debug:function-name} \mopt{\var{n}])]
-Returns the function name from the current or specified frame.
-
-\item[(\code{debug:pc} \mopt{\var{frame}})]
-Returns the index of the instruction for the function in the current or
-specified frame. This is useful in conjunction with \code{disassemble}.
-The pc returned points to the instruction after the one that was fatal.
-\end{comment}
-\end{Lentry}
-
-%%\f
-%%\node Breakpoint Commands, Function Tracing, Information Commands, The Debugger
-\section{Breakpoint Commands}
-
-\cmucl{} supports setting of breakpoints inside compiled functions and
-stepping of compiled code. Breakpoints can only be set at at known
-locations (\pxlref{unknown-locations}), so these commands are largely
-useless unless the \code{debug} optimize quality is at least \code{2}
-(\pxlref{debugger-policy}). These commands manipulate breakpoints:
-\begin{Lentry}
-\item[\code{breakpoint} \var{location} \mstar{\var{option} \var{value}}]
-%%\hfill\\
-Set a breakpoint in some function. \var{location} may be an integer
-code location number (as displayed by \code{list-locations}) or a
-keyword. The keyword can be used to indicate setting a breakpoint at
-the function start (\kwd{start}, \kwd{s}) or function end
-(\kwd{end}, \kwd{e}). The \code{breakpoint} command has
-\kwd{condition}, \kwd{break}, \kwd{print} and \kwd{function}
-options which work similarly to the \code{trace} options.
-
-\item[\code{list-locations} (or \code{ll}) \mopt{\var{function}}]%%\hfill\\
-List all the code locations in the current frame's function, or in
-\var{function} if it is supplied. The display format is the code
-location number, a colon and then the source form for that location:
-\begin{example}
-3: (1- N)
-\end{example}
-If consecutive locations have the same source, then a numeric range like
-\code{3-5:} will be printed. For example, a default function call has a
-known location both immediately before and after the call, which would
-result in two code locations with the same source. The listed function
-becomes the new default function for breakpoint setting (via the
-\code{breakpoint}) command.
-
-\item[\code{list-breakpoints} (or \code{lb})]%%\hfill\\
-List all currently active breakpoints with their breakpoint number.
-
-\item[\code{delete-breakpoint} (or \code{db}) \mopt{\var{number}}]%%\hfill\\
-Delete a breakpoint specified by its breakpoint number. If no number is
-specified, delete all breakpoints.
-
-\item[\code{step}]%%\hfill\\
-Step to the next possible breakpoint location in the current function.
-This always steps over function calls, instead of stepping into them
-\end{Lentry}
-
-\begin{comment}
-* Breakpoint Example::
-\end{comment}
-
-%%\node Breakpoint Example, , Breakpoint Commands, Breakpoint Commands
-\subsection{Breakpoint Example}
-
-Consider this definition of the factorial function:
-\begin{lisp}
-(defun ! (n)
- (if (zerop n)
- 1
- (* n (! (1- n)))))
-\end{lisp}
-This debugger session demonstrates the use of breakpoints:
-\begin{example}
-common-lisp-user> (break) ; Invoke debugger
-
-Break
-
-Restarts:
- 0: [CONTINUE] Return from BREAK.
- 1: [ABORT ] Return to Top-Level.
-
-Debug (type H for help)
-
-(INTERACTIVE-EVAL (BREAK))
-0] ll #'!
-0: #'(LAMBDA (N) (BLOCK ! (IF # 1 #)))
-1: (ZEROP N)
-2: (* N (! (1- N)))
-3: (1- N)
-4: (! (1- N))
-5: (* N (! (1- N)))
-6: #'(LAMBDA (N) (BLOCK ! (IF # 1 #)))
-0] br 2
-(* N (! (1- N)))
-1: 2 in !
-Added.
-0] q
-
-common-lisp-user> (! 10) ; Call the function
-
-*Breakpoint hit*
-
-Restarts:
- 0: [CONTINUE] Return from BREAK.
- 1: [ABORT ] Return to Top-Level.
-
-Debug (type H for help)
-
-(! 10) ; We are now in first call (arg 10) before the multiply
-Source: (* N (! (1- N)))
-3] st
-
-*Step*
-
-(! 10) ; We have finished evaluation of (1- n)
-Source: (1- N)
-3] st
-
-*Breakpoint hit*
-
-Restarts:
- 0: [CONTINUE] Return from BREAK.
- 1: [ABORT ] Return to Top-Level.
-
-Debug (type H for help)
-
-(! 9) ; We hit the breakpoint in the recursive call
-Source: (* N (! (1- N)))
-3]
-\end{example}
-
-
-
-%%\f
-%%\node Function Tracing, Specials, Breakpoint Commands, The Debugger
-\section{Function Tracing}
-\cindex{tracing}
-\cpsubindex{function}{tracing}
-
-The tracer causes selected functions to print their arguments and
-their results whenever they are called. Options allow conditional
-printing of the trace information and conditional breakpoints on
-function entry or exit.
-
-\begin{defmac}{}{trace}{%
- \args{\mstar{option global-value} \mstar{name \mstar{option
- value}}}}
-
- \code{trace} is a debugging tool that prints information when
- specified functions are called. In its simplest form:
- \begin{example}
- (trace \var{name-1} \var{name-2} ...)
- \end{example}
- \code{trace} causes a printout on \vindexed{trace-output} each time
- that one of the named functions is entered or returns (the
- \var{names} are not evaluated.) Trace output is indented according
- to the number of pending traced calls, and this trace depth is
- printed at the beginning of each line of output. Printing verbosity
- of arguments and return values is controlled by
- \vindexed{debug-print-level} and \vindexed{debug-print-length}.
-
- If no \var{names} or \var{options} are are given, \code{trace}
- returns the list of all currently traced functions,
- \code{*traced-function-list*}.
-
- Trace options can cause the normal printout to be suppressed, or
- cause extra information to be printed. Each option is a pair of an
- option keyword and a value form. Options may be interspersed with
- function names. Options only affect tracing of the function whose
- name they appear immediately after. Global options are specified
- before the first name, and affect all functions traced by a given
- use of \code{trace}. If an already traced function is traced again,
- any new options replace the old options. The following options are
- defined:
- \begin{Lentry}
- \item[\kwd{condition} \var{form}, \kwd{condition-after} \var{form},
- \kwd{condition-all} \var{form}] If \kwd{condition} is specified,
- then \code{trace} does nothing unless \var{form} evaluates to true
- at the time of the call. \kwd{condition-after} is similar, but
- suppresses the initial printout, and is tested when the function
- returns. \kwd{condition-all} tries both before and after.
-
- \item[\kwd{wherein} \var{names}] If specified, \var{names} is a
- function name or list of names. \code{trace} does nothing unless
- a call to one of those functions encloses the call to this
- function (i.e. it would appear in a backtrace.) Anonymous
- functions have string names like \code{"DEFUN FOO"}.
-
- \item[\kwd{break} \var{form}, \kwd{break-after} \var{form},
- \kwd{break-all} \var{form}] If specified, and \var{form} evaluates
- to true, then the debugger is invoked at the start of the
- function, at the end of the function, or both, according to the
- respective option.
-
- \item[\kwd{print} \var{form}, \kwd{print-after} \var{form},
- \kwd{print-all} \var{form}] In addition to the usual printout, the
- result of evaluating \var{form} is printed at the start of the
- function, at the end of the function, or both, according to the
- respective option. Multiple print options cause multiple values
- to be printed.
-
- \item[\kwd{function} \var{function-form}] This is a not really an
- option, but rather another way of specifying what function to
- trace. The \var{function-form} is evaluated immediately, and the
- resulting function is traced.
-
- \item[\kwd{encapsulate \mgroup{:default | t | nil}}] In \cmucl,
- tracing can be done either by temporarily redefining the function
- name (encapsulation), or using breakpoints. When breakpoints are
- used, the function object itself is destructively modified to
- cause the tracing action. The advantage of using breakpoints is
- that tracing works even when the function is anonymously called
- via \code{funcall}.
-
- When \kwd{encapsulate} is true, tracing is done via encapsulation.
- \kwd{default} is the default, and means to use encapsulation for
- interpreted functions and funcallable instances, breakpoints
- otherwise. When encapsulation is used, forms are {\it not}
- evaluated in the function's lexical environment, but
- \code{debug:arg} can still be used.
- \end{Lentry}
-
- \kwd{condition}, \kwd{break} and \kwd{print} forms are evaluated in
- the lexical environment of the called function; \code{debug:var} and
- \code{debug:arg} can be used. The \code{-after} and \code{-all}
- forms are evaluated in the null environment.
-\end{defmac}
-
-\begin{defmac}{}{untrace}{ \args{\amprest{} \var{function-names}}}
-
- This macro turns off tracing for the specified functions, and
- removes their names from \code{*traced-function-list*}. If no
- \var{function-names} are given, then all currently traced functions
- are untraced.
-\end{defmac}
-
-\begin{defvar}{extensions:}{traced-function-list}
-
- A list of function names maintained and used by \code{trace},
- \code{untrace}, and \code{untrace-all}. This list should contain
- the names of all functions currently being traced.
-\end{defvar}
-
-\begin{defvar}{extensions:}{max-trace-indentation}
-
- The maximum number of spaces which should be used to indent trace
- printout. This variable is initially set to 40.
-\end{defvar}
-
-\begin{comment}
-* Encapsulation Functions::
-\end{comment}
-
-%%\node Encapsulation Functions, , Function Tracing, Function Tracing
-\subsection{Encapsulation Functions}
-\cindex{encapsulation}
-\cindex{advising}
-
-The encapsulation functions provide a mechanism for intercepting the
-arguments and results of a function. \code{encapsulate} changes the
-function definition of a symbol, and saves it so that it can be
-restored later. The new definition normally calls the original
-definition. The \clisp{} \findexed{fdefinition} function always returns
-the original definition, stripping off any encapsulation.
-
-The original definition of the symbol can be restored at any time by
-the \code{unencapsulate} function. \code{encapsulate} and \code{unencapsulate}
-allow a symbol to be multiply encapsulated in such a way that different
-encapsulations can be completely transparent to each other.
-
-Each encapsulation has a type which may be an arbitrary lisp object.
-If a symbol has several encapsulations of different types, then any
-one of them can be removed without affecting more recent ones.
-A symbol may have more than one encapsulation of the same type, but
-only the most recent one can be undone.
-
-\begin{defun}{extensions:}{encapsulate}{%
- \args{\var{symbol} \var{type} \var{body}}}
-
- Saves the current definition of \var{symbol}, and replaces it with a
- function which returns the result of evaluating the form,
- \var{body}. \var{Type} is an arbitrary lisp object which is the
- type of encapsulation.
-
- When the new function is called, the following variables are bound
- for the evaluation of \var{body}:
- \begin{Lentry}
-
- \item[\code{extensions:argument-list}] A list of the arguments to
- the function.
-
- \item[\code{extensions:basic-definition}] The unencapsulated
- definition of the function.
- \end{Lentry}
- The unencapsulated definition may be called with the original
- arguments by including the form
- \begin{lisp}
- (apply extensions:basic-definition extensions:argument-list)
- \end{lisp}
-
- \code{encapsulate} always returns \var{symbol}.
-\end{defun}
-
-\begin{defun}{extensions:}{unencapsulate}{\args{\var{symbol} \var{type}}}
-
- Undoes \var{symbol}'s most recent encapsulation of type \var{type}.
- \var{Type} is compared with \code{eq}. Encapsulations of other
- types are left in place.
-\end{defun}
-
-\begin{defun}{extensions:}{encapsulated-p}{%
- \args{\var{symbol} \var{type}}}
-
- Returns \true{} if \var{symbol} has an encapsulation of type
- \var{type}. Returns \nil{} otherwise. \var{type} is compared with
- \code{eq}.
-\end{defun}
-
-%%\f
-\begin{comment}
-section{The Single Stepper}
-
-\begin{defmac}{}{step}{ \args{\var{form}}}
-
- Evaluates form with single stepping enabled or if \var{form} is
- \code{T}, enables stepping until explicitly disabled. Stepping can
- be disabled by quitting to the lisp top level, or by evaluating the
- form \w{\code{(step ())}}.
-
- While stepping is enabled, every call to eval will prompt the user
- for a single character command. The prompt is the form which is
- about to be \code{eval}ed. It is printed with \code{*print-level*}
- and \code{*print-length*} bound to \code{*step-print-level*} and
- \code{*step-print-length*}. All interaction is done through the
- stream \code{*query-io*}. Because of this, the stepper can not be
- used in Hemlock eval mode. When connected to a slave Lisp, the
- stepper can be used from Hemlock.
-
- The commands are:
- \begin{Lentry}
-
- \item[\key{n} (next)] Evaluate the expression with stepping still
- enabled.
-
- \item[\key{s} (skip)] Evaluate the expression with stepping
- disabled.
-
- \item[\key{q} (quit)] Evaluate the expression, but disable all
- further stepping inside the current call to \code{step}.
-
- \item[\key{p} (print)] Print current form. (does not use
- \code{*step-print-level*} or \code{*step-print-length*}.)
-
- \item[\key{b} (break)] Enter break loop, and then prompt for the
- command again when the break loop returns.
-
- \item[\key{e} (eval)] Prompt for and evaluate an arbitrary
- expression. The expression is evaluated with stepping disabled.
-
- \item[\key{?} (help)] Prints a brief list of the commands.
-
- \item[\key{r} (return)] Prompt for an arbitrary value to return as
- result of the current call to eval.
-
- \item[\key{g}] Throw to top level.
- \end{Lentry}
-\end{defmac}
-
-\begin{defvar}{extensions:}{step-print-level}
- \defvarx[extensions:]{step-print-length}
-
- \code{*print-level*} and \code{*print-length*} are bound to these
- values while printing the current form. \code{*step-print-level*}
- and \code{*step-print-length*} are initially bound to 4 and 5,
- respectively.
-\end{defvar}
-
-\begin{defvar}{extensions:}{max-step-indentation}
-
- Step indents the prompts to highlight the nesting of the evaluation.
- This variable contains the maximum number of spaces to use for
- indenting. Initially set to 40.
-\end{defvar}
-
-\end{comment}
-
-%%\f
-%%\node Specials, , Function Tracing, The Debugger
-\section{Specials}
-These are the special variables that control the debugger action.
-
-\begin{changebar}
-\begin{defvar}{debug:}{debug-print-level}
- \defvarx[debug:]{debug-print-length}
-
- \code{*print-level*} and \code{*print-length*} are bound to these
- values during the execution of some debug commands. When evaluating
- arbitrary expressions in the debugger, the normal values of
- \code{*print-level*} and \code{*print-length*} are in effect. These
- variables are initially set to 3 and 5, respectively.
-\end{defvar}
-\end{changebar}
-
-%%\f
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/compiler.ms}
-
-
-%%\node The Compiler, Advanced Compiler Use and Efficiency Hints, The Debugger, Top
-\chapter{The Compiler} \hide{ -*- Dictionary: cmu-user -*-}
-
-\begin{comment}
-* Compiler Introduction::
-* Calling the Compiler::
-* Compilation Units::
-* Interpreting Error Messages::
-* Types in Python::
-* Getting Existing Programs to Run::
-* Compiler Policy::
-* Open Coding and Inline Expansion::
-\end{comment}
-
-%%\node Compiler Introduction, Calling the Compiler, The Compiler, The Compiler
-\section{Compiler Introduction}
-
-This chapter contains information about the compiler that every \cmucl{} user
-should be familiar with. Chapter \ref{advanced-compiler} goes into greater
-depth, describing ways to use more advanced features.
-
-The \cmucl{} compiler (also known as \Python{}) has many features
-that are seldom or never supported by conventional \llisp{}
-compilers:
-\begin{itemize}
-
-\item Source level debugging of compiled code (see chapter
- \ref{debugger}.)
-
-\item Type error compiler warnings for type errors detectable at
- compile time.
-
-\item Compiler error messages that provide a good indication of where
- the error appeared in the source.
-
-\item Full run-time checking of all potential type errors, with
- optimization of type checks to minimize the cost.
-
-\item Scheme-like features such as proper tail recursion and extensive
- source-level optimization.
-
-\item Advanced tuning and optimization features such as comprehensive
- efficiency notes, flow analysis, and untagged number representations
- (see chapter \ref{advanced-compiler}.)
-\end{itemize}
-
-
-%%\f
-%%\node Calling the Compiler, Compilation Units, Compiler Introduction, The Compiler
-\section{Calling the Compiler}
-\cindex{compiling}
-Functions may be compiled using \code{compile}, \code{compile-file}, or
-\code{compile-from-stream}.
-
-\begin{defun}{}{compile}{ \args{\var{name} \ampoptional{} \var{definition}}}
-
- This function compiles the function whose name is \var{name}. If
- \var{name} is \false, the compiled function object is returned. If
- \var{definition} is supplied, it should be a lambda expression that
- is to be compiled and then placed in the function cell of
- \var{name}. As per the proposed X3J13 cleanup
- ``compile-argument-problems'', \var{definition} may also be an
- interpreted function.
-
- The return values are as per the proposed X3J13 cleanup
- ``compiler-diagnostics''. The first value is the function name or
- function object. The second value is \false{} if no compiler
- diagnostics were issued, and \true{} otherwise. The third value is
- \false{} if no compiler diagnostics other than style warnings were
- issued. A non-\false{} value indicates that there were ``serious''
- compiler diagnostics issued, or that other conditions of type
- \tindexed{error} or \tindexed{warning} (but not
- \tindexed{style-warning}) were signaled during compilation.
-\end{defun}
-
-
-\begin{defun}{}{compile-file}{
- \args{\var{input-pathname}
- \keys{\kwd{output-file} \kwd{error-file} \kwd{trace-file}}
- \morekeys{\kwd{error-output} \kwd{verbose} \kwd{print} \kwd{progress}}
- \yetmorekeys{\kwd{load} \kwd{block-compile} \kwd{entry-points}}
- \yetmorekeys{\kwd{byte-compile}}}}
-
- The \cmucl{} \code{compile-file} is extended through the addition of
- several new keywords and an additional interpretation of
- \var{input-pathname}:
- \begin{Lentry}
-
- \item[\var{input-pathname}] If this argument is a list of input
- files, rather than a single input pathname, then all the source
- files are compiled into a single object file. In this case, the
- name of the first file is used to determine the default output
- file names. This is especially useful in combination with
- \var{block-compile}.
-
- \item[\kwd{output-file}] This argument specifies the name of the
- output file. \true{} gives the default name, \false{} suppresses
- the output file.
-
- \item[\kwd{error-file}] A listing of all the error output is
- directed to this file. If there are no errors, then no error file
- is produced (and any existing error file is deleted.) \true{}
- gives \w{"\var{name}\code{.err}"} (the default), and \false{}
- suppresses the output file.
-
- \item[\kwd{error-output}] If \true{} (the default), then error
- output is sent to \code{*error-output*}. If a stream, then output
- is sent to that stream instead. If \false, then error output is
- suppressed. Note that this error output is in addition to (but
- the same as) the output placed in the \var{error-file}.
-
- \item[\kwd{verbose}] If \true{} (the default), then the compiler
- prints to error output at the start and end of compilation of each
- file. See \varref{compile-verbose}.
-
- \item[\kwd{print}] If \true{} (the default), then the compiler
- prints to error output when each function is compiled. See
- \varref{compile-print}.
-
- \item[\kwd{progress}] If \true{} (default \false{}), then the
- compiler prints to error output progress information about the
- phases of compilation of each function. This is a CMU extension
- that is useful mainly in large block compilations. See
- \varref{compile-progress}.
-
- \item[\kwd{trace-file}] If \true{}, several of the intermediate
- representations (including annotated assembly code) are dumped out
- to this file. \true{} gives \w{"\var{name}\code{.trace}"}. Trace
- output is off by default. \xlref{trace-files}.
-
- \item[\kwd{load}] If \true{}, load the resulting output file.
-
- \item[\kwd{block-compile}] Controls the compile-time resolution of
- function calls. By default, only self-recursive calls are
- resolved, unless an \code{ext:block-start} declaration appears in
- the source file. \xlref{compile-file-block}.
-
- \item[\kwd{entry-points}] If non-null, then this is a list of the
- names of all functions in the file that should have global
- definitions installed (because they are referenced in other
- files.) \xlref{compile-file-block}.
-
- \item[\kwd{byte-compile}] If \true{}, compiling to a compact
- interpreted byte code is enabled. Possible values are \true{},
- \false{}, and \kwd{maybe} (the default.) See
- \varref{byte-compile-default} and \xlref{byte-compile}.
- \end{Lentry}
-
- The return values are as per the proposed X3J13 cleanup
- ``compiler-diagnostics''. The first value from \code{compile-file}
- is the truename of the output file, or \false{} if the file could
- not be created. The interpretation of the second and third values
- is described above for \code{compile}.
-\end{defun}
-
-\begin{defvar}{}{compile-verbose}
- \defvarx{compile-print}
- \defvarx{compile-progress}
-
- These variables determine the default values for the \kwd{verbose},
- \kwd{print} and \kwd{progress} arguments to \code{compile-file}.
-\end{defvar}
-
-\begin{defun}{extensions:}{compile-from-stream}{%
- \args{\var{input-stream}
- \keys{\kwd{error-stream}}
- \morekeys{\kwd{trace-stream}}
- \yetmorekeys{\kwd{block-compile} \kwd{entry-points}}
- \yetmorekeys{\kwd{byte-compile}}}}
-
- This function is similar to \code{compile-file}, but it takes all
- its arguments as streams. It reads \llisp{} code from
- \var{input-stream} until end of file is reached, compiling into the
- current environment. This function returns the same two values as
- the last two values of \code{compile}. No output files are
- produced.
-\end{defun}
-
-
-%%\f
-%%\node Compilation Units, Interpreting Error Messages, Calling the Compiler, The Compiler
-\section{Compilation Units}
-\cpsubindex{compilation}{units}
-
-\cmucl{} supports the \code{with-compilation-unit} macro added to the
-language by the proposed X3J13 ``with-compilation-unit'' compiler
-cleanup. This provides a mechanism for eliminating spurious undefined
-warnings when there are forward references across files, and also
-provides a standard way to access compiler extensions.
-
-\begin{defmac}{}{with-compilation-unit}{%
- \args{(\mstar{\var{key} \var{value}}) \mstar{\var{form}}}}
-
- This macro evaluates the \var{forms} in an environment that causes
- warnings for undefined variables, functions and types to be delayed
- until all the forms have been evaluated. Each keyword \var{value}
- is an evaluated form. These keyword options are recognized:
- \begin{Lentry}
-
- \item[\kwd{override}] If uses of \code{with-compilation-unit} are
- dynamically nested, the outermost use will take precedence,
- suppressing printing of undefined warnings by inner uses.
- However, when the \code{override} option is true this shadowing is
- inhibited; an inner use will print summary warnings for the
- compilations within the inner scope.
-
- \item[\kwd{optimize}] This is a CMU extension that specifies of the
- ``global'' compilation policy for the dynamic extent of the body.
- The argument should evaluate to an \code{optimize} declare form,
- like:
- \begin{lisp}
- (optimize (speed 3) (safety 0))
- \end{lisp}
- \xlref{optimize-declaration}
-
- \item[\kwd{optimize-interface}] Similar to \kwd{optimize}, but
- specifies the compilation policy for function interfaces (argument
- count and type checking) for the dynamic extent of the body.
- \xlref{optimize-interface-declaration}.
-
- \item[\kwd{context-declarations}] This is a CMU extension that
- pattern-matches on function names, automatically splicing in any
- appropriate declarations at the head of the function definition.
- \xlref{context-declarations}.
- \end{Lentry}
-\end{defmac}
-
-\begin{comment}
-* Undefined Warnings::
-\end{comment}
-
-%%\node Undefined Warnings, , Compilation Units, Compilation Units
-\subsection{Undefined Warnings}
-
-\cindex{undefined warnings}
-Warnings about undefined variables, functions and types are delayed until the
-end of the current compilation unit. The compiler entry functions
-(\code{compile}, etc.) implicitly use \code{with-compilation-unit}, so undefined
-warnings will be printed at the end of the compilation unless there is an
-enclosing \code{with-compilation-unit}. In order the gain the benefit of this
-mechanism, you should wrap a single \code{with-compilation-unit} around the calls
-to \code{compile-file}, i.e.:
-\begin{lisp}
-(with-compilation-unit ()
- (compile-file "file1")
- (compile-file "file2")
- ...)
-\end{lisp}
-
-Unlike for functions and types, undefined warnings for variables are
-not suppressed when a definition (e.g. \code{defvar}) appears after
-the reference (but in the same compilation unit.) This is because
-doing special declarations out of order just doesn't
-work\dash{}although early references will be compiled as special,
-bindings will be done lexically.
-
-Undefined warnings are printed with full source context
-(\pxlref{error-messages}), which tremendously simplifies the problem
-of finding undefined references that resulted from macroexpansion.
-After printing detailed information about the undefined uses of each
-name, \code{with-compilation-unit} also prints summary listings of the
-names of all the undefined functions, types and variables.
-
-\begin{defvar}{}{undefined-warning-limit}
-
- This variable controls the number of undefined warnings for each
- distinct name that are printed with full source context when the
- compilation unit ends. If there are more undefined references than
- this, then they are condensed into a single warning:
- \begin{example}
- Warning: \var{count} more uses of undefined function \var{name}.
- \end{example}
- When the value is \code{0}, then the undefined warnings are not
- broken down by name at all: only the summary listing of undefined
- names is printed.
-\end{defvar}
-
-%%\f
-%%\node Interpreting Error Messages, Types in Python, Compilation Units, The Compiler
-\section{Interpreting Error Messages}
-\label{error-messages}
-\cpsubindex{error messages}{compiler}
-\cindex{compiler error messages}
-
-One of \Python{}'s unique features is the level of source location
-information it provides in error messages. The error messages contain
-a lot of detail in a terse format, to they may be confusing at first.
-Error messages will be illustrated using this example program:
-\begin{lisp}
-(defmacro zoq (x)
- `(roq (ploq (+ ,x 3))))
-
-(defun foo (y)
- (declare (symbol y))
- (zoq y))
-\end{lisp}
-The main problem with this program is that it is trying to add \code{3} to a
-symbol. Note also that the functions \code{roq} and \code{ploq} aren't defined
-anywhere.
-
-\begin{comment}
-* The Parts of the Error Message::
-* The Original and Actual Source::
-* The Processing Path::
-* Error Severity::
-* Errors During Macroexpansion::
-* Read Errors::
-* Error Message Parameterization::
-\end{comment}
-
-%%\node The Parts of the Error Message, The Original and Actual Source, Interpreting Error Messages, Interpreting Error Messages
-\subsection{The Parts of the Error Message}
-
-The compiler will produce this warning:
-\begin{example}
-File: /usr/me/stuff.lisp
-
-In: DEFUN FOO
- (ZOQ Y)
---> ROQ PLOQ +
-==>
- Y
-Warning: Result is a SYMBOL, not a NUMBER.
-\end{example}
-In this example we see each of the six possible parts of a compiler error
-message:
-\begin{Lentry}
-
-\item[\w{\code{File: /usr/me/stuff.lisp}}] This is the \var{file} that
- the compiler read the relevant code from. The file name is
- displayed because it may not be immediately obvious when there is an
- error during compilation of a large system, especially when
- \code{with-compilation-unit} is used to delay undefined warnings.
-
-\item[\w{\code{In: DEFUN FOO}}] This is the \var{definition} or
- top-level form responsible for the error. It is obtained by taking
- the first two elements of the enclosing form whose first element is
- a symbol beginning with ``\code{DEF}''. If there is no enclosing
- \w{\var{def}mumble}, then the outermost form is used. If there are
- multiple \w{\var{def}mumbles}, then they are all printed from the
- out in, separated by \code{$=>$}'s. In this example, the problem
- was in the \code{defun} for \code{foo}.
-
-\item[\w{\code{(ZOQ Y)}}] This is the \i{original source} form
- responsible for the error. Original source means that the form
- directly appeared in the original input to the compiler, i.e. in the
- lambda passed to \code{compile} or the top-level form read from the
- source file. In this example, the expansion of the \code{zoq} macro
- was responsible for the error.
-
-\item[\w{\code{--$>$ ROQ PLOQ +}} ] This is the \i{processing path}
- that the compiler used to produce the errorful code. The processing
- path is a representation of the evaluated forms enclosing the actual
- source that the compiler encountered when processing the original
- source. The path is the first element of each form, or the form
- itself if the form is not a list. These forms result from the
- expansion of macros or source-to-source transformation done by the
- compiler. In this example, the enclosing evaluated forms are the
- calls to \code{roq}, \code{ploq} and \code{+}. These calls resulted
- from the expansion of the \code{zoq} macro.
-
-\item[\code{==$>$ Y}] This is the \i{actual source} responsible for
- the error. If the actual source appears in the explanation, then we
- print the next enclosing evaluated form, instead of printing the
- actual source twice. (This is the form that would otherwise have
- been the last form of the processing path.) In this example, the
- problem is with the evaluation of the reference to the variable
- \code{y}.
-
-\item[\w{\code{Warning: Result is a SYMBOL, not a NUMBER.}}] This is
- the \var{explanation} the problem. In this example, the problem is
- that \code{y} evaluates to a \code{symbol}, but is in a context
- where a number is required (the argument to \code{+}).
-\end{Lentry}
-
-Note that each part of the error message is distinctively marked:
-\begin{itemize}
-
-\item \code{File:} and \code{In:} mark the file and definition,
- respectively.
-
-\item The original source is an indented form with no prefix.
-
-\item Each line of the processing path is prefixed with \code{--$>$}.
-
-\item The actual source form is indented like the original source, but
- is marked by a preceding \code{==$>$} line. This is like the
- ``macroexpands to'' notation used in \cltl.
-
-\item The explanation is prefixed with the error severity
- (\pxlref{error-severity}), either \code{Error:}, \code{Warning:}, or
- \code{Note:}.
-\end{itemize}
-
-
-Each part of the error message is more specific than the preceding
-one. If consecutive error messages are for nearby locations, then the
-front part of the error messages would be the same. In this case, the
-compiler omits as much of the second message as in common with the
-first. For example:
-\begin{example}
-File: /usr/me/stuff.lisp
-
-In: DEFUN FOO
- (ZOQ Y)
---> ROQ
-==>
- (PLOQ (+ Y 3))
-Warning: Undefined function: PLOQ
-
-==>
- (ROQ (PLOQ (+ Y 3)))
-Warning: Undefined function: ROQ
-\end{example}
-In this example, the file, definition and original source are
-identical for the two messages, so the compiler omits them in the
-second message. If consecutive messages are entirely identical, then
-the compiler prints only the first message, followed by:
-\begin{example}
-[Last message occurs \var{repeats} times]
-\end{example}
-where \var{repeats} is the number of times the message was given.
-
-If the source was not from a file, then no file line is printed. If
-the actual source is the same as the original source, then the
-processing path and actual source will be omitted. If no forms
-intervene between the original source and the actual source, then the
-processing path will also be omitted.
-
-%%\f
-%%\node The Original and Actual Source, The Processing Path, The Parts of the Error Message, Interpreting Error Messages
-\subsection{The Original and Actual Source}
-\cindex{original source}
-\cindex{actual source}
-
-The \i{original source} displayed will almost always be a list. If the actual
-source for an error message is a symbol, the original source will be the
-immediately enclosing evaluated list form. So even if the offending symbol
-does appear in the original source, the compiler will print the enclosing list
-and then print the symbol as the actual source (as though the symbol were
-introduced by a macro.)
-
-When the \i{actual source} is displayed (and is not a symbol), it will always
-be code that resulted from the expansion of a macro or a source-to-source
-compiler optimization. This is code that did not appear in the original
-source program; it was introduced by the compiler.
-
-Keep in mind that when the compiler displays a source form in an error message,
-it always displays the most specific (innermost) responsible form. For
-example, compiling this function:
-\begin{lisp}
-(defun bar (x)
- (let (a)
- (declare (fixnum a))
- (setq a (foo x))
- a))
-\end{lisp}
-Gives this error message:
-\begin{example}
-In: DEFUN BAR
- (LET (A) (DECLARE (FIXNUM A)) (SETQ A (FOO X)) A)
-Warning: The binding of A is not a FIXNUM:
- NIL
-\end{example}
-This error message is not saying ``there's a problem somewhere in this
-\code{let}''\dash{}it is saying that there is a problem with the
-\code{let} itself. In this example, the problem is that \code{a}'s
-\false{} initial value is not a \code{fixnum}.
-
-%%\f
-%%\node The Processing Path, Error Severity, The Original and Actual Source, Interpreting Error Messages
-\subsection{The Processing Path}
-\cindex{processing path}
-\cindex{macroexpansion}
-\cindex{source-to-source transformation}
-
-The processing path is mainly useful for debugging macros, so if you don't
-write macros, you can ignore the processing path. Consider this example:
-\begin{lisp}
-(defun foo (n)
- (dotimes (i n *undefined*)))
-\end{lisp}
-Compiling results in this error message:
-\begin{example}
-In: DEFUN FOO
- (DOTIMES (I N *UNDEFINED*))
---> DO BLOCK LET TAGBODY RETURN-FROM
-==>
- (PROGN *UNDEFINED*)
-Warning: Undefined variable: *UNDEFINED*
-\end{example}
-Note that \code{do} appears in the processing path. This is because \code{dotimes}
-expands into:
-\begin{lisp}
-(do ((i 0 (1+ i)) (#:g1 n))
- ((>= i #:g1) *undefined*)
- (declare (type unsigned-byte i)))
-\end{lisp}
-The rest of the processing path results from the expansion of \code{do}:
-\begin{lisp}
-(block nil
- (let ((i 0) (#:g1 n))
- (declare (type unsigned-byte i))
- (tagbody (go #:g3)
- #:g2 (psetq i (1+ i))
- #:g3 (unless (>= i #:g1) (go #:g2))
- (return-from nil (progn *undefined*)))))
-\end{lisp}
-In this example, the compiler descended into the \code{block},
-\code{let}, \code{tagbody} and \code{return-from} to reach the
-\code{progn} printed as the actual source. This is a place where the
-``actual source appears in explanation'' rule was applied. The
-innermost actual source form was the symbol \code{*undefined*} itself,
-but that also appeared in the explanation, so the compiler backed out
-one level.
-
-%%\f
-%%\node Error Severity, Errors During Macroexpansion, The Processing Path, Interpreting Error Messages
-\subsection{Error Severity}
-\label{error-severity}
-\cindex{severity of compiler errors}
-\cindex{compiler error severity}
-
-There are three levels of compiler error severity:
-\begin{Lentry}
-
-\item[Error] This severity is used when the compiler encounters a
- problem serious enough to prevent normal processing of a form.
- Instead of compiling the form, the compiler compiles a call to
- \code{error}. Errors are used mainly for signaling syntax errors.
- If an error happens during macroexpansion, the compiler will handle
- it. The compiler also handles and attempts to proceed from read
- errors.
-
-\item[Warning] Warnings are used when the compiler can prove that
- something bad will happen if a portion of the program is executed,
- but the compiler can proceed by compiling code that signals an error
- at runtime if the problem has not been fixed:
- \begin{itemize}
-
- \item Violation of type declarations, or
-
- \item Function calls that have the wrong number of arguments or
- malformed keyword argument lists, or
-
- \item Referencing a variable declared \code{ignore}, or unrecognized
- declaration specifiers.
- \end{itemize}
-
- In the language of the \clisp{} standard, these are situations where
- the compiler can determine that a situation with undefined
- consequences or that would cause an error to be signaled would
- result at runtime.
-
-\item[Note] Notes are used when there is something that seems a bit
- odd, but that might reasonably appear in correct programs.
-\end{Lentry}
-Note that the compiler does not fully conform to the proposed X3J13
-``compiler-diagnostics'' cleanup. Errors, warnings and notes mostly
-correspond to errors, warnings and style-warnings, but many things
-that the cleanup considers to be style-warnings are printed as
-warnings rather than notes. Also, warnings, style-warnings and most
-errors aren't really signaled using the condition system.
-
-%%\f
-%%\node Errors During Macroexpansion, Read Errors, Error Severity, Interpreting Error Messages
-\subsection{Errors During Macroexpansion}
-\cpsubindex{macroexpansion}{errors during}
-
-The compiler handles errors that happen during macroexpansion, turning
-them into compiler errors. If you want to debug the error (to debug a
-macro), you can set \code{*break-on-signals*} to \code{error}. For
-example, this definition:
-\begin{lisp}
-(defun foo (e l)
- (do ((current l (cdr current))
- ((atom current) nil))
- (when (eq (car current) e) (return current))))
-\end{lisp}
-gives this error:
-\begin{example}
-In: DEFUN FOO
- (DO ((CURRENT L #) (# NIL)) (WHEN (EQ # E) (RETURN CURRENT)) )
-Error: (during macroexpansion)
-
-Error in function LISP::DO-DO-BODY.
-DO step variable is not a symbol: (ATOM CURRENT)
-\end{example}
-
-
-%%\f
-%%\node Read Errors, Error Message Parameterization, Errors During Macroexpansion, Interpreting Error Messages
-\subsection{Read Errors}
-\cpsubindex{read errors}{compiler}
-
-The compiler also handles errors while reading the source. For example:
-\begin{example}
-Error: Read error at 2:
- "(,/\back{foo})"
-Error in function LISP::COMMA-MACRO.
-Comma not inside a backquote.
-\end{example}
-The ``\code{at 2}'' refers to the character position in the source file at
-which the error was signaled, which is generally immediately after the
-erroneous text. The next line, ``\code{(,/\back{foo})}'', is the line in
-the source that contains the error file position. The ``\code{/\back{} }''
-indicates the error position within that line (in this example,
-immediately after the offending comma.)
-
-When in \hemlock{} (or any other EMACS-like editor), you can go to a
-character position with:
-\begin{example}
-M-< C-u \var{position} C-f
-\end{example}
-Note that if the source is from a \hemlock{} buffer, then the position
-is relative to the start of the compiled region or \code{defun}, not the
-file or buffer start.
-
-After printing a read error message, the compiler attempts to recover from the
-error by backing up to the start of the enclosing top-level form and reading
-again with \code{*read-suppress*} true. If the compiler can recover from the
-error, then it substitutes a call to \code{cerror} for the unreadable form and
-proceeds to compile the rest of the file normally.
-
-If there is a read error when the file position is at the end of the file
-(i.e., an unexpected EOF error), then the error message looks like this:
-\begin{example}
-Error: Read error in form starting at 14:
- "(defun test ()"
-Error in function LISP::FLUSH-WHITESPACE.
-EOF while reading #<Stream for file "/usr/me/test.lisp">
-\end{example}
-In this case, ``\code{starting at 14}'' indicates the character
-position at which the compiler started reading, i.e. the position
-before the start of the form that was missing the closing delimiter.
-The line \w{"\code{(defun test ()}"} is first line after the starting
-position that the compiler thinks might contain the unmatched open
-delimiter.
-
-%%\f
-%%\node Error Message Parameterization, , Read Errors, Interpreting Error Messages
-\subsection{Error Message Parameterization}
-\cpsubindex{error messages}{verbosity}
-\cpsubindex{verbosity}{of error messages}
-
-There is some control over the verbosity of error messages. See also
-\varref{undefined-warning-limit}, \code{*efficiency-note-limit*} and
-\varref{efficiency-note-cost-threshold}.
-
-\begin{defvar}{}{enclosing-source-cutoff}
-
- This variable specifies the number of enclosing actual source forms
- that are printed in full, rather than in the abbreviated processing
- path format. Increasing the value from its default of \code{1}
- allows you to see more of the guts of the macroexpanded source,
- which is useful when debugging macros.
-\end{defvar}
-
-\begin{defvar}{}{error-print-length}
- \defvarx{error-print-level}
-
- These variables are the print level and print length used in
- printing error messages. The default values are \code{5} and
- \code{3}. If null, the global values of \code{*print-level*} and
- \code{*print-length*} are used.
-\end{defvar}
-
-\begin{defmac}{extensions:}{def-source-context}{%
- \args{\var{name} \var{lambda-list} \mstar{form}}}
-
- This macro defines how to extract an abbreviated source context from
- the \var{name}d form when it appears in the compiler input.
- \var{lambda-list} is a \code{defmacro} style lambda-list used to
- parse the arguments. The \var{body} should return a list of
- subforms that can be printed on about one line. There are
- predefined methods for \code{defstruct}, \code{defmethod}, etc. If
- no method is defined, then the first two subforms are returned.
- Note that this facility implicitly determines the string name
- associated with anonymous functions.
-\end{defmac}
-
-%%\f
-%%\node Types in Python, Getting Existing Programs to Run, Interpreting Error Messages, The Compiler
-\section{Types in Python}
-\cpsubindex{types}{in python}
-
-A big difference between \Python{} and all other \llisp{} compilers
-is the approach to type checking and amount of knowledge about types:
-\begin{itemize}
-
-\item \Python{} treats type declarations much differently that other
- Lisp compilers do. \Python{} doesn't blindly believe type
- declarations; it considers them assertions about the program that
- should be checked.
-
-\item \Python{} also has a tremendously greater knowledge of the
- \clisp{} type system than other compilers. Support is incomplete
- only for the \code{not}, \code{and} and \code{satisfies} types.
-\end{itemize}
-See also sections \ref{advanced-type-stuff} and \ref{type-inference}.
-
-%%\f
-\begin{comment}
-* Compile Time Type Errors::
-* Precise Type Checking::
-* Weakened Type Checking::
-\end{comment}
-
-%%\node Compile Time Type Errors, Precise Type Checking, Types in Python, Types in Python
-\subsection{Compile Time Type Errors}
-\cindex{compile time type errors}
-\cpsubindex{type checking}{at compile time}
-
-If the compiler can prove at compile time that some portion of the
-program cannot be executed without a type error, then it will give a
-warning at compile time. It is possible that the offending code would
-never actually be executed at run-time due to some higher level
-consistency constraint unknown to the compiler, so a type warning
-doesn't always indicate an incorrect program. For example, consider
-this code fragment:
-\begin{lisp}
-(defun raz (foo)
- (let ((x (case foo
- (:this 13)
- (:that 9)
- (:the-other 42))))
- (declare (fixnum x))
- (foo x)))
-\end{lisp}
-Compilation produces this warning:
-\begin{example}
-In: DEFUN RAZ
- (CASE FOO (:THIS 13) (:THAT 9) (:THE-OTHER 42))
---> LET COND IF COND IF COND IF
-==>
- (COND)
-Warning: This is not a FIXNUM:
- NIL
-\end{example}
-In this case, the warning is telling you that if \code{foo} isn't any
-of \kwd{this}, \kwd{that} or \kwd{the-other}, then \code{x} will be
-initialized to \false, which the \code{fixnum} declaration makes
-illegal. The warning will go away if \code{ecase} is used instead of
-\code{case}, or if \kwd{the-other} is changed to \true.
-
-This sort of spurious type warning happens moderately often in the
-expansion of complex macros and in inline functions. In such cases,
-there may be dead code that is impossible to correctly execute. The
-compiler can't always prove this code is dead (could never be
-executed), so it compiles the erroneous code (which will always signal
-an error if it is executed) and gives a warning.
-
-\begin{defun}{extensions:}{required-argument}{}
-
- This function can be used as the default value for keyword arguments
- that must always be supplied. Since it is known by the compiler to
- never return, it will avoid any compile-time type warnings that
- would result from a default value inconsistent with the declared
- type. When this function is called, it signals an error indicating
- that a required keyword argument was not supplied. This function is
- also useful for \code{defstruct} slot defaults corresponding to
- required arguments. \xlref{empty-type}.
-
- Although this function is a CMU extension, it is relatively harmless
- to use it in otherwise portable code, since you can easily define it
- yourself:
- \begin{lisp}
- (defun required-argument ()
- (error "A required keyword argument was not supplied."))
- \end{lisp}
-\end{defun}
-
-Type warnings are inhibited when the
-\code{extensions:inhibit-warnings} optimization quality is \code{3}
-(\pxlref{compiler-policy}.) This can be used in a local declaration
-to inhibit type warnings in a code fragment that has spurious
-warnings.
-
-%%\f
-%%\node Precise Type Checking, Weakened Type Checking, Compile Time Type Errors, Types in Python
-\subsection{Precise Type Checking}
-\label{precise-type-checks}
-\cindex{precise type checking}
-\cpsubindex{type checking}{precise}
-
-With the default compilation policy, all type
-assertions\footnote{There are a few circumstances where a type
- declaration is discarded rather than being used as type assertion.
- This doesn't affect safety much, since such discarded declarations
- are also not believed to be true by the compiler.} are precisely
-checked. Precise checking means that the check is done as though
-\code{typep} had been called with the exact type specifier that
-appeared in the declaration. \Python{} uses \var{policy} to determine
-whether to trust type assertions (\pxlref{compiler-policy}). Type
-assertions from declarations are indistinguishable from the type
-assertions on arguments to built-in functions. In \Python, adding
-type declarations makes code safer.
-
-If a variable is declared to be \w{\code{(integer 3 17)}}, then its
-value must always always be an integer between \code{3} and \code{17}.
-If multiple type declarations apply to a single variable, then all the
-declarations must be correct; it is as though all the types were
-intersected producing a single \code{and} type specifier.
-
-Argument type declarations are automatically enforced. If you declare
-the type of a function argument, a type check will be done when that
-function is called. In a function call, the called function does the
-argument type checking, which means that a more restrictive type
-assertion in the calling function (e.g., from \code{the}) may be lost.
-
-The types of structure slots are also checked. The value of a
-structure slot must always be of the type indicated in any \kwd{type}
-slot option.\footnote{The initial value need not be of this type as
- long as the corresponding argument to the constructor is always
- supplied, but this will cause a compile-time type warning unless
- \code{required-argument} is used.} Because of precise type checking,
-the arguments to slot accessors are checked to be the correct type of
-structure.
-
-In traditional \llisp{} compilers, not all type assertions are
-checked, and type checks are not precise. Traditional compilers
-blindly trust explicit type declarations, but may check the argument
-type assertions for built-in functions. Type checking is not precise,
-since the argument type checks will be for the most general type legal
-for that argument. In many systems, type declarations suppress what
-little type checking is being done, so adding type declarations makes
-code unsafe. This is a problem since it discourages writing type
-declarations during initial coding. In addition to being more error
-prone, adding type declarations during tuning also loses all the
-benefits of debugging with checked type assertions.
-
-To gain maximum benefit from \Python{}'s type checking, you should
-always declare the types of function arguments and structure slots as
-precisely as possible. This often involves the use of \code{or},
-\code{member} and other list-style type specifiers. Paradoxically,
-even though adding type declarations introduces type checks, it
-usually reduces the overall amount of type checking. This is
-especially true for structure slot type declarations.
-
-\Python{} uses the \code{safety} optimization quality (rather than
-presence or absence of declarations) to choose one of three levels of
-run-time type error checking: \pxlref{optimize-declaration}.
-\xlref{advanced-type-stuff} for more information about types in
-\Python.
-
-%%\f
-%%\node Weakened Type Checking, , Precise Type Checking, Types in Python
-\subsection{Weakened Type Checking}
-\label{weakened-type-checks}
-\cindex{weakened type checking}
-\cpsubindex{type checking}{weakened}
-
-When the value for the \code{speed} optimization quality is greater
-than \code{safety}, and \code{safety} is not \code{0}, then type
-checking is weakened to reduce the speed and space penalty. In
-structure-intensive code this can double the speed, yet still catch
-most type errors. Weakened type checks provide a level of safety
-similar to that of ``safe'' code in other \llisp{} compilers.
-
-A type check is weakened by changing the check to be for some
-convenient supertype of the asserted type. For example,
-\code{\w{(integer 3 17)}} is changed to \code{fixnum},
-\code{\w{(simple-vector 17)}} to \code{simple-vector}, and structure
-types are changed to \code{structure}. A complex check like:
-\begin{example}
-(or node hunk (member :foo :bar :baz))
-\end{example}
-will be omitted entirely (i.e., the check is weakened to \code{*}.) If
-a precise check can be done for no extra cost, then no weakening is
-done.
-
-Although weakened type checking is similar to type checking done by
-other compilers, it is sometimes safer and sometimes less safe.
-Weakened checks are done in the same places is precise checks, so all
-the preceding discussion about where checking is done still applies.
-Weakened checking is sometimes somewhat unsafe because although the
-check is weakened, the precise type is still input into type
-inference. In some contexts this will result in type inferences not
-justified by the weakened check, and hence deletion of some type
-checks that would be done by conventional compilers.
-
-For example, if this code was compiled with weakened checks:
-\begin{lisp}
-(defstruct foo
- (a nil :type simple-string))
-
-(defstruct bar
- (a nil :type single-float))
-
-(defun myfun (x)
- (declare (type bar x))
- (* (bar-a x) 3.0))
-\end{lisp}
-and \code{myfun} was passed a \code{foo}, then no type error would be
-signaled, and we would try to multiply a \code{simple-vector} as
-though it were a float (with unpredictable results.) This is because
-the check for \code{bar} was weakened to \code{structure}, yet when
-compiling the call to \code{bar-a}, the compiler thinks it knows it
-has a \code{bar}.
-
-Note that normally even weakened type checks report the precise type
-in error messages. For example, if \code{myfun}'s \code{bar} check is
-weakened to \code{structure}, and the argument is \false{}, then the
-error will be:
-\begin{example}
-Type-error in MYFUN:
- NIL is not of type BAR
-\end{example}
-However, there is some speed and space cost for signaling a precise
-error, so the weakened type is reported if the \code{speed}
-optimization quality is \code{3} or \code{debug} quality is less than
-\code{1}:
-\begin{example}
-Type-error in MYFUN:
- NIL is not of type STRUCTURE
-\end{example}
-\xlref{optimize-declaration} for further discussion of the
-\code{optimize} declaration.
-
-%%\f
-%%\node Getting Existing Programs to Run, Compiler Policy, Types in Python, The Compiler
-\section{Getting Existing Programs to Run}
-\cpsubindex{existing programs}{to run}
-\cpsubindex{types}{portability}
-\cindex{compatibility with other Lisps}
-
-Since \Python{} does much more comprehensive type checking than other
-Lisp compilers, \Python{} will detect type errors in many programs
-that have been debugged using other compilers. These errors are
-mostly incorrect declarations, although compile-time type errors can
-find actual bugs if parts of the program have never been tested.
-
-Some incorrect declarations can only be detected by run-time type
-checking. It is very important to initially compile programs with
-full type checks and then test this version. After the checking
-version has been tested, then you can consider weakening or
-eliminating type checks. \b{This applies even to previously debugged
- programs.} \Python{} does much more type inference than other
-\llisp{} compilers, so believing an incorrect declaration does much
-more damage.
-
-The most common problem is with variables whose initial value doesn't
-match the type declaration. Incorrect initial values will always be
-flagged by a compile-time type error, and they are simple to fix once
-located. Consider this code fragment:
-\begin{example}
-(prog (foo)
- (declare (fixnum foo))
- (setq foo ...)
- ...)
-\end{example}
-Here the variable \code{foo} is given an initial value of \false, but
-is declared to be a \code{fixnum}. Even if it is never read, the
-initial value of a variable must match the declared type. There are
-two ways to fix this problem. Change the declaration:
-\begin{example}
-(prog (foo)
- (declare (type (or fixnum null) foo))
- (setq foo ...)
- ...)
-\end{example}
-or change the initial value:
-\begin{example}
-(prog ((foo 0))
- (declare (fixnum foo))
- (setq foo ...)
- ...)
-\end{example}
-It is generally preferable to change to a legal initial value rather
-than to weaken the declaration, but sometimes it is simpler to weaken
-the declaration than to try to make an initial value of the
-appropriate type.
-
-
-Another declaration problem occasionally encountered is incorrect
-declarations on \code{defmacro} arguments. This probably usually
-happens when a function is converted into a macro. Consider this
-macro:
-\begin{lisp}
-(defmacro my-1+ (x)
- (declare (fixnum x))
- `(the fixnum (1+ ,x)))
-\end{lisp}
-Although legal and well-defined \clisp, this meaning of this
-definition is almost certainly not what the writer intended. For
-example, this call is illegal:
-\begin{lisp}
-(my-1+ (+ 4 5))
-\end{lisp}
-The call is illegal because the argument to the macro is \w{\code{(+ 4
- 5)}}, which is a \code{list}, not a \code{fixnum}. Because of
-macro semantics, it is hardly ever useful to declare the types of
-macro arguments. If you really want to assert something about the
-type of the result of evaluating a macro argument, then put a
-\code{the} in the expansion:
-\begin{lisp}
-(defmacro my-1+ (x)
- `(the fixnum (1+ (the fixnum ,x))))
-\end{lisp}
-In this case, it would be stylistically preferable to change this
-macro back to a function and declare it inline. Macros have no
-efficiency advantage over inline functions when using \Python.
-\xlref{inline-expansion}.
-
-
-Some more subtle problems are caused by incorrect declarations that
-can't be detected at compile time. Consider this code:
-\begin{example}
-(do ((pos 0 (position #\back{a} string :start (1+ pos))))
- ((null pos))
- (declare (fixnum pos))
- ...)
-\end{example}
-Although \code{pos} is almost always a \code{fixnum}, it is \false{}
-at the end of the loop. If this example is compiled with full type
-checks (the default), then running it will signal a type error at the
-end of the loop. If compiled without type checks, the program will go
-into an infinite loop (or perhaps \code{position} will complain
-because \w{\code{(1+ nil)}} isn't a sensible start.) Why? Because if
-you compile without type checks, the compiler just quietly believes
-the type declaration. Since \code{pos} is always a \code{fixnum}, it
-is never \nil, so \w{\code{(null pos)}} is never true, and the loop
-exit test is optimized away. Such errors are sometimes flagged by
-unreachable code notes (\pxlref{dead-code-notes}), but it is still
-important to initially compile any system with full type checks, even
-if the system works fine when compiled using other compilers.
-
-In this case, the fix is to weaken the type declaration to
-\w{\code{(or fixnum null)}}.\footnote{Actually, this declaration is
- totally unnecessary in \Python, since it already knows
- \code{position} returns a non-negative \code{fixnum} or \false.}
-Note that there is usually little performance penalty for weakening a
-declaration in this way. Any numeric operations in the body can still
-assume the variable is a \code{fixnum}, since \false{} is not a legal
-numeric argument. Another possible fix would be to say:
-\begin{example}
-(do ((pos 0 (position #\back{a} string :start (1+ pos))))
- ((null pos))
- (let ((pos pos))
- (declare (fixnum pos))
- ...))
-\end{example}
-This would be preferable in some circumstances, since it would allow a
-non-standard representation to be used for the local \code{pos}
-variable in the loop body (see section \ref{ND-variables}.)
-
-In summary, remember that \i{all} values that a variable \i{ever}
-has must be of the declared type, and that you should test using safe
-code initially.
-%%\f
-%%\node Compiler Policy, Open Coding and Inline Expansion, Getting Existing Programs to Run, The Compiler
-\section{Compiler Policy}
-\label{compiler-policy}
-\cpsubindex{policy}{compiler}
-\cindex{compiler policy}
-
-The policy is what tells the compiler \var{how} to compile a program.
-This is logically (and often textually) distinct from the program
-itself. Broad control of policy is provided by the \code{optimize}
-declaration; other declarations and variables control more specific
-aspects of compilation.
-
-%%\f
-\begin{comment}
-* The Optimize Declaration::
-* The Optimize-Interface Declaration::
-\end{comment}
-
-%%\node The Optimize Declaration, The Optimize-Interface Declaration, Compiler Policy, Compiler Policy
-\subsection{The Optimize Declaration}
-\label{optimize-declaration}
-\cindex{optimize declaration}
-\cpsubindex{declarations}{\code{optimize}}
-
-The \code{optimize} declaration recognizes six different
-\var{qualities}. The qualities are conceptually independent aspects
-of program performance. In reality, increasing one quality tends to
-have adverse effects on other qualities. The compiler compares the
-relative values of qualities when it needs to make a trade-off; i.e.,
-if \code{speed} is greater than \code{safety}, then improve speed at
-the cost of safety.
-
-The default for all qualities (except \code{debug}) is \code{1}.
-Whenever qualities are equal, ties are broken according to a broad
-idea of what a good default environment is supposed to be. Generally
-this downplays \code{speed}, \code{compile-speed} and \code{space} in
-favor of \code{safety} and \code{debug}. Novice and casual users
-should stick to the default policy. Advanced users often want to
-improve speed and memory usage at the cost of safety and
-debuggability.
-
-If the value for a quality is \code{0} or \code{3}, then it may have a
-special interpretation. A value of \code{0} means ``totally
-unimportant'', and a \code{3} means ``ultimately important.'' These
-extreme optimization values enable ``heroic'' compilation strategies
-that are not always desirable and sometimes self-defeating.
-Specifying more than one quality as \code{3} is not desirable, since
-it doesn't tell the compiler which quality is most important.
-
-
-These are the optimization qualities:
-\begin{Lentry}
-
-\item[\code{speed}] \cindex{speed optimization quality}How fast the
- program should is run. \code{speed 3} enables some optimizations
- that hurt debuggability.
-
-\item[\code{compilation-speed}] \cindex{compilation-speed optimization
- quality}How fast the compiler should run. Note that increasing
- this above \code{safety} weakens type checking.
-
-\item[\code{space}] \cindex{space optimization quality}How much space
- the compiled code should take up. Inline expansion is mostly
- inhibited when \code{space} is greater than \code{speed}. A value
- of \code{0} enables promiscuous inline expansion. Wide use of a
- \code{0} value is not recommended, as it may waste so much space
- that run time is slowed. \xlref{inline-expansion} for a discussion
- of inline expansion.
-
-\item[\code{debug}] \cindex{debug optimization quality}How debuggable
- the program should be. The quality is treated differently from the
- other qualities: each value indicates a particular level of debugger
- information; it is not compared with the other qualities.
- \xlref{debugger-policy} for more details.
-
-\item[\code{safety}] \cindex{safety optimization quality}How much
- error checking should be done. If \code{speed}, \code{space} or
- \code{compilation-speed} is more important than \code{safety}, then
- type checking is weakened (\pxlref{weakened-type-checks}). If
- \code{safety} if \code{0}, then no run time error checking is done.
- In addition to suppressing type checks, \code{0} also suppresses
- argument count checking, unbound-symbol checking and array bounds
- checks.
-
-\item[\code{extensions:inhibit-warnings}] \cindex{inhibit-warnings
- optimization quality}This is a CMU extension that determines how
- little (or how much) diagnostic output should be printed during
- compilation. This quality is compared to other qualities to
- determine whether to print style notes and warnings concerning those
- qualities. If \code{speed} is greater than \code{inhibit-warnings},
- then notes about how to improve speed will be printed, etc. The
- default value is \code{1}, so raising the value for any standard
- quality above its default enables notes for that quality. If
- \code{inhibit-warnings} is \code{3}, then all notes and most
- non-serious warnings are inhibited. This is useful with
- \code{declare} to suppress warnings about unavoidable problems.
-\end{Lentry}
-
-%%\node The Optimize-Interface Declaration, , The Optimize Declaration, Compiler Policy
-\subsection{The Optimize-Interface Declaration}
-\label{optimize-interface-declaration}
-\cindex{optimize-interface declaration}
-\cpsubindex{declarations}{\code{optimize-interface}}
-
-The \code{extensions:optimize-interface} declaration is identical in
-syntax to the \code{optimize} declaration, but it specifies the policy
-used during compilation of code the compiler automatically generates
-to check the number and type of arguments supplied to a function. It
-is useful to specify this policy separately, since even thoroughly
-debugged functions are vulnerable to being passed the wrong arguments.
-The \code{optimize-interface} declaration can specify that arguments
-should be checked even when the general \code{optimize} policy is
-unsafe.
-
-Note that this argument checking is the checking of user-supplied
-arguments to any functions defined within the scope of the
-declaration, \code{not} the checking of arguments to \llisp{}
-primitives that appear in those definitions.
-
-The idea behind this declaration is that it allows the definition of
-functions that appear fully safe to other callers, but that do no
-internal error checking. Of course, it is possible that arguments may
-be invalid in ways other than having incorrect type. Functions
-compiled unsafely must still protect themselves against things like
-user-supplied array indices that are out of bounds and improper lists.
-See also the \kwd{context-declarations} option to
-\macref{with-compilation-unit}.
-
-%%\f
-%%\node Open Coding and Inline Expansion, , Compiler Policy, The Compiler
-\section{Open Coding and Inline Expansion}
-\label{open-coding}
-\cindex{open-coding}
-\cindex{inline expansion}
-\cindex{static functions}
-
-Since \clisp{} forbids the redefinition of standard functions\footnote{See the
-proposed X3J13 ``lisp-symbol-redefinition'' cleanup.}, the compiler can have
-special knowledge of these standard functions embedded in it. This special
-knowledge is used in various ways (open coding, inline expansion, source
-transformation), but the implications to the user are basically the same:
-\begin{itemize}
-
-\item Attempts to redefine standard functions may be frustrated, since
- the function may never be called. Although it is technically
- illegal to redefine standard functions, users sometimes want to
- implicitly redefine these functions when they are debugging using
- the \code{trace} macro. Special-casing of standard functions can be
- inhibited using the \code{notinline} declaration.
-
-\item The compiler can have multiple alternate implementations of
- standard functions that implement different trade-offs of speed,
- space and safety. This selection is based on the compiler policy,
- \pxlref{compiler-policy}.
-\end{itemize}
-
-
-When a function call is \i{open coded}, inline code whose effect is
-equivalent to the function call is substituted for that function call.
-When a function call is \i{closed coded}, it is usually left as is,
-although it might be turned into a call to a different function with
-different arguments. As an example, if \code{nthcdr} were to be open
-coded, then
-\begin{lisp}
-(nthcdr 4 foobar)
-\end{lisp}
-might turn into
-\begin{lisp}
-(cdr (cdr (cdr (cdr foobar))))
-\end{lisp}
-or even
-\begin{lisp}
-(do ((i 0 (1+ i))
- (list foobar (cdr foobar)))
- ((= i 4) list))
-\end{lisp}
-
-If \code{nth} is closed coded, then
-\begin{lisp}
-(nth x l)
-\end{lisp}
-might stay the same, or turn into something like:
-\begin{lisp}
-(car (nthcdr x l))
-\end{lisp}
-
-In general, open coding sacrifices space for speed, but some functions (such as
-\code{car}) are so simple that they are always open-coded. Even when not
-open-coded, a call to a standard function may be transformed into a different
-function call (as in the last example) or compiled as \i{static call}. Static
-function call uses a more efficient calling convention that forbids
-redefinition.
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/efficiency.ms}
-
-
-
-\hide{ -*- Dictionary: cmu-user -*- }
-%%\node Advanced Compiler Use and Efficiency Hints, UNIX Interface, The Compiler, Top
-\chapter{Advanced Compiler Use and Efficiency Hints}
-\begin{center}
-\b{By Robert MacLachlan}
-\end{center}
-\vspace{1 cm}
-\label{advanced-compiler}
-
-\begin{comment}
-* Advanced Compiler Introduction::
-* More About Types in Python::
-* Type Inference::
-* Source Optimization::
-* Tail Recursion::
-* Local Call::
-* Block Compilation::
-* Inline Expansion::
-* Byte Coded Compilation::
-* Object Representation::
-* Numbers::
-* General Efficiency Hints::
-* Efficiency Notes::
-* Profiling::
-\end{comment}
-
-%%\node Advanced Compiler Introduction, More About Types in Python, Advanced Compiler Use and Efficiency Hints, Advanced Compiler Use and Efficiency Hints
-\section{Advanced Compiler Introduction}
-
-In \cmucl, as is any language on any computer, the path to efficient
-code starts with good algorithms and sensible programming techniques,
-but to avoid inefficiency pitfalls, you need to know some of this
-implementation's quirks and features. This chapter is mostly a fairly
-long and detailed overview of what optimizations \python{} does.
-Although there are the usual negative suggestions of inefficient
-features to avoid, the main emphasis is on describing the things that
-programmers can count on being efficient.
-
-The optimizations described here can have the effect of speeding up
-existing programs written in conventional styles, but the potential
-for new programming styles that are clearer and less error-prone is at
-least as significant. For this reason, several sections end with a
-discussion of the implications of these optimizations for programming
-style.
-
-\begin{comment}
-* Types::
-* Optimization::
-* Function Call::
-* Representation of Objects::
-* Writing Efficient Code::
-\end{comment}
-
-%%\node Types, Optimization, Advanced Compiler Introduction, Advanced Compiler Introduction
-\subsection{Types}
-
-Python's support for types is unusual in three major ways:
-\begin{itemize}
-
-\item Precise type checking encourages the specific use of type
- declarations as a form of run-time consistency checking. This
- speeds development by localizing type errors and giving more
- meaningful error messages. \xlref{precise-type-checks}. \python{}
- produces completely safe code; optimized type checking maintains
- reasonable efficiency on conventional hardware
- (\pxlref{type-check-optimization}.)
-
-\item Comprehensive support for the \clisp{} type system makes complex
- type specifiers useful. Using type specifiers such as \code{or} and
- \code{member} has both efficiency and robustness advantages.
- \xlref{advanced-type-stuff}.
-
-\item Type inference eliminates the need for some declarations, and
- also aids compile-time detection of type errors. Given detailed
- type declarations, type inference can often eliminate type checks
- and enable more efficient object representations and code sequences.
- Checking all types results in fewer type checks. See sections
- \ref{type-inference} and \ref{non-descriptor}.
-\end{itemize}
-
-
-%%\node Optimization, Function Call, Types, Advanced Compiler Introduction
-\subsection{Optimization}
-
-The main barrier to efficient Lisp programs is not that there is no
-efficient way to code the program in Lisp, but that it is difficult to
-arrive at that efficient coding. Common Lisp is a highly complex
-language, and usually has many semantically equivalent ``reasonable''
-ways to code a given problem. It is desirable to make all of these
-equivalent solutions have comparable efficiency so that programmers
-don't have to waste time discovering the most efficient solution.
-
-Source level optimization increases the number of efficient ways to
-solve a problem. This effect is much larger than the increase in the
-efficiency of the ``best'' solution. Source level optimization
-transforms the original program into a more efficient (but equivalent)
-program. Although the optimizer isn't doing anything the programmer
-couldn't have done, this high-level optimization is important because:
-\begin{itemize}
-
-\item The programmer can code simply and directly, rather than
- obfuscating code to please the compiler.
-
-\item When presented with a choice of similar coding alternatives, the
- programmer can chose whichever happens to be most convenient,
- instead of worrying about which is most efficient.
-\end{itemize}
-
-Source level optimization eliminates the need for macros to optimize
-their expansion, and also increases the effectiveness of inline
-expansion. See sections \ref{source-optimization} and
-\ref{inline-expansion}.
-
-Efficient support for a safer programming style is the biggest
-advantage of source level optimization. Existing tuned programs
-typically won't benefit much from source optimization, since their
-source has already been optimized by hand. However, even tuned
-programs tend to run faster under \python{} because:
-\begin{itemize}
-
-\item Low level optimization and register allocation provides modest
- speedups in any program.
-
-\item Block compilation and inline expansion can reduce function call
- overhead, but may require some program restructuring. See sections
- \ref{inline-expansion}, \ref{local-call} and
- \ref{block-compilation}.
-
-\item Efficiency notes will point out important type declarations that
- are often missed even in highly tuned programs.
- \xlref{efficiency-notes}.
-
-\item Existing programs can be compiled safely without prohibitive
- speed penalty, although they would be faster and safer with added
- declarations. \xlref{type-check-optimization}.
-
-\item The context declaration mechanism allows both space and runtime
- of large systems to be reduced without sacrificing robustness by
- semi-automatically varying compilation policy without addition any
- \code{optimize} declarations to the source.
- \xlref{context-declarations}.
-
-\item Byte compilation can be used to dramatically reduce the size of
- code that is not speed-critical. \xlref{byte-compile}
-\end{itemize}
-
-
-%%\node Function Call, Representation of Objects, Optimization, Advanced Compiler Introduction
-\subsection{Function Call}
-
-The sort of symbolic programs generally written in \llisp{} often
-favor recursion over iteration, or have inner loops so complex that
-they involve multiple function calls. Such programs spend a larger
-fraction of their time doing function calls than is the norm in other
-languages; for this reason \llisp{} implementations strive to make the
-general (or full) function call as inexpensive as possible. \python{}
-goes beyond this by providing two good alternatives to full call:
-\begin{itemize}
-
-\item Local call resolves function references at compile time,
- allowing better calling sequences and optimization across function
- calls. \xlref{local-call}.
-
-\item Inline expansion totally eliminates call overhead and allows
- many context dependent optimizations. This provides a safe and
- efficient implementation of operations with function semantics,
- eliminating the need for error-prone macro definitions or manual
- case analysis. Although most \clisp{} implementations support
- inline expansion, it becomes a more powerful tool with \python{}'s
- source level optimization. See sections \ref{source-optimization}
- and \ref{inline-expansion}.
-\end{itemize}
-
-
-Generally, \python{} provides simple implementations for simple uses
-of function call, rather than having only a single calling convention.
-These features allow a more natural programming style:
-\begin{itemize}
-
-\item Proper tail recursion. \xlref{tail-recursion}
-
-\item Relatively efficient closures.
-
-\item A \code{funcall} that is as efficient as normal named call.
-
-\item Calls to local functions such as from \code{labels} are
- optimized:
-\begin{itemize}
-
-\item Control transfer is a direct jump.
-
-\item The closure environment is passed in registers rather than heap
- allocated.
-
-\item Keyword arguments and multiple values are implemented more
- efficiently.
-\end{itemize}
-
-\xlref{local-call}.
-\end{itemize}
-
-%%\node Representation of Objects, Writing Efficient Code, Function Call, Advanced Compiler Introduction
-\subsection{Representation of Objects}
-
-Sometimes traditional \llisp{} implementation techniques compare so
-poorly to the techniques used in other languages that \llisp{} can
-become an impractical language choice. Terrible inefficiencies appear
-in number-crunching programs, since \llisp{} numeric operations often
-involve number-consing and generic arithmetic. \python{} supports
-efficient natural representations for numbers (and some other types),
-and allows these efficient representations to be used in more
-contexts. \python{} also provides good efficiency notes that warn
-when a crucial declaration is missing.
-
-See section \ref{non-descriptor} for more about object representations and
-numeric types. Also \pxlref{efficiency-notes} about efficiency notes.
-
-%%\node Writing Efficient Code, , Representation of Objects, Advanced Compiler Introduction
-\subsection{Writing Efficient Code}
-\label{efficiency-overview}
-
-Writing efficient code that works is a complex and prolonged process.
-It is important not to get so involved in the pursuit of efficiency
-that you lose sight of what the original problem demands. Remember
-that:
-\begin{itemize}
-
-\item The program should be correct\dash{}it doesn't matter how
- quickly you get the wrong answer.
-
-\item Both the programmer and the user will make errors, so the
- program must be robust\dash{}it must detect errors in a way that
- allows easy correction.
-
-\item A small portion of the program will consume most of the
- resources, with the bulk of the code being virtually irrelevant to
- efficiency considerations. Even experienced programmers familiar
- with the problem area cannot reliably predict where these ``hot
- spots'' will be.
-\end{itemize}
-
-
-
-The best way to get efficient code that is still worth using, is to separate
-coding from tuning. During coding, you should:
-\begin{itemize}
-
-\item Use a coding style that aids correctness and robustness without
- being incompatible with efficiency.
-
-\item Choose appropriate data structures that allow efficient
- algorithms and object representations
- (\pxlref{object-representation}). Try to make interfaces abstract
- enough so that you can change to a different representation if
- profiling reveals a need.
-
-\item Whenever you make an assumption about a function argument or
- global data structure, add consistency assertions, either with type
- declarations or explicit uses of \code{assert}, \code{ecase}, etc.
-\end{itemize}
-
-During tuning, you should:
-\begin{itemize}
-
-\item Identify the hot spots in the program through profiling (section
- \ref{profiling}.)
-
-\item Identify inefficient constructs in the hot spot with efficiency
- notes, more profiling, or manual inspection of the source. See
- sections \ref{general-efficiency} and \ref{efficiency-notes}.
-
-\item Add declarations and consider the application of optimizations.
- See sections \ref{local-call}, \ref{inline-expansion} and
- \ref{non-descriptor}.
-
-\item If all else fails, consider algorithm or data structure changes.
- If you did a good job coding, changes will be easy to introduce.
-\end{itemize}
-
-
-
-%%\f
-%%\node More About Types in Python, Type Inference, Advanced Compiler Introduction, Advanced Compiler Use and Efficiency Hints
-\section{More About Types in Python}
-\label{advanced-type-stuff}
-\cpsubindex{types}{in python}
-
-This section goes into more detail describing what types and declarations are
-recognized by \python. The area where \python{} differs most radically from
-previous \llisp{} compilers is in its support for types:
-\begin{itemize}
-
-\item Precise type checking helps to find bugs at run time.
-
-\item Compile-time type checking helps to find bugs at compile time.
-
-\item Type inference minimizes the need for generic operations, and
- also increases the efficiency of run time type checking and the
- effectiveness of compile time type checking.
-
-\item Support for detailed types provides a wealth of opportunity for
- operation-specific type inference and optimization.
-\end{itemize}
-
-
-
-\begin{comment}
-* More Types Meaningful::
-* Canonicalization::
-* Member Types::
-* Union Types::
-* The Empty Type::
-* Function Types::
-* The Values Declaration::
-* Structure Types::
-* The Freeze-Type Declaration::
-* Type Restrictions::
-* Type Style Recommendations::
-\end{comment}
-
-%%\node More Types Meaningful, Canonicalization, More About Types in Python, More About Types in Python
-\subsection{More Types Meaningful}
-
-\clisp{} has a very powerful type system, but conventional \llisp{}
-implementations typically only recognize the small set of types
-special in that implementation. In these systems, there is an
-unfortunate paradox: a declaration for a relatively general type like
-\code{fixnum} will be recognized by the compiler, but a highly
-specific declaration such as \code{\w{(integer 3 17)}} is totally
-ignored.
-
-This is obviously a problem, since the user has to know how to specify
-the type of an object in the way the compiler wants it. A very
-minimal (but rarely satisfied) criterion for type system support is
-that it be no worse to make a specific declaration than to make a
-general one. \python{} goes beyond this by exploiting a number of
-advantages obtained from detailed type information.
-
-Using more restrictive types in declarations allows the compiler to do
-better type inference and more compile-time type checking. Also, when
-type declarations are considered to be consistency assertions that
-should be verified (conditional on policy), then complex types are
-useful for making more detailed assertions.
-
-Python ``understands'' the list-style \code{or}, \code{member},
-\code{function}, array and number type specifiers. Understanding
-means that:
-\begin{itemize}
-
-\item If the type contains more information than is used in a
- particular context, then the extra information is simply ignored,
- rather than derailing type inference.
-
-\item In many contexts, the extra information from these type
- specifier is used to good effect. In particular, type checking in
- \code{Python} is \var{precise}, so these complex types can be used
- in declarations to make interesting assertions about functions and
- data structures (\pxlref{precise-type-checks}.) More specific
- declarations also aid type inference and reduce the cost for type
- checking.
-\end{itemize}
-
-For related information, \pxlref{numeric-types} for numeric types, and
-section \ref{array-types} for array types.
-
-
-%%\node Canonicalization, Member Types, More Types Meaningful, More About Types in Python
-\subsection{Canonicalization}
-\cpsubindex{types}{equivalence}
-\cindex{canonicalization of types}
-\cindex{equivalence of types}
-
-When given a type specifier, \python{} will often rewrite it into a
-different (but equivalent) type. This is the mechanism that \python{}
-uses for detecting type equivalence. For example, in \python{}'s
-canonical representation, these types are equivalent:
-\begin{example}
-(or list (member :end)) \myequiv (or cons (member nil :end))
-\end{example}
-This has two implications for the user:
-\begin{itemize}
-
-\item The standard symbol type specifiers for \code{atom},
- \code{null}, \code{fixnum}, etc., are in no way magical. The
- \tindexed{null} type is actually defined to be \code{\w{(member
- nil)}}, \tindexed{list} is \code{\w{(or cons null)}}, and
- \tindexed{fixnum} is \code{\w{(signed-byte 30)}}.
-
-\item When the compiler prints out a type, it may not look like the
- type specifier that originally appeared in the program. This is
- generally not a problem, but it must be taken into consideration
- when reading compiler error messages.
-\end{itemize}
-
-
-%%\node Member Types, Union Types, Canonicalization, More About Types in Python
-\subsection{Member Types}
-\cindex{member types}
-
-The \tindexed{member} type specifier can be used to represent
-``symbolic'' values, analogous to the enumerated types of Pascal. For
-example, the second value of \code{find-symbol} has this type:
-\begin{lisp}
-(member :internal :external :inherited nil)
-\end{lisp}
-Member types are very useful for expressing consistency constraints on data
-structures, for example:
-\begin{lisp}
-(defstruct ice-cream
- (flavor :vanilla :type (member :vanilla :chocolate :strawberry)))
-\end{lisp}
-Member types are also useful in type inference, as the number of members can
-sometimes be pared down to one, in which case the value is a known constant.
-
-%%\node Union Types, The Empty Type, Member Types, More About Types in Python
-\subsection{Union Types}
-\cindex{union (\code{or}) types}
-\cindex{or (union) types}
-
-The \tindexed{or} (union) type specifier is understood, and is
-meaningfully applied in many contexts. The use of \code{or} allows
-assertions to be made about types in dynamically typed programs. For
-example:
-\begin{lisp}
-(defstruct box
- (next nil :type (or box null))
- (top :removed :type (or box-top (member :removed))))
-\end{lisp}
-The type assertion on the \code{top} slot ensures that an error will be signaled
-when there is an attempt to store an illegal value (such as \kwd{rmoved}.)
-Although somewhat weak, these union type assertions provide a useful input into
-type inference, allowing the cost of type checking to be reduced. For example,
-this loop is safely compiled with no type checks:
-\begin{lisp}
-(defun find-box-with-top (box)
- (declare (type (or box null) box))
- (do ((current box (box-next current)))
- ((null current))
- (unless (eq (box-top current) :removed)
- (return current))))
-\end{lisp}
-
-Union types are also useful in type inference for representing types that are
-partially constrained. For example, the result of this expression:
-\begin{lisp}
-(if foo
- (logior x y)
- (list x y))
-\end{lisp}
-can be expressed as \code{\w{(or integer cons)}}.
-
-%%\node The Empty Type, Function Types, Union Types, More About Types in Python
-\subsection{The Empty Type}
-\label{empty-type}
-\cindex{NIL type}
-\cpsubindex{empty type}{the}
-\cpsubindex{errors}{result type of}
-
-The type \false{} is also called the empty type, since no object is of
-type \false{}. The union of no types, \code{(or)}, is also empty.
-\python{}'s interpretation of an expression whose type is \false{} is
-that the expression never yields any value, but rather fails to
-terminate, or is thrown out of. For example, the type of a call to
-\code{error} or a use of \code{return} is \false{}. When the type of
-an expression is empty, compile-time type warnings about its value are
-suppressed; presumably somebody else is signaling an error. If a
-function is declared to have return type \false{}, but does in fact
-return, then (in safe compilation policies) a ``\code{NIL Function
- returned}'' error will be signaled. See also the function
-\funref{required-argument}.
-
-%%\node Function Types, The Values Declaration, The Empty Type, More About Types in Python
-\subsection{Function Types}
-\label{function-types}
-\cpsubindex{function}{types}
-\cpsubindex{types}{function}
-
-\findexed{function} types are understood in the restrictive sense, specifying:
-\begin{itemize}
-
-\item The argument syntax that the function must be called with. This
- is information about what argument counts are acceptable, and which
- keyword arguments are recognized. In \python, warnings about
- argument syntax are a consequence of function type checking.
-
-\item The types of the argument values that the caller must pass. If
- the compiler can prove that some argument to a call is of a type
- disallowed by the called function's type, then it will give a
- compile-time type warning. In addition to being used for
- compile-time type checking, these type assertions are also used as
- output type assertions in code generation. For example, if
- \code{foo} is declared to have a \code{fixnum} argument, then the
- \code{1+} in \w{\code{(foo (1+ x))}} is compiled with knowledge that
- the result must be a fixnum.
-
-\item The types the values that will be bound to argument variables in
- the function's definition. Declaring a function's type with
- \code{ftype} implicitly declares the types of the arguments in the
- definition. \python{} checks for consistency between the definition
- and the \code{ftype} declaration. Because of precise type checking,
- an error will be signaled when a function is called with an
- argument of the wrong type.
-
-\item The type of return value(s) that the caller can expect. This
- information is a useful input to type inference. For example, if a
- function is declared to return a \code{fixnum}, then when a call to
- that function appears in an expression, the expression will be
- compiled with knowledge that the call will return a \code{fixnum}.
-
-\item The type of return value(s) that the definition must return.
- The result type in an \code{ftype} declaration is treated like an
- implicit \code{the} wrapped around the body of the definition. If
- the definition returns a value of the wrong type, an error will be
- signaled. If the compiler can prove that the function returns the
- wrong type, then it will give a compile-time warning.
-\end{itemize}
-
-This is consistent with the new interpretation of function types and
-the \code{ftype} declaration in the proposed X3J13
-``function-type-argument-type-semantics'' cleanup. Note also, that if
-you don't explicitly declare the type of a function using a global
-\code{ftype} declaration, then \python{} will compute a function type
-from the definition, providing a degree of inter-routine type
-inference, \pxlref{function-type-inference}.
-
-%%\node The Values Declaration, Structure Types, Function Types, More About Types in Python
-\subsection{The Values Declaration}
-\cindex{values declaration}
-
-\cmucl{} supports the \code{values} declaration as an extension to
-\clisp. The syntax is {\code{(values \var{type1}
- \var{type2}$\ldots$\var{typen})}}. This declaration is
-semantically equivalent to a \code{the} form wrapped around the body
-of the special form in which the \code{values} declaration appears.
-The advantage of \code{values} over \findexed{the} is purely
-syntactic\dash{}it doesn't introduce more indentation. For example:
-\begin{example}
-(defun foo (x)
- (declare (values single-float))
- (ecase x
- (:this ...)
- (:that ...)
- (:the-other ...)))
-\end{example}
-is equivalent to:
-\begin{example}
-(defun foo (x)
- (the single-float
- (ecase x
- (:this ...)
- (:that ...)
- (:the-other ...))))
-\end{example}
-and
-\begin{example}
-(defun floor (number &optional (divisor 1))
- (declare (values integer real))
- ...)
-\end{example}
-is equivalent to:
-\begin{example}
-(defun floor (number &optional (divisor 1))
- (the (values integer real)
- ...))
-\end{example}
-In addition to being recognized by \code{lambda} (and hence by
-\code{defun}), the \code{values} declaration is recognized by all the
-other special forms with bodies and declarations: \code{let},
-\code{let*}, \code{labels} and \code{flet}. Macros with declarations
-usually splice the declarations into one of the above forms, so they
-will accept this declaration too, but the exact effect of a
-\code{values} declaration will depend on the macro.
-
-If you declare the types of all arguments to a function, and also
-declare the return value types with \code{values}, you have described
-the type of the function. \python{} will use this argument and result
-type information to derive a function type that will then be applied
-to calls of the function (\pxlref{function-types}.) This provides a
-way to declare the types of functions that is much less syntactically
-awkward than using the \code{ftype} declaration with a \code{function}
-type specifier.
-
-Although the \code{values} declaration is non-standard, it is
-relatively harmless to use it in otherwise portable code, since any
-warning in non-CMU implementations can be suppressed with the standard
-\code{declaration} proclamation.
-
-%%\node Structure Types, The Freeze-Type Declaration, The Values Declaration, More About Types in Python
-\subsection{Structure Types}
-\label{structure-types}
-\cindex{structure types}
-\cindex{defstruct types}
-\cpsubindex{types}{structure}
-
-Because of precise type checking, structure types are much better supported by
-Python than by conventional compilers:
-\begin{itemize}
-
-\item The structure argument to structure accessors is precisely
- checked\dash{}if you call \code{foo-a} on a \code{bar}, an error
- will be signaled.
-
-\item The types of slot values are precisely checked\dash{}if you pass
- the wrong type argument to a constructor or a slot setter, then an
- error will be signaled.
-\end{itemize}
-This error checking is tremendously useful for detecting bugs in
-programs that manipulate complex data structures.
-
-An additional advantage of checking structure types and enforcing slot
-types is that the compiler can safely believe slot type declarations.
-\python{} effectively moves the type checking from the slot access to
-the slot setter or constructor call. This is more efficient since
-caller of the setter or constructor often knows the type of the value,
-entirely eliminating the need to check the value's type. Consider
-this example:
-\begin{lisp}
-(defstruct coordinate
- (x nil :type single-float)
- (y nil :type single-float))
-
-(defun make-it ()
- (make-coordinate :x 1.0 :y 1.0))
-
-(defun use-it (it)
- (declare (type coordinate it))
- (sqrt (expt (coordinate-x it) 2) (expt (coordinate-y it) 2)))
-\end{lisp}
-\code{make-it} and \code{use-it} are compiled with no checking on the
-types of the float slots, yet \code{use-it} can use
-\code{single-float} arithmetic with perfect safety. Note that
-\code{make-coordinate} must still check the values of \code{x} and
-\code{y} unless the call is block compiled or inline expanded
-(\pxlref{local-call}.) But even without this advantage, it is almost
-always more efficient to check slot values on structure
-initialization, since slots are usually written once and read many
-times.
-
-%%\node The Freeze-Type Declaration, Type Restrictions, Structure Types, More About Types in Python
-\subsection{The Freeze-Type Declaration}
-\cindex{freeze-type declaration}
-\label{freeze-type}
-
-The \code{extensions:freeze-type} declaration is a CMU extension that
-enables more efficient compilation of user-defined types by asserting
-that the definition is not going to change. This declaration may only
-be used globally (with \code{declaim} or \code{proclaim}). Currently
-\code{freeze-type} only affects structure type testing done by
-\code{typep}, \code{typecase}, etc. Here is an example:
-\begin{lisp}
-(declaim (freeze-type foo bar))
-\end{lisp}
-This asserts that the types \code{foo} and \code{bar} and their
-subtypes are not going to change. This allows more efficient type
-testing, since the compiler can open-code a test for all possible
-subtypes, rather than having to examine the type hierarchy at
-run-time.
-
-%%\node Type Restrictions, Type Style Recommendations, The Freeze-Type Declaration, More About Types in Python
-\subsection{Type Restrictions}
-\cpsubindex{types}{restrictions on}
-
-Avoid use of the \code{and}, \code{not} and \code{satisfies} types in
-declarations, since type inference has problems with them. When these
-types do appear in a declaration, they are still checked precisely,
-but the type information is of limited use to the compiler.
-\code{and} types are effective as long as the intersection can be
-canonicalized to a type that doesn't use \code{and}. For example:
-\begin{example}
-(and fixnum unsigned-byte)
-\end{example}
-is fine, since it is the same as:
-\begin{example}
-(integer 0 \var{most-positive-fixnum})
-\end{example}
-but this type:
-\begin{example}
-(and symbol (not (member :end)))
-\end{example}
-will not be fully understood by type interference since the \code{and}
-can't be removed by canonicalization.
-
-Using any of these type specifiers in a type test with \code{typep} or
-\code{typecase} is fine, since as tests, these types can be translated
-into the \code{and} macro, the \code{not} function or a call to the
-satisfies predicate.
-
-%%\node Type Style Recommendations, , Type Restrictions, More About Types in Python
-\subsection{Type Style Recommendations}
-\cindex{style recommendations}
-
-Python provides good support for some currently unconventional ways of
-using the \clisp{} type system. With \python, it is desirable to make
-declarations as precise as possible, but type inference also makes
-some declarations unnecessary. Here are some general guidelines for
-maximum robustness and efficiency:
-\begin{itemize}
-
-\item Declare the types of all function arguments and structure slots
- as precisely as possible (while avoiding \code{not}, \code{and} and
- \code{satisfies}). Put these declarations in during initial coding
- so that type assertions can find bugs for you during debugging.
-
-\item Use the \tindexed{member} type specifier where there are a small
- number of possible symbol values, for example: \w{\code{(member :red
- :blue :green)}}.
-
-\item Use the \tindexed{or} type specifier in situations where the
- type is not certain, but there are only a few possibilities, for
- example: \w{\code{(or list vector)}}.
-
-\item Declare integer types with the tightest bounds that you can,
- such as \code{\w{(integer 3 7)}}.
-
-\item Define \findexed{deftype} or \findexed{defstruct} types before
- they are used. Definition after use is legal (producing no
- ``undefined type'' warnings), but type tests and structure
- operations will be compiled much less efficiently.
-
-\item Use the \code{extensions:freeze-type} declaration to speed up
- type testing for structure types which won't have new subtypes added
- later. \xlref{freeze-type}
-
-\item In addition to declaring the array element type and simpleness,
- also declare the dimensions if they are fixed, for example:
- \begin{example}
- (simple-array single-float (1024 1024))
- \end{example}
- This bounds information allows array indexing for multi-dimensional
- arrays to be compiled much more efficiently, and may also allow
- array bounds checking to be done at compile time.
- \xlref{array-types}.
-
-\item Avoid use of the \findexed{the} declaration within expressions.
- Not only does it clutter the code, but it is also almost worthless
- under safe policies. If the need for an output type assertion is
- revealed by efficiency notes during tuning, then you can consider
- \code{the}, but it is preferable to constrain the argument types
- more, allowing the compiler to prove the desired result type.
-
-\item Don't bother declaring the type of \findexed{let} or other
- non-argument variables unless the type is non-obvious. If you
- declare function return types and structure slot types, then the
- type of a variable is often obvious both to the programmer and to
- the compiler. An important case where the type isn't obvious, and a
- declaration is appropriate, is when the value for a variable is
- pulled out of untyped structure (e.g., the result of \code{car}), or
- comes from some weakly typed function, such as \code{read}.
-
-\item Declarations are sometimes necessary for integer loop variables,
- since the compiler can't always prove that the value is of a good
- integer type. These declarations are best added during tuning, when
- an efficiency note indicates the need.
-\end{itemize}
-
-
-%%\f
-%%\node Type Inference, Source Optimization, More About Types in Python, Advanced Compiler Use and Efficiency Hints
-\section{Type Inference}
-\label{type-inference}
-\cindex{type inference}
-\cindex{inference of types}
-\cindex{derivation of types}
-
-Type inference is the process by which the compiler tries to figure
-out the types of expressions and variables, given an inevitable lack
-of complete type information. Although \python{} does much more type
-inference than most \llisp{} compilers, remember that the more precise
-and comprehensive type declarations are, the more type inference will
-be able to do.
-
-\begin{comment}
-* Variable Type Inference::
-* Local Function Type Inference::
-* Global Function Type Inference::
-* Operation Specific Type Inference::
-* Dynamic Type Inference::
-* Type Check Optimization::
-\end{comment}
-
-%%\node Variable Type Inference, Local Function Type Inference, Type Inference, Type Inference
-\subsection{Variable Type Inference}
-\label{variable-type-inference}
-
-The type of a variable is the union of the types of all the
-definitions. In the degenerate case of a let, the type of the
-variable is the type of the initial value. This inferred type is
-intersected with any declared type, and is then propagated to all the
-variable's references. The types of \findexed{multiple-value-bind}
-variables are similarly inferred from the types of the individual
-values of the values form.
-
-If multiple type declarations apply to a single variable, then all the
-declarations must be correct; it is as though all the types were intersected
-producing a single \tindexed{and} type specifier. In this example:
-\begin{example}
-(defmacro my-dotimes ((var count) &body body)
- `(do ((,var 0 (1+ ,var)))
- ((>= ,var ,count))
- (declare (type (integer 0 *) ,var))
- ,@body))
-
-(my-dotimes (i ...)
- (declare (fixnum i))
- ...)
-\end{example}
-the two declarations for \code{i} are intersected, so \code{i} is
-known to be a non-negative fixnum.
-
-In practice, this type inference is limited to lets and local
-functions, since the compiler can't analyze all the calls to a global
-function. But type inference works well enough on local variables so
-that it is often unnecessary to declare the type of local variables.
-This is especially likely when function result types and structure
-slot types are declared. The main areas where type inference breaks
-down are:
-\begin{itemize}
-
-\item When the initial value of a variable is a untyped expression,
- such as \code{\w{(car x)}}, and
-
-\item When the type of one of the variable's definitions is a function
- of the variable's current value, as in: \code{(setq x (1+ x))}
-\end{itemize}
-
-
-%%\node Local Function Type Inference, Global Function Type Inference, Variable Type Inference, Type Inference
-\subsection{Local Function Type Inference}
-\cpsubindex{local call}{type inference}
-
-The types of arguments to local functions are inferred in the same was
-as any other local variable; the type is the union of the argument
-types across all the calls to the function, intersected with the
-declared type. If there are any assignments to the argument
-variables, the type of the assigned value is unioned in as well.
-
-The result type of a local function is computed in a special way that
-takes tail recursion (\pxlref{tail-recursion}) into consideration.
-The result type is the union of all possible return values that aren't
-tail-recursive calls. For example, \python{} will infer that the
-result type of this function is \code{integer}:
-\begin{lisp}
-(defun ! (n res)
- (declare (integer n res))
- (if (zerop n)
- res
- (! (1- n) (* n res))))
-\end{lisp}
-Although this is a rather obvious result, it becomes somewhat less
-trivial in the presence of mutual tail recursion of multiple
-functions. Local function result type inference interacts with the
-mechanisms for ensuring proper tail recursion mentioned in section
-\ref{local-call-return}.
-
-%%\node Global Function Type Inference, Operation Specific Type Inference, Local Function Type Inference, Type Inference
-\subsection{Global Function Type Inference}
-\label{function-type-inference}
-\cpsubindex{function}{type inference}
-
-As described in section \ref{function-types}, a global function type
-(\tindexed{ftype}) declaration places implicit type assertions on the
-call arguments, and also guarantees the type of the return value. So
-wherever a call to a declared function appears, there is no doubt as
-to the types of the arguments and return value. Furthermore,
-\python{} will infer a function type from the function's definition if
-there is no \code{ftype} declaration. Any type declarations on the
-argument variables are used as the argument types in the derived
-function type, and the compiler's best guess for the result type of
-the function is used as the result type in the derived function type.
-
-This method of deriving function types from the definition implicitly assumes
-that functions won't be redefined at run-time. Consider this example:
-\begin{lisp}
-(defun foo-p (x)
- (let ((res (and (consp x) (eq (car x) 'foo))))
- (format t "It is ~:[not ~;~]foo." res)))
-
-(defun frob (it)
- (if (foo-p it)
- (setf (cadr it) 'yow!)
- (1+ it)))
-\end{lisp}
-
-Presumably, the programmer really meant to return \code{res} from
-\code{foo-p}, but he seems to have forgotten. When he tries to call
-do \code{\w{(frob (list 'foo nil))}}, \code{frob} will flame out when
-it tries to add to a \code{cons}. Realizing his error, he fixes
-\code{foo-p} and recompiles it. But when he retries his test case, he
-is baffled because the error is still there. What happened in this
-example is that \python{} proved that the result of \code{foo-p} is
-\code{null}, and then proceeded to optimize away the \code{setf} in
-\code{frob}.
-
-Fortunately, in this example, the error is detected at compile time
-due to notes about unreachable code (\pxlref{dead-code-notes}.)
-Still, some users may not want to worry about this sort of problem
-during incremental development, so there is a variable to control
-deriving function types.
-
-\begin{defvar}{extensions:}{derive-function-types}
-
- If true (the default), argument and result type information derived
- from compilation of \code{defun}s is used when compiling calls to
- that function. If false, only information from \code{ftype}
- proclamations will be used.
-\end{defvar}
-
-%%\node Operation Specific Type Inference, Dynamic Type Inference, Global Function Type Inference, Type Inference
-\subsection{Operation Specific Type Inference}
-\label{operation-type-inference}
-\cindex{operation specific type inference}
-\cindex{arithmetic type inference}
-\cpsubindex{numeric}{type inference}
-
-Many of the standard \clisp{} functions have special type inference
-procedures that determine the result type as a function of the
-argument types. For example, the result type of \code{aref} is the
-array element type. Here are some other examples of type inferences:
-\begin{lisp}
-(logand x #xFF) \result{} (unsigned-byte 8)
-
-(+ (the (integer 0 12) x) (the (integer 0 1) y)) \result{} (integer 0 13)
-
-(ash (the (unsigned-byte 16) x) -8) \result{} (unsigned-byte 8)
-\end{lisp}
-
-%%\node Dynamic Type Inference, Type Check Optimization, Operation Specific Type Inference, Type Inference
-\subsection{Dynamic Type Inference}
-\label{constraint-propagation}
-\cindex{dynamic type inference}
-\cindex{conditional type inference}
-\cpsubindex{type inference}{dynamic}
-
-Python uses flow analysis to infer types in dynamically typed
-programs. For example:
-\begin{example}
-(ecase x
- (list (length x))
- ...)
-\end{example}
-Here, the compiler knows the argument to \code{length} is a list,
-because the call to \code{length} is only done when \code{x} is a
-list. The most significant efficiency effect of inference from
-assertions is usually in type check optimization.
-
-
-Dynamic type inference has two inputs: explicit conditionals and
-implicit or explicit type assertions. Flow analysis propagates these
-constraints on variable type to any code that can be executed only
-after passing though the constraint. Explicit type constraints come
-from \findexed{if}s where the test is either a lexical variable or a
-function of lexical variables and constants, where the function is
-either a type predicate, a numeric comparison or \code{eq}.
-
-If there is an \code{eq} (or \code{eql}) test, then the compiler will
-actually substitute one argument for the other in the true branch.
-For example:
-\begin{lisp}
-(when (eq x :yow!) (return x))
-\end{lisp}
-becomes:
-\begin{lisp}
-(when (eq x :yow!) (return :yow!))
-\end{lisp}
-This substitution is done when one argument is a constant, or one
-argument has better type information than the other. This
-transformation reveals opportunities for constant folding or
-type-specific optimizations. If the test is against a constant, then
-the compiler can prove that the variable is not that constant value in
-the false branch, or \w{\code{(not (member :yow!))}} in the example
-above. This can eliminate redundant tests, for example:
-\begin{example}
-(if (eq x nil)
- ...
- (if x a b))
-\end{example}
-is transformed to this:
-\begin{example}
-(if (eq x nil)
- ...
- a)
-\end{example}
-Variables appearing as \code{if} tests are interpreted as
-\code{\w{(not (eq \var{var} nil))}} tests. The compiler also converts
-\code{=} into \code{eql} where possible. It is difficult to do
-inference directly on \code{=} since it does implicit coercions.
-
-When there is an explicit \code{$<$} or \code{$>$} test on
-\begin{changebar}
- numeric
-\end{changebar}
-variables, the compiler makes inferences about the ranges the
-variables can assume in the true and false branches. This is mainly
-useful when it proves that the values are small enough in magnitude to
-allow open-coding of arithmetic operations. For example, in many uses
-of \code{dotimes} with a \code{fixnum} repeat count, the compiler
-proves that fixnum arithmetic can be used.
-
-Implicit type assertions are quite common, especially if you declare
-function argument types. Dynamic inference from implicit type
-assertions sometimes helps to disambiguate programs to a useful
-degree, but is most noticeable when it detects a dynamic type error.
-For example:
-\begin{lisp}
-(defun foo (x)
- (+ (car x) x))
-\end{lisp}
-results in this warning:
-\begin{example}
-In: DEFUN FOO
- (+ (CAR X) X)
-==>
- X
-Warning: Result is a LIST, not a NUMBER.
-\end{example}
-
-Note that \llisp{}'s dynamic type checking semantics make dynamic type
-inference useful even in programs that aren't really dynamically
-typed, for example:
-\begin{lisp}
-(+ (car x) (length x))
-\end{lisp}
-Here, \code{x} presumably always holds a list, but in the absence of a
-declaration the compiler cannot assume \code{x} is a list simply
-because list-specific operations are sometimes done on it. The
-compiler must consider the program to be dynamically typed until it
-proves otherwise. Dynamic type inference proves that the argument to
-\code{length} is always a list because the call to \code{length} is
-only done after the list-specific \code{car} operation.
-
-
-%%\node Type Check Optimization, , Dynamic Type Inference, Type Inference
-\subsection{Type Check Optimization}
-\label{type-check-optimization}
-\cpsubindex{type checking}{optimization}
-\cpsubindex{optimization}{type check}
-
-Python backs up its support for precise type checking by minimizing
-the cost of run-time type checking. This is done both through type
-inference and though optimizations of type checking itself.
-
-Type inference often allows the compiler to prove that a value is of
-the correct type, and thus no type check is necessary. For example:
-\begin{lisp}
-(defstruct foo a b c)
-(defstruct link
- (foo (required-argument) :type foo)
- (next nil :type (or link null)))
-
-(foo-a (link-foo x))
-\end{lisp}
-Here, there is no need to check that the result of \code{link-foo} is
-a \code{foo}, since it always is. Even when some type checks are
-necessary, type inference can often reduce the number:
-\begin{example}
-(defun test (x)
- (let ((a (foo-a x))
- (b (foo-b x))
- (c (foo-c x)))
- ...))
-\end{example}
-In this example, only one \w{\code{(foo-p x)}} check is needed. This
-applies to a lesser degree in list operations, such as:
-\begin{lisp}
-(if (eql (car x) 3) (cdr x) y)
-\end{lisp}
-Here, we only have to check that \code{x} is a list once.
-
-Since \python{} recognizes explicit type tests, code that explicitly
-protects itself against type errors has little introduced overhead due
-to implicit type checking. For example, this loop compiles with no
-implicit checks checks for \code{car} and \code{cdr}:
-\begin{lisp}
-(defun memq (e l)
- (do ((current l (cdr current)))
- ((atom current) nil)
- (when (eq (car current) e) (return current))))
-\end{lisp}
-
-\cindex{complemented type checks}
-Python reduces the cost of checks that must be done through an
-optimization called \var{complementing}. A complemented check for
-\var{type} is simply a check that the value is not of the type
-\w{\code{(not \var{type})}}. This is only interesting when something
-is known about the actual type, in which case we can test for the
-complement of \w{\code{(and \var{known-type} (not \var{type}))}}, or
-the difference between the known type and the assertion. An example:
-\begin{lisp}
-(link-foo (link-next x))
-\end{lisp}
-Here, we change the type check for \code{link-foo} from a test for
-\code{foo} to a test for:
-\begin{lisp}
-(not (and (or foo null) (not foo)))
-\end{lisp}
-or more simply \w{\code{(not null)}}. This is probably the most
-important use of complementing, since the situation is fairly common,
-and a \code{null} test is much cheaper than a structure type test.
-
-Here is a more complicated example that illustrates the combination of
-complementing with dynamic type inference:
-\begin{lisp}
-(defun find-a (a x)
- (declare (type (or link null) x))
- (do ((current x (link-next current)))
- ((null current) nil)
- (let ((foo (link-foo current)))
- (when (eq (foo-a foo) a) (return foo)))))
-\end{lisp}
-This loop can be compiled with no type checks. The \code{link} test
-for \code{link-foo} and \code{link-next} is complemented to
-\w{\code{(not null)}}, and then deleted because of the explicit
-\code{null} test. As before, no check is necessary for \code{foo-a},
-since the \code{link-foo} is always a \code{foo}. This sort of
-situation shows how precise type checking combined with precise
-declarations can actually result in reduced type checking.
-
-%%\f
-%%\node Source Optimization, Tail Recursion, Type Inference, Advanced Compiler Use and Efficiency Hints
-\section{Source Optimization}
-\label{source-optimization}
-\cindex{optimization}
-
-This section describes source-level transformations that \python{} does on
-programs in an attempt to make them more efficient. Although source-level
-optimizations can make existing programs more efficient, the biggest advantage
-of this sort of optimization is that it makes it easier to write efficient
-programs. If a clean, straightforward implementation is can be transformed
-into an efficient one, then there is no need for tricky and dangerous hand
-optimization.
-
-\begin{comment}
-* Let Optimization::
-* Constant Folding::
-* Unused Expression Elimination::
-* Control Optimization::
-* Unreachable Code Deletion::
-* Multiple Values Optimization::
-* Source to Source Transformation::
-* Style Recommendations::
-\end{comment}
-
-%%\node Let Optimization, Constant Folding, Source Optimization, Source Optimization
-\subsection{Let Optimization}
-\label{let-optimization}
-
-\cindex{let optimization} \cpsubindex{optimization}{let}
-
-The primary optimization of let variables is to delete them when they
-are unnecessary. Whenever the value of a let variable is a constant,
-a constant variable or a constant (local or non-notinline) function,
-the variable is deleted, and references to the variable are replaced
-with references to the constant expression. This is useful primarily
-in the expansion of macros or inline functions, where argument values
-are often constant in any given call, but are in general non-constant
-expressions that must be bound to preserve order of evaluation. Let
-variable optimization eliminates the need for macros to carefully
-avoid spurious bindings, and also makes inline functions just as
-efficient as macros.
-
-A particularly interesting class of constant is a local function.
-Substituting for lexical variables that are bound to a function can
-substantially improve the efficiency of functional programming styles,
-for example:
-\begin{lisp}
-(let ((a #'(lambda (x) (zow x))))
- (funcall a 3))
-\end{lisp}
-effectively transforms to:
-\begin{lisp}
-(zow 3)
-\end{lisp}
-This transformation is done even when the function is a closure, as in:
-\begin{lisp}
-(let ((a (let ((y (zug)))
- #'(lambda (x) (zow x y)))))
- (funcall a 3))
-\end{lisp}
-becoming:
-\begin{lisp}
-(zow 3 (zug))
-\end{lisp}
-
-A constant variable is a lexical variable that is never assigned to,
-always keeping its initial value. Whenever possible, avoid setting
-lexical variables\dash{}instead bind a new variable to the new value.
-Except for loop variables, it is almost always possible to avoid
-setting lexical variables. This form:
-\begin{example}
-(let ((x (f x)))
- ...)
-\end{example}
-is \var{more} efficient than this form:
-\begin{example}
-(setq x (f x))
-...
-\end{example}
-Setting variables makes the program more difficult to understand, both
-to the compiler and to the programmer. \python{} compiles assignments
-at least as efficiently as any other \llisp{} compiler, but most let
-optimizations are only done on constant variables.
-
-Constant variables with only a single use are also optimized away,
-even when the initial value is not constant.\footnote{The source
- transformation in this example doesn't represent the preservation of
- evaluation order implicit in the compiler's internal representation.
- Where necessary, the back end will reintroduce temporaries to
- preserve the semantics.} For example, this expansion of
-\code{incf}:
-\begin{lisp}
-(let ((#:g3 (+ x 1)))
- (setq x #:G3))
-\end{lisp}
-becomes:
-\begin{lisp}
-(setq x (+ x 1))
-\end{lisp}
-The type semantics of this transformation are more important than the
-elimination of the variable itself. Consider what happens when
-\code{x} is declared to be a \code{fixnum}; after the transformation,
-the compiler can compile the addition knowing that the result is a
-\code{fixnum}, whereas before the transformation the addition would
-have to allow for fixnum overflow.
-
-Another variable optimization deletes any variable that is never read.
-This causes the initial value and any assigned values to be unused,
-allowing those expressions to be deleted if they have no side-effects.
-
-Note that a let is actually a degenerate case of local call
-(\pxlref{let-calls}), and that let optimization can be done on calls
-that weren't created by a let. Also, local call allows an applicative
-style of iteration that is totally assignment free.
-
-%%\node Constant Folding, Unused Expression Elimination, Let Optimization, Source Optimization
-\subsection{Constant Folding}
-\cindex{constant folding}
-\cpsubindex{folding}{constant}
-
-Constant folding is an optimization that replaces a call of constant
-arguments with the constant result of that call. Constant folding is
-done on all standard functions for which it is legal. Inline
-expansion allows folding of any constant parts of the definition, and
-can be done even on functions that have side-effects.
-
-It is convenient to rely on constant folding when programming, as in this
-example:
-\begin{example}
-(defconstant limit 42)
-
-(defun foo ()
- (... (1- limit) ...))
-\end{example}
-Constant folding is also helpful when writing macros or inline
-functions, since it usually eliminates the need to write a macro that
-special-cases constant arguments.
-
-\cindex{constant-function declaration} Constant folding of a user
-defined function is enabled by the \code{extensions:constant-function}
-proclamation. In this example:
-\begin{example}
-(declaim (ext:constant-function myfun))
-(defun myexp (x y)
- (declare (single-float x y))
- (exp (* (log x) y)))
-
- ... (myexp 3.0 1.3) ...
-\end{example}
-The call to \code{myexp} is constant-folded to \code{4.1711674}.
-
-
-%%\node Unused Expression Elimination, Control Optimization, Constant Folding, Source Optimization
-\subsection{Unused Expression Elimination}
-\cindex{unused expression elimination}
-\cindex{dead code elimination}
-
-If the value of any expression is not used, and the expression has no
-side-effects, then it is deleted. As with constant folding, this
-optimization applies most often when cleaning up after inline
-expansion and other optimizations. Any function declared an
-\code{extensions:constant-function} is also subject to unused
-expression elimination.
-
-Note that \python{} will eliminate parts of unused expressions known
-to be side-effect free, even if there are other unknown parts. For
-example:
-\begin{lisp}
-(let ((a (list (foo) (bar))))
- (if t
- (zow)
- (raz a)))
-\end{lisp}
-becomes:
-\begin{lisp}
-(progn (foo) (bar))
-(zow)
-\end{lisp}
-
-
-%%\node Control Optimization, Unreachable Code Deletion, Unused Expression Elimination, Source Optimization
-\subsection{Control Optimization}
-\cindex{control optimization}
-\cpsubindex{optimization}{control}
-
-The most important optimization of control is recognizing when an
-\findexed{if} test is known at compile time, then deleting the
-\code{if}, the test expression, and the unreachable branch of the
-\code{if}. This can be considered a special case of constant folding,
-although the test doesn't have to be truly constant as long as it is
-definitely not \false. Note also, that type inference propagates the
-result of an \code{if} test to the true and false branches,
-\pxlref{constraint-propagation}.
-
-A related \code{if} optimization is this transformation:\footnote{Note
- that the code for \code{x} and \code{y} isn't actually replicated.}
-\begin{lisp}
-(if (if a b c) x y)
-\end{lisp}
-into:
-\begin{lisp}
-(if a
- (if b x y)
- (if c x y))
-\end{lisp}
-The opportunity for this sort of optimization usually results from a
-conditional macro. For example:
-\begin{lisp}
-(if (not a) x y)
-\end{lisp}
-is actually implemented as this:
-\begin{lisp}
-(if (if a nil t) x y)
-\end{lisp}
-which is transformed to this:
-\begin{lisp}
-(if a
- (if nil x y)
- (if t x y))
-\end{lisp}
-which is then optimized to this:
-\begin{lisp}
-(if a y x)
-\end{lisp}
-Note that due to \python{}'s internal representations, the
-\code{if}\dash{}\code{if} situation will be recognized even if other
-forms are wrapped around the inner \code{if}, like:
-\begin{example}
-(if (let ((g ...))
- (loop
- ...
- (return (not g))
- ...))
- x y)
-\end{example}
-
-In \python, all the \clisp{} macros really are macros, written in
-terms of \code{if}, \code{block} and \code{tagbody}, so user-defined
-control macros can be just as efficient as the standard ones.
-\python{} emits basic blocks using a heuristic that minimizes the
-number of unconditional branches. The code in a \code{tagbody} will
-not be emitted in the order it appeared in the source, so there is no
-point in arranging the code to make control drop through to the
-target.
-
-%%\node Unreachable Code Deletion, Multiple Values Optimization, Control Optimization, Source Optimization
-\subsection{Unreachable Code Deletion}
-\label{dead-code-notes}
-\cindex{unreachable code deletion}
-\cindex{dead code elimination}
-
-Python will delete code whenever it can prove that the code can never be
-executed. Code becomes unreachable when:
-\begin{itemize}
-
-\item
-An \code{if} is optimized away, or
-
-\item
-There is an explicit unconditional control transfer such as \code{go} or
-\code{return-from}, or
-
-\item
-The last reference to a local function is deleted (or there never was any
-reference.)
-\end{itemize}
-
-
-When code that appeared in the original source is deleted, the compiler prints
-a note to indicate a possible problem (or at least unnecessary code.) For
-example:
-\begin{lisp}
-(defun foo ()
- (if t
- (write-line "True.")
- (write-line "False.")))
-\end{lisp}
-will result in this note:
-\begin{example}
-In: DEFUN FOO
- (WRITE-LINE "False.")
-Note: Deleting unreachable code.
-\end{example}
-
-It is important to pay attention to unreachable code notes, since they often
-indicate a subtle type error. For example:
-\begin{example}
-(defstruct foo a b)
-
-(defun lose (x)
- (let ((a (foo-a x))
- (b (if x (foo-b x) :none)))
- ...))
-\end{example}
-results in this note:
-\begin{example}
-In: DEFUN LOSE
- (IF X (FOO-B X) :NONE)
-==>
- :NONE
-Note: Deleting unreachable code.
-\end{example}
-The \kwd{none} is unreachable, because type inference knows that the argument
-to \code{foo-a} must be a \code{foo}, and thus can't be \false. Presumably the
-programmer forgot that \code{x} could be \false{} when he wrote the binding for
-\code{a}.
-
-Here is an example with an incorrect declaration:
-\begin{lisp}
-(defun count-a (string)
- (do ((pos 0 (position #\back{a} string :start (1+ pos)))
- (count 0 (1+ count)))
- ((null pos) count)
- (declare (fixnum pos))))
-\end{lisp}
-This time our note is:
-\begin{example}
-In: DEFUN COUNT-A
- (DO ((POS 0 #) (COUNT 0 #))
- ((NULL POS) COUNT)
- (DECLARE (FIXNUM POS)))
---> BLOCK LET TAGBODY RETURN-FROM PROGN
-==>
- COUNT
-Note: Deleting unreachable code.
-\end{example}
-The problem here is that \code{pos} can never be null since it is declared a
-\code{fixnum}.
-
-It takes some experience with unreachable code notes to be able to
-tell what they are trying to say. In non-obvious cases, the best
-thing to do is to call the function in a way that should cause the
-unreachable code to be executed. Either you will get a type error, or
-you will find that there truly is no way for the code to be executed.
-
-Not all unreachable code results in a note:
-\begin{itemize}
-
-\item A note is only given when the unreachable code textually appears
- in the original source. This prevents spurious notes due to the
- optimization of macros and inline functions, but sometimes also
- foregoes a note that would have been useful.
-
-\item Since accurate source information is not available for non-list
- forms, there is an element of heuristic in determining whether or
- not to give a note about an atom. Spurious notes may be given when
- a macro or inline function defines a variable that is also present
- in the calling function. Notes about \false{} and \true{} are never
- given, since it is too easy to confuse these constants in expanded
- code with ones in the original source.
-
-\item Notes are only given about code unreachable due to control flow.
- There is no note when an expression is deleted because its value is
- unused, since this is a common consequence of other optimizations.
-\end{itemize}
-
-
-Somewhat spurious unreachable code notes can also result when a macro
-inserts multiple copies of its arguments in different contexts, for
-example:
-\begin{lisp}
-(defmacro t-and-f (var form)
- `(if ,var ,form ,form))
-
-(defun foo (x)
- (t-and-f x (if x "True." "False.")))
-\end{lisp}
-results in these notes:
-\begin{example}
-In: DEFUN FOO
- (IF X "True." "False.")
-==>
- "False."
-Note: Deleting unreachable code.
-
-==>
- "True."
-Note: Deleting unreachable code.
-\end{example}
-It seems like it has deleted both branches of the \code{if}, but it has really
-deleted one branch in one copy, and the other branch in the other copy. Note
-that these messages are only spurious in not satisfying the intent of the rule
-that notes are only given when the deleted code appears in the original source;
-there is always \var{some} code being deleted when a unreachable code note is
-printed.
-
-
-%%\node Multiple Values Optimization, Source to Source Transformation, Unreachable Code Deletion, Source Optimization
-\subsection{Multiple Values Optimization}
-\cindex{multiple value optimization}
-\cpsubindex{optimization}{multiple value}
-
-Within a function, \python{} implements uses of multiple values
-particularly efficiently. Multiple values can be kept in arbitrary
-registers, so using multiple values doesn't imply stack manipulation
-and representation conversion. For example, this code:
-\begin{example}
-(let ((a (if x (foo x) u))
- (b (if x (bar x) v)))
- ...)
-\end{example}
-is actually more efficient written this way:
-\begin{example}
-(multiple-value-bind
- (a b)
- (if x
- (values (foo x) (bar x))
- (values u v))
- ...)
-\end{example}
-
-Also, \pxlref{local-call-return} for information on how local call
-provides efficient support for multiple function return values.
-
-
-%%\node Source to Source Transformation, Style Recommendations, Multiple Values Optimization, Source Optimization
-\subsection{Source to Source Transformation}
-\cindex{source-to-source transformation}
-\cpsubindex{transformation}{source-to-source}
-
-The compiler implements a number of operation-specific optimizations as
-source-to-source transformations. You will often see unfamiliar code in error
-messages, for example:
-\begin{lisp}
-(defun my-zerop () (zerop x))
-\end{lisp}
-gives this warning:
-\begin{example}
-In: DEFUN MY-ZEROP
- (ZEROP X)
-==>
- (= X 0)
-Warning: Undefined variable: X
-\end{example}
-The original \code{zerop} has been transformed into a call to
-\code{=}. This transformation is indicated with the same \code{==$>$}
-used to mark macro and function inline expansion. Although it can be
-confusing, display of the transformed source is important, since
-warnings are given with respect to the transformed source. This a
-more obscure example:
-\begin{lisp}
-(defun foo (x) (logand 1 x))
-\end{lisp}
-gives this efficiency note:
-\begin{example}
-In: DEFUN FOO
- (LOGAND 1 X)
-==>
- (LOGAND C::Y C::X)
-Note: Forced to do static-function Two-arg-and (cost 53).
- Unable to do inline fixnum arithmetic (cost 1) because:
- The first argument is a INTEGER, not a FIXNUM.
- etc.
-\end{example}
-Here, the compiler commuted the call to \code{logand}, introducing
-temporaries. The note complains that the \var{first} argument is not
-a \code{fixnum}, when in the original call, it was the second
-argument. To make things more confusing, the compiler introduced
-temporaries called \code{c::x} and \code{c::y} that are bound to
-\code{y} and \code{1}, respectively.
-
-You will also notice source-to-source optimizations when efficiency
-notes are enabled (\pxlref{efficiency-notes}.) When the compiler is
-unable to do a transformation that might be possible if there was more
-information, then an efficiency note is printed. For example,
-\code{my-zerop} above will also give this efficiency note:
-\begin{example}
-In: DEFUN FOO
- (ZEROP X)
-==>
- (= X 0)
-Note: Unable to optimize because:
- Operands might not be the same type, so can't open code.
-\end{example}
-
-%%\node Style Recommendations, , Source to Source Transformation, Source Optimization
-\subsection{Style Recommendations}
-\cindex{style recommendations}
-
-Source level optimization makes possible a clearer and more relaxed programming
-style:
-\begin{itemize}
-
-\item Don't use macros purely to avoid function call. If you want an
- inline function, write it as a function and declare it inline. It's
- clearer, less error-prone, and works just as well.
-
-\item Don't write macros that try to ``optimize'' their expansion in
- trivial ways such as avoiding binding variables for simple
- expressions. The compiler does these optimizations too, and is less
- likely to make a mistake.
-
-\item Make use of local functions (i.e., \code{labels} or \code{flet})
- and tail-recursion in places where it is clearer. Local function
- call is faster than full call.
-
-\item Avoid setting local variables when possible. Binding a new
- \code{let} variable is at least as efficient as setting an existing
- variable, and is easier to understand, both for the compiler and the
- programmer.
-
-\item Instead of writing similar code over and over again so that it
- can be hand customized for each use, define a macro or inline
- function, and let the compiler do the work.
-\end{itemize}
-
-
-%%\f
-%%\node Tail Recursion, Local Call, Source Optimization, Advanced Compiler Use and Efficiency Hints
-\section{Tail Recursion}
-\label{tail-recursion}
-\cindex{tail recursion}
-\cindex{recursion}
-
-A call is tail-recursive if nothing has to be done after the the call
-returns, i.e. when the call returns, the returned value is immediately
-returned from the calling function. In this example, the recursive
-call to \code{myfun} is tail-recursive:
-\begin{lisp}
-(defun myfun (x)
- (if (oddp (random x))
- (isqrt x)
- (myfun (1- x))))
-\end{lisp}
-
-Tail recursion is interesting because it is form of recursion that can be
-implemented much more efficiently than general recursion. In general, a
-recursive call requires the compiler to allocate storage on the stack at
-run-time for every call that has not yet returned. This memory consumption
-makes recursion unacceptably inefficient for representing repetitive algorithms
-having large or unbounded size. Tail recursion is the special case of
-recursion that is semantically equivalent to the iteration constructs normally
-used to represent repetition in programs. Because tail recursion is equivalent
-to iteration, tail-recursive programs can be compiled as efficiently as
-iterative programs.
-
-So why would you want to write a program recursively when you can write it
-using a loop? Well, the main answer is that recursion is a more general
-mechanism, so it can express some solutions simply that are awkward to write as
-a loop. Some programmers also feel that recursion is a stylistically
-preferable way to write loops because it avoids assigning variables.
-For example, instead of writing:
-\begin{lisp}
-(defun fun1 (x)
- something-that-uses-x)
-
-(defun fun2 (y)
- something-that-uses-y)
-
-(do ((x something (fun2 (fun1 x))))
- (nil))
-\end{lisp}
-You can write:
-\begin{lisp}
-(defun fun1 (x)
- (fun2 something-that-uses-x))
-
-(defun fun2 (y)
- (fun1 something-that-uses-y))
-
-(fun1 something)
-\end{lisp}
-The tail-recursive definition is actually more efficient, in addition to being
-(arguably) clearer. As the number of functions and the complexity of their
-call graph increases, the simplicity of using recursion becomes compelling.
-Consider the advantages of writing a large finite-state machine with separate
-tail-recursive functions instead of using a single huge \code{prog}.
-
-It helps to understand how to use tail recursion if you think of a
-tail-recursive call as a \code{psetq} that assigns the argument values to the
-called function's variables, followed by a \code{go} to the start of the called
-function. This makes clear an inherent efficiency advantage of tail-recursive
-call: in addition to not having to allocate a stack frame, there is no need to
-prepare for the call to return (e.g., by computing a return PC.)
-
-Is there any disadvantage to tail recursion? Other than an increase
-in efficiency, the only way you can tell that a call has been compiled
-tail-recursively is if you use the debugger. Since a tail-recursive
-call has no stack frame, there is no way the debugger can print out
-the stack frame representing the call. The effect is that backtrace
-will not show some calls that would have been displayed in a
-non-tail-recursive implementation. In practice, this is not as bad as
-it sounds\dash{}in fact it isn't really clearly worse, just different.
-\xlref{debug-tail-recursion} for information about the debugger
-implications of tail recursion.
-
-In order to ensure that tail-recursion is preserved in arbitrarily
-complex calling patterns across separately compiled functions, the
-compiler must compile any call in a tail-recursive position as a
-tail-recursive call. This is done regardless of whether the program
-actually exhibits any sort of recursive calling pattern. In this
-example, the call to \code{fun2} will always be compiled as a
-tail-recursive call:
-\begin{lisp}
-(defun fun1 (x)
- (fun2 x))
-\end{lisp}
-So tail recursion doesn't necessarily have anything to do with recursion
-as it is normally thought of. \xlref{local-tail-recursion} for more
-discussion of using tail recursion to implement loops.
-
-\begin{comment}
-* Tail Recursion Exceptions::
-\end{comment}
-
-%%\node Tail Recursion Exceptions, , Tail Recursion, Tail Recursion
-\subsection{Tail Recursion Exceptions}
-
-Although \python{} is claimed to be ``properly'' tail-recursive, some
-might dispute this, since there are situations where tail recursion is
-inhibited:
-\begin{itemize}
-
-\item When the call is enclosed by a special binding, or
-
-\item When the call is enclosed by a \code{catch} or
- \code{unwind-protect}, or
-
-\item When the call is enclosed by a \code{block} or \code{tagbody}
- and the block name or \code{go} tag has been closed over.
-\end{itemize}
-These dynamic extent binding forms inhibit tail recursion because they
-allocate stack space to represent the binding. Shallow-binding
-implementations of dynamic scoping also require cleanup code to be
-evaluated when the scope is exited.
-
-%%\f
-%%\node Local Call, Block Compilation, Tail Recursion, Advanced Compiler Use and Efficiency Hints
-\section{Local Call}
-\label{local-call}
-\cindex{local call}
-\cpsubindex{call}{local}
-\cpsubindex{function call}{local}
-
-Python supports two kinds of function call: full call and local call.
-Full call is the standard calling convention; its late binding and
-generality make \llisp{} what it is, but create unavoidable overheads.
-When the compiler can compile the calling function and the called
-function simultaneously, it can use local call to avoid some of the
-overhead of full call. Local call is really a collection of
-compilation strategies. If some aspect of call overhead is not needed
-in a particular local call, then it can be omitted. In some cases,
-local call can be totally free. Local call provides two main
-advantages to the user:
-\begin{itemize}
-
-\item Local call makes the use of the lexical function binding forms
- \findexed{flet} and \findexed{labels} much more efficient. A local
- call is always faster than a full call, and in many cases is much
- faster.
-
-\item Local call is a natural approach to \i{block compilation}, a
- compilation technique that resolves function references at compile
- time. Block compilation speeds function call, but increases
- compilation times and prevents function redefinition.
-\end{itemize}
-
-
-\begin{comment}
-* Self-Recursive Calls::
-* Let Calls::
-* Closures::
-* Local Tail Recursion::
-* Return Values::
-\end{comment}
-
-%%\node Self-Recursive Calls, Let Calls, Local Call, Local Call
-\subsection{Self-Recursive Calls}
-\cpsubindex{recursion}{self}
-
-Local call is used when a function defined by \code{defun} calls itself. For
-example:
-\begin{lisp}
-(defun fact (n)
- (if (zerop n)
- 1
- (* n (fact (1- n)))))
-\end{lisp}
-This use of local call speeds recursion, but can also complicate
-debugging, since \findexed{trace} will only show the first call to
-\code{fact}, and not the recursive calls. This is because the
-recursive calls directly jump to the start of the function, and don't
-indirect through the \code{symbol-function}. Self-recursive local
-call is inhibited when the \kwd{block-compile} argument to
-\code{compile-file} is \false{} (\pxlref{compile-file-block}.)
-
-%%\node Let Calls, Closures, Self-Recursive Calls, Local Call
-\subsection{Let Calls}
-\label{let-calls}
-Because local call avoids unnecessary call overheads, the compiler
-internally uses local call to implement some macros and special forms
-that are not normally thought of as involving a function call. For
-example, this \code{let}:
-\begin{example}
-(let ((a (foo))
- (b (bar)))
- ...)
-\end{example}
-is internally represented as though it was macroexpanded into:
-\begin{example}
-(funcall #'(lambda (a b)
- ...)
- (foo)
- (bar))
-\end{example}
-This implementation is acceptable because the simple cases of local
-call (equivalent to a \code{let}) result in good code. This doesn't
-make \code{let} any more efficient, but does make local calls that are
-semantically the same as \code{let} much more efficient than full
-calls. For example, these definitions are all the same as far as the
-compiler is concerned:
-\begin{example}
-(defun foo ()
- ...some other stuff...
- (let ((a something))
- ...some stuff...))
-
-(defun foo ()
- (flet ((localfun (a)
- ...some stuff...))
- ...some other stuff...
- (localfun something)))
-
-(defun foo ()
- (let ((funvar #'(lambda (a)
- ...some stuff...)))
- ...some other stuff...
- (funcall funvar something)))
-\end{example}
-
-Although local call is most efficient when the function is called only
-once, a call doesn't have to be equivalent to a \code{let} to be more
-efficient than full call. All local calls avoid the overhead of
-argument count checking and keyword argument parsing, and there are a
-number of other advantages that apply in many common situations.
-\xlref{let-optimization} for a discussion of the optimizations done on
-let calls.
-
-%%\node Closures, Local Tail Recursion, Let Calls, Local Call
-\subsection{Closures}
-\cindex{closures}
-
-Local call allows for much more efficient use of closures, since the
-closure environment doesn't need to be allocated on the heap, or even
-stored in memory at all. In this example, there is no penalty for
-\code{localfun} referencing \code{a} and \code{b}:
-\begin{lisp}
-(defun foo (a b)
- (flet ((localfun (x)
- (1+ (* a b x))))
- (if (= a b)
- (localfun (- x))
- (localfun x))))
-\end{lisp}
-In local call, the compiler effectively passes closed-over values as
-extra arguments, so there is no need for you to ``optimize'' local
-function use by explicitly passing in lexically visible values.
-Closures may also be subject to let optimization
-(\pxlref{let-optimization}.)
-
-Note: indirect value cells are currently always allocated on the heap
-when a variable is both assigned to (with \code{setq} or \code{setf})
-and closed over, regardless of whether the closure is a local function
-or not. This is another reason to avoid setting variables when you
-don't have to.
-
-%%\node Local Tail Recursion, Return Values, Closures, Local Call
-\subsection{Local Tail Recursion}
-\label{local-tail-recursion}
-\cindex{tail recursion}
-\cpsubindex{recursion}{tail}
-
-Tail-recursive local calls are particularly efficient, since they are
-in effect an assignment plus a control transfer. Scheme programmers
-write loops with tail-recursive local calls, instead of using the
-imperative \code{go} and \code{setq}. This has not caught on in the
-\clisp{} community, since conventional \llisp{} compilers don't
-implement local call. In \python, users can choose to write loops
-such as:
-\begin{lisp}
-(defun ! (n)
- (labels ((loop (n total)
- (if (zerop n)
- total
- (loop (1- n) (* n total)))))
- (loop n 1)))
-\end{lisp}
-
-\begin{defmac}{extensions:}{iterate}{%
- \args{\var{name} (\mstar{(\var{var} \var{initial-value})})
- \mstar{\var{declaration}} \mstar{\var{form}}}}
-
- This macro provides syntactic sugar for using \findexed{labels} to
- do iteration. It creates a local function \var{name} with the
- specified \var{var}s as its arguments and the \var{declaration}s and
- \var{form}s as its body. This function is then called with the
- \var{initial-values}, and the result of the call is return from the
- macro.
-
- Here is our factorial example rewritten using \code{iterate}:
-
- \begin{lisp}
- (defun ! (n)
- (iterate loop
- ((n n)
- (total 1))
- (if (zerop n)
- total
- (loop (1- n) (* n total)))))
- \end{lisp}
-
- The main advantage of using \code{iterate} over \code{do} is that
- \code{iterate} naturally allows stepping to be done differently
- depending on conditionals in the body of the loop. \code{iterate}
- can also be used to implement algorithms that aren't really
- iterative by simply doing a non-tail call. For example, the
- standard recursive definition of factorial can be written like this:
-\begin{lisp}
-(iterate fact
- ((n n))
- (if (zerop n)
- 1
- (* n (fact (1- n)))))
-\end{lisp}
-\end{defmac}
-
-%%\node Return Values, , Local Tail Recursion, Local Call
-\subsection{Return Values}
-\label{local-call-return}
-\cpsubindex{return values}{local call}
-\cpsubindex{local call}{return values}
-
-One of the more subtle costs of full call comes from allowing
-arbitrary numbers of return values. This overhead can be avoided in
-local calls to functions that always return the same number of values.
-For efficiency reasons (as well as stylistic ones), you should write
-functions so that they always return the same number of values. This
-may require passing extra \false{} arguments to \code{values} in some
-cases, but the result is more efficient, not less so.
-
-When efficiency notes are enabled (\pxlref{efficiency-notes}), and the
-compiler wants to use known values return, but can't prove that the
-function always returns the same number of values, then it will print
-a note like this:
-\begin{example}
-In: DEFUN GRUE
- (DEFUN GRUE (X) (DECLARE (FIXNUM X)) (COND (# #) (# NIL) (T #)))
-Note: Return type not fixed values, so can't use known return convention:
- (VALUES (OR (INTEGER -536870912 -1) NULL) &REST T)
-\end{example}
-
-In order to implement proper tail recursion in the presence of known
-values return (\pxlref{tail-recursion}), the compiler sometimes must
-prove that multiple functions all return the same number of values.
-When this can't be proven, the compiler will print a note like this:
-\begin{example}
-In: DEFUN BLUE
- (DEFUN BLUE (X) (DECLARE (FIXNUM X)) (COND (# #) (# #) (# #) (T #)))
-Note: Return value count mismatch prevents known return from
- these functions:
- BLUE
- SNOO
-\end{example}
-\xlref{number-local-call} for the interaction between local call
-and the representation of numeric types.
-
-%%\f
-%%\node Block Compilation, Inline Expansion, Local Call, Advanced Compiler Use and Efficiency Hints
-\section{Block Compilation}
-\label{block-compilation}
-\cindex{block compilation}
-\cpsubindex{compilation}{block}
-
-Block compilation allows calls to global functions defined by
-\findexed{defun} to be compiled as local calls. The function call
-can be in a different top-level form than the \code{defun}, or even in a
-different file.
-
-In addition, block compilation allows the declaration of the \i{entry points}
-to the block compiled portion. An entry point is any function that may be
-called from outside of the block compilation. If a function is not an entry
-point, then it can be compiled more efficiently, since all calls are known at
-compile time. In particular, if a function is only called in one place, then
-it will be let converted. This effectively inline expands the function, but
-without the code duplication that results from defining the function normally
-and then declaring it inline.
-
-The main advantage of block compilation is that it it preserves efficiency in
-programs even when (for readability and syntactic convenience) they are broken
-up into many small functions. There is absolutely no overhead for calling a
-non-entry point function that is defined purely for modularity (i.e. called
-only in one place.)
-
-Block compilation also allows the use of non-descriptor arguments and return
-values in non-trivial programs (\pxlref{number-local-call}).
-
-\begin{comment}
-* Block Compilation Semantics::
-* Block Compilation Declarations::
-* Compiler Arguments::
-* Practical Difficulties::
-* Context Declarations::
-* Context Declaration Example::
-\end{comment}
-
-%%\node Block Compilation Semantics, Block Compilation Declarations, Block Compilation, Block Compilation
-\subsection{Block Compilation Semantics}
-
-The effect of block compilation can be envisioned as the compiler turning all
-the \code{defun}s in the block compilation into a single \code{labels} form:
-\begin{example}
-(declaim (start-block fun1 fun3))
-
-(defun fun1 ()
- ...)
-
-(defun fun2 ()
- ...
- (fun1)
- ...)
-
-(defun fun3 (x)
- (if x
- (fun1)
- (fun2)))
-
-(declaim (end-block))
-\end{example}
-becomes:
-\begin{example}
-(labels ((fun1 ()
- ...)
- (fun2 ()
- ...
- (fun1)
- ...)
- (fun3 (x)
- (if x
- (fun1)
- (fun2))))
- (setf (fdefinition 'fun1) #'fun1)
- (setf (fdefinition 'fun3) #'fun3))
-\end{example}
-Calls between the block compiled functions are local calls, so changing the
-global definition of \code{fun1} will have no effect on what \code{fun2} does;
-\code{fun2} will keep calling the old \code{fun1}.
-
-The entry points \code{fun1} and \code{fun3} are still installed in
-the \code{symbol-function} as the global definitions of the functions,
-so a full call to an entry point works just as before. However,
-\code{fun2} is not an entry point, so it is not globally defined. In
-addition, \code{fun2} is only called in one place, so it will be let
-converted.
-
-
-%%\node Block Compilation Declarations, Compiler Arguments, Block Compilation Semantics, Block Compilation
-\subsection{Block Compilation Declarations}
-\cpsubindex{declarations}{block compilation}
-\cindex{start-block declaration}
-\cindex{end-block declaration}
-
-The \code{extensions:start-block} and \code{extensions:end-block}
-declarations allow fine-grained control of block compilation. These
-declarations are only legal as a global declarations (\code{declaim}
-or \code{proclaim}).
-
-\noindent
-\vspace{1 em}
-The \code{start-block} declaration has this syntax:
-\begin{example}
-(start-block \mstar{\var{entry-point-name}})
-\end{example}
-When processed by the compiler, this declaration marks the start of
-block compilation, and specifies the entry points to that block. If
-no entry points are specified, then \var{all} functions are made into
-entry points. If already block compiling, then the compiler ends the
-current block and starts a new one.
-
-\noindent
-\vspace{1 em}
-The \code{end-block} declaration has no arguments:
-\begin{lisp}
-(end-block)
-\end{lisp}
-The \code{end-block} declaration ends a block compilation unit without
-starting a new one. This is useful mainly when only a portion of a file
-is worth block compiling.
-
-%%\node Compiler Arguments, Practical Difficulties, Block Compilation Declarations, Block Compilation
-\subsection{Compiler Arguments}
-\label{compile-file-block}
-\cpsubindex{compile-file}{block compilation arguments}
-
-The \kwd{block-compile} and \kwd{entry-points} arguments to
-\code{extensions:compile-from-stream} and \funref{compile-file} provide overall
-control of block compilation, and allow block compilation without requiring
-modification of the program source.
-
-There are three possible values of the \kwd{block-compile} argument:
-\begin{Lentry}
-
-\item[\false{}] Do no compile-time resolution of global function
- names, not even for self-recursive calls. This inhibits any
- \code{start-block} declarations appearing in the file, allowing all
- functions to be incrementally redefined.
-
-\item[\true{}] Start compiling in block compilation mode. This is
- mainly useful for block compiling small files that contain no
- \code{start-block} declarations. See also the \kwd{entry-points}
- argument.
-
-\item[\kwd{specified}] Start compiling in form-at-a-time mode, but
- exploit \code{start-block} declarations and compile self-recursive
- calls as local calls. Normally \kwd{specified} is the default for
- this argument (see \varref{block-compile-default}.)
-\end{Lentry}
-
-The \kwd{entry-points} argument can be used in conjunction with
-\w{\kwd{block-compile} \true{}} to specify the entry-points to a
-block-compiled file. If not specified or \nil, all global functions
-will be compiled as entry points. When \kwd{block-compile} is not
-\true, this argument is ignored.
-
-\begin{defvar}{}{block-compile-default}
-
- This variable determines the default value for the
- \kwd{block-compile} argument to \code{compile-file} and
- \code{compile-from-stream}. The initial value of this variable is
- \kwd{specified}, but \false{} is sometimes useful for totally
- inhibiting block compilation.
-\end{defvar}
-
-%%\node Practical Difficulties, Context Declarations, Compiler Arguments, Block Compilation
-\subsection{Practical Difficulties}
-
-The main problem with block compilation is that the compiler uses
-large amounts of memory when it is block compiling. This places an
-upper limit on the amount of code that can be block compiled as a
-unit. To make best use of block compilation, it is necessary to
-locate the parts of the program containing many internal calls, and
-then add the appropriate \code{start-block} declarations. When writing
-new code, it is a good idea to put in block compilation declarations
-from the very beginning, since writing block declarations correctly
-requires accurate knowledge of the program's function call structure.
-If you want to initially develop code with full incremental
-redefinition, you can compile with \varref{block-compile-default} set to
-\false.
-
-Note if a \code{defun} appears in a non-null lexical environment, then
-calls to it cannot be block compiled.
-
-Unless files are very small, it is probably impractical to block compile
-multiple files as a unit by specifying a list of files to \code{compile-file}.
-Semi-inline expansion (\pxlref{semi-inline}) provides another way to
-extend block compilation across file boundaries.
-%%\f
-%%\node Context Declarations, Context Declaration Example, Practical Difficulties, Block Compilation
-\subsection{Context Declarations}
-\label{context-declarations}
-\cindex{context sensitive declarations}
-\cpsubindex{declarations}{context-sensitive}
-
-\cmucl{} has a context-sensitive declaration mechanism which is useful
-because it allows flexible control of the compilation policy in large
-systems without requiring changes to the source files. The primary
-use of this feature is to allow the exported interfaces of a system to
-be compiled more safely than the system internals. The context used
-is the name being defined and the kind of definition (function, macro,
-etc.)
-
-The \kwd{context-declarations} option to \macref{with-compilation-unit} has
-dynamic scope, affecting all compilation done during the evaluation of the
-body. The argument to this option should evaluate to a list of lists of the
-form:
-\begin{example}
-(\var{context-spec} \mplus{\var{declare-form}})
-\end{example}
-In the indicated context, the specified declare forms are inserted at
-the head of each definition. The declare forms for all contexts that
-match are appended together, with earlier declarations getting
-precedence over later ones. A simple example:
-\begin{example}
- :context-declarations
- '((:external (declare (optimize (safety 2)))))
-\end{example}
-This will cause all functions that are named by external symbols to be
-compiled with \code{safety 2}.
-
-The full syntax of context specs is:
-\begin{Lentry}
-
-\item[\kwd{internal}, \kwd{external}] True if the symbol is internal
- (external) in its home package.
-
-\item[\kwd{uninterned}] True if the symbol has no home package.
-
-\item[\code{\w{(:package \mstar{\var{package-name}})}}] True if the
- symbol's home package is in any of the named packages (false if
- uninterned.)
-
-\item[\kwd{anonymous}] True if the function doesn't have any
- interesting name (not \code{defmacro}, \code{defun}, \code{labels}
- or \code{flet}).
-
-\item[\kwd{macro}, \kwd{function}] \kwd{macro} is a global
- (\code{defmacro}) macro. \kwd{function} is anything else.
-
-\item[\kwd{local}, \kwd{global}] \kwd{local} is a \code{labels} or
- \code{flet}. \kwd{global} is anything else.
-
-\item[\code{\w{(:or \mstar{\var{context-spec}})}}] True when any
- supplied \var{context-spec} is true.
-
-\item[\code{\w{(:and \mstar{\var{context-spec}})}}] True only when all
- supplied \var{context-spec}s are true.
-
-\item[\code{\w{(:not \mstar{\var{context-spec}})}}] True when
- \var{context-spec} is false.
-
-\item[\code{\w{(:member \mstar{\var{name}})}}] True when the defined
- name is one of these names (\code{equal} test.)
-
-\item[\code{\w{(:match \mstar{\var{pattern}})}}] True when any of the
- patterns is a substring of the name. The name is wrapped with
- \code{\$}'s, so ``\code{\$FOO}'' matches names beginning with
- ``\code{FOO}'', etc.
-\end{Lentry}
-
-%%\node Context Declaration Example, , Context Declarations, Block Compilation
-\subsection{Context Declaration Example}
-
-Here is a more complex example of \code{with-compilation-unit} options:
-\begin{example}
-:optimize '(optimize (speed 2) (space 2) (inhibit-warnings 2)
- (debug 1) (safety 0))
-:optimize-interface '(optimize-interface (safety 1) (debug 1))
-:context-declarations
-'(((:or :external (:and (:match "\%") (:match "SET")))
- (declare (optimize-interface (safety 2))))
- ((:or (:and :external :macro)
- (:match "\$PARSE-"))
- (declare (optimize (safety 2)))))
-\end{example}
-The \code{optimize} and \code{extensions:optimize-interface}
-declarations (\pxlref{optimize-declaration}) set up the global
-compilation policy. The bodies of functions are to be compiled
-completely unsafe (\code{safety 0}), but argument count and weakened
-argument type checking is to be done when a function is called
-(\code{speed 2 safety 1}).
-
-The first declaration specifies that all functions that are external
-or whose names contain both ``\code{\%}'' and ``\code{SET}'' are to be
-compiled compiled with completely safe interfaces (\code{safety 2}).
-The reason for this particular \kwd{match} rule is that \code{setf}
-inverse functions in this system tend to have both strings in their
-name somewhere. We want \code{setf} inverses to be safe because they
-are implicitly called by users even though their name is not exported.
-
-The second declaration makes external macros or functions whose names
-start with ``\code{PARSE-}'' have safe bodies (as well as interfaces).
-This is desirable because a syntax error in a macro may cause a type
-error inside the body. The \kwd{match} rule is used because macros
-often have auxiliary functions whose names begin with this string.
-
-This particular example is used to build part of the standard \cmucl{}
-system. Note however, that context declarations must be set up
-according to the needs and coding conventions of a particular system;
-different parts of \cmucl{} are compiled with different context
-declarations, and your system will probably need its own declarations.
-In particular, any use of the \kwd{match} option depends on naming
-conventions used in coding.
-
-%%\f
-%%\node Inline Expansion, Byte Coded Compilation, Block Compilation, Advanced Compiler Use and Efficiency Hints
-\section{Inline Expansion}
-\label{inline-expansion}
-\cindex{inline expansion}
-\cpsubindex{expansion}{inline}
-\cpsubindex{call}{inline}
-\cpsubindex{function call}{inline}
-\cpsubindex{optimization}{function call}
-
-Python can expand almost any function inline, including functions
-with keyword arguments. The only restrictions are that keyword
-argument keywords in the call must be constant, and that global
-function definitions (\code{defun}) must be done in a null lexical
-environment (not nested in a \code{let} or other binding form.) Local
-functions (\code{flet}) can be inline expanded in any environment.
-Combined with \python{}'s source-level optimization, inline expansion
-can be used for things that formerly required macros for efficient
-implementation. In \python, macros don't have any efficiency
-advantage, so they need only be used where a macro's syntactic
-flexibility is required.
-
-Inline expansion is a compiler optimization technique that reduces
-the overhead of a function call by simply not doing the call:
-instead, the compiler effectively rewrites the program to appear as
-though the definition of the called function was inserted at each
-call site. In \llisp, this is straightforwardly expressed by
-inserting the \code{lambda} corresponding to the original definition:
-\begin{lisp}
-(proclaim '(inline my-1+))
-(defun my-1+ (x) (+ x 1))
-
-(my-1+ someval) \result{} ((lambda (x) (+ x 1)) someval)
-\end{lisp}
-
-When the function expanded inline is large, the program after inline
-expansion may be substantially larger than the original program. If
-the program becomes too large, inline expansion hurts speed rather
-than helping it, since hardware resources such as physical memory and
-cache will be exhausted. Inline expansion is called for:
-\begin{itemize}
-
-\item When profiling has shown that a relatively simple function is
- called so often that a large amount of time is being wasted in the
- calling of that function (as opposed to running in that function.)
- If a function is complex, it will take a long time to run relative
- the time spent in call, so the speed advantage of inline expansion
- is diminished at the same time the space cost of inline expansion is
- increased. Of course, if a function is rarely called, then the
- overhead of calling it is also insignificant.
-
-\item With functions so simple that they take less space to inline
- expand than would be taken to call the function (such as
- \code{my-1+} above.) It would require intimate knowledge of the
- compiler to be certain when inline expansion would reduce space, but
- it is generally safe to inline expand functions whose definition is
- a single function call, or a few calls to simple \clisp{} functions.
-\end{itemize}
-
-
-In addition to this speed/space tradeoff from inline expansion's
-avoidance of the call, inline expansion can also reveal opportunities
-for optimization. \python{}'s extensive source-level optimization can
-make use of context information from the caller to tremendously
-simplify the code resulting from the inline expansion of a function.
-
-The main form of caller context is local information about the actual
-argument values: what the argument types are and whether the arguments
-are constant. Knowledge about argument types can eliminate run-time
-type tests (e.g., for generic arithmetic.) Constant arguments in a
-call provide opportunities for constant folding optimization after
-inline expansion.
-
-A hidden way that constant arguments are often supplied to functions
-is through the defaulting of unsupplied optional or keyword arguments.
-There can be a huge efficiency advantage to inline expanding functions
-that have complex keyword-based interfaces, such as this definition of
-the \code{member} function:
-\begin{lisp}
-(proclaim '(inline member))
-(defun member (item list &key
- (key #'identity)
- (test #'eql testp)
- (test-not nil notp))
- (do ((list list (cdr list)))
- ((null list) nil)
- (let ((car (car list)))
- (if (cond (testp
- (funcall test item (funcall key car)))
- (notp
- (not (funcall test-not item (funcall key car))))
- (t
- (funcall test item (funcall key car))))
- (return list)))))
-
-\end{lisp}
-After inline expansion, this call is simplified to the obvious code:
-\begin{lisp}
-(member a l :key #'foo-a :test #'char=) \result{}
-
-(do ((list list (cdr list)))
- ((null list) nil)
- (let ((car (car list)))
- (if (char= item (foo-a car))
- (return list))))
-\end{lisp}
-In this example, there could easily be more than an order of magnitude
-improvement in speed. In addition to eliminating the original call to
-\code{member}, inline expansion also allows the calls to \code{char=}
-and \code{foo-a} to be open-coded. We go from a loop with three tests
-and two calls to a loop with one test and no calls.
-
-\xlref{source-optimization} for more discussion of source level
-optimization.
-
-\begin{comment}
-* Inline Expansion Recording::
-* Semi-Inline Expansion::
-* The Maybe-Inline Declaration::
-\end{comment}
-
-%%\node Inline Expansion Recording, Semi-Inline Expansion, Inline Expansion, Inline Expansion
-\subsection{Inline Expansion Recording}
-\cindex{recording of inline expansions}
-
-Inline expansion requires that the source for the inline expanded function to
-be available when calls to the function are compiled. The compiler doesn't
-remember the inline expansion for every function, since that would take an
-excessive about of space. Instead, the programmer must tell the compiler to
-record the inline expansion before the definition of the inline expanded
-function is compiled. This is done by globally declaring the function inline
-before the function is defined, by using the \code{inline} and
-\code{extensions:maybe-inline} (\pxlref{maybe-inline-declaration})
-declarations.
-
-In addition to recording the inline expansion of inline functions at the time
-the function is compiled, \code{compile-file} also puts the inline expansion in
-the output file. When the output file is loaded, the inline expansion is made
-available for subsequent compilations; there is no need to compile the
-definition again to record the inline expansion.
-
-If a function is declared inline, but no expansion is recorded, then the
-compiler will give an efficiency note like:
-\begin{example}
-Note: MYFUN is declared inline, but has no expansion.
-\end{example}
-When you get this note, check that the \code{inline} declaration and the
-definition appear before the calls that are to be inline expanded. This note
-will also be given if the inline expansion for a \code{defun} could not be
-recorded because the \code{defun} was in a non-null lexical environment.
-
-%%\node Semi-Inline Expansion, The Maybe-Inline Declaration, Inline Expansion Recording, Inline Expansion
-\subsection{Semi-Inline Expansion}
-\label{semi-inline}
-
-Python supports \var{semi-inline} functions. Semi-inline expansion
-shares a single copy of a function across all the calls in a component
-by converting the inline expansion into a local function
-(\pxlref{local-call}.) This takes up less space when there are
-multiple calls, but also provides less opportunity for context
-dependent optimization. When there is only one call, the result is
-identical to normal inline expansion. Semi-inline expansion is done
-when the \code{space} optimization quality is \code{0}, and the
-function has been declared \code{extensions:maybe-inline}.
-
-This mechanism of inline expansion combined with local call also
-allows recursive functions to be inline expanded. If a recursive
-function is declared \code{inline}, calls will actually be compiled
-semi-inline. Although recursive functions are often so complex that
-there is little advantage to semi-inline expansion, it can still be
-useful in the same sort of cases where normal inline expansion is
-especially advantageous, i.e. functions where the calling context can
-help a lot.
-
-%%\node The Maybe-Inline Declaration, , Semi-Inline Expansion, Inline Expansion
-\subsection{The Maybe-Inline Declaration}
-\label{maybe-inline-declaration}
-\cindex{maybe-inline declaration}
-
-The \code{extensions:maybe-inline} declaration is a \cmucl{}
-extension. It is similar to \code{inline}, but indicates that inline
-expansion may sometimes be desirable, rather than saying that inline
-expansion should almost always be done. When used in a global
-declaration, \code{extensions:maybe-inline} causes the expansion for
-the named functions to be recorded, but the functions aren't actually
-inline expanded unless \code{space} is \code{0} or the function is
-eventually (perhaps locally) declared \code{inline}.
-
-Use of the \code{extensions:maybe-inline} declaration followed by the
-\code{defun} is preferable to the standard idiom of:
-\begin{lisp}
-(proclaim '(inline myfun))
-(defun myfun () ...)
-(proclaim '(notinline myfun))
-
-;;; \i{Any calls to \code{myfun} here are not inline expanded.}
-
-(defun somefun ()
- (declare (inline myfun))
- ;;
- ;; \i{Calls to \code{myfun} here are inline expanded.}
- ...)
-\end{lisp}
-The problem with using \code{notinline} in this way is that in
-\clisp{} it does more than just suppress inline expansion, it also
-forbids the compiler to use any knowledge of \code{myfun} until a
-later \code{inline} declaration overrides the \code{notinline}. This
-prevents compiler warnings about incorrect calls to the function, and
-also prevents block compilation.
-
-The \code{extensions:maybe-inline} declaration is used like this:
-\begin{lisp}
-(proclaim '(extensions:maybe-inline myfun))
-(defun myfun () ...)
-
-;;; \i{Any calls to \code{myfun} here are not inline expanded.}
-
-(defun somefun ()
- (declare (inline myfun))
- ;;
- ;; \i{Calls to \code{myfun} here are inline expanded.}
- ...)
-
-(defun someotherfun ()
- (declare (optimize (space 0)))
- ;;
- ;; \i{Calls to \code{myfun} here are expanded semi-inline.}
- ...)
-\end{lisp}
-In this example, the use of \code{extensions:maybe-inline} causes the
-expansion to be recorded when the \code{defun} for \code{somefun} is
-compiled, and doesn't waste space through doing inline expansion by
-default. Unlike \code{notinline}, this declaration still allows the
-compiler to assume that the known definition really is the one that
-will be called when giving compiler warnings, and also allows the
-compiler to do semi-inline expansion when the policy is appropriate.
-
-When the goal is merely to control whether inline expansion is done by
-default, it is preferable to use \code{extensions:maybe-inline} rather
-than \code{notinline}. The \code{notinline} declaration should be
-reserved for those special occasions when a function may be redefined
-at run-time, so the compiler must be told that the obvious definition
-of a function is not necessarily the one that will be in effect at the
-time of the call.
-
-%%\f
-%%\node Byte Coded Compilation, Object Representation, Inline Expansion, Advanced Compiler Use and Efficiency Hints
-\section{Byte Coded Compilation}
-\label{byte-compile}
-\cindex{byte coded compilation}
-\cindex{space optimization}
-
-\Python{} supports byte compilation to reduce the size of Lisp
-programs by allowing functions to be compiled more compactly. Byte
-compilation provides an extreme speed/space tradeoff: byte code is
-typically six times more compact than native code, but runs fifty
-times (or more) slower. This is about ten times faster than the
-standard interpreter, which is itself considered fast in comparison to
-other \clisp{} interpreters.
-
-Large Lisp systems (such as \cmucl{} itself) often have large amounts
-of user-interface code, compile-time (macro) code, debugging code, or
-rarely executed special-case code. This code is a good target for
-byte compilation: very little time is spent running in it, but it can
-take up quite a bit of space. Straight-line code with many function
-calls is much more suitable than inner loops.
-
-When byte-compiling, the compiler compiles about twice as fast, and
-can produce a hardware independent object file (\file{.bytef} type.)
-This file can be loaded like a normal fasl file on any implementation
-of CMU CL with the same byte-ordering (DEC PMAX has \file{.lbytef}
-type.)
-
-The decision to byte compile or native compile can be done on a
-per-file or per-code-object basis. The \kwd{byte-compile} argument to
-\funref{compile-file} has these possible values:
-\begin{Lentry}
-\item[\false{}] Don't byte compile anything in this file.
-
-\item[\true{}] Byte compile everything in this file and produce a
- processor-independent \file{.bytef} file.
-
-\item[\kwd{maybe}] Produce a normal fasl file, but byte compile any
- functions for which the \code{speed} optimization quality is
- \code{0} and the \code{debug} quality is not greater than \code{1}.
-\end{Lentry}
-
-\begin{defvar}{extensions:}{byte-compile-top-level}
-
- If this variable is true (the default) and the \kwd{byte-compile}
- argument to \code{compile-file} is \kwd{maybe}, then byte compile
- top-level code (code outside of any \code{defun}, \code{defmethod},
- etc.)
-\end{defvar}
-
-\begin{defvar}{extensions:}{byte-compile-default}
-
- This variable determines the default value for the
- \kwd{byte-compile} argument to \code{compile-file}, initially
- \kwd{maybe}.
-\end{defvar}
-
-%%\f
-%%\node Object Representation, Numbers, Byte Coded Compilation, Advanced Compiler Use and Efficiency Hints
-\section{Object Representation}
-\label{object-representation}
-\cindex{object representation}
-\cpsubindex{representation}{object}
-\cpsubindex{efficiency}{of objects}
-
-A somewhat subtle aspect of writing efficient \clisp{} programs is
-choosing the correct data structures so that the underlying objects
-can be implemented efficiently. This is partly because of the need
-for multiple representations for a given value
-(\pxlref{non-descriptor}), but is also due to the sheer number of
-object types that \clisp{} has built in. The number of possible
-representations complicates the choice of a good representation
-because semantically similar objects may vary in their efficiency
-depending on how the program operates on them.
-
-\begin{comment}
-* Think Before You Use a List::
-* Structure Representation::
-* Arrays::
-* Vectors::
-* Bit-Vectors::
-* Hashtables::
-\end{comment}
-
-%%\node Think Before You Use a List, Structure Representation, Object Representation, Object Representation
-\subsection{Think Before You Use a List}
-\cpsubindex{lists}{efficiency of}
-
-Although Lisp's creator seemed to think that it was for LISt Processing, the
-astute observer may have noticed that the chapter on list manipulation makes up
-less that three percent of \i{Common Lisp: the Language II}. The language has
-grown since Lisp 1.5\dash{}new data types supersede lists for many purposes.
-
-%%\node Structure Representation, Arrays, Think Before You Use a List, Object Representation
-\subsection{Structure Representation}
-\cpsubindex{structure types}{efficiency of} One of the best ways of
-building complex data structures is to define appropriate structure
-types using \findexed{defstruct}. In \python, access of structure
-slots is always at least as fast as list or vector access, and is
-usually faster. In comparison to a list representation of a tuple,
-structures also have a space advantage.
-
-Even if structures weren't more efficient than other representations, structure
-use would still be attractive because programs that use structures in
-appropriate ways are much more maintainable and robust than programs written
-using only lists. For example:
-\begin{lisp}
-(rplaca (caddr (cadddr x)) (caddr y))
-\end{lisp}
-could have been written using structures in this way:
-\begin{lisp}
-(setf (beverage-flavor (astronaut-beverage x)) (beverage-flavor y))
-\end{lisp}
-The second version is more maintainable because it is easier to
-understand what it is doing. It is more robust because structures
-accesses are type checked. An \code{astronaut} will never be confused
-with a \code{beverage}, and the result of \code{beverage-flavor} is
-always a flavor. See sections \ref{structure-types} and
-\ref{freeze-type} for more information about structure types.
-\xlref{type-inference} for a number of examples that make clear the
-advantages of structure typing.
-
-Note that the structure definition should be compiled before any uses
-of its accessors or type predicate so that these function calls can be
-efficiently open-coded.
-
-%%\node Arrays, Vectors, Structure Representation, Object Representation
-\subsection{Arrays}
-\label{array-types}
-\cpsubindex{arrays}{efficiency of}
-
-Arrays are often the most efficient representation for collections of objects
-because:
-\begin{itemize}
-
-\item Array representations are often the most compact. An array is
- always more compact than a list containing the same number of
- elements.
-
-\item Arrays allow fast constant-time access.
-
-\item Arrays are easily destructively modified, which can reduce
- consing.
-
-\item Array element types can be specialized, which reduces both
- overall size and consing (\pxlref{specialized-array-types}.)
-\end{itemize}
-
-
-Access of arrays that are not of type \code{simple-array} is less
-efficient, so declarations are appropriate when an array is of a
-simple type like \code{simple-string} or \code{simple-bit-vector}.
-Arrays are almost always simple, but the compiler may not be able to
-prove simpleness at every use. The only way to get a non-simple array
-is to use the \kwd{displaced-to}, \kwd{fill-pointer} or
-\code{adjustable} arguments to \code{make-array}. If you don't use
-these hairy options, then arrays can always be declared to be simple.
-
-Because of the many specialized array types and the possibility of
-non-simple arrays, array access is much like generic arithmetic
-(\pxlref{generic-arithmetic}). In order for array accesses to be
-efficiently compiled, the element type and simpleness of the array
-must be known at compile time. If there is inadequate information,
-the compiler is forced to call a generic array access routine. You
-can detect inefficient array accesses by enabling efficiency notes,
-\pxlref{efficiency-notes}.
-
-%%\node Vectors, Bit-Vectors, Arrays, Object Representation
-\subsection{Vectors}
-\cpsubindex{vectors}{efficiency of}
-
-Vectors (one dimensional arrays) are particularly useful, since in
-addition to their obvious array-like applications, they are also well
-suited to representing sequences. In comparison to a list
-representation, vectors are faster to access and take up between two
-and sixty-four times less space (depending on the element type.) As
-with arbitrary arrays, the compiler needs to know that vectors are not
-complex, so you should use \code{simple-string} in preference to
-\code{string}, etc.
-
-The only advantage that lists have over vectors for representing
-sequences is that it is easy to change the length of a list, add to it
-and remove items from it. Likely signs of archaic, slow lisp code are
-\code{nth} and \code{nthcdr}. If you are using these functions you
-should probably be using a vector.
-
-%%\node Bit-Vectors, Hashtables, Vectors, Object Representation
-\subsection{Bit-Vectors}
-\cpsubindex{bit-vectors}{efficiency of}
-
-Another thing that lists have been used for is set manipulation. In
-applications where there is a known, reasonably small universe of
-items bit-vectors can be used to improve performance. This is much
-less convenient than using lists, because instead of symbols, each
-element in the universe must be assigned a numeric index into the bit
-vector. Using a bit-vector will nearly always be faster, and can be
-tremendously faster if the number of elements in the set is not small.
-The logical operations on \code{simple-bit-vector}s are efficient,
-since they operate on a word at a time.
-
-
-%%\node Hashtables, , Bit-Vectors, Object Representation
-\subsection{Hashtables}
-\cpsubindex{hash-tables}{efficiency of}
-
-Hashtables are an efficient and general mechanism for maintaining associations
-such as the association between an object and its name. Although hashtables
-are usually the best way to maintain associations, efficiency and style
-considerations sometimes favor the use of an association list (a-list).
-
-\code{assoc} is fairly fast when the \var{test} argument is \code{eq}
-or \code{eql} and there are only a few elements, but the time goes up
-in proportion with the number of elements. In contrast, the
-hash-table lookup has a somewhat higher overhead, but the speed is
-largely unaffected by the number of entries in the table. For an
-\code{equal} hash-table or alist, hash-tables have an even greater
-advantage, since the test is more expensive. Whatever you do, be sure
-to use the most restrictive test function possible.
-
-The style argument observes that although hash-tables and alists
-overlap in function, they do not do all things equally well.
-\begin{itemize}
-
-\item Alists are good for maintaining scoped environments. They were
- originally invented to implement scoping in the Lisp interpreter,
- and are still used for this in \python. With an alist one can
- non-destructively change an association simply by consing a new
- element on the front. This is something that cannot be done with
- hash-tables.
-
-\item Hashtables are good for maintaining a global association. The
- value associated with an entry can easily be changed with
- \code{setf}. With an alist, one has to go through contortions,
- either \code{rplacd}'ing the cons if the entry exists, or pushing a
- new one if it doesn't. The side-effecting nature of hash-table
- operations is an advantage here.
-\end{itemize}
-
-
-Historically, symbol property lists were often used for global name
-associations. Property lists provide an awkward and error-prone
-combination of name association and record structure. If you must use
-the property list, please store all the related values in a single
-structure under a single property, rather than using many properties.
-This makes access more efficient, and also adds a modicum of typing
-and abstraction. \xlref{advanced-type-stuff} for information on types
-in \cmucl.
-
-%%\f
-%%\node Numbers, General Efficiency Hints, Object Representation, Advanced Compiler Use and Efficiency Hints
-\section{Numbers}
-\label{numeric-types}
-\cpsubindex{numeric}{types}
-\cpsubindex{types}{numeric}
-
-Numbers are interesting because numbers are one of the few \llisp{} data types
-that have direct support in conventional hardware. If a number can be
-represented in the way that the hardware expects it, then there is a big
-efficiency advantage.
-
-Using hardware representations is problematical in \llisp{} due to
-dynamic typing (where the type of a value may be unknown at compile
-time.) It is possible to compile code for statically typed portions
-of a \llisp{} program with efficiency comparable to that obtained in
-statically typed languages such as C, but not all \llisp{}
-implementations succeed. There are two main barriers to efficient
-numerical code in \llisp{}:
-\begin{itemize}
-
-\item The compiler must prove that the numerical expression is in fact
- statically typed, and
-
-\item The compiler must be able to somehow reconcile the conflicting
- demands of the hardware mandated number representation with the
- \llisp{} requirements of dynamic typing and garbage-collecting
- dynamic storage allocation.
-\end{itemize}
-
-Because of its type inference (\pxlref{type-inference}) and efficiency
-notes (\pxlref{efficiency-notes}), \python{} is better than
-conventional \llisp{} compilers at ensuring that numerical expressions
-are statically typed. Python also goes somewhat farther than existing
-compilers in the area of allowing native machine number
-representations in the presence of garbage collection.
-
-\begin{comment}
-* Descriptors::
-* Non-Descriptor Representations::
-* Variables::
-* Generic Arithmetic::
-* Fixnums::
-* Word Integers::
-* Floating Point Efficiency::
-* Specialized Arrays::
-* Specialized Structure Slots::
-* Interactions With Local Call::
-* Representation of Characters::
-\end{comment}
-
-%%\node Descriptors, Non-Descriptor Representations, Numbers, Numbers
-\subsection{Descriptors}
-\cpsubindex{descriptors}{object}
-\cindex{object representation}
-\cpsubindex{representation}{object}
-\cpsubindex{consing}{overhead of}
-
-\llisp{}'s dynamic typing requires that it be possible to represent
-any value with a fixed length object, known as a \var{descriptor}.
-This fixed-length requirement is implicit in features such as:
-\begin{itemize}
-
-\item Data types (like \code{simple-vector}) that can contain any type
- of object, and that can be destructively modified to contain
- different objects (of possibly different types.)
-
-\item Functions that can be called with any type of argument, and that
- can be redefined at run time.
-\end{itemize}
-
-In order to save space, a descriptor is invariably represented as a
-single word. Objects that can be directly represented in the
-descriptor itself are said to be \var{immediate}. Descriptors for
-objects larger than one word are in reality pointers to the memory
-actually containing the object.
-
-Representing objects using pointers has two major disadvantages:
-\begin{itemize}
-
-\item The memory pointed to must be allocated on the heap, so it must
- eventually be freed by the garbage collector. Excessive heap
- allocation of objects (or ``consing'') is inefficient in several
- ways. \xlref{consing}.
-
-\item Representing an object in memory requires the compiler to emit
- additional instructions to read the actual value in from memory, and
- then to write the value back after operating on it.
-\end{itemize}
-
-The introduction of garbage collection makes things even worse, since
-the garbage collector must be able to determine whether a descriptor
-is an immediate object or a pointer. This requires that a few bits in
-each descriptor be dedicated to the garbage collector. The loss of a
-few bits doesn't seem like much, but it has a major efficiency
-implication\dash{}objects whose natural machine representation is a
-full word (integers and single-floats) cannot have an immediate
-representation. So the compiler is forced to use an unnatural
-immediate representation (such as \code{fixnum}) or a natural pointer
-representation (with the attendant consing overhead.)
-
-
-%%\node Non-Descriptor Representations, Variables, Descriptors, Numbers
-\subsection{Non-Descriptor Representations}
-\label{non-descriptor}
-\cindex{non-descriptor representations}
-\cindex{stack numbers}
-
-From the discussion above, we can see that the standard descriptor
-representation has many problems, the worst being number consing.
-\llisp{} compilers try to avoid these descriptor efficiency problems by using
-\var{non-descriptor} representations. A compiler that uses non-descriptor
-representations can compile this function so that it does no number consing:
-\begin{lisp}
-(defun multby (vec n)
- (declare (type (simple-array single-float (*)) vec)
- (single-float n))
- (dotimes (i (length vec))
- (setf (aref vec i)
- (* n (aref vec i)))))
-\end{lisp}
-If a descriptor representation were used, each iteration of the loop might
-cons two floats and do three times as many memory references.
-
-As its negative definition suggests, the range of possible non-descriptor
-representations is large. The performance improvement from non-descriptor
-representation depends upon both the number of types that have non-descriptor
-representations and the number of contexts in which the compiler is forced to
-use a descriptor representation.
-
-Many \llisp{} compilers support non-descriptor representations for
-float types such as \code{single-float} and \code{double-float}
-(section \ref{float-efficiency}.) \python{} adds support for full
-word integers (\pxlref{word-integers}), characters
-(\pxlref{characters}) and system-area pointers (unconstrained
-pointers, \pxlref{system-area-pointers}.) Many \llisp{} compilers
-support non-descriptor representations for variables (section
-\ref{ND-variables}) and array elements (section
-\ref{specialized-array-types}.) \python{} adds support for
-non-descriptor arguments and return values in local call
-(\pxlref{number-local-call}) and structure slots (\pxlref{raw-slots}).
-
-%%\node Variables, Generic Arithmetic, Non-Descriptor Representations, Numbers
-\subsection{Variables}
-\label{ND-variables}
-\cpsubindex{variables}{non-descriptor}
-\cpsubindex{type declarations}{variable}
-\cpsubindex{efficiency}{of numeric variables}
-
-In order to use a non-descriptor representation for a variable or
-expression intermediate value, the compiler must be able to prove that
-the value is always of a particular type having a non-descriptor
-representation. Type inference (\pxlref{type-inference}) often needs
-some help from user-supplied declarations. The best kind of type
-declaration is a variable type declaration placed at the binding
-point:
-\begin{lisp}
-(let ((x (car l)))
- (declare (single-float x))
- ...)
-\end{lisp}
-Use of \code{the}, or of variable declarations not at the binding form
-is insufficient to allow non-descriptor representation of the
-variable\dash{}with these declarations it is not certain that all
-values of the variable are of the right type. It is sometimes useful
-to introduce a gratuitous binding that allows the compiler to change
-to a non-descriptor representation, like:
-\begin{lisp}
-(etypecase x
- ((signed-byte 32)
- (let ((x x))
- (declare (type (signed-byte 32) x))
- ...))
- ...)
-\end{lisp}
-The declaration on the inner \code{x} is necessary here due to a phase
-ordering problem. Although the compiler will eventually prove that
-the outer \code{x} is a \w{\code{(signed-byte 32)}} within that
-\code{etypecase} branch, the inner \code{x} would have been optimized
-away by that time. Declaring the type makes let optimization more
-cautious.
-
-Note that storing a value into a global (or \code{special}) variable
-always forces a descriptor representation. Wherever possible, you
-should operate only on local variables, binding any referenced globals
-to local variables at the beginning of the function, and doing any
-global assignments at the end.
-
-Efficiency notes signal use of inefficient representations, so
-programmer's needn't continuously worry about the details of
-representation selection (\pxlref{representation-eff-note}.)
-
-%%\node Generic Arithmetic, Fixnums, Variables, Numbers
-\subsection{Generic Arithmetic}
-\label{generic-arithmetic}
-\cindex{generic arithmetic}
-\cpsubindex{arithmetic}{generic}
-\cpsubindex{numeric}{operation efficiency}
-
-In \clisp, arithmetic operations are \var{generic}.\footnote{As Steele
- notes in CLTL II, this is a generic conception of generic, and is
- not to be confused with the CLOS concept of a generic function.}
-The \code{+} function can be passed \code{fixnum}s, \code{bignum}s,
-\code{ratio}s, and various kinds of \code{float}s and
-\code{complex}es, in any combination. In addition to the inherent
-complexity of \code{bignum} and \code{ratio} operations, there is also
-a lot of overhead in just figuring out which operation to do and what
-contagion and canonicalization rules apply. The complexity of generic
-arithmetic is so great that it is inconceivable to open code it.
-Instead, the compiler does a function call to a generic arithmetic
-routine, consuming many instructions before the actual computation
-even starts.
-
-This is ridiculous, since even \llisp{} programs do a lot of
-arithmetic, and the hardware is capable of doing operations on small
-integers and floats with a single instruction. To get acceptable
-efficiency, the compiler special-cases uses of generic arithmetic that
-are directly implemented in the hardware. In order to open code
-arithmetic, several constraints must be met:
-\begin{itemize}
-
-\item All the arguments must be known to be a good type of number.
-
-\item The result must be known to be a good type of number.
-
-\item Any intermediate values such as the result of \w{\code{(+ a b)}}
- in the call \w{\code{(+ a b c)}} must be known to be a good type of
- number.
-
-\item All the above numbers with good types must be of the \var{same}
- good type. Don't try to mix integers and floats or different float
- formats.
-\end{itemize}
-
-The ``good types'' are \w{\code{(signed-byte 32)}},
-\w{\code{(unsigned-byte 32)}}, \code{single-float} and
-\code{double-float}. See sections \ref{fixnums}, \ref{word-integers}
-and \ref{float-efficiency} for more discussion of good numeric types.
-
-\code{float} is not a good type, since it might mean either
-\code{single-float} or \code{double-float}. \code{integer} is not a
-good type, since it might mean \code{bignum}. \code{rational} is not
-a good type, since it might mean \code{ratio}. Note however that
-these types are still useful in declarations, since type inference may
-be able to strengthen a weak declaration into a good one, when it
-would be at a loss if there was no declaration at all
-(\pxlref{type-inference}). The \code{integer} and
-\code{unsigned-byte} (or non-negative integer) types are especially
-useful in this regard, since they can often be strengthened to a good
-integer type.
-
-Arithmetic with \code{complex} numbers is inefficient in comparison to
-float and integer arithmetic. Complex numbers are always represented
-with a pointer descriptor (causing consing overhead), and complex
-arithmetic is always closed coded using the general generic arithmetic
-functions. But arithmetic with complex types such as:
-\begin{lisp}
-(complex float)
-(complex fixnum)
-\end{lisp}
-is still faster than \code{bignum} or \code{ratio} arithmetic, since the
-implementation is much simpler.
-
-Note: don't use \code{/} to divide integers unless you want the
-overhead of rational arithmetic. Use \code{truncate} even when you
-know that the arguments divide evenly.
-
-You don't need to remember all the rules for how to get open-coded
-arithmetic, since efficiency notes will tell you when and where there
-is a problem\dash{}\pxlref{efficiency-notes}.
-
-
-%%\node Fixnums, Word Integers, Generic Arithmetic, Numbers
-\subsection{Fixnums}
-\label{fixnums}
-\cindex{fixnums}
-\cindex{bignums}
-
-A fixnum is a ``FIXed precision NUMber''. In modern \llisp{}
-implementations, fixnums can be represented with an immediate
-descriptor, so operating on fixnums requires no consing or memory
-references. Clever choice of representations also allows some
-arithmetic operations to be done on fixnums using hardware supported
-word-integer instructions, somewhat reducing the speed penalty for
-using an unnatural integer representation.
-
-It is useful to distinguish the \code{fixnum} type from the fixnum
-representation of integers. In \python, there is absolutely nothing
-magical about the \code{fixnum} type in comparison to other finite
-integer types. \code{fixnum} is equivalent to (is defined with
-\code{deftype} to be) \w{\code{(signed-byte 30)}}. \code{fixnum} is
-simply the largest subset of integers that \i{can be represented}
-using an immediate fixnum descriptor.
-
-Unlike in other \clisp{} compilers, it is in no way desirable to use
-the \code{fixnum} type in declarations in preference to more
-restrictive integer types such as \code{bit}, \w{\code{(integer -43
- 7)}} and \w{\code{(unsigned-byte 8)}}. Since Python does
-understand these integer types, it is preferable to use the more
-restrictive type, as it allows better type inference
-(\pxlref{operation-type-inference}.)
-
-The small, efficient fixnum is contrasted with bignum, or ``BIG
-NUMber''. This is another descriptor representation for integers, but
-this time a pointer representation that allows for arbitrarily large
-integers. Bignum operations are less efficient than fixnum
-operations, both because of the consing and memory reference overheads
-of a pointer descriptor, and also because of the inherent complexity
-of extended precision arithmetic. While fixnum operations can often
-be done with a single instruction, bignum operations are so complex
-that they are always done using generic arithmetic.
-
-A crucial point is that the compiler will use generic arithmetic if it
-can't \var{prove} that all the arguments, intermediate values, and
-results are fixnums. With bounded integer types such as
-\code{fixnum}, the result type proves to be especially problematical,
-since these types are not closed under common arithmetic operations
-such as \code{+}, \code{-}, \code{*} and \code{/}. For example,
-\w{\code{(1+ (the fixnum x))}} does not necessarily evaluate to a
-\code{fixnum}. Bignums were added to \llisp{} to get around this
-problem, but they really just transform the correctness problem ``if
-this add overflows, you will get the wrong answer'' to the efficiency
-problem ``if this add \var{might} overflow then your program will run
-slowly (because of generic arithmetic.)''
-
-There is just no getting around the fact that the hardware only
-directly supports short integers. To get the most efficient open
-coding, the compiler must be able to prove that the result is a good
-integer type. This is an argument in favor of using more restrictive
-integer types: \w{\code{(1+ (the fixnum x))}} may not always be a
-\code{fixnum}, but \w{\code{(1+ (the (unsigned-byte 8) x))}} always
-is. Of course, you can also assert the result type by putting in lots
-of \code{the} declarations and then compiling with \code{safety}
-\code{0}.
-
-%%\node Word Integers, Floating Point Efficiency, Fixnums, Numbers
-\subsection{Word Integers}
-\label{word-integers}
-\cindex{word integers}
-
-Python is unique in its efficient implementation of arithmetic
-on full-word integers through non-descriptor representations and open coding.
-Arithmetic on any subtype of these types:
-\begin{lisp}
-(signed-byte 32)
-(unsigned-byte 32)
-\end{lisp}
-is reasonably efficient, although subtypes of \code{fixnum} remain
-somewhat more efficient.
-
-If a word integer must be represented as a descriptor, then the
-\code{bignum} representation is used, with its associated consing
-overhead. The support for word integers in no way changes the
-language semantics, it just makes arithmetic on small bignums vastly
-more efficient. It is fine to do arithmetic operations with mixed
-\code{fixnum} and word integer operands; just declare the most
-specific integer type you can, and let the compiler decide what
-representation to use.
-
-In fact, to most users, the greatest advantage of word integer
-arithmetic is that it effectively provides a few guard bits on the
-fixnum representation. If there are missing assertions on
-intermediate values in a fixnum expression, the intermediate results
-can usually be proved to fit in a word. After the whole expression is
-evaluated, there will often be a fixnum assertion on the final result,
-allowing creation of a fixnum result without even checking for
-overflow.
-
-The remarks in section \ref{fixnums} about fixnum result type also
-apply to word integers; you must be careful to give the compiler
-enough information to prove that the result is still a word integer.
-This time, though, when we blow out of word integers we land in into
-generic bignum arithmetic, which is much worse than sleazing from
-\code{fixnum}s to word integers. Note that mixing
-\w{\code{(unsigned-byte 32)}} arguments with arguments of any signed
-type (such as \code{fixnum}) is a no-no, since the result might not be
-unsigned.
-
-%%\node Floating Point Efficiency, Specialized Arrays, Word Integers, Numbers
-\subsection{Floating Point Efficiency}
-\label{float-efficiency}
-\cindex{floating point efficiency}
-
-Arithmetic on objects of type \code{single-float} and \code{double-float} is
-efficiently implemented using non-descriptor representations and open coding.
-As for integer arithmetic, the arguments must be known to be of the same float
-type. Unlike for integer arithmetic, the results and intermediate values
-usually take care of themselves due to the rules of float contagion, i.e.
-\w{\code{(1+ (the single-float x))}} is always a \code{single-float}.
-
-Although they are not specially implemented, \code{short-float} and
-\code{long-float} are also acceptable in declarations, since they are
-synonyms for the \code{single-float} and \code{double-float} types,
-respectively.
-
-\begin{changebar}
- Some versions of CMU Common Lisp include extra support for floating
- point arithmetic. In particular, if \code{*features*} includes
- \kwd{propagate-float-type}, list-style float type specifiers such as
- \w{\code{(single-float 0.0 1.0)}} will be used to good effect.
-
- For example, in this function,
- \begin{example}
- (defun square (x)
- (declare (type (single-float 0f0 10f0)))
- (* x x))
- \end{example}
- \Python{} can deduce that the
- return type of the function \code{square} is \w{\code{(single-float
- 0f0 100f0)}}.
-
- Many union types are also supported so that
- \begin{example}
- (+ (the (or (integer 1 1) (integer 5 5)) x)
- (the (or (integer 10 10) (integer 20 20)) y))
- \end{example}
- has the inferred type \code{(or (integer 11 11) (integer 15 15)
- (integer 21 21) (integer 25 25))}. This also works for
- floating-point numbers. Member types, however, are not because in
- general the member elements do not have to be numbers. Thus,
- instead of \code{(member 1 4)}, you should write \code{(or (integer
- 1 1) (integer 4 4))}.
-
- In addition, if \kwd{propagate-fun-type} is in \code{*features*},
- \Python{} knows how to infer types for many mathematical functions
- including square root, exponential and logarithmic functions,
- trignometric functions and their inverses, and hyperbolic functions
- and their inverses. For numeric code, this can greatly enhance
- efficiency by allowing the compiler to use specialized versions of
- the functions instead of the generic versions. The greatest benefit
- of this type inference is determining that the result of the
- function is real-valued number instead of possibly being
- a complex-valued number.
-
- For example, consider the function
- \begin{example}
- (defun fun (x)
- (declare (type (single-float 0f0 100f0) x))
- (values (sqrt x) (log x 10f0)))
- \end{example}
- With this declaration, the compiler can determine that the argument
- to \code{sqrt} and \code{log} are always non-negative so that the result
- is always a \code{single-float}. In fact, the return type for this
- function is derived to be \code{(values (single-float 0f0 10f0)
- (single-float * 2f0))}.
-
- If the declaration were reduced to just \w{\code{(declare
- single-float x)}}, the argument to \code{sqrt} and \code{log}
- could be negative. This forces the use of the generic versions of
- these functions because the result could be a complex number.
-
- Union types are not yet supported for functions.
-
- We note, however, that proper interval arithmetic is not fully
- implemented in the compiler so the inferred types may be slightly in
- error due to round-off errors. This round-off error could
- accumulate to cause the compiler to erroneously deduce the result
- type and cause code to be removed as being
- unreachable.\footnote{This, however, has not actually happened, but
- it is a possibility.}%
- Thus, the declarations should only be precise enough for the
- compiler to deduce that a real-valued argument to a function would
- produce a real-valued result. The efficiency notes
- (\pxlref{representation-eff-note}) from the compiler will guide you
- on what declarations might be useful.
-\end{changebar}
-
-When a float must be represented as a descriptor, a pointer representation is
-used, creating consing overhead. For this reason, you should try to avoid
-situations (such as full call and non-specialized data structures) that force a
-descriptor representation. See sections \ref{specialized-array-types},
-\ref{raw-slots} and \ref{number-local-call}.
-
-\xlref{ieee-float} for information on the extensions to support IEEE
-floating point.
-
-%%\node Specialized Arrays, Specialized Structure Slots, Floating Point Efficiency, Numbers
-\subsection{Specialized Arrays}
-\label{specialized-array-types}
-\cindex{specialized array types}
-\cpsubindex{array types}{specialized}
-\cpsubindex{types}{specialized array}
-
-\clisp{} supports specialized array element types through the
-\kwd{element-type} argument to \code{make-array}. When an array has a
-specialized element type, only elements of that type can be stored in
-the array. From this restriction comes two major efficiency
-advantages:
-\begin{itemize}
-
-\item A specialized array can save space by packing multiple elements
- into a single word. For example, a \code{base-char} array can have
- 4 elements per word, and a \code{bit} array can have 32. This
- space-efficient representation is possible because it is not
- necessary to separately indicate the type of each element.
-
-\item The elements in a specialized array can be given the same
- non-descriptor representation as the one used in registers and on
- the stack, eliminating the need for representation conversions when
- reading and writing array elements. For objects with pointer
- descriptor representations (such as floats and word integers) there
- is also a substantial consing reduction because it is not necessary
- to allocate a new object every time an array element is modified.
-\end{itemize}
-
-
-These are the specialized element types currently supported:
-\begin{lisp}
-bit
-(unsigned-byte 2)
-(unsigned-byte 4)
-(unsigned-byte 8)
-(unsigned-byte 16)
-(unsigned-byte 32)
-base-character
-single-float
-double-float
-\end{lisp}
-\begin{changebar}
-%% New stuff
-Some versions of \cmucl{}\footnote{Currently, this includes the X86
- and Sparc versions which are compiled with the \kwd{signed-array}
- feature.} also support the following specialized element types:
-\begin{lisp}
-(signed-byte 8)
-(signed-byte 16)
-(signed-byte 30)
-(signed-byte 32)
-\end{lisp}
-\end{changebar}
-Although a \code{simple-vector} can hold any type of object, \true{}
-should still be considered a specialized array type, since arrays with
-element type \true{} are specialized to hold descriptors.
-
-
-
-When using non-descriptor representations, it is particularly
-important to make sure that array accesses are open-coded, since in
-addition to the generic operation overhead, efficiency is lost when
-the array element is converted to a descriptor so that it can be
-passed to (or from) the generic access routine. You can detect
-inefficient array accesses by enabling efficiency notes,
-\pxlref{efficiency-notes}. \xlref{array-types}.
-
-%%\node Specialized Structure Slots, Interactions With Local Call, Specialized Arrays, Numbers
-\subsection{Specialized Structure Slots}
-\label{raw-slots}
-\cpsubindex{structure types}{numeric slots}
-\cindex{specialized structure slots}
-
-Structure slots declared by the \kwd{type} \code{defstruct} slot option
-to have certain known numeric types are also given non-descriptor
-representations. These types (and subtypes of these types) are supported:
-\begin{lisp}
-(unsigned-byte 32)
-single-float
-double-float
-\end{lisp}
-
-The primary advantage of specialized slot representations is a large
-reduction spurious memory allocation and access overhead of programs
-that intensively use these types.
-
-%%\node Interactions With Local Call, Representation of Characters, Specialized Structure Slots, Numbers
-\subsection{Interactions With Local Call}
-\label{number-local-call}
-\cpsubindex{local call}{numeric operands}
-\cpsubindex{call}{numeric operands}
-\cindex{numbers in local call}
-
-Local call has many advantages (\pxlref{local-call}); one relevant to
-our discussion here is that local call extends the usefulness of
-non-descriptor representations. If the compiler knows from the
-argument type that an argument has a non-descriptor representation,
-then the argument will be passed in that representation. The easiest
-way to ensure that the argument type is known at compile time is to
-always declare the argument type in the called function, like:
-\begin{lisp}
-(defun 2+f (x)
- (declare (single-float x))
- (+ x 2.0))
-\end{lisp}
-The advantages of passing arguments and return values in a non-descriptor
-representation are the same as for non-descriptor representations in general:
-reduced consing and memory access (\pxlref{non-descriptor}.) This
-extends the applicative programming styles discussed in section
-\ref{local-call} to numeric code. Also, if source files are kept reasonably
-small, block compilation can be used to reduce number consing to a minimum.
-
-Note that non-descriptor return values can only be used with the known return
-convention (section \ref{local-call-return}.) If the compiler can't prove that
-a function always returns the same number of values, then it must use the
-unknown values return convention, which requires a descriptor representation.
-Pay attention to the known return efficiency notes to avoid number consing.
-
-%%\node Representation of Characters, , Interactions With Local Call, Numbers
-\subsection{Representation of Characters}
-\label{characters}
-\cindex{characters}
-\cindex{strings}
-
-Python also uses a non-descriptor representation for characters when
-convenient. This improves the efficiency of string manipulation, but is
-otherwise pretty invisible; characters have an immediate descriptor
-representation, so there is not a great penalty for converting a character to a
-descriptor. Nonetheless, it may sometimes be helpful to declare
-character-valued variables as \code{base-character}.
-
-%%\f
-%%\node General Efficiency Hints, Efficiency Notes, Numbers, Advanced Compiler Use and Efficiency Hints
-\section{General Efficiency Hints}
-\label{general-efficiency}
-\cpsubindex{efficiency}{general hints}
-
-This section is a summary of various implementation costs and ways to get
-around them. These hints are relatively unrelated to the use of the \python{}
-compiler, and probably also apply to most other \llisp{} implementations. In
-each section, there are references to related in-depth discussion.
-
-\begin{comment}
-* Compile Your Code::
-* Avoid Unnecessary Consing::
-* Complex Argument Syntax::
-* Mapping and Iteration::
-* Trace Files and Disassembly::
-\end{comment}
-
-%%\node Compile Your Code, Avoid Unnecessary Consing, General Efficiency Hints, General Efficiency Hints
-\subsection{Compile Your Code}
-\cpsubindex{compilation}{why to}
-
-At this point, the advantages of compiling code relative to running it
-interpreted probably need not be emphasized too much, but remember that
-in \cmucl, compiled code typically runs hundreds of times faster than
-interpreted code. Also, compiled (\code{fasl}) files load significantly faster
-than source files, so it is worthwhile compiling files which are loaded many
-times, even if the speed of the functions in the file is unimportant.
-
-Even disregarding the efficiency advantages, compiled code is as good or better
-than interpreted code. Compiled code can be debugged at the source level (see
-chapter \ref{debugger}), and compiled code does more error checking. For these
-reasons, the interpreter should be regarded mainly as an interactive command
-interpreter, rather than as a programming language implementation.
-
-\b{Do not} be concerned about the performance of your program until you
-see its speed compiled. Some techniques that make compiled code run
-faster make interpreted code run slower.
-
-%%\node Avoid Unnecessary Consing, Complex Argument Syntax, Compile Your Code, General Efficiency Hints
-\subsection{Avoid Unnecessary Consing}
-\label{consing}
-\cindex{consing}
-\cindex{garbage collection}
-\cindex{memory allocation}
-\cpsubindex{efficiency}{of memory use}
-
-
-Consing is another name for allocation of storage, as done by the
-\code{cons} function (hence its name.) \code{cons} is by no means the
-only function which conses\dash{}so does \code{make-array} and many
-other functions. Arithmetic and function call can also have hidden
-consing overheads. Consing hurts performance in the following ways:
-\begin{itemize}
-
-\item Consing reduces memory access locality, increasing paging
- activity.
-
-\item Consing takes time just like anything else.
-
-\item Any space allocated eventually needs to be reclaimed, either by
- garbage collection or by starting a new \code{lisp} process.
-\end{itemize}
-
-
-Consing is not undiluted evil, since programs do things other than
-consing, and appropriate consing can speed up the real work. It would
-certainly save time to allocate a vector of intermediate results that
-are reused hundreds of times. Also, if it is necessary to copy a
-large data structure many times, it may be more efficient to update
-the data structure non-destructively; this somewhat increases update
-overhead, but makes copying trivial.
-
-Note that the remarks in section \ref{efficiency-overview} about the
-importance of separating tuning from coding also apply to consing
-overhead. The majority of consing will be done by a small portion of
-the program. The consing hot spots are even less predictable than the
-CPU hot spots, so don't waste time and create bugs by doing
-unnecessary consing optimization. During initial coding, avoid
-unnecessary side-effects and cons where it is convenient. If
-profiling reveals a consing problem, \var{then} go back and fix the
-hot spots.
-
-\xlref{non-descriptor} for a discussion of how to avoid number consing
-in \python.
-
-
-%%\node Complex Argument Syntax, Mapping and Iteration, Avoid Unnecessary Consing, General Efficiency Hints
-\subsection{Complex Argument Syntax}
-\cpsubindex{argument syntax}{efficiency}
-\cpsubindex{efficiency}{of argument syntax}
-\cindex{keyword argument efficiency}
-\cindex{rest argument efficiency}
-
-Common Lisp has very powerful argument passing mechanisms. Unfortunately, two
-of the most powerful mechanisms, rest arguments and keyword arguments, have a
-significant performance penalty:
-\begin{itemize}
-
-\item
-With keyword arguments, the called function has to parse the supplied keywords
-by iterating over them and checking them against the desired keywords.
-
-\item
-With rest arguments, the function must cons a list to hold the arguments. If a
-function is called many times or with many arguments, large amounts of memory
-will be allocated.
-\end{itemize}
-
-Although rest argument consing is worse than keyword parsing, neither problem
-is serious unless thousands of calls are made to such a function. The use of
-keyword arguments is strongly encouraged in functions with many arguments or
-with interfaces that are likely to be extended, and rest arguments are often
-natural in user interface functions.
-
-Optional arguments have some efficiency advantage over keyword
-arguments, but their syntactic clumsiness and lack of extensibility
-has caused many \clisp{} programmers to abandon use of optionals
-except in functions that have obviously simple and immutable
-interfaces (such as \code{subseq}), or in functions that are only
-called in a few places. When defining an interface function to be
-used by other programmers or users, use of only required and keyword
-arguments is recommended.
-
-Parsing of \code{defmacro} keyword and rest arguments is done at
-compile time, so a macro can be used to provide a convenient syntax
-with an efficient implementation. If the macro-expanded form contains
-no keyword or rest arguments, then it is perfectly acceptable in inner
-loops.
-
-Keyword argument parsing overhead can also be avoided by use of inline
-expansion (\pxlref{inline-expansion}) and block compilation (section
-\ref{block-compilation}.)
-
-Note: the compiler open-codes most heavily used system functions which have
-keyword or rest arguments, so that no run-time overhead is involved.
-
-%%\node Mapping and Iteration, Trace Files and Disassembly, Complex Argument Syntax, General Efficiency Hints
-\subsection{Mapping and Iteration}
-\cpsubindex{mapping}{efficiency of}
-
-One of the traditional \llisp{} programming styles is a highly applicative one,
-involving the use of mapping functions and many lists to store intermediate
-results. To compute the sum of the square-roots of a list of numbers, one
-might say:
-\begin{lisp}
-(apply #'+ (mapcar #'sqrt list-of-numbers))
-\end{lisp}
-
-This programming style is clear and elegant, but unfortunately results
-in slow code. There are two reasons why:
-\begin{itemize}
-
-\item The creation of lists of intermediate results causes much
- consing (see \ref{consing}).
-
-\item Each level of application requires another scan down the list.
- Thus, disregarding other effects, the above code would probably take
- twice as long as a straightforward iterative version.
-\end{itemize}
-
-
-An example of an iterative version of the same code:
-\begin{lisp}
-(do ((num list-of-numbers (cdr num))
- (sum 0 (+ (sqrt (car num)) sum)))
- ((null num) sum))
-\end{lisp}
-
-See sections \ref{variable-type-inference} and \ref{let-optimization}
-for a discussion of the interactions of iteration constructs with type
-inference and variable optimization. Also, section
-\ref{local-tail-recursion} discusses an applicative style of
-iteration.
-
-%%\node Trace Files and Disassembly, , Mapping and Iteration, General Efficiency Hints
-\subsection{Trace Files and Disassembly}
-\label{trace-files}
-\cindex{trace files}
-\cindex{assembly listing}
-\cpsubindex{listing files}{trace}
-\cindex{Virtual Machine (VM, or IR2) representation}
-\cindex{implicit continuation representation (IR1)}
-\cpsubindex{continuations}{implicit representation}
-
-In order to write efficient code, you need to know the relative costs
-of different operations. The main reason why writing efficient
-\llisp{} code is difficult is that there are so many operations, and
-the costs of these operations vary in obscure context-dependent ways.
-Although efficiency notes point out some problem areas, the only way
-to ensure generation of the best code is to look at the assembly code
-output.
-
-The \code{disassemble} function is a convenient way to get the assembly code for a
-function, but it can be very difficult to interpret, since the correspondence
-with the original source code is weak. A better (but more awkward) option is
-to use the \kwd{trace-file} argument to \code{compile-file} to generate a trace
-file.
-
-A trace file is a dump of the compiler's internal representations,
-including annotated assembly code. Each component in the program gets
-four pages in the trace file (separated by ``\code{$\hat{ }L$}''):
-\begin{itemize}
-
-\item The implicit-continuation (or IR1) representation of the
- optimized source. This is a dump of the flow graph representation
- used for ``source level'' optimizations. As you will quickly
- notice, it is not really very close to the source. This
- representation is not very useful to even sophisticated users.
-
-\item The Virtual Machine (VM, or IR2) representation of the program.
- This dump represents the generated code as sequences of ``Virtual
- OPerations'' (VOPs.) This representation is intermediate between
- the source and the assembly code\dash{}each VOP corresponds fairly
- directly to some primitive function or construct, but a given VOP
- also has a fairly predictable instruction sequence. An operation
- (such as \code{+}) may have multiple implementations with different
- cost and applicability. The choice of a particular VOP such as
- \code{+/fixnum} or \code{+/single-float} represents this choice of
- implementation. Once you are familiar with it, the VM
- representation is probably the most useful for determining what
- implementation has been used.
-
-\item An assembly listing, annotated with the VOP responsible for
- generating the instructions. This listing is useful for figuring
- out what a VOP does and how it is implemented in a particular
- context, but its large size makes it more difficult to read.
-
-\item A disassembly of the generated code, which has all
- pseudo-operations expanded out, but is not annotated with VOPs.
-\end{itemize}
-
-
-Note that trace file generation takes much space and time, since the trace file
-is tens of times larger than the source file. To avoid huge confusing trace
-files and much wasted time, it is best to separate the critical program portion
-into its own file and then generate the trace file from this small file.
-
-%%\f
-%%\node Efficiency Notes, Profiling, General Efficiency Hints, Advanced Compiler Use and Efficiency Hints
-\section{Efficiency Notes}
-\label{efficiency-notes}
-\cindex{efficiency notes}
-\cpsubindex{notes}{efficiency}
-\cindex{tuning}
-
-Efficiency notes are messages that warn the user that the compiler has
-chosen a relatively inefficient implementation for some operation.
-Usually an efficiency note reflects the compiler's desire for more
-type information. If the type of the values concerned is known to the
-programmer, then additional declarations can be used to get a more
-efficient implementation.
-
-Efficiency notes are controlled by the
-\code{extensions:inhibit-warnings} (\pxlref{optimize-declaration})
-optimization quality. When \code{speed} is greater than
-\code{extensions:inhibit-warnings}, efficiency notes are enabled.
-Note that this implicitly enables efficiency notes whenever
-\code{speed} is increased from its default of \code{1}.
-
-Consider this program with an obscure missing declaration:
-\begin{lisp}
-(defun eff-note (x y z)
- (declare (fixnum x y z))
- (the fixnum (+ x y z)))
-\end{lisp}
-If compiled with \code{\w{(speed 3) (safety 0)}}, this note is given:
-\begin{example}
-In: DEFUN EFF-NOTE
- (+ X Y Z)
-==>
- (+ (+ X Y) Z)
-Note: Forced to do inline (signed-byte 32) arithmetic (cost 3).
- Unable to do inline fixnum arithmetic (cost 2) because:
- The first argument is a (INTEGER -1073741824 1073741822),
- not a FIXNUM.
-\end{example}
-This efficiency note tells us that the result of the intermediate
-computation \code{\w{(+ x y)}} is not known to be a \code{fixnum}, so
-the addition of the intermediate sum to \code{z} must be done less
-efficiently. This can be fixed by changing the definition of
-\code{eff-note}:
-\begin{lisp}
-(defun eff-note (x y z)
- (declare (fixnum x y z))
- (the fixnum (+ (the fixnum (+ x y)) z)))
-\end{lisp}
-
-\begin{comment}
-* Type Uncertainty::
-* Efficiency Notes and Type Checking::
-* Representation Efficiency Notes::
-* Verbosity Control::
-\end{comment}
-
-%%\node Type Uncertainty, Efficiency Notes and Type Checking, Efficiency Notes, Efficiency Notes
-\subsection{Type Uncertainty}
-\cpsubindex{types}{uncertainty}
-\cindex{uncertainty of types}
-
-The main cause of inefficiency is the compiler's lack of adequate
-information about the types of function argument and result values.
-Many important operations (such as arithmetic) have an inefficient
-general (generic) case, but have efficient implementations that can
-usually be used if there is sufficient argument type information.
-
-Type efficiency notes are given when a value's type is uncertain.
-There is an important distinction between values that are \i{not
- known} to be of a good type (uncertain) and values that are \i{known
- not} to be of a good type. Efficiency notes are given mainly for
-the first case (uncertain types.) If it is clear to the compiler that
-that there is not an efficient implementation for a particular
-function call, then an efficiency note will only be given if the
-\code{extensions:inhibit-warnings} optimization quality is \code{0}
-(\pxlref{optimize-declaration}.)
-
-In other words, the default efficiency notes only suggest that you add
-declarations, not that you change the semantics of your program so
-that an efficient implementation will apply. For example, compilation
-of this form will not give an efficiency note:
-\begin{lisp}
-(elt (the list l) i)
-\end{lisp}
-even though a vector access is more efficient than indexing a list.
-
-%%\node Efficiency Notes and Type Checking, Representation Efficiency Notes, Type Uncertainty, Efficiency Notes
-\subsection{Efficiency Notes and Type Checking}
-\cpsubindex{type checking}{efficiency of}
-\cpsubindex{efficiency}{of type checking}
-\cpsubindex{optimization}{type check}
-
-It is important that the \code{eff-note} example above used
-\w{\code{(safety 0)}}. When type checking is enabled, you may get apparently
-spurious efficiency notes. With \w{\code{(safety 1)}}, the note has this extra
-line on the end:
-\begin{example}
-The result is a (INTEGER -1610612736 1610612733), not a FIXNUM.
-\end{example}
-This seems strange, since there is a \code{the} declaration on the result of that
-second addition.
-
-In fact, the inefficiency is real, and is a consequence of \python{}'s
-treating declarations as assertions to be verified. The compiler
-can't assume that the result type declaration is true\dash{}it must
-generate the result and then test whether it is of the appropriate
-type.
-
-In practice, this means that when you are tuning a program to run
-without type checks, you should work from the efficiency notes
-generated by unsafe compilation. If you want code to run efficiently
-with type checking, then you should pay attention to all the
-efficiency notes that you get during safe compilation. Since user
-supplied output type assertions (e.g., from \code{the}) are
-disregarded when selecting operation implementations for safe code,
-you must somehow give the compiler information that allows it to prove
-that the result truly must be of a good type. In our example, it
-could be done by constraining the argument types more:
-\begin{lisp}
-(defun eff-note (x y z)
- (declare (type (unsigned-byte 18) x y z))
- (+ x y z))
-\end{lisp}
-Of course, this declaration is acceptable only if the arguments to \code{eff-note}
-always \var{are} \w{\code{(unsigned-byte 18)}} integers.
-
-%%\node Representation Efficiency Notes, Verbosity Control, Efficiency Notes and Type Checking, Efficiency Notes
-\subsection{Representation Efficiency Notes}
-\label{representation-eff-note}
-\cindex{representation efficiency notes}
-\cpsubindex{efficiency notes}{for representation}
-\cindex{object representation efficiency notes}
-\cindex{stack numbers}
-\cindex{non-descriptor representations}
-\cpsubindex{descriptor representations}{forcing of}
-
-When operating on values that have non-descriptor representations
-(\pxlref{non-descriptor}), there can be a substantial time and consing
-penalty for converting to and from descriptor representations. For
-this reason, the compiler gives an efficiency note whenever it is
-forced to do a representation coercion more expensive than
-\varref{efficiency-note-cost-threshold}.
-
-Inefficient representation coercions may be due to type uncertainty,
-as in this example:
-\begin{lisp}
-(defun set-flo (x)
- (declare (single-float x))
- (prog ((var 0.0))
- (setq var (gorp))
- (setq var x)
- (return var)))
-\end{lisp}
-which produces this efficiency note:
-\begin{example}
-In: DEFUN SET-FLO
- (SETQ VAR X)
-Note: Doing float to pointer coercion (cost 13) from X to VAR.
-\end{example}
-The variable \code{var} is not known to always hold values of type
-\code{single-float}, so a descriptor representation must be used for its value.
-In sort of situation, and adding a declaration will eliminate the inefficiency.
-
-Often inefficient representation conversions are not due to type
-uncertainty\dash{}instead, they result from evaluating a
-non-descriptor expression in a context that requires a descriptor
-result:
-\begin{itemize}
-
-\item Assignment to or initialization of any data structure other than
- a specialized array (\pxlref{specialized-array-types}), or
-
-\item Assignment to a \code{special} variable, or
-
-\item Passing as an argument or returning as a value in any function
- call that is not a local call (\pxlref{number-local-call}.)
-\end{itemize}
-
-If such inefficient coercions appear in a ``hot spot'' in the program, data
-structures redesign or program reorganization may be necessary to improve
-efficiency. See sections \ref{block-compilation}, \ref{numeric-types} and
-\ref{profiling}.
-
-Because representation selection is done rather late in compilation,
-the source context in these efficiency notes is somewhat vague, making
-interpretation more difficult. This is a fairly straightforward
-example:
-\begin{lisp}
-(defun cf+ (x y)
- (declare (single-float x y))
- (cons (+ x y) t))
-\end{lisp}
-which gives this efficiency note:
-\begin{example}
-In: DEFUN CF+
- (CONS (+ X Y) T)
-Note: Doing float to pointer coercion (cost 13), for:
- The first argument of CONS.
-\end{example}
-The source context form is almost always the form that receives the value being
-coerced (as it is in the preceding example), but can also be the source form
-which generates the coerced value. Compiling this example:
-\begin{lisp}
-(defun if-cf+ (x y)
- (declare (single-float x y))
- (cons (if (grue) (+ x y) (snoc)) t))
-\end{lisp}
-produces this note:
-\begin{example}
-In: DEFUN IF-CF+
- (+ X Y)
-Note: Doing float to pointer coercion (cost 13).
-\end{example}
-
-In either case, the note's text explanation attempts to include
-additional information about what locations are the source and
-destination of the coercion. Here are some example notes:
-\begin{example}
- (IF (GRUE) X (SNOC))
-Note: Doing float to pointer coercion (cost 13) from X.
-
- (SETQ VAR X)
-Note: Doing float to pointer coercion (cost 13) from X to VAR.
-\end{example}
-Note that the return value of a function is also a place to which coercions may
-have to be done:
-\begin{example}
- (DEFUN F+ (X Y) (DECLARE (SINGLE-FLOAT X Y)) (+ X Y))
-Note: Doing float to pointer coercion (cost 13) to "<return value>".
-\end{example}
-Sometimes the compiler is unable to determine a name for the source or
-destination, in which case the source context is the only clue.
-
-
-%%\node Verbosity Control, , Representation Efficiency Notes, Efficiency Notes
-\subsection{Verbosity Control}
-\cpsubindex{verbosity}{of efficiency notes}
-\cpsubindex{efficiency notes}{verbosity}
-
-These variables control the verbosity of efficiency notes:
-
-\begin{defvar}{}{efficiency-note-cost-threshold}
-
- Before printing some efficiency notes, the compiler compares the
- value of this variable to the difference in cost between the chosen
- implementation and the best potential implementation. If the
- difference is not greater than this limit, then no note is printed.
- The units are implementation dependent; the initial value suppresses
- notes about ``trivial'' inefficiencies. A value of \code{1} will
- note any inefficiency.
-\end{defvar}
-
-\begin{defvar}{}{efficiency-note-limit}
-
- When printing some efficiency notes, the compiler reports possible
- efficient implementations. The initial value of \code{2} prevents
- excessively long efficiency notes in the common case where there is
- no type information, so all implementations are possible.
-\end{defvar}
-
-%%\f
-%%\node Profiling, , Efficiency Notes, Advanced Compiler Use and Efficiency Hints
-\section{Profiling}
-
-\cindex{profiling}
-\cindex{timing}
-\cindex{consing}
-\cindex{tuning}
-\label{profiling}
-
-The first step in improving a program's performance is to profile the
-activity of the program to find where it spends its time. The best
-way to do this is to use the profiling utility found in the
-\code{profile} package. This package provides a macro \code{profile}
-that encapsulates functions with statistics gathering code.
-
-\begin{comment}
-* Profile Interface::
-* Profiling Techniques::
-* Nested or Recursive Calls::
-* Clock resolution::
-* Profiling overhead::
-* Additional Timing Utilities::
-* A Note on Timing::
-* Benchmarking Techniques::
-\end{comment}
-
-%%\node Profile Interface, Profiling Techniques, Profiling, Profiling
-\subsection{Profile Interface}
-
-\begin{defvar}{profile:}{timed-functions}
-
- This variable holds a list of all functions that are currently being
- profiled.
-\end{defvar}
-
-\begin{defmac}{profile:}{profile}{%
- \args{\mstar{\var{name} \mor \kwd{callers} \code{t}}}}
-
- This macro wraps profiling code around the named functions. As in
- \code{trace}, the \var{name}s are not evaluated. If a function is
- already profiled, then the function is unprofiled and reprofiled
- (useful to notice function redefinition.) A warning is printed for
- each name that is not a defined function.
-
- If \kwd{callers \var{t}} is specified, then each function that calls
- this function is recorded along with the number of calls made.
-\end{defmac}
-
-\begin{defmac}{profile:}{unprofile}{%
- \args{\mstar{\var{name}}}}
-
- This macro removes profiling code from the named functions. If no
- \var{name}s are supplied, all currently profiled functions are
- unprofiled.
-\end{defmac}
-
-\begin{changebar}
- \begin{defmac}{profile:}{profile-all}{%
- \args{\keys{\kwd{package} \kwd{callers-p}}}}
-
- This macro in effect calls \code{profile:profile} for each
- function in the specified package which defaults to
- \code{*package*}. \kwd{callers-p} has the same meaning as in
- \code{profile:profile}.
- \end{defmac}
-\end{changebar}
-
-\begin{defmac}{profile:}{report-time}{\args{\mstar{\var{name}}}}
-
- This macro prints a report for each \var{name}d function of the
- following information:
- \begin{itemize}
- \item The total CPU time used in that function for all calls,
-
- \item the total number of bytes consed in that function for all
- calls,
-
- \item the total number of calls,
-
- \item the average amount of CPU time per call.
- \end{itemize}
- Summary totals of the CPU time, consing and calls columns are
- printed. An estimate of the profiling overhead is also printed (see
- below). If no \var{name}s are supplied, then the times for all
- currently profiled functions are printed.
-\end{defmac}
-
-\begin{defmac}{}{reset-time}{\args{\mstar{\var{name}}}}
-
- This macro resets the profiling counters associated with the
- \var{name}d functions. If no \var{name}s are supplied, then all
- currently profiled functions are reset.
-\end{defmac}
-
-
-%%\node Profiling Techniques, Nested or Recursive Calls, Profile Interface, Profiling
-\subsection{Profiling Techniques}
-
-Start by profiling big pieces of a program, then carefully choose which
-functions close to, but not in, the inner loop are to be profiled next.
-Avoid profiling functions that are called by other profiled functions, since
-this opens the possibility of profiling overhead being included in the reported
-times.
-
-If the per-call time reported is less than 1/10 second, then consider the clock
-resolution and profiling overhead before you believe the time. It may be that
-you will need to run your program many times in order to average out to a
-higher resolution.
-
-
-%%\node Nested or Recursive Calls, Clock resolution, Profiling Techniques, Profiling
-\subsection{Nested or Recursive Calls}
-
-The profiler attempts to compensate for nested or recursive calls. Time and
-consing overhead will be charged to the dynamically innermost (most recent)
-call to a profiled function. So profiling a subfunction of a profiled function
-will cause the reported time for the outer function to decrease. However if an
-inner function has a large number of calls, some of the profiling overhead may
-``leak'' into the reported time for the outer function. In general, be wary of
-profiling short functions that are called many times.
-
-%%\node Clock resolution, Profiling overhead, Nested or Recursive Calls, Profiling
-\subsection{Clock resolution}
-
-Unless you are very lucky, the length of your machine's clock ``tick'' is
-probably much longer than the time it takes simple function to run. For
-example, on the IBM RT, the clock resolution is 1/50 second. This means that
-if a function is only called a few times, then only the first couple decimal
-places are really meaningful.
-
-Note however, that if a function is called many times, then the statistical
-averaging across all calls should result in increased resolution. For example,
-on the IBM RT, if a function is called a thousand times, then a resolution of
-tens of microseconds can be expected.
-
-%%\node Profiling overhead, Additional Timing Utilities, Clock resolution, Profiling
-\subsection{Profiling overhead}
-
-The added profiling code takes time to run every time that the profiled
-function is called, which can disrupt the attempt to collect timing
-information. In order to avoid serious inflation of the times for functions
-that take little time to run, an estimate of the overhead due to profiling is
-subtracted from the times reported for each function.
-
-Although this correction works fairly well, it is not totally accurate,
-resulting in times that become increasingly meaningless for functions with
-short runtimes. This is only a concern when the estimated profiling overhead
-is many times larger than reported total CPU time.
-
-The estimated profiling overhead is not represented in the reported total CPU
-time. The sum of total CPU time and the estimated profiling overhead should be
-close to the total CPU time for the entire profiling run (as determined by the
-\code{time} macro.) Time unaccounted for is probably being used by functions that
-you forgot to profile.
-
-%%\node Additional Timing Utilities, A Note on Timing, Profiling overhead, Profiling
-\subsection{Additional Timing Utilities}
-
-\begin{defmac}{}{time}{ \args{\var{form}}}
-
- This macro evaluates \var{form}, prints some timing and memory
- allocation information to \code{*trace-output*}, and returns any
- values that \var{form} returns. The timing information includes
- real time, user run time, and system run time. This macro executes
- a form and reports the time and consing overhead. If the
- \code{time} form is not compiled (e.g. it was typed at top-level),
- then \code{compile} will be called on the form to give more accurate
- timing information. If you really want to time interpreted speed,
- you can say:
-\begin{lisp}
-(time (eval '\var{form}))
-\end{lisp}
-Things that execute fairly quickly should be timed more than once,
-since there may be more paging overhead in the first timing. To
-increase the accuracy of very short times, you can time multiple
-evaluations:
-\begin{lisp}
-(time (dotimes (i 100) \var{form}))
-\end{lisp}
-\end{defmac}
-
-\begin{defun}{extensions:}{get-bytes-consed}{}
-
- This function returns the number of bytes allocated since the first
- time you called it. The first time it is called it returns zero.
- The above profiling routines use this to report consing information.
-\end{defun}
-
-\begin{defvar}{extensions:}{gc-run-time}
-
- This variable accumulates the run-time consumed by garbage
- collection, in the units returned by
- \findexed{get-internal-run-time}.
-\end{defvar}
-
-\begin{defconst}{}{internal-time-units-per-second}
-The value of internal-time-units-per-second is 100.
-\end{defconst}
-
-%%\node A Note on Timing, Benchmarking Techniques, Additional Timing Utilities, Profiling
-\subsection{A Note on Timing}
-\cpsubindex{CPU time}{interpretation of}
-\cpsubindex{run time}{interpretation of}
-\cindex{interpretation of run time}
-
-There are two general kinds of timing information provided by the
-\code{time} macro and other profiling utilities: real time and run
-time. Real time is elapsed, wall clock time. It will be affected in
-a fairly obvious way by any other activity on the machine. The more
-other processes contending for CPU and memory, the more real time will
-increase. This means that real time measurements are difficult to
-replicate, though this is less true on a dedicated workstation. The
-advantage of real time is that it is real. It tells you really how
-long the program took to run under the benchmarking conditions. The
-problem is that you don't know exactly what those conditions were.
-
-Run time is the amount of time that the processor supposedly spent
-running the program, as opposed to waiting for I/O or running other
-processes. ``User run time'' and ``system run time'' are numbers
-reported by the Unix kernel. They are supposed to be a measure of how
-much time the processor spent running your ``user'' program (which
-will include GC overhead, etc.), and the amount of time that the
-kernel spent running ``on your behalf.''
-
-Ideally, user time should be totally unaffected by benchmarking
-conditions; in reality user time does depend on other system activity,
-though in rather non-obvious ways.
-
-System time will clearly depend on benchmarking conditions. In Lisp
-benchmarking, paging activity increases system run time (but not by as much
-as it increases real time, since the kernel spends some time waiting for
-the disk, and this is not run time, kernel or otherwise.)
-
-In my experience, the biggest trap in interpreting kernel/user run time is
-to look only at user time. In reality, it seems that the \var{sum} of kernel
-and user time is more reproducible. The problem is that as system activity
-increases, there is a spurious \var{decrease} in user run time. In effect, as
-paging, etc., increases, user time leaks into system time.
-
-So, in practice, the only way to get truly reproducible results is to run
-with the same competing activity on the system. Try to run on a machine
-with nobody else logged in, and check with ``ps aux'' to see if there are any
-system processes munching large amounts of CPU or memory. If the ratio
-between real time and the sum of user and system time varies much between
-runs, then you have a problem.
-
-%%\node Benchmarking Techniques, , A Note on Timing, Profiling
-\subsection{Benchmarking Techniques}
-\cindex{benchmarking techniques}
-
-Given these imperfect timing tools, how do should you do benchmarking? The
-answer depends on whether you are trying to measure improvements in the
-performance of a single program on the same hardware, or if you are trying to
-compare the performance of different programs and/or different hardware.
-
-For the first use (measuring the effect of program modifications with
-constant hardware), you should look at \var{both} system+user and real time to
-understand what effect the change had on CPU use, and on I/O (including
-paging.) If you are working on a CPU intensive program, the change in
-system+user time will give you a moderately reproducible measure of
-performance across a fairly wide range of system conditions. For a CPU
-intensive program, you can think of system+user as ``how long it would have
-taken to run if I had my own machine.'' So in the case of comparing CPU
-intensive programs, system+user time is relatively real, and reasonable to
-use.
-
-For programs that spend a substantial amount of their time paging, you
-really can't predict elapsed time under a given operating condition without
-benchmarking in that condition. User or system+user time may be fairly
-reproducible, but it is also relatively meaningless, since in a paging or
-I/O intensive program, the program is spending its time waiting, not
-running, and system time and user time are both measures of run time.
-A change that reduces run time might increase real time by increasing
-paging.
-
-Another common use for benchmarking is comparing the performance of
-the same program on different hardware. You want to know which
-machine to run your program on. For comparing different machines
-(operating systems, etc.), the only way to compare that makes sense is
-to set up the machines in \var{exactly} the way that they will
-\var{normally} be run, and then measure \var{real} time. If the
-program will normally be run along with X, then run X. If the program
-will normally be run on a dedicated workstation, then be sure nobody
-else is on the benchmarking machine. If the program will normally be
-run on a machine with three other Lisp jobs, then run three other Lisp
-jobs. If the program will normally be run on a machine with 8meg of
-memory, then run with 8meg. Here, ``normal'' means ``normal for that
-machine''. If you the choice of an unloaded RT or a heavily loaded
-PMAX, do your benchmarking on an unloaded RT and a heavily loaded
-PMAX.
-
-If you have a program you believe to be CPU intensive, then you might be
-tempted to compare ``run'' times across systems, hoping to get a meaningful
-result even if the benchmarking isn't done under the expected running
-condition. Don't to this, for two reasons:
-\begin{itemize}
-
-\item The operating systems might not compute run time in the same
- way.
-
-\item Under the real running condition, the program might not be CPU
- intensive after all.
-\end{itemize}
-
-
-In the end, only real time means anything\dash{}it is the amount of time you
-have to wait for the result. The only valid uses for run time are:
-\begin{itemize}
-
-\item To develop insight into the program. For example, if run time
- is much less than elapsed time, then you are probably spending lots
- of time paging.
-
-\item To evaluate the relative performance of CPU intensive programs
- in the same environment.
-\end{itemize}
-
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/Unix.ms}
-
-
-
-%%\node UNIX Interface, Event Dispatching with SERVE-EVENT, Advanced Compiler Use and Efficiency Hints, Top
-\chapter{UNIX Interface}
-\label{unix-interface}
-\begin{center}
-\b{By Robert MacLachlan, Skef Wholey,}
-\end{center}
-\begin{center}
-\b{Bill Chiles, and William Lott}
-\end{center}
-
-CMU Common Lisp attempts to make the full power of the underlying
-environment available to the Lisp programmer. This is done using
-combination of hand-coded interfaces and foreign function calls to C
-libraries. Although the techniques differ, the style of interface is
-similar. This chapter provides an overview of the facilities
-available and general rules for using them, as well as describing
-specific features in detail. It is assumed that the reader has a
-working familiarity with Mach, Unix and X, as well as access to the
-standard system documentation.
-
-\begin{comment}
-* Reading the Command Line::
-* Lisp Equivalents for C Routines::
-* Type Translations::
-* System Area Pointers::
-* Unix System Calls::
-* File Descriptor Streams::
-* Making Sense of Mach Return Codes::
-* Unix Interrupts::
-\end{comment}
-
-
-%%\node Reading the Command Line, Useful Variables, UNIX Interface, UNIX Interface
-\section{Reading the Command Line}
-
-The shell parses the command line with which Lisp is invoked, and
-passes a data structure containing the parsed information to Lisp.
-This information is then extracted from that data structure and put
-into a set of Lisp data structures.
-
-\begin{defvar}{extensions:}{command-line-strings}
- \defvarx[extensions:]{command-line-utility-name}
- \defvarx[extensions:]{command-line-words}
- \defvarx[extensions:]{command-line-switches}
-
- The value of \code{*command-line-words*} is a list of strings that
- make up the command line, one word per string. The first word on
- the command line, i.e. the name of the program invoked (usually
- \code{lisp}) is stored in \code{*command-line-utility-name*}. The
- value of \code{*command-line-switches*} is a list of
- \code{command-line-switch} structures, with a structure for each
- word on the command line starting with a hyphen. All the command
- line words between the program name and the first switch are stored
- in \code{*command-line-words*}.
-\end{defvar}
-
-The following functions may be used to examine \code{command-line-switch}
-structures.
-\begin{defun}{extensions:}{cmd-switch-name}{\args{\var{switch}}}
-
- Returns the name of the switch, less the preceding hyphen and
- trailing equal sign (if any).
-\end{defun}
-\begin{defun}{extensions:}{cmd-switch-value}{\args{\var{switch}}}
-
- Returns the value designated using an embedded equal sign, if any.
- If the switch has no equal sign, then this is null.
-\end{defun}
-\begin{defun}{extensions:}{cmd-switch-words}{\args{\var{switch}}}
-
- Returns a list of the words between this switch and the next switch
- or the end of the command line.
-\end{defun}
-\begin{defun}{extensions:}{cmd-switch-arg}{\args{\var{switch}}}
-
- Returns the first non-null value from \code{cmd-switch-value}, the
- first element in \code{cmd-switch-words}, or the first word in
- \var{command-line-words}.
-\end{defun}
-
-\begin{defun}{extensions:}{get-command-line-switch}{\args{\var{sname}}}
-
- This function takes the name of a switch as a string and returns the
- value of the switch given on the command line. If no value was
- specified, then any following words are returned. If there are no
- following words, then \true{} is returned. If the switch was not
- specified, then \false{} is returned.
-\end{defun}
-
-\begin{defmac}{extensions:}{defswitch}{%
- \args{\var{name} \ampoptional{} \var{function}}}
-
- This macro causes \var{function} to be called when the switch
- \var{name} appears in the command line. Name is a simple-string
- that does not begin with a hyphen (unless the switch name really
- does begin with one.)
-
- If \var{function} is not supplied, then the switch is parsed into
- \var{command-line-switches}, but otherwise ignored. This suppresses
- the undefined switch warning which would otherwise take place. THe
- warning can also be globally suppressed by
- \var{complain-about-illegal-switches}.
-\end{defmac}
-
-%%\node Useful Variables, Lisp Equivalents for C Routines, Reading the Command Line, UNIX Interface
-
-\section{Useful Variables}
-
-\begin{defvar}{system:}{stdin}
- \defvarx[system:]{stdout} \defvarx[system:]{stderr}
-
- Streams connected to the standard input, output and error file
- descriptors.
-\end{defvar}
-
-\begin{defvar}{system:}{tty}
-
- A stream connected to \file{/dev/tty}.
-\end{defvar}
-
-%%\node Lisp Equivalents for C Routines, Type Translations, Useful Variables, UNIX Interface
-\section{Lisp Equivalents for C Routines}
-
-The UNIX documentation describes the system interface in terms of C
-procedure headers. The corresponding Lisp function will have a somewhat
-different interface, since Lisp argument passing conventions and
-datatypes are different.
-
-The main difference in the argument passing conventions is that Lisp does not
-support passing values by reference. In Lisp, all argument and results are
-passed by value. Interface functions take some fixed number of arguments and
-return some fixed number of values. A given ``parameter'' in the C
-specification will appear as an argument, return value, or both, depending on
-whether it is an In parameter, Out parameter, or In/Out parameter. The basic
-transformation one makes to come up with the Lisp equivalent of a C routine is
-to remove the Out parameters from the call, and treat them as extra return
-values. In/Out parameters appear both as arguments and return values. Since
-Out and In/Out parameters are only conventions in C, you must determine the
-usage from the documentation.
-
-
-Thus, the C routine declared as
-\begin{example}
-kern_return_t lookup(servport, portsname, portsid)
- port servport;
- char *portsname;
- int *portsid; /* out */
- {
- ...
- *portsid = <expression to compute portsid field>
- return(KERN_SUCCESS);
- }
-\end{example}
-has as its Lisp equivalent something like
-\begin{lisp}
-(defun lookup (ServPort PortsName)
- ...
- (values
- success
- <expression to compute portsid field>))
-\end{lisp}
-If there are multiple out or in-out arguments, then there are multiple
-additional returns values.
-
-Fortunately, CMU Common Lisp programmers rarely have to worry about the
-nuances of this translation process, since the names of the arguments and
-return values are documented in a way so that the \code{describe} function
-(and the \Hemlock{} \code{Describe Function Call} command, invoked with
-\b{C-M-Shift-A}) will list this information. Since the names of arguments
-and return values are usually descriptive, the information that
-\code{describe} prints is usually all one needs to write a
-call. Most programmers use this on-line documentation nearly
-all of the time, and thereby avoid the need to handle bulky
-manuals and perform the translation from barbarous tongues.
-
-%%\node Type Translations, System Area Pointers, Lisp Equivalents for C Routines, UNIX Interface
-\section{Type Translations}
-\cindex{aliens}
-\cpsubindex{types}{alien}
-\cpsubindex{types}{foreign language}
-
-Lisp data types have very different representations from those used by
-conventional languages such as C. Since the system interfaces are
-designed for conventional languages, Lisp must translate objects to and
-from the Lisp representations. Many simple objects have a direct
-translation: integers, characters, strings and floating point numbers
-are translated to the corresponding Lisp object. A number of types,
-however, are implemented differently in Lisp for reasons of clarity and
-efficiency.
-
-Instances of enumerated types are expressed as keywords in Lisp.
-Records, arrays, and pointer types are implemented with the \Alien{}
-facility (see page \pageref{aliens}.) Access functions are defined
-for these types which convert fields of records, elements of arrays,
-or data referenced by pointers into Lisp objects (possibly another
-object to be referenced with another access function).
-
-One should dispose of \Alien{} objects created by constructor
-functions or returned from remote procedure calls when they are no
-longer of any use, freeing the virtual memory associated with that
-object. Since \alien{}s contain pointers to non-Lisp data, the
-garbage collector cannot do this itself. If the memory
-was obtained from \funref{make-alien} or from a foreign function call
-to a routine that used \code{malloc}, then \funref{free-alien} should
-be used. If the \alien{} was created
-using MACH memory allocation (e.g. \code{vm\_allocate}), then the
-storage should be freed using \code{vm\_deallocate}.
-
-%%\node System Area Pointers, Unix System Calls, Type Translations, UNIX Interface
-\section{System Area Pointers}
-\label{system-area-pointers}
-
-\cindex{pointers}\cpsubindex{malloc}{C function}\cpsubindex{free}{C function}
-Note that in some cases an address is represented by a Lisp integer, and in
-other cases it is represented by a real pointer. Pointers are usually used
-when an object in the current address space is being referred to. The MACH
-virtual memory manipulation calls must use integers, since in principle the
-address could be in any process, and Lisp cannot abide random pointers.
-Because these types are represented differently in Lisp, one must explicitly
-coerce between these representations.
-
-System Area Pointers (SAPs) provide a mechanism that bypasses the
-\Alien{} type system and accesses virtual memory directly. A SAP is a
-raw byte pointer into the \code{lisp} process address space. SAPs are
-represented with a pointer descriptor, so SAP creation can cause
-consing. However, the compiler uses a non-descriptor representation
-for SAPs when possible, so the consing overhead is generally minimal.
-\xlref{non-descriptor}.
-
-\begin{defun}{system:}{sap-int}{\args{\var{sap}}}
- \defunx[system:]{int-sap}{\args{\var{int}}}
-
- The function \code{sap-int} is used to generate an integer
- corresponding to the system area pointer, suitable for passing to
- the kernel interfaces (which want all addresses specified as
- integers). The function \code{int-sap} is used to do the opposite
- conversion. The integer representation of a SAP is the byte offset
- of the SAP from the start of the address space.
-\end{defun}
-
-\begin{defun}{system:}{sap+}{\args{\var{sap} \var{offset}}}
-
- This function adds a byte \var{offset} to \var{sap}, returning a new
- SAP.
-\end{defun}
-
-\begin{defun}{system:}{sap-ref-8}{\args{\var{sap} \var{offset}}}
- \defunx[system:]{sap-ref-16}{\args{\var{sap} \var{offset}}}
- \defunx[system:]{sap-ref-32}{\args{\var{sap} \var{offset}}}
-
- These functions return the 8, 16 or 32 bit unsigned integer at
- \var{offset} from \var{sap}. The \var{offset} is always a byte
- offset, regardless of the number of bits accessed. \code{setf} may
- be used with the these functions to deposit values into virtual
- memory.
-\end{defun}
-
-\begin{defun}{system:}{signed-sap-ref-8}{\args{\var{sap} \var{offset}}}
- \defunx[system:]{signed-sap-ref-16}{\args{\var{sap} \var{offset}}}
- \defunx[system:]{signed-sap-ref-32}{\args{\var{sap} \var{offset}}}
-
- These functions are the same as the above unsigned operations,
- except that they sign-extend, returning a negative number if the
- high bit is set.
-\end{defun}
-
-%%\node Unix System Calls, File Descriptor Streams, System Area Pointers, UNIX Interface
-\section{Unix System Calls}
-
-You probably won't have much cause to use them, but all the Unix system
-calls are available. The Unix system call functions are in the
-\code{Unix} package. The name of the interface for a particular system
-call is the name of the system call prepended with \code{unix-}. The
-system usually defines the associated constants without any prefix name.
-To find out how to use a particular system call, try using
-\code{describe} on it. If that is unhelpful, look at the source in
-\file{syscall.lisp} or consult your system maintainer.
-
-The Unix system calls indicate an error by returning \false{} as the
-first value and the Unix error number as the second value. If the call
-succeeds, then the first value will always be non-\nil, often \code{t}.
-
-\begin{defun}{Unix:}{get-unix-error-msg}{\args{\var{error}}}
-
- This function returns a string describing the Unix error number
- \var{error}.
-\end{defun}
-
-%%\node File Descriptor Streams, Making Sense of Mach Return Codes, Unix System Calls, UNIX Interface
-\section{File Descriptor Streams}
-
-Many of the UNIX system calls return file descriptors. Instead of using other
-UNIX system calls to perform I/O on them, you can create a stream around them.
-For this purpose, fd-streams exist. See also \funref{read-n-bytes}.
-
-\begin{defun}{system:}{make-fd-stream}{%
- \args{\var{descriptor}} \keys{\kwd{input} \kwd{output}
- \kwd{element-type}} \morekeys{\kwd{buffering} \kwd{name}
- \kwd{file} \kwd{original}} \yetmorekeys{\kwd{delete-original}
- \kwd{auto-close}} \yetmorekeys{\kwd{timeout} \kwd{pathname}}}
-
- This function creates a file descriptor stream using
- \var{descriptor}. If \kwd{input} is non-\nil, input operations are
- allowed. If \kwd{output} is non-\nil, output operations are
- allowed. The default is input only. These keywords are defined:
- \begin{Lentry}
- \item[\kwd{element-type}] is the type of the unit of transaction for
- the stream, which defaults to \code{string-char}. See the Common
- Lisp description of \code{open} for valid values.
-
- \item[\kwd{buffering}] is the kind of output buffering desired for
- the stream. Legal values are \kwd{none} for no buffering,
- \kwd{line} for buffering up to each newline, and \kwd{full} for
- full buffering.
-
- \item[\kwd{name}] is a simple-string name to use for descriptive
- purposes when the system prints an fd-stream. When printing
- fd-streams, the system prepends the streams name with \code{Stream
- for }. If \var{name} is unspecified, it defaults to a string
- containing \var{file} or \var{descriptor}, in order of preference.
-
- \item[\kwd{file}, \kwd{original}] \var{file} specifies the defaulted
- namestring of the associated file when creating a file stream
- (must be a \code{simple-string}). \var{original} is the
- \code{simple-string} name of a backup file containing the original
- contents of \var{file} while writing \var{file}.
-
- When you abort the stream by passing \true{} to \code{close} as
- the second argument, if you supplied both \var{file} and
- \var{original}, \code{close} will rename the \var{original} name
- to the \var{file} name. When you \code{close} the stream
- normally, if you supplied \var{original}, and
- \var{delete-original} is non-\nil, \code{close} deletes
- \var{original}. If \var{auto-close} is true (the default), then
- \var{descriptor} will be closed when the stream is garbage
- collected.
-
- \item[\kwd{pathname}]: The original pathname passed to open and
- returned by \code{pathname}; not defaulted or translated.
-
- \item[\kwd{timeout}] if non-null, then \var{timeout} is an integer
- number of seconds after which an input wait should time out. If a
- read does time out, then the \code{system:io-timeout} condition is
- signalled.
- \end{Lentry}
-\end{defun}
-
-\begin{defun}{system:}{fd-stream-p}{\args{\var{object}}}
-
- This function returns \true{} if \var{object} is an fd-stream, and
- \nil{} if not. Obsolete: use the portable \code{(typep x
- 'file-stream)}.
-\end{defun}
-
-\begin{defun}{system:}{fd-stream-fd}{\args{\var{stream}}}
-
- This returns the file descriptor associated with \var{stream}.
-\end{defun}
-
-
-%%\node Making Sense of Mach Return Codes, Unix Interrupts, File Descriptor Streams, UNIX Interface
-\section{Making Sense of Mach Return Codes}
-
-Whenever a remote procedure call returns a Unix error code (such as
-\code{kern\_return\_t}), it is usually prudent to check that code to
-see if the call was successful. To relieve the programmer of the
-hassle of testing this value himself, and to centralize the
-information about the meaning of non-success return codes, CMU Common
-Lisp provides a number of macros and functions. See also
-\funref{get-unix-error-msg}.
-
-\begin{defun}{system:}{gr-error}{%
- \args{\var{function} \var{gr} \ampoptional{} \var{context}}}
-
- Signals a Lisp error, printing a message indicating that the call to
- the specified \var{function} failed, with the return code \var{gr}.
- If supplied, the \var{context} string is printed after the
- \var{function} name and before the string associated with the
- \var{gr}. For example:
-\begin{example}
-* (gr-error 'nukegarbage 3 "lost big")
-
-Error in function GR-ERROR:
-NUKEGARBAGE lost big, no space.
-Proceed cases:
-0: Return to Top-Level.
-Debug (type H for help)
-(Signal #<Conditions:Simple-Error.5FDE0>)
-0]
-\end{example}
-\end{defun}
-
-\begin{defmac}{system:}{gr-call}{\args{\var{function} \amprest{} \var{args}}}
- \defmacx[system:]{gr-call*}{\args{\var{function} \amprest{} \var{args}}}
-
- These macros can be used to call a function and automatically check
- the GeneralReturn code and signal an appropriate error in case of
- non-successful return. \code{gr-call} returns \false{} if no error
- occurs, while \code{gr-call*} returns the second value of the
- function called.
-\begin{example}
-* (gr-call mach:port_allocate *task-self*)
-NIL
-*
-\end{example}
-\end{defmac}
-
-\begin{defmac}{system:}{gr-bind}{
- \args{\code{(}\mstar{\var{var}}\code{)}
- \code{(}\var{function} \mstar{\var{arg}}\code{)}
- \mstar{\var{form}}}}
-
- This macro can be used much like \code{multiple-value-bind} to bind
- the \var{var}s to return values resulting from calling the
- \var{function} with the given \var{arg}s. The first return value is
- not bound to a variable, but is checked as a GeneralReturn code, as
- in \code{gr-call}.
-\begin{example}
-* (gr-bind (port_list port_list_cnt)
- (mach:port_select *task-self*)
- (format t "The port count is ~S." port_list_cnt)
- port_list)
-The port count is 0.
-#<Alien value>
-*
-\end{example}
-\end{defmac}
-
-%%\node Unix Interrupts, , Making Sense of Mach Return Codes, UNIX Interface
-\section{Unix Interrupts}
-
-\cindex{unix interrupts} \cindex{interrupts}
-CMU Common Lisp allows access to all the Unix signals that can be generated
-under Unix. It should be noted that if this capability is abused, it is
-possible to completely destroy the running Lisp. The following macros and
-functions allow access to the Unix interrupt system. The signal names as
-specified in section 2 of the \i{Unix Programmer's Manual} are exported
-from the Unix package.
-
-\begin{comment}
-* Changing Interrupt Handlers::
-* Examples of Signal Handlers::
-\end{comment}
-
-%%\node Changing Interrupt Handlers, Examples of Signal Handlers, Unix Interrupts, Unix Interrupts
-\subsection{Changing Interrupt Handlers}
-\label{signal-handlers}
-
-\begin{defmac}{system:}{with-enabled-interrupts}{
- \args{\var{specs} \amprest{} \var{body}}}
-
- This macro should be called with a list of signal specifications,
- \var{specs}. Each element of \var{specs} should be a list of
- two\hide{ or three} elements: the first should be the Unix signal
- for which a handler should be established, the second should be a
- function to be called when the signal is received\hide{, and the
- third should be an optional character used to generate the signal
- from the keyboard. This last item is only useful for the SIGINT,
- SIGQUIT, and SIGTSTP signals.} One or more signal handlers can be
- established in this way. \code{with-enabled-interrupts} establishes
- the correct signal handlers and then executes the forms in
- \var{body}. The forms are executed in an unwind-protect so that the
- state of the signal handlers will be restored to what it was before
- the \code{with-enabled-interrupts} was entered. A signal handler
- function specified as NIL will set the Unix signal handler to the
- default which is normally either to ignore the signal or to cause a
- core dump depending on the particular signal.
-\end{defmac}
-
-\begin{defmac}{system:}{without-interrupts}{\args{\amprest{} \var{body}}}
-
- It is sometimes necessary to execute a piece a code that can not be
- interrupted. This macro the forms in \var{body} with interrupts
- disabled. Note that the Unix interrupts are not actually disabled,
- rather they are queued until after \var{body} has finished
- executing.
-\end{defmac}
-
-\begin{defmac}{system:}{with-interrupts}{\args{\amprest{} \var{body}}}
-
- When executing an interrupt handler, the system disables interrupts,
- as if the handler was wrapped in in a \code{without-interrupts}.
- The macro \code{with-interrupts} can be used to enable interrupts
- while the forms in \var{body} are evaluated. This is useful if
- \var{body} is going to enter a break loop or do some long
- computation that might need to be interrupted.
-\end{defmac}
-
-\begin{defmac}{system:}{without-hemlock}{\args{\amprest{} \var{body}}}
-
- For some interrupts, such as SIGTSTP (suspend the Lisp process and
- return to the Unix shell) it is necessary to leave Hemlock and then
- return to it. This macro executes the forms in \var{body} after
- exiting Hemlock. When \var{body} has been executed, control is
- returned to Hemlock.
-\end{defmac}
-
-\begin{defun}{system:}{enable-interrupt}{%
- \args{\var{signal} \var{function}\hide{ \ampoptional{}
- \var{character}}}}
-
- This function establishes \var{function} as the handler for
- \var{signal}.
- \hide{The optional \var{character} can be specified
- for the SIGINT, SIGQUIT, and SIGTSTP signals and causes that
- character to generate the appropriate signal from the keyboard.}
- Unless you want to establish a global signal handler, you should use
- the macro \code{with-enabled-interrupts} to temporarily establish a
- signal handler. \hide{Without \var{character},}
- \code{enable-interrupt} returns the old function associated with the
- signal. \hide{When \var{character} is specified for SIGINT,
- SIGQUIT, or SIGTSTP, it returns the old character code.}
-\end{defun}
-
-\begin{defun}{system:}{ignore-interrupt}{\args{\var{signal}}}
-
- Ignore-interrupt sets the Unix signal mechanism to ignore
- \var{signal} which means that the Lisp process will never see the
- signal. Ignore-interrupt returns the old function associated with
- the signal or \false{} if none is currently defined.
-\end{defun}
-
-\begin{defun}{system:}{default-interrupt}{\args{\var{signal}}}
-
- Default-interrupt can be used to tell the Unix signal mechanism to
- perform the default action for \var{signal}. For details on what
- the default action for a signal is, see section 2 of the \i{Unix
- Programmer's Manual}. In general, it is likely to ignore the
- signal or to cause a core dump.
-\end{defun}
-
-%%\node Examples of Signal Handlers, , Changing Interrupt Handlers, Unix Interrupts
-\subsection{Examples of Signal Handlers}
-
-The following code is the signal handler used by the Lisp system for the
-SIGINT signal.
-\begin{lisp}
-(defun ih-sigint (signal code scp)
- (declare (ignore signal code scp))
- (without-hemlock
- (with-interrupts
- (break "Software Interrupt" t))))
-\end{lisp}
-The \code{without-hemlock} form is used to make sure that Hemlock is exited before
-a break loop is entered. The \code{with-interrupts} form is used to enable
-interrupts because the user may want to generate an interrupt while in the
-break loop. Finally, break is called to enter a break loop, so the user
-can look at the current state of the computation. If the user proceeds
-from the break loop, the computation will be restarted from where it was
-interrupted.
-
-The following function is the Lisp signal handler for the SIGTSTP signal
-which suspends a process and returns to the Unix shell.
-\begin{lisp}
-(defun ih-sigtstp (signal code scp)
- (declare (ignore signal code scp))
- (without-hemlock
- (Unix:unix-kill (Unix:unix-getpid) Unix:sigstop)))
-\end{lisp}
-Lisp uses this interrupt handler to catch the SIGTSTP signal because it is
-necessary to get out of Hemlock in a clean way before returning to the shell.
-
-To set up these interrupt handlers, the following is recommended:
-\begin{lisp}
-(with-enabled-interrupts ((Unix:SIGINT #'ih-sigint)
- (Unix:SIGTSTP #'ih-sigtstp))
- <user code to execute with the above signal handlers enabled.>
-)
-\end{lisp}
-
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/server.ms}
-
-%%\node Event Dispatching with SERVE-EVENT, Alien Objects, UNIX Interface, Top
-\chapter{Event Dispatching with SERVE-EVENT}
-\begin{center}
-\b{By Bill Chiles and Robert MacLachlan}
-\end{center}
-
-It is common to have multiple activities simultaneously operating in the same
-Lisp process. Furthermore, Lisp programmers tend to expect a flexible
-development environment. It must be possible to load and modify application
-programs without requiring modifications to other running programs. CMU Common
-Lisp achieves this by having a central scheduling mechanism based on an
-event-driven, object-oriented paradigm.
-
-An \var{event} is some interesting happening that should cause the Lisp process
-to wake up and do something. These events include X events and activity on
-Unix file descriptors. The object-oriented mechanism is only available with
-the first two, and it is optional with X events as described later in this
-chapter. In an X event, the window ID is the object capability and the X event
-type is the operation code. The Unix file descriptor input mechanism simply
-consists of an association list of a handler to call when input shows up on a
-particular file descriptor.
-
-
-\begin{comment}
-* Object Sets::
-* The SERVE-EVENT Function::
-* Using SERVE-EVENT with Unix File Descriptors::
-* Using SERVE-EVENT with the CLX Interface to X::
-* A SERVE-EVENT Example::
-\end{comment}
-
-%%\node Object Sets, The SERVE-EVENT Function, Event Dispatching with SERVE-EVENT, Event Dispatching with SERVE-EVENT
-\section{Object Sets}
-\label{object-sets}
-\cindex{object sets}
-An \i{object set} is a collection of objects that have the same implementation
-for each operation. Externally the object is represented by the object
-capability and the operation is represented by the operation code. Within
-Lisp, the object is represented by an arbitrary Lisp object, and the
-implementation for the operation is represented by an arbitrary Lisp function.
-The object set mechanism maintains this translation from the external to the
-internal representation.
-
-\begin{defun}{system:}{make-object-set}{%
- \args{\var{name} \ampoptional{} \var{default-handler}}}
-
- This function makes a new object set. \var{Name} is a string used
- only for purposes of identifying the object set when it is printed.
- \var{Default-handler} is the function used as a handler when an
- undefined operation occurs on an object in the set. You can define
- operations with the \code{serve-}\var{operation} functions exported
- the \code{extensions} package for X events
- (\pxlref{x-serve-mumbles}). Objects are added with
- \code{system:add-xwindow-object}. Initially the object set has no
- objects and no defined operations.
-\end{defun}
-
-\begin{defun}{system:}{object-set-operation}{%
- \args{\var{object-set} \var{operation-code}}}
-
- This function returns the handler function that is the
- implementation of the operation corresponding to
- \var{operation-code} in \var{object-set}. When set with
- \code{setf}, the setter function establishes the new handler. The
- \code{serve-}\var{operation} functions exported from the
- \code{extensions} package for X events (\pxlref{x-serve-mumbles})
- call this on behalf of the user when announcing a new operation for
- an object set.
-\end{defun}
-
-\begin{defun}{system:}{add-xwindow-object}{%
- \args{\var{window} \var{object} \var{object-set}}}
-
- These functions add \var{port} or \var{window} to \var{object-set}.
- \var{Object} is an arbitrary Lisp object that is associated with the
- \var{port} or \var{window} capability. \var{Window} is a CLX
- window. When an event occurs, \code{system:serve-event} passes
- \var{object} as an argument to the handler function.
-\end{defun}
-
-
-%%\node The SERVE-EVENT Function, Using SERVE-EVENT with Unix File Descriptors, Object Sets, Event Dispatching with SERVE-EVENT
-\section{The SERVE-EVENT Function}
-
-The \code{system:serve-event} function is the standard way for an application
-to wait for something to happen. For example, the Lisp system calls
-\code{system:serve-event} when it wants input from X or a terminal stream.
-The idea behind \code{system:serve-event} is that it knows the appropriate
-action to take when any interesting event happens. If an application calls
-\code{system:serve-event} when it is idle, then any other applications with
-pending events can run. This allows several applications to run ``at the
-same time'' without interference, even though there is only one thread of
-control. Note that if an application is waiting for input of any kind,
-then other applications will get events.
-
-\begin{defun}{system:}{serve-event}{\args{\ampoptional{} \var{timeout}}}
-
- This function waits for an event to happen and then dispatches to
- the correct handler function. If specified, \var{timeout} is the
- number of seconds to wait before timing out. A time out of zero
- seconds is legal and causes \code{system:serve-event} to poll for
- any events immediately available for processing.
- \code{system:serve-event} returns \true{} if it serviced at least
- one event, and \nil{} otherwise. Depending on the application, when
- \code{system:serve-event} returns \true, you might want to call it
- repeatedly with a timeout of zero until it returns \nil.
-
- If input is available on any designated file descriptor, then this
- calls the appropriate handler function supplied by
- \code{system:add-fd-handler}.
-
- Since events for many different applications may arrive
- simultaneously, an application waiting for a specific event must
- loop on \code{system:serve-event} until the desired event happens.
- Since programs such as \hemlock{} call \code{system:serve-event} for
- input, applications usually do not need to call
- \code{system:serve-event} at all; \hemlock{} allows other
- application's handlers to run when it goes into an input wait.
-\end{defun}
-
-\begin{defun}{system:}{serve-all-events}{\args{\ampoptional{} \var{timeout}}}
-
- This function is similar to \code{system:serve-event}, except it
- serves all the pending events rather than just one. It returns
- \true{} if it serviced at least one event, and \nil{} otherwise.
-\end{defun}
-
-
-%%\node Using SERVE-EVENT with Unix File Descriptors, Using SERVE-EVENT with the CLX Interface to X, The SERVE-EVENT Function, Event Dispatching with SERVE-EVENT
-\section{Using SERVE-EVENT with Unix File Descriptors}
-Object sets are not available for use with file descriptors, as there are
-only two operations possible on file descriptors: input and output.
-Instead, a handler for either input or output can be registered with
-\code{system:serve-event} for a specific file descriptor. Whenever any input
-shows up, or output is possible on this file descriptor, the function
-associated with the handler for that descriptor is funcalled with the
-descriptor as it's single argument.
-
-\begin{defun}{system:}{add-fd-handler}{%
- \args{\var{fd} \var{direction} \var{function}}}
-
- This function installs and returns a new handler for the file
- descriptor \var{fd}. \var{direction} can be either \kwd{input} if
- the system should invoke the handler when input is available or
- \kwd{output} if the system should invoke the handler when output is
- possible. This returns a unique object representing the handler,
- and this is a suitable argument for \code{system:remove-fd-handler}
- \var{function} must take one argument, the file descriptor.
-\end{defun}
-
-\begin{defun}{system:}{remove-fd-handler}{\args{\var{handler}}}
-
- This function removes \var{handler}, that \code{add-fd-handler} must
- have previously returned.
-\end{defun}
-
-\begin{defmac}{system:}{with-fd-handler}{%
- \args{(\var{direction} \var{fd} \var{function})
- \mstar{\var{form}}}}
-
- This macro executes the supplied forms with a handler installed
- using \var{fd}, \var{direction}, and \var{function}. See
- \code{system:add-fd-handler}.
-\end{defmac}
-
-\begin{defun}{system:}{wait-until-fd-usable}{%
- \args{\var{direction} \var{fd} \ampoptional{} \var{timeout}}}
-
- This function waits for up to \var{timeout} seconds for \var{fd} to
- become usable for \var{direction} (either \kwd{input} or
- \kwd{output}). If \var{timeout} is \nil{} or unspecified, this
- waits forever.
-\end{defun}
-
-\begin{defun}{system:}{invalidate-descriptor}{\args{\var{fd}}}
-
- This function removes all handlers associated with \var{fd}. This
- should only be used in drastic cases (such as I/O errors, but not
- necessarily EOF). Normally, you should use \code{remove-fd-handler}
- to remove the specific handler.
-\end{defun}
-
-\begin{comment}
-
-section{Using SERVE-EVENT with Matchmaker Interfaces}
-\label{ipc-serve-mumbles}
-Remember from section \ref{object-sets}, an object set is a collection of
-objects, ports in this case, with some set of operations, message ID's, with
-corresponding implementations, the same handler functions.
-
-Matchmaker uses the object set operations to implement servers. For
-each server interface \i{XXX}, Matchmaker defines a function,
-\code{serve-}\i{XXX}, of two arguments, an object set and a function.
-The \code{serve-}\i{XXX} function establishes the function as the
-implementation of the \i{XXX} operation in the object set. Recall
-from section \ref{object-sets}, \code{system:add-port-object}
-associates some Lisp object with a port in an object set. When
-\code{system:serve-event} notices activity on a port, it calls the
-function given to \code{serve-}\i{XXX} with the object given to
-\code{system:add-port-object} and the input parameters specified in
-the message definition. The return values from the function are used
-as the output parameters for the message, if any.
-\code{serve-}\i{XXX} functions are also generated for each \i{server
- message} and asynchronous user interface.
-
-To use a Lisp server:
-\begin{itemize}
-
-\item Create an object set.
-
-\item Define some operations on it using the \code{serve-}\i{XXX}
- functions.
-
-\item Create an object for every port on which you receive requests.
-
-\item Call \code{system:serve-event} to service an RPC request.
-\end{itemize}
-
-
-Object sets allow many servers in the same Lisp to operate without knowing
-about each other. There can be multiple implementations of the same interface
-with different operation handlers established in distinct object sets. This
-property is especially useful when handling emergency messages.
-
-\end{comment}
-
-%%\node Using SERVE-EVENT with the CLX Interface to X, A SERVE-EVENT Example, Using SERVE-EVENT with Unix File Descriptors, Event Dispatching with SERVE-EVENT
-\section{Using SERVE-EVENT with the CLX Interface to X}
-\label{x-serve-mumbles}
-Remember from section \ref{object-sets}, an object set is a collection of
-objects, CLX windows in this case, with some set of operations, event keywords,
-with corresponding implementations, the same handler functions. Since X allows
-multiple display connections from a given process, you can avoid using object
-sets if every window in an application or display connection behaves the same.
-If a particular X application on a single display connection has windows that
-want to handle certain events differently, then using object sets is a
-convenient way to organize this since you need some way to map the window/event
-combination to the appropriate functionality.
-
-The following is a discussion of functions exported from the \code{extensions}
-package that facilitate handling CLX events through \code{system:serve-event}.
-The first two routines are useful regardless of whether you use
-\code{system:serve-event}:
-\begin{defun}{ext:}{open-clx-display}{%
- \args{\ampoptional{} \var{string}}}
-
- This function parses \var{string} for an X display specification
- including display and screen numbers. \var{String} defaults to the
- following:
- \begin{example}
- (cdr (assoc :display ext:*environment-list* :test #'eq))
- \end{example}
- If any field in the display specification is missing, this signals
- an error. \code{ext:open-clx-display} returns the CLX display and
- screen.
-\end{defun}
-
-\begin{defun}{ext:}{flush-display-events}{\args{\var{display}}}
-
- This function flushes all the events in \var{display}'s event queue
- including the current event, in case the user calls this from within
- an event handler.
-\end{defun}
-
-
-\begin{comment}
-* Without Object Sets::
-* With Object Sets::
-\end{comment}
-
-%%\node Without Object Sets, With Object Sets, Using SERVE-EVENT with the CLX Interface to X, Using SERVE-EVENT with the CLX Interface to X
-\subsection{Without Object Sets}
-Since most applications that use CLX, can avoid the complexity of object sets,
-these routines are described in a separate section. The routines described in
-the next section that use the object set mechanism are based on these
-interfaces.
-
-\begin{defun}{ext:}{enable-clx-event-handling}{%
- \args{\var{display} \var{handler}}}
-
- This function causes \code{system:serve-event} to notice when there
- is input on \var{display}'s connection to the X11 server. When this
- happens, \code{system:serve-event} invokes \var{handler} on
- \var{display} in a dynamic context with an error handler bound that
- flushes all events from \var{display} and returns. By returning,
- the error handler declines to handle the error, but it will have
- cleared all events; thus, entering the debugger will not result in
- infinite errors due to streams that wait via
- \code{system:serve-event} for input. Calling this repeatedly on the
- same \var{display} establishes \var{handler} as a new handler,
- replacing any previous one for \var{display}.
-\end{defun}
-
-\begin{defun}{ext:}{disable-clx-event-handling}{\args{\var{display}}}
-
- This function undoes the effect of
- \code{ext:enable-clx-event-handling}.
-\end{defun}
-
-\begin{defmac}{ext:}{with-clx-event-handling}{%
- \args{(\var{display} \var{handler}) \mstar{form}}}
-
- This macro evaluates each \var{form} in a context where
- \code{system:serve-event} invokes \var{handler} on \var{display}
- whenever there is input on \var{display}'s connection to the X
- server. This destroys any previously established handler for
- \var{display}.
-\end{defmac}
-
-
-%%\node With Object Sets, , Without Object Sets, Using SERVE-EVENT with the CLX Interface to X
-\subsection{With Object Sets}
-This section discusses the use of object sets and
-\code{system:serve-event} to handle CLX events. This is necessary
-when a single X application has distinct windows that want to handle
-the same events in different ways. Basically, you need some way of
-asking for a given window which way you want to handle some event
-because this event is handled differently depending on the window.
-Object sets provide this feature.
-
-For each CLX event-key symbol-name \i{XXX} (for example,
-\var{key-press}), there is a function \code{serve-}\i{XXX} of two
-arguments, an object set and a function. The \code{serve-}\i{XXX}
-function establishes the function as the handler for the \kwd{XXX}
-event in the object set. Recall from section \ref{object-sets},
-\code{system:add-xwindow-object} associates some Lisp object with a
-CLX window in an object set. When \code{system:serve-event} notices
-activity on a window, it calls the function given to
-\code{ext:enable-clx-event-handling}. If this function is
-\code{ext:object-set-event-handler}, it calls the function given to
-\code{serve-}\i{XXX}, passing the object given to
-\code{system:add-xwindow-object} and the event's slots as well as a
-couple other arguments described below.
-
-To use object sets in this way:
-\begin{itemize}
-
-\item Create an object set.
-
-\item Define some operations on it using the \code{serve-}\i{XXX}
- functions.
-
-\item Add an object for every window on which you receive requests.
- This can be the CLX window itself or some structure more meaningful
- to your application.
-
-\item Call \code{system:serve-event} to service an X event.
-\end{itemize}
-
-
-\begin{defun}{ext:}{object-set-event-handler}{%
- \args{\var{display}}}
-
- This function is a suitable argument to
- \code{ext:enable-clx-event-handling}. The actual event handlers
- defined for particular events within a given object set must take an
- argument for every slot in the appropriate event. In addition to
- the event slots, \code{ext:object-set-event-handler} passes the
- following arguments:
- \begin{itemize}
- \item The object, as established by
- \code{system:add-xwindow-object}, on which the event occurred.
- \item event-key, see \code{xlib:event-case}.
- \item send-event-p, see \code{xlib:event-case}.
- \end{itemize}
-
- Describing any \code{ext:serve-}\var{event-key-name} function, where
- \var{event-key-name} is an event-key symbol-name (for example,
- \code{ext:serve-key-press}), indicates exactly what all the
- arguments are in their correct order.
-
-%% \begin{comment}
-%% \code{ext:object-set-event-handler} ignores \kwd{no-exposure}
-%% events on pixmaps, issuing a warning if one occurs. It is only
-%% prepared to dispatch events for windows.
-%% \end{comment}
-
- When creating an object set for use with
- \code{ext:object-set-event-handler}, specify
- \code{ext:default-clx-event-handler} as the default handler for
- events in that object set. If no default handler is specified, and
- the system invokes the default default handler, it will cause an
- error since this function takes arguments suitable for handling port
- messages.
-\end{defun}
-
-
-%%\node A SERVE-EVENT Example, , Using SERVE-EVENT with the CLX Interface to X, Event Dispatching with SERVE-EVENT
-\section{A SERVE-EVENT Example}
-This section contains two examples using \code{system:serve-event}. The first
-one does not use object sets, and the second, slightly more complicated one
-does.
-
-
-\begin{comment}
-* Without Object Sets Example::
-* With Object Sets Example::
-\end{comment}
-
-%%\node Without Object Sets Example, With Object Sets Example, A SERVE-EVENT Example, A SERVE-EVENT Example
-\subsection{Without Object Sets Example}
-This example defines an input handler for a CLX display connection. It only
-recognizes \kwd{key-press} events. The body of the example loops over
-\code{system:serve-event} to get input.
-
-\begin{lisp}
-(in-package "SERVER-EXAMPLE")
-
-(defun my-input-handler (display)
- (xlib:event-case (display :timeout 0)
- (:key-press (event-window code state)
- (format t "KEY-PRESSED (Window = ~D) = ~S.~%"
- (xlib:window-id event-window)
- ;; See Hemlock Command Implementor's Manual for convenient
- ;; input mapping function.
- (ext:translate-character display code state))
- ;; Make XLIB:EVENT-CASE discard the event.
- t)))
-\end{lisp}
-\begin{lisp}
-(defun server-example ()
- "An example of using the SYSTEM:SERVE-EVENT function and object sets to
- handle CLX events."
- (let* ((display (ext:open-clx-display))
- (screen (display-default-screen display))
- (black (screen-black-pixel screen))
- (white (screen-white-pixel screen))
- (window (create-window :parent (screen-root screen)
- :x 0 :y 0 :width 200 :height 200
- :background white :border black
- :border-width 2
- :event-mask
- (xlib:make-event-mask :key-press))))
- ;; Wrap code in UNWIND-PROTECT, so we clean up after ourselves.
- (unwind-protect
- (progn
- ;; Enable event handling on the display.
- (ext:enable-clx-event-handling display #'my-input-handler)
- ;; Map the windows to the screen.
- (map-window window)
- ;; Make sure we send all our requests.
- (display-force-output display)
- ;; Call serve-event for 100,000 events or immediate timeouts.
- (dotimes (i 100000) (system:serve-event)))
- ;; Disable event handling on this display.
- (ext:disable-clx-event-handling display)
- ;; Get rid of the window.
- (destroy-window window)
- ;; Pick off any events the X server has already queued for our
- ;; windows, so we don't choke since SYSTEM:SERVE-EVENT is no longer
- ;; prepared to handle events for us.
- (loop
- (unless (deleting-window-drop-event *display* window)
- (return)))
- ;; Close the display.
- (xlib:close-display display))))
-
-(defun deleting-window-drop-event (display win)
- "Check for any events on win. If there is one, remove it from the
- event queue and return t; otherwise, return nil."
- (xlib:display-finish-output display)
- (let ((result nil))
- (xlib:process-event
- display :timeout 0
- :handler #'(lambda (&key event-window &allow-other-keys)
- (if (eq event-window win)
- (setf result t)
- nil)))
- result))
-\end{lisp}
-
-
-%%\node With Object Sets Example, , Without Object Sets Example, A SERVE-EVENT Example
-\subsection{With Object Sets Example}
-This example involves more work, but you get a little more for your effort. It
-defines two objects, \code{input-box} and \code{slider}, and establishes a
-\kwd{key-press} handler for each object, \code{key-pressed} and
-\code{slider-pressed}. We have two object sets because we handle events on the
-windows manifesting these objects differently, but the events come over the
-same display connection.
-
-\begin{lisp}
-(in-package "SERVER-EXAMPLE")
-
-(defstruct (input-box (:print-function print-input-box)
- (:constructor make-input-box (display window)))
- "Our program knows about input-boxes, and it doesn't care how they
- are implemented."
- display ; The CLX display on which my input-box is displayed.
- window) ; The CLX window in which the user types.
-;;;
-(defun print-input-box (object stream n)
- (declare (ignore n))
- (format stream "#<Input-Box ~S>" (input-box-display object)))
-
-(defvar *input-box-windows*
- (system:make-object-set "Input Box Windows"
- #'ext:default-clx-event-handler))
-
-(defun key-pressed (input-box event-key event-window root child
- same-screen-p x y root-x root-y modifiers time
- key-code send-event-p)
- "This is our :key-press event handler."
- (declare (ignore event-key root child same-screen-p x y
- root-x root-y time send-event-p))
- (format t "KEY-PRESSED (Window = ~D) = ~S.~%"
- (xlib:window-id event-window)
- ;; See Hemlock Command Implementor's Manual for convenient
- ;; input mapping function.
- (ext:translate-character (input-box-display input-box)
- key-code modifiers)))
-;;;
-(ext:serve-key-press *input-box-windows* #'key-pressed)
-\end{lisp}
-\begin{lisp}
-(defstruct (slider (:print-function print-slider)
- (:include input-box)
- (:constructor %make-slider
- (display window window-width max)))
- "Our program knows about sliders too, and these provide input values
- zero to max."
- bits-per-value ; bits per discrete value up to max.
- max) ; End value for slider.
-;;;
-(defun print-slider (object stream n)
- (declare (ignore n))
- (format stream "#<Slider ~S 0..~D>"
- (input-box-display object)
- (1- (slider-max object))))
-;;;
-(defun make-slider (display window max)
- (%make-slider display window
- (truncate (xlib:drawable-width window) max)
- max))
-
-(defvar *slider-windows*
- (system:make-object-set "Slider Windows"
- #'ext:default-clx-event-handler))
-
-(defun slider-pressed (slider event-key event-window root child
- same-screen-p x y root-x root-y modifiers time
- key-code send-event-p)
- "This is our :key-press event handler for sliders. Probably this is
- a mouse thing, but for simplicity here we take a character typed."
- (declare (ignore event-key root child same-screen-p x y
- root-x root-y time send-event-p))
- (format t "KEY-PRESSED (Window = ~D) = ~S --> ~D.~%"
- (xlib:window-id event-window)
- ;; See Hemlock Command Implementor's Manual for convenient
- ;; input mapping function.
- (ext:translate-character (input-box-display slider)
- key-code modifiers)
- (truncate x (slider-bits-per-value slider))))
-;;;
-(ext:serve-key-press *slider-windows* #'slider-pressed)
-\end{lisp}
-\begin{lisp}
-(defun server-example ()
- "An example of using the SYSTEM:SERVE-EVENT function and object sets to
- handle CLX events."
- (let* ((display (ext:open-clx-display))
- (screen (display-default-screen display))
- (black (screen-black-pixel screen))
- (white (screen-white-pixel screen))
- (iwindow (create-window :parent (screen-root screen)
- :x 0 :y 0 :width 200 :height 200
- :background white :border black
- :border-width 2
- :event-mask
- (xlib:make-event-mask :key-press)))
- (swindow (create-window :parent (screen-root screen)
- :x 0 :y 300 :width 200 :height 50
- :background white :border black
- :border-width 2
- :event-mask
- (xlib:make-event-mask :key-press)))
- (input-box (make-input-box display iwindow))
- (slider (make-slider display swindow 15)))
- ;; Wrap code in UNWIND-PROTECT, so we clean up after ourselves.
- (unwind-protect
- (progn
- ;; Enable event handling on the display.
- (ext:enable-clx-event-handling display
- #'ext:object-set-event-handler)
- ;; Add the windows to the appropriate object sets.
- (system:add-xwindow-object iwindow input-box
- *input-box-windows*)
- (system:add-xwindow-object swindow slider
- *slider-windows*)
- ;; Map the windows to the screen.
- (map-window iwindow)
- (map-window swindow)
- ;; Make sure we send all our requests.
- (display-force-output display)
- ;; Call server for 100,000 events or immediate timeouts.
- (dotimes (i 100000) (system:serve-event)))
- ;; Disable event handling on this display.
- (ext:disable-clx-event-handling display)
- (delete-window iwindow display)
- (delete-window swindow display)
- ;; Close the display.
- (xlib:close-display display))))
-\end{lisp}
-\begin{lisp}
-(defun delete-window (window display)
- ;; Remove the windows from the object sets before destroying them.
- (system:remove-xwindow-object window)
- ;; Destroy the window.
- (destroy-window window)
- ;; Pick off any events the X server has already queued for our
- ;; windows, so we don't choke since SYSTEM:SERVE-EVENT is no longer
- ;; prepared to handle events for us.
- (loop
- (unless (deleting-window-drop-event display window)
- (return))))
-
-(defun deleting-window-drop-event (display win)
- "Check for any events on win. If there is one, remove it from the
- event queue and return t; otherwise, return nil."
- (xlib:display-finish-output display)
- (let ((result nil))
- (xlib:process-event
- display :timeout 0
- :handler #'(lambda (&key event-window &allow-other-keys)
- (if (eq event-window win)
- (setf result t)
- nil)))
- result))
-\end{lisp}
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/alien.ms}
-
-%%\node Alien Objects, Interprocess Communication under LISP, Event Dispatching with SERVE-EVENT, Top
-\chapter{Alien Objects}
-\label{aliens}
-\begin{center}
-\b{By Robert MacLachlan and William Lott}
-\end{center}
-\vspace{1 cm}
-
-\begin{comment}
-* Introduction to Aliens::
-* Alien Types::
-* Alien Operations::
-* Alien Variables::
-* Alien Data Structure Example::
-* Loading Unix Object Files::
-* Alien Function Calls::
-* Step-by-Step Alien Example::
-\end{comment}
-
-%%\node Introduction to Aliens, Alien Types, Alien Objects, Alien Objects
-\section{Introduction to Aliens}
-
-Because of Lisp's emphasis on dynamic memory allocation and garbage
-collection, Lisp implementations use unconventional memory representations
-for objects. This representation mismatch creates problems when a Lisp
-program must share objects with programs written in another language. There
-are three different approaches to establishing communication:
-\begin{itemize}
-\item The burden can be placed on the foreign program (and programmer) by
-requiring the use of Lisp object representations. The main difficulty with
-this approach is that either the foreign program must be written with Lisp
-interaction in mind, or a substantial amount of foreign ``glue'' code must be
-written to perform the translation.
-
-\item The Lisp system can automatically convert objects back and forth
-between the Lisp and foreign representations. This is convenient, but
-translation becomes prohibitively slow when large or complex data structures
-must be shared.
-
-\item The Lisp program can directly manipulate foreign objects through the
-use of extensions to the Lisp language. Most Lisp systems make use of
-this approach, but the language for describing types and expressing
-accesses is often not powerful enough for complex objects to be easily
-manipulated.
-\end{itemize}
-\cmucl{} relies primarily on the automatic conversion and direct manipulation
-approaches: Aliens of simple scalar types are automatically converted,
-while complex types are directly manipulated in their foreign
-representation. Any foreign objects that can't automatically be
-converted into Lisp values are represented by objects of type
-\code{alien-value}. Since Lisp is a dynamically typed language, even
-foreign objects must have a run-time type; this type information is
-provided by encapsulating the raw pointer to the foreign data within an
-\code{alien-value} object.
-
-The Alien type language and operations are most similar to those of the
-C language, but Aliens can also be used when communicating with most
-other languages that can be linked with C.
-
-%%\f
-%%\node Alien Types, Alien Operations, Introduction to Aliens, Alien Objects
-\section{Alien Types}
-
-Alien types have a description language based on nested list structure. For
-example:
-\begin{example}
-struct foo \{
- int a;
- struct foo *b[100];
-\};
-\end{example}
-has the corresponding Alien type:
-\begin{lisp}
-(struct foo
- (a int)
- (b (array (* (struct foo)) 100)))
-\end{lisp}
-
-
-\begin{comment}
-* Defining Alien Types::
-* Alien Types and Lisp Types::
-* Alien Type Specifiers::
-* The C-Call Package::
-\end{comment}
-
-%%\node Defining Alien Types, Alien Types and Lisp Types, Alien Types, Alien Types
-\subsection{Defining Alien Types}
-
-Types may be either named or anonymous. With structure and union
-types, the name is part of the type specifier, allowing recursively
-defined types such as:
-\begin{lisp}
-(struct foo (a (* (struct foo))))
-\end{lisp}
-An anonymous structure or union type is specified by using the name
-\nil. The \funref{with-alien} macro defines a local scope which
-``captures'' any named type definitions. Other types are not
-inherently named, but can be given named abbreviations using
-\code{def-alien-type}.
-
-\begin{defmac}{alien:}{def-alien-type}{name type}
-
- This macro globally defines \var{name} as a shorthand for the Alien
- type \var{type}. When introducing global structure and union type
- definitions, \var{name} may be \nil, in which case the name to
- define is taken from the type's name.
-\end{defmac}
-
-
-%%\node Alien Types and Lisp Types, Alien Type Specifiers, Defining Alien Types, Alien Types
-\subsection{Alien Types and Lisp Types}
-
-The Alien types form a subsystem of the \cmucl{} type system. An
-\code{alien} type specifier provides a way to use any Alien type as a
-Lisp type specifier. For example
-\begin{lisp}
-(typep foo '(alien (* int)))
-\end{lisp}
-can be used to determine whether \code{foo} is a pointer to an
-\code{int}. \code{alien} type specifiers can be used in the same ways
-as ordinary type specifiers (like \code{string}.) Alien type
-declarations are subject to the same precise type checking as any
-other declaration (section \xlref{precise-type-checks}.)
-
-Note that the Alien type system overlaps with normal Lisp type
-specifiers in some cases. For example, the type specifier
-\code{(alien single-float)} is identical to \code{single-float}, since
-Alien floats are automatically converted to Lisp floats. When
-\code{type-of} is called on an Alien value that is not automatically
-converted to a Lisp value, then it will return an \code{alien} type
-specifier.
-
-%%\node Alien Type Specifiers, The C-Call Package, Alien Types and Lisp Types, Alien Types
-\subsection{Alien Type Specifiers}
-
-Some Alien type names are \clisp symbols, but the names are
-still exported from the \code{alien} package, so it is legal to say
-\code{alien:single-float}. These are the basic Alien type specifiers:
-
-\begin{deftp}{Alien type}{*}{%
- \args{\var{type}}}
-
- A pointer to an object of the specified \var{type}. If \var{type}
- is \true, then it means a pointer to anything, similar to
- ``\code{void *}'' in ANSI C. Currently, the only way to detect a
- null pointer is:
-\begin{lisp}
- (zerop (sap-int (alien-sap \var{ptr})))
-\end{lisp}
-\xlref{system-area-pointers}
-\end{deftp}
-
-\begin{deftp}{Alien type}{array}{\var{type} \mstar{\var{dimension}}}
-
- An array of the specified \var{dimensions}, holding elements of type
- \var{type}. Note that \code{(* int)} and \code{(array int)} are
- considered to be different types when type checking is done; pointer
- and array types must be explicitly coerced using \code{cast}.
-
- Arrays are accessed using \code{deref}, passing the indices as
- additional arguments. Elements are stored in column-major order (as
- in C), so the first dimension determines only the size of the memory
- block, and not the layout of the higher dimensions. An array whose
- first dimension is variable may be specified by using \nil{} as the
- first dimension. Fixed-size arrays can be allocated as array
- elements, structure slots or \code{with-alien} variables. Dynamic
- arrays can only be allocated using \funref{make-alien}.
-\end{deftp}
-
-\begin{deftp}{Alien type}{struct}{\var{name}
- \mstar{(\var{field} \var{type} \mopt{\var{bits}})}}
-
- A structure type with the specified \var{name} and \var{fields}.
- Fields are allocated at the same positions used by the
- implementation's C compiler. \var{bits} is intended for C-like bit
- field support, but is currently unused. If \var{name} is \false,
- then the type is anonymous.
-
- If a named Alien \code{struct} specifier is passed to
- \funref{def-alien-type} or \funref{with-alien}, then this defines,
- respectively, a new global or local Alien structure type. If no
- \var{fields} are specified, then the fields are taken from the
- current (local or global) Alien structure type definition of
- \var{name}.
-\end{deftp}
-
-\begin{deftp}{Alien type}{union}{\var{name}
- \mstar{(\var{field} \var{type} \mopt{\var{bits}})}}
-
- Similar to \code{struct}, but defines a union type. All fields are
- allocated at the same offset, and the size of the union is the size
- of the largest field. The programmer must determine which field is
- active from context.
-\end{deftp}
-
-\begin{deftp}{Alien type}{enum}{\var{name} \mstar{\var{spec}}}
-
- An enumeration type that maps between integer values and keywords.
- If \var{name} is \false, then the type is anonymous. Each
- \var{spec} is either a keyword, or a list \code{(\var{keyword}
- \var{value})}. If \var{integer} is not supplied, then it defaults
- to one greater than the value for the preceding spec (or to zero if
- it is the first spec.)
-\end{deftp}
-
-\begin{deftp}{Alien type}{signed}{\mopt{\var{bits}}}
- A signed integer with the specified number of bits precision. The
- upper limit on integer precision is determined by the machine's word
- size. If no size is specified, the maximum size will be used.
-\end{deftp}
-
-\begin{deftp}{Alien type}{integer}{\mopt{\var{bits}}}
- Identical to \code{signed}---the distinction between \code{signed}
- and \code{integer} is purely stylistic.
-\end{deftp}
-
-\begin{deftp}{Alien type}{unsigned}{\mopt{\var{bits}}}
- Like \code{signed}, but specifies an unsigned integer.
-\end{deftp}
-
-\begin{deftp}{Alien type}{boolean}{\mopt{\var{bits}}}
- Similar to an enumeration type that maps \code{0} to \false{} and
- all other values to \true. \var{bits} determines the amount of
- storage allocated to hold the truth value.
-\end{deftp}
-
-\begin{deftp}{Alien type}{single-float}{}
- A floating-point number in IEEE single format.
-\end{deftp}
-
-\begin{deftp}{Alien type}{double-float}{}
- A floating-point number in IEEE double format.
-\end{deftp}
-
-\begin{deftp}{Alien type}{function}{\var{result-type} \mstar{\var{arg-type}}}
- \label{alien-function-types}
- A Alien function that takes arguments of the specified
- \var{arg-types} and returns a result of type \var{result-type}.
- Note that the only context where a \code{function} type is directly
- specified is in the argument to \code{alien-funcall} (see section
- \funref{alien-funcall}.) In all other contexts, functions are
- represented by function pointer types: \code{(* (function ...))}.
-\end{deftp}
-
-\begin{deftp}{Alien type}{system-area-pointer}{}
- A pointer which is represented in Lisp as a
- \code{system-area-pointer} object (\pxlref{system-area-pointers}.)
-\end{deftp}
-
-%%\node The C-Call Package, , Alien Type Specifiers, Alien Types
-\subsection{The C-Call Package}
-
-The \code{c-call} package exports these type-equivalents to the C type
-of the same name: \code{char}, \code{short}, \code{int}, \code{long},
-\code{unsigned-char}, \code{unsigned-short}, \code{unsigned-int},
-\code{unsigned-long}, \code{float}, \code{double}. \code{c-call} also
-exports these types:
-
-\begin{deftp}{Alien type}{void}{}
- This type is used in function types to declare that no useful value
- is returned. Evaluation of an \code{alien-funcall} form will return
- zero values.
-\end{deftp}
-
-\begin{deftp}{Alien type}{c-string}{}
- This type is similar to \code{(* char)}, but is interpreted as a
- null-terminated string, and is automatically converted into a Lisp
- string when accessed. If the pointer is C \code{NULL} (or 0), then
- accessing gives Lisp \false.
-
- Assigning a Lisp string to a \code{c-string} structure field or
- variable stores the contents of the string to the memory already
- pointed to by that variable. When an Alien of type \code{(* char)}
- is assigned to a \code{c-string}, then the \code{c-string} pointer
- is assigned to. This allows \code{c-string} pointers to be
- initialized. For example:
-\begin{lisp}
- (def-alien-type nil (struct foo (str c-string)))
-
- (defun make-foo (str) (let ((my-foo (make-alien (struct foo))))
- (setf (slot my-foo 'str) (make-alien char (length str))) (setf (slot
- my-foo 'str) str) my-foo))
-\end{lisp}
-Storing Lisp \false{} writes C \code{NULL} to the \code{c-string}
-pointer.
-\end{deftp}
-
-%%\f
-%%\node Alien Operations, Alien Variables, Alien Types, Alien Objects
-\section{Alien Operations}
-
-This section describes the basic operations on Alien values.
-
-\begin{comment}
-* Alien Access Operations::
-* Alien Coercion Operations::
-* Alien Dynamic Allocation::
-\end{comment}
-
-%%\node Alien Access Operations, Alien Coercion Operations, Alien Operations, Alien Operations
-\subsection{Alien Access Operations}
-
-\begin{defun}{alien:}{deref}{\args{\var{pointer-or-array} \amprest \var{indices}}}
-
- This function returns the value pointed to by an Alien pointer or
- the value of an Alien array element. If a pointer, an optional
- single index can be specified to give the equivalent of C pointer
- arithmetic; this index is scaled by the size of the type pointed to.
- If an array, the number of indices must be the same as the number of
- dimensions in the array type. \code{deref} can be set with
- \code{setf} to assign a new value.
-\end{defun}
-
-\begin{defun}{alien:}{slot}{\args{\var{struct-or-union} \var{slot-name}}}
-
- This function extracts the value of slot \var{slot-name} from the an
- Alien \code{struct} or \code{union}. If \var{struct-or-union} is a
- pointer to a structure or union, then it is automatically
- dereferenced. This can be set with \code{setf} to assign a new
- value. Note that \var{slot-name} is evaluated, and need not be a
- compile-time constant (but only constant slot accesses are
- efficiently compiled.)
-\end{defun}
-
-%%\node Alien Coercion Operations, Alien Dynamic Allocation, Alien Access Operations, Alien Operations
-\subsection{Alien Coercion Operations}
-
-\begin{defmac}{alien:}{addr}{\var{alien-expr}}
-
- This macro returns a pointer to the location specified by
- \var{alien-expr}, which must be either an Alien variable, a use of
- \code{deref}, a use of \code{slot}, or a use of
- \funref{extern-alien}.
-\end{defmac}
-
-\begin{defmac}{alien:}{cast}{\var{alien} \var{new-type}}
-
- This macro converts \var{alien} to a new Alien with the specified
- \var{new-type}. Both types must be an Alien pointer, array or
- function type. Note that the result is not \code{eq} to the
- argument, but does refer to the same data bits.
-\end{defmac}
-
-\begin{defmac}{alien:}{sap-alien}{\var{sap} \var{type}}
- \defunx[alien:]{alien-sap}{\var{alien-value}}
-
- \code{sap-alien} converts \var{sap} (a system area pointer
- \pxlref{system-area-pointers}) to an Alien value with the specified
- \var{type}. \var{type} is not evaluated.
-
-\code{alien-sap} returns the SAP which points to \var{alien-value}'s
-data.
-
-The \var{type} to \code{sap-alien} and the type of the \var{alien-value} to
-\code{alien-sap} must some Alien pointer, array or record type.
-\end{defmac}
-
-%%\node Alien Dynamic Allocation, , Alien Coercion Operations, Alien Operations
-\subsection{Alien Dynamic Allocation}
-
-Dynamic Aliens are allocated using the \code{malloc} library, so foreign code
-can call \code{free} on the result of \code{make-alien}, and Lisp code can
-call \code{free-alien} on objects allocated by foreign code.
-
-\begin{defmac}{alien:}{make-alien}{\var{type} \mopt{\var{size}}}
-
- This macro returns a dynamically allocated Alien of the specified
- \var{type} (which is not evaluated.) The allocated memory is not
- initialized, and may contain arbitrary junk. If supplied,
- \var{size} is an expression to evaluate to compute the size of the
- allocated object. There are two major cases:
- \begin{itemize}
- \item When \var{type} is an array type, an array of that type is
- allocated and a \var{pointer} to it is returned. Note that you
- must use \code{deref} to change the result to an array before you
- can use \code{deref} to read or write elements:
- \begin{lisp}
- (defvar *foo* (make-alien (array char 10)))
-
- (type-of *foo*) \result{} (alien (* (array (signed 8) 10)))
-
- (setf (deref (deref foo) 0) 10) \result{} 10
- \end{lisp}
- If supplied, \var{size} is used as the first dimension for the
- array.
-
- \item When \var{type} is any other type, then then an object for
- that type is allocated, and a \var{pointer} to it is returned. So
- \code{(make-alien int)} returns a \code{(* int)}. If \var{size}
- is specified, then a block of that many objects is allocated, with
- the result pointing to the first one.
- \end{itemize}
-\end{defmac}
-
-\begin{defun}{alien:}{free-alien}{\var{alien}}
-
- This function frees the storage for \var{alien} (which must have
- been allocated with \code{make-alien} or \code{malloc}.)
-\end{defun}
-
-See also \funref{with-alien}, which stack-allocates Aliens.
-
-%%\f
-%%\node Alien Variables, Alien Data Structure Example, Alien Operations, Alien Objects
-\section{Alien Variables}
-
-Both local (stack allocated) and external (C global) Alien variables are
-supported.
-
-\begin{comment}
-* Local Alien Variables::
-* External Alien Variables::
-\end{comment}
-
-%%\node Local Alien Variables, External Alien Variables, Alien Variables, Alien Variables
-\subsection{Local Alien Variables}
-
-\begin{defmac}{alien:}{with-alien}{\mstar{(\var{name} \var{type}
- \mopt{\var{initial-value}})} \mstar{form}}
-
- This macro establishes local alien variables with the specified
- Alien types and names for dynamic extent of the body. The variable
- \var{names} are established as symbol-macros; the bindings have
- lexical scope, and may be assigned with \code{setq} or \code{setf}.
- This form is analogous to defining a local variable in C: additional
- storage is allocated, and the initial value is copied.
-
- \code{with-alien} also establishes a new scope for named structures
- and unions. Any \var{type} specified for a variable may contain
- name structure or union types with the slots specified. Within the
- lexical scope of the binding specifiers and body, a locally defined
- structure type \var{foo} can be referenced by its name using:
-\begin{lisp}
- (struct foo)
-\end{lisp}
-\end{defmac}
-
-%%\node External Alien Variables, , Local Alien Variables, Alien Variables
-\subsection{External Alien Variables}
-\label{external-aliens}
-
-External Alien names are strings, and Lisp names are symbols. When an
-external Alien is represented using a Lisp variable, there must be a
-way to convert from one name syntax into the other. The macros
-\code{extern-alien}, \code{def-alien-variable} and
-\funref{def-alien-routine} use this conversion heuristic:
-\begin{itemize}
-\item Alien names are converted to Lisp names by uppercasing and
- replacing underscores with hyphens.
-
-\item Conversely, Lisp names are converted to Alien names by
- lowercasing and replacing hyphens with underscores.
-
-\item Both the Lisp symbol and Alien string names may be separately
- specified by using a list of the form:
-\begin{lisp}
- (\var{alien-string} \var{lisp-symbol})
-\end{lisp}
-\end{itemize}
-
-\begin{defmac}{alien:}{def-alien-variable}{\var{name} \var{type}}
-
- This macro defines \var{name} as an external Alien variable of the
- specified Alien \var{type}. \var{name} and \var{type} are not
- evaluated. The Lisp name of the variable (see above) becomes a
- global Alien variable in the Lisp namespace. Global Alien variables
- are effectively ``global symbol macros''; a reference to the
- variable fetches the contents of the external variable. Similarly,
- setting the variable stores new contents---the new contents must be
- of the declared \var{type}.
-
- For example, it is often necessary to read the global C variable
- \code{errno} to determine why a particular function call failed. It
- is possible to define errno and make it accessible from Lisp by the
- following:
-\begin{lisp}
-(def-alien-variable "errno" int)
-
-;; Now it is possible to get the value of the C variable errno simply by
-;; referencing that Lisp variable:
-;;
-(print errno)
-\end{lisp}
-\end{defmac}
-
-\begin{defmac}{alien:}{extern-alien}{\var{name} \var{type}}
-
- This macro returns an Alien with the specified \var{type} which
- points to an externally defined value. \var{name} is not evaluated,
- and may be specified either as a string or a symbol. \var{type} is
- an unevaluated Alien type specifier.
-\end{defmac}
-
-%%\f
-%%\node Alien Data Structure Example, Loading Unix Object Files, Alien Variables, Alien Objects
-\section{Alien Data Structure Example}
-
-Now that we have Alien types, operations and variables, we can manipulate
-foreign data structures. This C declaration can be translated into the
-following Alien type:
-\begin{lisp}
-struct foo \{
- int a;
- struct foo *b[100];
-\};
-
- \myequiv
-
-(def-alien-type nil
- (struct foo
- (a int)
- (b (array (* (struct foo)) 100))))
-\end{lisp}
-
-With this definition, the following C expression can be translated in this way:
-\begin{example}
-struct foo f;
-f.b[7].a
-
- \myequiv
-
-(with-alien ((f (struct foo)))
- (slot (deref (slot f 'b) 7) 'a)
- ;;
- ;; Do something with f...
- )
-\end{example}
-
-
-Or consider this example of an external C variable and some accesses:
-\begin{example}
-struct c_struct \{
- short x, y;
- char a, b;
- int z;
- c_struct *n;
-\};
-
-extern struct c_struct *my_struct;
-
-my_struct->x++;
-my_struct->a = 5;
-my_struct = my_struct->n;
-\end{example}
-which can be made be manipulated in Lisp like this:
-\begin{lisp}
-(def-alien-type nil
- (struct c-struct
- (x short)
- (y short)
- (a char)
- (b char)
- (z int)
- (n (* c-struct))))
-
-(def-alien-variable "my_struct" (* c-struct))
-
-(incf (slot my-struct 'x))
-(setf (slot my-struct 'a) 5)
-(setq my-struct (slot my-struct 'n))
-\end{lisp}
-
-
-%%\f
-%%\node Loading Unix Object Files, Alien Function Calls, Alien Data Structure Example, Alien Objects
-\section{Loading Unix Object Files}
-
-Foreign object files are loaded into the running Lisp process by
-\code{load-foreign}. First, it runs the linker on the files and
-libraries, creating an absolute Unix object file. This object file is
-then loaded into into the currently running Lisp. The external
-symbols defining routines and variables are made available for future
-external references (e.g. by \code{extern-alien}.)
-\code{load-foreign} must be run before any of the defined symbols are
-referenced.
-
-Note that if a Lisp core image is saved (using \funref{save-lisp}), all
-loaded foreign code is lost when the image is restarted.
-
-\begin{defun}{alien:}{load-foreign}{%
- \args{\var{files} \keys{\kwd{libraries} \kwd{base-file} \kwd{env}}}}
-
- \var{files} is a \code{simple-string} or list of
- \code{simple-string}s specifying the names of the object files.
- \var{libraries} is a list of \code{simple-string}s specifying
- libraries in a format that \code{ld}, the Unix linker, expects. The
- default value for \var{libraries} is \code{("-lc")} (i.e., the
- standard C library). \var{base-file} is the file to use for the
- initial symbol table information. The default is the Lisp start up
- code: \file{path:lisp}. \var{env} should be a list of simple
- strings in the format of Unix environment variables (i.e.,
- \code{\var{A}=\var{B}}, where \var{A} is an environment variable and
- \var{B} is its value). The default value for \var{env} is the
- environment information available at the time Lisp was invoked.
- Unless you are certain that you want to change this, you should just
- use the default.
-\end{defun}
-
-%%\f
-%%\node Alien Function Calls, Step-by-Step Alien Example, Loading Unix Object Files, Alien Objects
-\section{Alien Function Calls}
-
-The foreign function call interface allows a Lisp program to call functions
-written in other languages. The current implementation of the foreign
-function call interface assumes a C calling convention and thus routines
-written in any language that adheres to this convention may be called from
-Lisp.
-
-Lisp sets up various interrupt handling routines and other environment
-information when it first starts up, and expects these to be in place at all
-times. The C functions called by Lisp should either not change the
-environment, especially the interrupt entry points, or should make sure
-that these entry points are restored when the C function returns to Lisp.
-If a C function makes changes without restoring things to the way they were
-when the C function was entered, there is no telling what will happen.
-
-\begin{comment}
-* alien-funcall:: The alien-funcall Primitive
-* def-alien-routine:: The def-alien-routine Macro
-* def-alien-routine Example::
-* Calling Lisp from C::
-\end{comment}
-
-%%\node alien-funcall, def-alien-routine, Alien Function Calls, Alien Function Calls
-\subsection{The alien-funcall Primitive}
-
-\begin{defun}{alien:}{alien-funcall}{%
- \args{\var{alien-function} \amprest{} \var{arguments}}}
-
- This function is the foreign function call primitive:
- \var{alien-function} is called with the supplied \var{arguments} and
- its value is returned. The \var{alien-function} is an arbitrary
- run-time expression; to call a constant function, use
- \funref{extern-alien} or \code{def-alien-routine}.
-
- The type of \var{alien-function} must be \code{(alien (function
- ...))} or \code{(alien (* (function ...)))},
- \xlref{alien-function-types}. The function type is used to
- determine how to call the function (as though it was declared with
- a prototype.) The type need not be known at compile time, but only
- known-type calls are efficiently compiled. Limitations:
- \begin{itemize}
- \item Structure type return values are not implemented.
- \item Passing of structures by value is not implemented.
- \end{itemize}
-\end{defun}
-
-Here is an example which allocates a \code{(struct foo)}, calls a foreign
-function to initialize it, then returns a Lisp vector of all the
-\code{(* (struct foo))} objects filled in by the foreign call:
-\begin{lisp}
-;;
-;; Allocate a foo on the stack.
-(with-alien ((f (struct foo)))
- ;;
- ;; Call some C function to fill in foo fields.
- (alien-funcall (extern-alien "mangle_foo" (function void (* foo)))
- (addr f))
- ;;
- ;; Find how many foos to use by getting the A field.
- (let* ((num (slot f 'a))
- (result (make-array num)))
- ;;
- ;; Get a pointer to the array so that we don't have to keep extracting it:
- (with-alien ((a (* (array (* (struct foo)) 100)) (addr (slot f 'b))))
- ;;
- ;; Loop over the first N elements and stash them in the result vector.
- (dotimes (i num)
- (setf (svref result i) (deref (deref a) i)))
- result)))
-\end{lisp}
-
-%%\node def-alien-routine, def-alien-routine Example, alien-funcall, Alien Function Calls
-\subsection{The def-alien-routine Macro}
-
-
-\begin{defmac}{alien:}{def-alien-routine}{\var{name} \var{result-type}
- \mstar{(\var{aname} \var{atype} \mopt{style})}}
-
- This macro is a convenience for automatically generating Lisp
- interfaces to simple foreign functions. The primary feature is the
- parameter style specification, which translates the C
- pass-by-reference idiom into additional return values.
-
- \var{name} is usually a string external symbol, but may also be a
- symbol Lisp name or a list of the foreign name and the Lisp name.
- If only one name is specified, the other is automatically derived,
- (\pxlref{external-aliens}.)
-
- \var{result-type} is the Alien type of the return value. Each
- remaining subform specifies an argument to the foreign function.
- \var{aname} is the symbol name of the argument to the constructed
- function (for documentation) and \var{atype} is the Alien type of
- corresponding foreign argument. The semantics of the actual call
- are the same as for \funref{alien-funcall}. \var{style} should be
- one of the following:
- \begin{Lentry}
- \item[\kwd{in}] specifies that the argument is passed by value.
- This is the default. \kwd{in} arguments have no corresponding
- return value from the Lisp function.
-
- \item[\kwd{out}] specifies a pass-by-reference output value. The
- type of the argument must be a pointer to a fixed sized object
- (such as an integer or pointer). \kwd{out} and \kwd{in-out}
- cannot be used with pointers to arrays, records or functions. An
- object of the correct size is allocated, and its address is passed
- to the foreign function. When the function returns, the contents
- of this location are returned as one of the values of the Lisp
- function.
-
- \item[\kwd{copy}] is similar to \kwd{in}, but the argument is copied
- to a pre-allocated object and a pointer to this object is passed
- to the foreign routine.
-
- \item[\kwd{in-out}] is a combination of \kwd{copy} and \kwd{out}.
- The argument is copied to a pre-allocated object and a pointer to
- this object is passed to the foreign routine. On return, the
- contents of this location is returned as an additional value.
- \end{Lentry}
- Any efficiency-critical foreign interface function should be inline
- expanded by preceding \code{def-alien-routine} with:
- \begin{lisp}
- (declaim (inline \var{lisp-name}))
- \end{lisp}
- In addition to avoiding the Lisp call overhead, this allows
- pointers, word-integers and floats to be passed using non-descriptor
- representations, avoiding consing (\pxlref{non-descriptor}.)
-\end{defmac}
-
-%%\node def-alien-routine Example, Calling Lisp from C, def-alien-routine, Alien Function Calls
-\subsection{def-alien-routine Example}
-
-Consider the C function \code{cfoo} with the following calling convention:
-\begin{example}
-cfoo (str, a, i)
- char *str;
- char *a; /* update */
- int *i; /* out */
-\{
-/* Body of cfoo. */
-\}
-\end{example}
-which can be described by the following call to \code{def-alien-routine}:
-\begin{lisp}
-(def-alien-routine "cfoo" void
- (str c-string)
- (a char :in-out)
- (i int :out))
-\end{lisp}
-The Lisp function \code{cfoo} will have two arguments (\var{str} and \var{a})
-and two return values (\var{a} and \var{i}).
-
-%%\node Calling Lisp from C, , def-alien-routine Example, Alien Function Calls
-\subsection{Calling Lisp from C}
-
-Calling Lisp functions from C is sometimes possible, but is rather hackish.
-See \code{funcall0} ... \code{funcall3} in the \file{lisp/arch.h}. The
-arguments must be valid CMU CL object descriptors (e.g. fixnums must be
-left-shifted by 2.) See \file{compiler/generic/objdef.lisp} or the derived
-file \file{lisp/internals.h} for details of the object representation.
-\file{lisp/internals.h} is mechanically generated, and is not part of the
-source distribution. It is distributed in the \file{docs/} directory of the
-binary distribution.
-
-Note that the garbage collector moves objects, and won't be able to fix up any
-references in C variables, so either turn GC off or don't keep Lisp pointers
-in C data unless they are to statically allocated objects. You can use
-\funref{purify} to place live data structures in static space so that they
-won't move during GC.
-
-\begin{changebar}
-\subsection{Accessing Lisp Arrays}
-
-Due to the way \cmucl{} manages memory, the amount of memory that can
-be dynamically allocated by \code{malloc} or \funref{make-alien} is
-limited\footnote{\cmucl{} mmaps a large piece of memory for it's own
- use and this memory is typically about 8 MB above the start of the C
- heap. Thus, only about 8 MB of memory can be dynamically
- allocated.}.
-
-To overcome this limitation, it is possible to access the content of
-Lisp arrays which are limited only by the amount of physical memory
-and swap space available. However, this technique is only useful if
-the foreign function takes pointers to memory instead of allocating
-memory for itself. In latter case, you will have to modify the
-foreign functions.
-
-This technique takes advantage of the fact that \cmucl{} has
-specialized array types (\pxlref{specialized-array-types}) that match
-a typical C array. For example, a \code{(simple-array double-float
- (100))} is stored in memory in essentially the same way as the C
-array \code{double x[100]} would be. The following function allows us
-to get the physical address of such a Lisp array:
-\begin{example}
-(defun array-data-address (array)
- "Return the physical address of where the actual data of an array is
-stored.
-
-ARRAY must be a specialized array type in CMU Lisp. This means ARRAY
-must be an array of one of the following types:
-
- double-float
- single-float
- (unsigned-byte 32)
- (unsigned-byte 16)
- (unsigned-byte 8)
- (signed-byte 32)
- (signed-byte 16)
- (signed-byte 8)
-"
- (declare (type (or #+signed-array (array (signed-byte 8))
- #+signed-array (array (signed-byte 16))
- #+signed-array (array (signed-byte 32))
- (array (unsigned-byte 8))
- (array (unsigned-byte 16))
- (array (unsigned-byte 32))
- (array single-float)
- (array double-float))
- array)
- (optimize (speed 3) (safety 0))
- (ext:optimize-interface (safety 3)))
- ;; WITH-ARRAY-DATA will get us to the actual data. However, because
- ;; the array could have been displaced, we need to know where the
- ;; data starts.
- (lisp::with-array-data ((data array)
- (start)
- (end))
- (declare (ignore end))
- ;; DATA is a specialized simple-array. Memory is laid out like this:
- ;;
- ;; byte offset Value
- ;; 0 type code (should be 70 for double-float vector)
- ;; 4 4 * number of elements in vector
- ;; 8 1st element of vector
- ;; ... ...
- ;;
- (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data))))
- (type-size (let ((type (array-element-type data)))
- (cond ((or (equal type '(signed-byte 8))
- (equal type '(unsigned-byte 8)))
- 1)
- ((or (equal type '(signed-byte 16))
- (equal type '(unsigned-byte 16)))
- 2)
- ((or (equal type '(signed-byte 32))
- (equal type '(unsigned-byte 32)))
- 4)
- ((equal type 'single-float)
- 4)
- ((equal type 'double-float)
- 8)
- (t
- (error "Unknown specialized array element type"))))))
- (declare (type (unsigned-byte 32) addr)
- (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3)))
- (system:int-sap (the (unsigned-byte 32)
- (+ addr (* type-size start)))))))
-\end{example}
-
-Assume we have the C function below that we wish to use:
-\begin{example}
- double dotprod(double* x, double* y, int n)
- \{
- int k;
- double sum = 0;
-
- for (k = 0; k < n; ++k) \{
- sum += x[k] * y[k];
- \}
- \}
-\end{example}
-The following example generates two large arrays in Lisp, and calls the C
-function to do the desired computation. This would not have been
-possible using \code{malloc} or \code{make-alien} since we need about
-16 MB of memory to hold the two arrays.
-\begin{example}
- (def-alien-routine "dotprod" double
- (x (* double-float) :in)
- (y (* double-float) :in)
- (n int :in))
-
- (let ((x (make-array 1000000 :element-type 'double-float))
- (y (make-array 1000000 :element-type 'double-float)))
- ;; Initialize X and Y somehow
- (let ((x-addr (system:int-sap (array-data-address x)))
- (y-addr (system:int-sap (array-data-address y))))
- (dotprod x-addr y-addr 1000000)))
-\end{example}
-In this example, it may be useful to wrap the inner \code{let}
-expression in an \code{unwind-protect} that first turns off garbage
-collection and then turns garbage collection on afterwards. This will
-prevent garbage collection from moving \code{x} and \code{y} after we
-have obtained the (now erroneous) addresses but before the call to
-\code{dotprod} is made.
-
-\end{changebar}
-%%\f
-%%\node Step-by-Step Alien Example, , Alien Function Calls, Alien Objects
-\section{Step-by-Step Alien Example}
-
-This section presents a complete example of an interface to a somewhat
-complicated C function. This example should give a fairly good idea
-of how to get the effect you want for almost any kind of C function.
-Suppose you have the following C function which you want to be able to
-call from Lisp in the file \file{test.c}:
-\begin{verbatim}
-struct c_struct
-{
- int x;
- char *s;
-};
-
-struct c_struct *c_function (i, s, r, a)
- int i;
- char *s;
- struct c_struct *r;
- int a[10];
-{
- int j;
- struct c_struct *r2;
-
- printf("i = %d\n", i);
- printf("s = %s\n", s);
- printf("r->x = %d\n", r->x);
- printf("r->s = %s\n", r->s);
- for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
- r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
- r2->x = i + 5;
- r2->s = "A C string";
- return(r2);
-};
-\end{verbatim}
-It is possible to call this function from Lisp using the file \file{test.lisp}
-whose contents is:
-\begin{lisp}
-;;; -*- Package: test-c-call -*-
-(in-package "TEST-C-CALL")
-(use-package "ALIEN")
-(use-package "C-CALL")
-
-;;; Define the record c-struct in Lisp.
-(def-alien-type nil
- (struct c-struct
- (x int)
- (s c-string)))
-
-;;; Define the Lisp function interface to the C routine. It returns a
-;;; pointer to a record of type c-struct. It accepts four parameters:
-;;; i, an int; s, a pointer to a string; r, a pointer to a c-struct
-;;; record; and a, a pointer to the array of 10 ints.
-;;;
-;;; The INLINE declaration eliminates some efficiency notes about heap
-;;; allocation of Alien values.
-(declaim (inline c-function))
-(def-alien-routine c-function
- (* (struct c-struct))
- (i int)
- (s c-string)
- (r (* (struct c-struct)))
- (a (array int 10)))
-
-;;; A function which sets up the parameters to the C function and
-;;; actually calls it.
-(defun call-cfun ()
- (with-alien ((ar (array int 10))
- (c-struct (struct c-struct)))
- (dotimes (i 10) ; Fill array.
- (setf (deref ar i) i))
- (setf (slot c-struct 'x) 20)
- (setf (slot c-struct 's) "A Lisp String")
-
- (with-alien ((res (* (struct c-struct))
- (c-function 5 "Another Lisp String" (addr c-struct) ar)))
- (format t "Returned from C function.~%")
- (multiple-value-prog1
- (values (slot res 'x)
- (slot res 's))
- ;;
- ;; Deallocate result \i{after} we are done using it.
- (free-alien res)))))
-\end{lisp}
-To execute the above example, it is necessary to compile the C routine as
-follows:
-\begin{example}
-cc -c test.c
-\end{example}
-In order to enable incremental loading with some linkers, you may need to say:
-\begin{example}
-cc -G 0 -c test.c
-\end{example}
-Once the C code has been compiled, you can start up Lisp and load it in:
-\begin{example}
-%lisp
-;;; Lisp should start up with its normal prompt.
-
-;;; Compile the Lisp file. This step can be done separately. You don't have
-;;; to recompile every time.
-* (compile-file "test.lisp")
-
-;;; Load the foreign object file to define the necessary symbols. This must
-;;; be done before loading any code that refers to these symbols. next block
-;;; of comments are actually the output of LOAD-FOREIGN. Different linkers
-;;; will give different warnings, but some warning about redefining the code
-;;; size is typical.
-* (load-foreign "test.o")
-
-;;; Running library:load-foreign.csh...
-;;; Loading object file...
-;;; Parsing symbol table...
-Warning: "_gp" moved from #x00C082C0 to #x00C08460.
-
-Warning: "end" moved from #x00C00340 to #x00C004E0.
-
-;;; o.k. now load the compiled Lisp object file.
-* (load "test")
-
-;;; Now we can call the routine that sets up the parameters and calls the C
-;;; function.
-* (test-c-call::call-cfun)
-
-;;; The C routine prints the following information to standard output.
-i = 5
-s = Another Lisp string
-r->x = 20
-r->s = A Lisp string
-a[0] = 0.
-a[1] = 1.
-a[2] = 2.
-a[3] = 3.
-a[4] = 4.
-a[5] = 5.
-a[6] = 6.
-a[7] = 7.
-a[8] = 8.
-a[9] = 9.
-;;; Lisp prints out the following information.
-Returned from C function.
-;;; Return values from the call to test-c-call::call-cfun.
-10
-"A C string"
-*
-\end{example}
-
-If any of the foreign functions do output, they should not be called from
-within Hemlock. Depending on the situation, various strange behavior occurs.
-Under X, the output goes to the window in which Lisp was started; on a
-terminal, the output will overwrite the Hemlock screen image; in a Hemlock
-slave, standard output is \file{/dev/null} by default, so any output is
-discarded.
-
-\hide{File:/afs/cs.cmu.edu/project/clisp/hackers/ram/docs/cmu-user/ipc.ms}
-
-%%\node Interprocess Communication under LISP, Debugger Programmer's Interface, Alien Objects, Top
-\chapter{Interprocess Communication under LISP}
-\begin{center}
-\b{Written by William Lott and Bill Chiles}
-\end{center}
-\label{remote}
-
-CMU Common Lisp offers a facility for interprocess communication (IPC)
-on top of using Unix system calls and the complications of that level
-of IPC. There is a simple remote-procedure-call (RPC) package build
-on top of TCP/IP sockets.
-
-
-\begin{comment}
-* The REMOTE Package::
-* The WIRE Package::
-* Out-Of-Band Data::
-\end{comment}
-
-%%\node The REMOTE Package, The WIRE Package, Interprocess Communication under LISP, Interprocess Communication under LISP
-\section{The REMOTE Package}
-The \code{remote} package provides simple RPC facility including
-interfaces for creating servers, connecting to already existing
-servers, and calling functions in other Lisp processes. The routines
-for establishing a connection between two processes,
-\code{create-request-server} and \code{connect-to-remote-server},
-return \var{wire} structures. A wire maintains the current state of
-a connection, and all the RPC forms require a wire to indicate where
-to send requests.
-
-
-\begin{comment}
-* Connecting Servers and Clients::
-* Remote Evaluations::
-* Remote Objects::
-* Host Addresses::
-\end{comment}
-
-%%\node Connecting Servers and Clients, Remote Evaluations, The REMOTE Package, The REMOTE Package
-\subsection{Connecting Servers and Clients}
-
-Before a client can connect to a server, it must know the network address on
-which the server accepts connections. Network addresses consist of a host
-address or name, and a port number. Host addresses are either a string of the
-form \code{VANCOUVER.SLISP.CS.CMU.EDU} or a 32 bit unsigned integer. Port
-numbers are 16 bit unsigned integers. Note: \var{port} in this context has
-nothing to do with Mach ports and message passing.
-
-When a process wants to receive connection requests (that is, become a
-server), it first picks an integer to use as the port. Only one server
-(Lisp or otherwise) can use a given port number on a given machine at
-any particular time. This can be an iterative process to find a free
-port: picking an integer and calling \code{create-request-server}. This
-function signals an error if the chosen port is unusable. You will
-probably want to write a loop using \code{handler-case}, catching
-conditions of type error, since this function does not signal more
-specific conditions.
-
-\begin{defun}{wire:}{create-request-server}{%
- \args{\var{port} \ampoptional{} \var{on-connect}}}
-
- \code{create-request-server} sets up the current Lisp to accept
- connections on the given port. If port is unavailable for any
- reason, this signals an error. When a client connects to this port,
- the acceptance mechanism makes a wire structure and invokes the
- \var{on-connect} function. Invoking this function has a couple
- purposes, and \var{on-connect} may be \nil{} in which case the
- system foregoes invoking any function at connect time.
-
- The \var{on-connect} function is both a hook that allows you access
- to the wire created by the acceptance mechanism, and it confirms the
- connection. This function takes two arguments, the wire and the
- host address of the connecting process. See the section on host
- addresses below. When \var{on-connect} is \nil, the request server
- allows all connections. When it is non-\nil, the function returns
- two values, whether to accept the connection and a function the
- system should call when the connection terminates. Either value may
- be \nil, but when the first value is \nil, the acceptance mechanism
- destroys the wire.
-
- \code{create-request-server} returns an object that
- \code{destroy-request-server} uses to terminate a connection.
-\end{defun}
-
-\begin{defun}{wire:}{destroy-request-server}{\args{\var{server}}}
-
- \code{destroy-request-server} takes the result of
- \code{create-request-server} and terminates that server. Any
- existing connections remain intact, but all additional connection
- attempts will fail.
-\end{defun}
-
-\begin{defun}{wire:}{connect-to-remote-server}{%
- \args{\var{host} \var{port} \ampoptional{} \var{on-death}}}
-
- \code{connect-to-remote-server} attempts to connect to a remote
- server at the given \var{port} on \var{host} and returns a wire
- structure if it is successful. If \var{on-death} is non-\nil, it is
- a function the system invokes when this connection terminates.
-\end{defun}
-
-
-%%\node Remote Evaluations, Remote Objects, Connecting Servers and Clients, The REMOTE Package
-\subsection{Remote Evaluations}
-After the server and client have connected, they each have a wire
-allowing function evaluation in the other process. This RPC mechanism
-has three flavors: for side-effect only, for a single value, and for
-multiple values.
-
-Only a limited number of data types can be sent across wires as
-arguments for remote function calls and as return values: integers
-inclusively less than 32 bits in length, symbols, lists, and
-\var{remote-objects} (\pxlref{remote-objs}). The system sends symbols
-as two strings, the package name and the symbol name, and if the
-package doesn't exist remotely, the remote process signals an error.
-The system ignores other slots of symbols. Lists may be any tree of
-the above valid data types. To send other data types you must
-represent them in terms of these supported types. For example, you
-could use \code{prin1-to-string} locally, send the string, and use
-\code{read-from-string} remotely.
-
-\begin{defmac}{wire:}{remote}{%
- \args{\var{wire} \mstar{call-specs}}}
-
- The \code{remote} macro arranges for the process at the other end of
- \var{wire} to invoke each of the functions in the \var{call-specs}.
- To make sure the system sends the remote evaluation requests over
- the wire, you must call \code{wire-force-output}.
-
- Each of \var{call-specs} looks like a function call textually, but
- it has some odd constraints and semantics. The function position of
- the form must be the symbolic name of a function. \code{remote}
- evaluates each of the argument subforms for each of the
- \var{call-specs} locally in the current context, sending these
- values as the arguments for the functions.
-
- Consider the following example:
-\begin{verbatim}
-(defun write-remote-string (str)
- (declare (simple-string str))
- (wire:remote wire
- (write-string str)))
-\end{verbatim}
- The value of \code{str} in the local process is passed over the wire
- with a request to invoke \code{write-string} on the value. The
- system does not expect to remotely evaluate \code{str} for a value
- in the remote process.
-\end{defmac}
-
-\begin{defun}{wire:}{wire-force-output}{\args{\var{wire}}}
-
- \code{wire-force-output} flushes all internal buffers associated
- with \var{wire}, sending the remote requests. This is necessary
- after a call to \code{remote}.
-\end{defun}
-
-\begin{defmac}{wire:}{remote-value}{\args{\var{wire} \var{call-spec}}}
-
- The \code{remote-value} macro is similar to the \code{remote} macro.
- \code{remote-value} only takes one \var{call-spec}, and it returns
- the value returned by the function call in the remote process. The
- value must be a valid type the system can send over a wire, and
- there is no need to call \code{wire-force-output} in conjunction
- with this interface.
-
- If client unwinds past the call to \code{remote-value}, the server
- continues running, but the system ignores the value the server sends
- back.
-
- If the server unwinds past the remotely requested call, instead of
- returning normally, \code{remote-value} returns two values, \nil{}
- and \true. Otherwise this returns the result of the remote
- evaluation and \nil.
-\end{defmac}
-
-\begin{defmac}{wire:}{remote-value-bind}{%
- \args{\var{wire} (\mstar{variable}) remote-form
- \mstar{local-forms}}}
-
- \code{remote-value-bind} is similar to \code{multiple-value-bind}
- except the values bound come from \var{remote-form}'s evaluation in
- the remote process. The \var{local-forms} execute in an implicit
- \code{progn}.
-
- If the client unwinds past the call to \code{remote-value-bind}, the
- server continues running, but the system ignores the values the
- server sends back.
-
- If the server unwinds past the remotely requested call, instead of
- returning normally, the \var{local-forms} never execute, and
- \code{remote-value-bind} returns \nil.
-\end{defmac}
-
-
-%%\node Remote Objects, Host Addresses, Remote Evaluations, The REMOTE Package
-\subsection{Remote Objects}
-\label{remote-objs}
-
-The wire mechanism only directly supports a limited number of data
-types for transmission as arguments for remote function calls and as
-return values: integers inclusively less than 32 bits in length,
-symbols, lists. Sometimes it is useful to allow remote processes to
-refer to local data structures without allowing the remote process
-to operate on the data. We have \var{remote-objects} to support
-this without the need to represent the data structure in terms of
-the above data types, to send the representation to the remote
-process, to decode the representation, to later encode it again, and
-to send it back along the wire.
-
-You can convert any Lisp object into a remote-object. When you send
-a remote-object along a wire, the system simply sends a unique token
-for it. In the remote process, the system looks up the token and
-returns a remote-object for the token. When the remote process
-needs to refer to the original Lisp object as an argument to a
-remote call back or as a return value, it uses the remote-object it
-has which the system converts to the unique token, sending that
-along the wire to the originating process. Upon receipt in the
-first process, the system converts the token back to the same
-(\code{eq}) remote-object.
-
-\begin{defun}{wire:}{make-remote-object}{\args{\var{object}}}
-
- \code{make-remote-object} returns a remote-object that has
- \var{object} as its value. The remote-object can be passed across
- wires just like the directly supported wire data types.
-\end{defun}
-
-\begin{defun}{wire:}{remote-object-p}{\args{\var{object}}}
-
- The function \code{remote-object-p} returns \true{} if \var{object}
- is a remote object and \nil{} otherwise.
-\end{defun}
-
-\begin{defun}{wire:}{remote-object-local-p}{\args{\var{remote}}}
-
- The function \code{remote-object-local-p} returns \true{} if
- \var{remote} refers to an object in the local process. This is can
- only occur if the local process created \var{remote} with
- \code{make-remote-object}.
-\end{defun}
-
-\begin{defun}{wire:}{remote-object-eq}{\args{\var{obj1} \var{obj2}}}
-
- The function \code{remote-object-eq} returns \true{} if \var{obj1} and
- \var{obj2} refer to the same (\code{eq}) lisp object, regardless of
- which process created the remote-objects.
-\end{defun}
-
-\begin{defun}{wire:}{remote-object-value}{\args{\var{remote}}}
-
- This function returns the original object used to create the given
- remote object. It is an error if some other process originally
- created the remote-object.
-\end{defun}
-
-\begin{defun}{wire:}{forget-remote-translation}{\args{\var{object}}}
-
- This function removes the information and storage necessary to
- translate remote-objects back into \var{object}, so the next
- \code{gc} can reclaim the memory. You should use this when you no
- longer expect to receive references to \var{object}. If some remote
- process does send a reference to \var{object},
- \code{remote-object-value} signals an error.
-\end{defun}
-
-
-%%\node Host Addresses, , Remote Objects, The REMOTE Package
-\subsection{Host Addresses}
-The operating system maintains a database of all the valid host
-addresses. You can use this database to convert between host names
-and addresses and vice-versa.
-
-\begin{defun}{ext:}{lookup-host-entry}{\args{\var{host}}}
-
- \code{lookup-host-entry} searches the database for the given
- \var{host} and returns a host-entry structure for it. If it fails
- to find \var{host} in the database, it returns \nil. \var{Host} is
- either the address (as an integer) or the name (as a string) of the
- desired host.
-\end{defun}
-
-\begin{defun}{ext:}{host-entry-name}{\args{\var{host-entry}}}
- \defunx[ext:]{host-entry-aliases}{\args{\var{host-entry}}}
- \defunx[ext:]{host-entry-addr-list}{\args{\var{host-entry}}}
- \defunx[ext:]{host-entry-addr}{\args{\var{host-entry}}}
-
- \code{host-entry-name}, \code{host-entry-aliases}, and
- \code{host-entry-addr-list} each return the indicated slot from the
- host-entry structure. \code{host-entry-addr} returns the primary
- (first) address from the list returned by
- \code{host-entry-addr-list}.
-\end{defun}
-
-
-%%\node The WIRE Package, Out-Of-Band Data, The REMOTE Package, Interprocess Communication under LISP
-\section{The WIRE Package}
-
-The \code{wire} package provides for sending data along wires. The
-\code{remote} package sits on top of this package. All data sent
-with a given output routine must be read in the remote process with
-the complementary fetching routine. For example, if you send so a
-string with \code{wire-output-string}, the remote process must know
-to use \code{wire-get-string}. To avoid rigid data transfers and
-complicated code, the interface supports sending
-\var{tagged} data. With tagged data, the system sends a tag
-announcing the type of the next data, and the remote system takes
-care of fetching the appropriate type.
-
-When using interfaces at the wire level instead of the RPC level,
-the remote process must read everything sent by these routines. If
-the remote process leaves any input on the wire, it will later
-mistake the data for an RPC request causing unknown lossage.
-
-\begin{comment}
-* Untagged Data::
-* Tagged Data::
-* Making Your Own Wires::
-\end{comment}
-
-%%\node Untagged Data, Tagged Data, The WIRE Package, The WIRE Package
-\subsection{Untagged Data}
-When using these routines both ends of the wire know exactly what types are
-coming and going and in what order. This data is restricted to the following
-types:
-\begin{itemize}
-
-\item
-8 bit unsigned bytes.
-
-\item
-32 bit unsigned bytes.
-
-\item
-32 bit integers.
-
-\item
-simple-strings less than 65535 in length.
-\end{itemize}
-
-
-\begin{defun}{wire:}{wire-output-byte}{\args{\var{wire} \var{byte}}}
- \defunx[wire:]{wire-get-byte}{\args{\var{wire}}}
- \defunx[wire:]{wire-output-number}{\args{\var{wire} \var{number}}}
- \defunx[wire:]{wire-get-number}{\args{\var{wire} \ampoptional{}
- \var{signed}}}
- \defunx[wire:]{wire-output-string}{\args{\var{wire} \var{string}}}
- \defunx[wire:]{wire-get-string}{\args{\var{wire}}}
-
- These functions either output or input an object of the specified
- data type. When you use any of these output routines to send data
- across the wire, you must use the corresponding input routine
- interpret the data.
-\end{defun}
-
-
-%%\node Tagged Data, Making Your Own Wires, Untagged Data, The WIRE Package
-\subsection{Tagged Data}
-When using these routines, the system automatically transmits and interprets
-the tags for you, so both ends can figure out what kind of data transfers
-occur. Sending tagged data allows a greater variety of data types: integers
-inclusively less than 32 bits in length, symbols, lists, and \var{remote-objects}
-(\pxlref{remote-objs}). The system sends symbols as two strings, the
-package name and the symbol name, and if the package doesn't exist remotely,
-the remote process signals an error. The system ignores other slots of
-symbols. Lists may be any tree of the above valid data types. To send other
-data types you must represent them in terms of these supported types. For
-example, you could use \code{prin1-to-string} locally, send the string, and use
-\code{read-from-string} remotely.
-
-\begin{defun}{wire:}{wire-output-object}{%
- \args{\var{wire} \var{object} \ampoptional{} \var{cache-it}}}
- \defunx[wire:]{wire-get-object}{\args{\var{wire}}}
-
- The function \code{wire-output-object} sends \var{object} over
- \var{wire} preceded by a tag indicating its type.
-
- If \var{cache-it} is non-\nil, this function only sends \var{object}
- the first time it gets \var{object}. Each end of the wire
- associates a token with \var{object}, similar to remote-objects,
- allowing you to send the object more efficiently on successive
- transmissions. \var{cache-it} defaults to \true{} for symbols and
- \nil{} for other types. Since the RPC level requires function
- names, a high-level protocol based on a set of function calls saves
- time in sending the functions' names repeatedly.
-
- The function \code{wire-get-object} reads the results of
- \code{wire-output-object} and returns that object.
-\end{defun}
-
-
-%%\node Making Your Own Wires, , Tagged Data, The WIRE Package
-\subsection{Making Your Own Wires}
-You can create wires manually in addition to the \code{remote} package's
-interface creating them for you. To create a wire, you need a Unix \i{file
-descriptor}. If you are unfamiliar with Unix file descriptors, see section 2 of
-the Unix manual pages.
-
-\begin{defun}{wire:}{make-wire}{\args{\var{descriptor}}}
-
- The function \code{make-wire} creates a new wire when supplied with
- the file descriptor to use for the underlying I/O operations.
-\end{defun}
-
-\begin{defun}{wire:}{wire-p}{\args{\var{object}}}
-
- This function returns \true{} if \var{object} is indeed a wire,
- \nil{} otherwise.
-\end{defun}
-
-\begin{defun}{wire:}{wire-fd}{\args{\var{wire}}}
-
- This function returns the file descriptor used by the \var{wire}.
-\end{defun}
-
-
-%%\node Out-Of-Band Data, , The WIRE Package, Interprocess Communication under LISP
-\section{Out-Of-Band Data}
-
-The TCP/IP protocol allows users to send data asynchronously, otherwise
-known as \var{out-of-band} data. When using this feature, the operating
-system interrupts the receiving process if this process has chosen to be
-notified about out-of-band data. The receiver can grab this input
-without affecting any information currently queued on the socket.
-Therefore, you can use this without interfering with any current
-activity due to other wire and remote interfaces.
-
-Unfortunately, most implementations of TCP/IP are broken, so use of
-out-of-band data is limited for safety reasons. You can only reliably
-send one character at a time.
-
-This routines in this section provide a mechanism for establishing
-handlers for out-of-band characters and for sending them out-of-band.
-These all take a Unix file descriptor instead of a wire, but you can
-fetch a wire's file descriptor with \code{wire-fd}.
-
-\begin{defun}{wire:}{add-oob-handler}{\args{\var{fd} \var{char} \var{handler}}}
-
- The function \code{add-oob-handler} arranges for \var{handler} to be
- called whenever \var{char} shows up as out-of-band data on the file
- descriptor \var{fd}.
-\end{defun}
-
-\begin{defun}{wire:}{remove-oob-handler}{\args{\var{fd} \var{char}}}
-
- This function removes the handler for the character \var{char} on
- the file descriptor \var{fd}.
-\end{defun}
-
-\begin{defun}{wire:}{remove-all-oob-handlers}{\args{\var{fd}}}
-
- This function removes all handlers for the file descriptor \var{fd}.
-\end{defun}
-
-\begin{defun}{wire:}{send-character-out-of-band}{\args{\var{fd} \var{char}}}
-
- This function Sends the character \var{char} down the file
- descriptor \var{fd} out-of-band.
-\end{defun}
-
-%%\f
-\hide{File:debug-int.tex}
-%%\node Debugger Programmer's Interface, Function Index, Interprocess Communication under LISP, Top
-\chapter{Debugger Programmer's Interface}
-\label{debug-internals}
-
-The debugger programmers interface is exported from from the
-\code{"DEBUG-INTERNALS"} or \code{"DI"} package. This is a CMU
-extension that allows debugging tools to be written without detailed
-knowledge of the compiler or run-time system.
-
-Some of the interface routines take a code-location as an argument. As
-described in the section on code-locations, some code-locations are
-unknown. When a function calls for a \var{basic-code-location}, it
-takes either type, but when it specifically names the argument
-\var{code-location}, the routine will signal an error if you give it an
-unknown code-location.
-
-\begin{comment}
-* DI Exceptional Conditions::
-* Debug-variables::
-* Frames::
-* Debug-functions::
-* Debug-blocks::
-* Breakpoints::
-* Code-locations::
-* Debug-sources::
-* Source Translation Utilities::
-\end{comment}
-
-%%\f
-%%\node DI Exceptional Conditions, Debug-variables, Debugger Programmer's Interface, Debugger Programmer's Interface
-\section{DI Exceptional Conditions}
-
-Some of these operations fail depending on the availability debugging
-information. In the most severe case, when someone saved a Lisp image
-stripping all debugging data structures, no operations are valid. In
-this case, even backtracing and finding frames is impossible. Some
-interfaces can simply return values indicating the lack of information,
-or their return values are naturally meaningful in light missing data.
-Other routines, as documented below, will signal
-\code{serious-condition}s when they discover awkward situations. This
-interface does not provide for programs to detect these situations other
-than by calling a routine that detects them and signals a condition.
-These are serious-conditions because the program using the interface
-must handle them before it can correctly continue execution. These
-debugging conditions are not errors since it is no fault of the
-programmers that the conditions occur.
-
-\begin{comment}
-* Debug-conditions::
-* Debug-errors::
-\end{comment}
-
-%%\node Debug-conditions, Debug-errors, DI Exceptional Conditions, DI Exceptional Conditions
-\subsection{Debug-conditions}
-
-The debug internals interface signals conditions when it can't adhere
-to its contract. These are serious-conditions because the program
-using the interface must handle them before it can correctly continue
-execution. These debugging conditions are not errors since it is no
-fault of the programmers that the conditions occur. The interface
-does not provide for programs to detect these situations other than
-calling a routine that detects them and signals a condition.
-
-
-\begin{deftp}{Condition}{debug-condition}{}
-
-This condition inherits from serious-condition, and all debug-conditions
-inherit from this. These must be handled, but they are not programmer errors.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{no-debug-info}{}
-
-This condition indicates there is absolutely no debugging information
-available.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{no-debug-function-returns}{}
-
-This condition indicates the system cannot return values from a frame since
-its debug-function lacks debug information details about returning values.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{no-debug-blocks}{}
-This condition indicates that a function was not compiled with debug-block
-information, but this information is necessary necessary for some requested
-operation.
-\end{deftp}
-
-\begin{deftp}{Condition}{no-debug-variables}{}
-Similar to \code{no-debug-blocks}, except that variable information was
-requested.
-\end{deftp}
-
-\begin{deftp}{Condition}{lambda-list-unavailable}{}
-Similar to \code{no-debug-blocks}, except that lambda list information was
-requested.
-\end{deftp}
-
-\begin{deftp}{Condition}{invalid-value}{}
-
-This condition indicates a debug-variable has \kwd{invalid} or \kwd{unknown}
-value in a particular frame.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{ambiguous-variable-name}{}
-
-This condition indicates a user supplied debug-variable name identifies more
-than one valid variable in a particular frame.
-\end{deftp}
-
-
-%%\node Debug-errors, , Debug-conditions, DI Exceptional Conditions
-\subsection{Debug-errors}
-
-These are programmer errors resulting from misuse of the debugging tools'
-programmers' interface. You could have avoided an occurrence of one of these
-by using some routine to check the use of the routine generating the error.
-
-
-\begin{deftp}{Condition}{debug-error}{}
-This condition inherits from error, and all user programming errors inherit
-from this condition.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{unhandled-debug-condition}{}
-This error results from a signalled \code{debug-condition} occurring
-without anyone handling it.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{unknown-code-location}{}
-This error indicates the invalid use of an unknown-code-location.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{unknown-debug-variable}{}
-
-This error indicates an attempt to use a debug-variable in conjunction with an
-inappropriate debug-function; for example, checking the variable's validity
-using a code-location in the wrong debug-function will signal this error.
-\end{deftp}
-
-
-\begin{deftp}{Condition}{frame-function-mismatch}{}
-
-This error indicates you called a function returned by
-\code{preprocess-for-eval}
-on a frame other than the one for which the function had been prepared.
-\end{deftp}
-
-
-%%\f
-%%\node Debug-variables, Frames, DI Exceptional Conditions, Debugger Programmer's Interface
-\section{Debug-variables}
-
-Debug-variables represent the constant information about where the system
-stores argument and local variable values. The system uniquely identifies with
-an integer every instance of a variable with a particular name and package. To
-access a value, you must supply the frame along with the debug-variable since
-these are particular to a function, not every instance of a variable on the
-stack.
-
-\begin{defun}{}{debug-variable-name}{\args{\var{debug-variable}}}
-
- This function returns the name of the \var{debug-variable}. The
- name is the name of the symbol used as an identifier when writing
- the code.
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-package}{\args{\var{debug-variable}}}
-
- This function returns the package name of the \var{debug-variable}.
- This is the package name of the symbol used as an identifier when
- writing the code.
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-symbol}{\args{\var{debug-variable}}}
-
- This function returns the symbol from interning
- \code{debug-variable-name} in the package named by
- \code{debug-variable-package}.
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-id}{\args{\var{debug-variable}}}
-
- This function returns the integer that makes \var{debug-variable}'s
- name and package name unique with respect to other
- \var{debug-variable}'s in the same function.
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-validity}{%
- \args{\var{debug-variable} \var{basic-code-location}}}
-
- This function returns three values reflecting the validity of
- \var{debug-variable}'s value at \var{basic-code-location}:
- \begin{Lentry}
- \item[\kwd{valid}] The value is known to be available.
- \item[\kwd{invalid}] The value is known to be unavailable.
- \item[\kwd{unknown}] The value's availability is unknown.
- \end{Lentry}
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-value}{\args{\var{debug-variable}
- \var{frame}}}
-
- This function returns the value stored for \var{debug-variable} in
- \var{frame}. The value may be invalid. This is \code{SETF}'able.
-\end{defun}
-
-
-\begin{defun}{}{debug-variable-valid-value}{%
- \args{\var{debug-variable} \var{frame}}}
-
- This function returns the value stored for \var{debug-variable} in
- \var{frame}. If the value is not \kwd{valid}, then this signals an
- \code{invalid-value} error.
-\end{defun}
-
-
-%%\f
-%%\node Frames, Debug-functions, Debug-variables, Debugger Programmer's Interface
-\section{Frames}
-
-Frames describe a particular call on the stack for a particular thread. This
-is the environment for name resolution, getting arguments and locals, and
-returning values. The stack conceptually grows up, so the top of the stack is
-the most recently called function.
-
-\code{top-frame}, \code{frame-down}, \code{frame-up}, and
-\code{frame-debug-function} can only fail when there is absolutely no
-debug information available. This can only happen when someone saved a
-Lisp image specifying that the system dump all debugging data.
-
-
-\begin{defun}{}{top-frame}{}
-
- This function never returns the frame for itself, always the frame
- before calling \code{top-frame}.
-\end{defun}
-
-
-\begin{defun}{}{frame-down}{\args{\var{frame}}}
-
- This returns the frame immediately below \var{frame} on the stack.
- When \var{frame} is the bottom of the stack, this returns \nil.
-\end{defun}
-
-
-\begin{defun}{}{frame-up}{\args{\var{frame}}}
-
- This returns the frame immediately above \var{frame} on the stack.
- When \var{frame} is the top of the stack, this returns \nil.
-\end{defun}
-
-
-\begin{defun}{}{frame-debug-function}{\args{\var{frame}}}
-
- This function returns the debug-function for the function whose call
- \var{frame} represents.
-\end{defun}
-
-
-\begin{defun}{}{frame-code-location}{\args{\var{frame}}}
-
- This function returns the code-location where \var{frame}'s
- debug-function will continue running when program execution returns
- to \var{frame}. If someone interrupted this frame, the result could
- be an unknown code-location.
-\end{defun}
-
-
-\begin{defun}{}{frame-catches}{\args{\var{frame}}}
-
- This function returns an a-list for all active catches in
- \var{frame} mapping catch tags to the code-locations at which the
- catch re-enters.
-\end{defun}
-
-
-\begin{defun}{}{eval-in-frame}{\args{\var{frame} \var{form}}}
-
- This evaluates \var{form} in \var{frame}'s environment. This can
- signal several different debug-conditions since its success relies
- on a variety of inexact debug information: \code{invalid-value},
- \code{ambiguous-variable-name}, \code{frame-function-mismatch}. See
- also \funref{preprocess-for-eval}.
-\end{defun}
-
-\begin{comment}
- \begin{defun}{}{return-from-frame}{\args{\var{frame} \var{values}}}
-
- This returns the elements in the list \var{values} as multiple
- values from \var{frame} as if the function \var{frame} represents
- returned these values. This signals a
- \code{no-debug-function-returns} condition when \var{frame}'s
- debug-function lacks information on returning values.
-
- \i{Not Yet Implemented}
- \end{defun}
-\end{comment}
-
-%%\f
-%%\node Debug-functions, Debug-blocks, Frames, Debugger Programmer's Interface
-\section {Debug-functions}
-
-Debug-functions represent the static information about a function determined at
-compile time---argument and variable storage, their lifetime information,
-etc. The debug-function also contains all the debug-blocks representing
-basic-blocks of code, and these contains information about specific
-code-locations in a debug-function.
-
-\begin{defmac}{}{do-debug-function-blocks}{%
- \args{(\var{block-var} \var{debug-function} \mopt{result-form})
- \mstar{form}}}
-
- This executes the forms in a context with \var{block-var} bound to
- each debug-block in \var{debug-function} successively.
- \var{Result-form} is an optional form to execute for a return value,
- and \code{do-debug-function-blocks} returns \nil if there is no
- \var{result-form}. This signals a \code{no-debug-blocks} condition
- when the \var{debug-function} lacks debug-block information.
-\end{defmac}
-
-
-\begin{defun}{}{debug-function-lambda-list}{\args{\var{debug-function}}}
-
- This function returns a list representing the lambda-list for
- \var{debug-function}. The list has the following structure:
- \begin{example}
- (required-var1 required-var2
- ...
- (:optional var3 suppliedp-var4)
- (:optional var5)
- ...
- (:rest var6) (:rest var7)
- ...
- (:keyword keyword-symbol var8 suppliedp-var9)
- (:keyword keyword-symbol var10)
- ...
- )
- \end{example}
- Each \code{var}\var{n} is a debug-variable; however, the symbol
- \kwd{deleted} appears instead whenever the argument remains
- unreferenced throughout \var{debug-function}.
-
- If there is no lambda-list information, this signals a
- \code{lambda-list-unavailable} condition.
-\end{defun}
-
-
-\begin{defmac}{}{do-debug-function-variables}{%
- \args{(\var{var} \var{debug-function} \mopt{result})
- \mstar{form}}}
-
- This macro executes each \var{form} in a context with \var{var}
- bound to each debug-variable in \var{debug-function}. This returns
- the value of executing \var{result} (defaults to \nil). This may
- iterate over only some of \var{debug-function}'s variables or none
- depending on debug policy; for example, possibly the compilation
- only preserved argument information.
-\end{defmac}
-
-
-\begin{defun}{}{debug-variable-info-available}{\args{\var{debug-function}}}
-
- This function returns whether there is any variable information for
- \var{debug-function}. This is useful for distinguishing whether
- there were no locals in a function or whether there was no variable
- information. For example, if \code{do-debug-function-variables}
- executes its forms zero times, then you can use this function to
- determine the reason.
-\end{defun}
-
-
-\begin{defun}{}{debug-function-symbol-variables}{%
- \args{\var{debug-function} \var{symbol}}}
-
- This function returns a list of debug-variables in
- \var{debug-function} having the same name and package as
- \var{symbol}. If \var{symbol} is uninterned, then this returns a
- list of debug-variables without package names and with the same name
- as \var{symbol}. The result of this function is limited to the
- availability of variable information in \var{debug-function}; for
- example, possibly \var{debug-function} only knows about its
- arguments.
-\end{defun}
-
-
-\begin{defun}{}{ambiguous-debug-variables}{%
- \args{\var{debug-function} \var{name-prefix-string}}}
-
- This function returns a list of debug-variables in
- \var{debug-function} whose names contain \var{name-prefix-string} as
- an initial substring. The result of this function is limited to the
- availability of variable information in \var{debug-function}; for
- example, possibly \var{debug-function} only knows about its
- arguments.
-\end{defun}
-
-
-\begin{defun}{}{preprocess-for-eval}{%
- \args{\var{form} \var{basic-code-location}}}
-
- This function returns a function of one argument that evaluates
- \var{form} in the lexical context of \var{basic-code-location}.
- This allows efficient repeated evaluation of \var{form} at a certain
- place in a function which could be useful for conditional breaking.
- This signals a \code{no-debug-variables} condition when the
- code-location's debug-function has no debug-variable information
- available. The returned function takes a frame as an argument. See
- also \funref{eval-in-frame}.
-\end{defun}
-
-
-\begin{defun}{}{function-debug-function}{\args{\var{function}}}
-
- This function returns a debug-function that represents debug
- information for \var{function}.
-\end{defun}
-
-
-\begin{defun}{}{debug-function-kind}{\args{\var{debug-function}}}
-
- This function returns the kind of function \var{debug-function}
- represents. The value is one of the following:
- \begin{Lentry}
- \item[\kwd{optional}] This kind of function is an entry point to an
- ordinary function. It handles optional defaulting, parsing
- keywords, etc.
- \item[\kwd{external}] This kind of function is an entry point to an
- ordinary function. It checks argument values and count and calls
- the defined function.
- \item[\kwd{top-level}] This kind of function executes one or more
- random top-level forms from a file.
- \item[\kwd{cleanup}] This kind of function represents the cleanup
- forms in an \code{unwind-protect}.
- \item[\nil] This kind of function is not one of the above; that is,
- it is not specially marked in any way.
- \end{Lentry}
-\end{defun}
-
-
-\begin{defun}{}{debug-function-function}{\args{\var{debug-function}}}
-
- This function returns the Common Lisp function associated with the
- \var{debug-function}. This returns \nil{} if the function is
- unavailable or is non-existent as a user callable function object.
-\end{defun}
-
-
-\begin{defun}{}{debug-function-name}{\args{\var{debug-function}}}
-
- This function returns the name of the function represented by
- \var{debug-function}. This may be a string or a cons; do not assume
- it is a symbol.
-\end{defun}
-
-
-%%\f
-%%\node Debug-blocks, Breakpoints, Debug-functions, Debugger Programmer's Interface
-\section{Debug-blocks}
-
-Debug-blocks contain information pertinent to a specific range of code in a
-debug-function.
-
-\begin{defmac}{}{do-debug-block-locations}{%
- \args{(\var{code-var} \var{debug-block} \mopt{result})
- \mstar{form}}}
-
- This macro executes each \var{form} in a context with \var{code-var}
- bound to each code-location in \var{debug-block}. This returns the
- value of executing \var{result} (defaults to \nil).
-\end{defmac}
-
-
-\begin{defun}{}{debug-block-successors}{\args{\var{debug-block}}}
-
- This function returns the list of possible code-locations where
- execution may continue when the basic-block represented by
- \var{debug-block} completes its execution.
-\end{defun}
-
-
-\begin{defun}{}{debug-block-elsewhere-p}{\args{\var{debug-block}}}
-
- This function returns whether \var{debug-block} represents elsewhere
- code. This is code the compiler has moved out of a function's code
- sequence for optimization reasons. Code-locations in these blocks
- are unsuitable for stepping tools, and the first code-location has
- nothing to do with a normal starting location for the block.
-\end{defun}
-
-
-%%\f
-%%\node Breakpoints, Code-locations, Debug-blocks, Debugger Programmer's Interface
-\section{Breakpoints}
-
-A breakpoint represents a function the system calls with the current frame when
-execution passes a certain code-location. A break point is active or inactive
-independent of its existence. They also have an extra slot for users to tag
-the breakpoint with information.
-
-\begin{defun}{}{make-breakpoint}{%
- \args{\var{hook-function} \var{what} \keys{\kwd{kind} \kwd{info}
- \kwd{function-end-cookie}}}}
-
- This function creates and returns a breakpoint. When program
- execution encounters the breakpoint, the system calls
- \var{hook-function}. \var{hook-function} takes the current frame
- for the function in which the program is running and the breakpoint
- object.
-
- \var{what} and \var{kind} determine where in a function the system
- invokes \var{hook-function}. \var{what} is either a code-location
- or a debug-function. \var{kind} is one of \kwd{code-location},
- \kwd{function-start}, or \kwd{function-end}. Since the starts and
- ends of functions may not have code-locations representing them,
- designate these places by supplying \var{what} as a debug-function
- and \var{kind} indicating the \kwd{function-start} or
- \kwd{function-end}. When \var{what} is a debug-function and
- \var{kind} is \kwd{function-end}, then hook-function must take two
- additional arguments, a list of values returned by the function and
- a function-end-cookie.
-
- \var{info} is information supplied by and used by the user.
-
- \var{function-end-cookie} is a function. To implement function-end
- breakpoints, the system uses starter breakpoints to establish the
- function-end breakpoint for each invocation of the function. Upon
- each entry, the system creates a unique cookie to identify the
- invocation, and when the user supplies a function for this argument,
- the system invokes it on the cookie. The system later invokes the
- function-end breakpoint hook on the same cookie. The user may save
- the cookie when passed to the function-end-cookie function for later
- comparison in the hook function.
-
- This signals an error if \var{what} is an unknown code-location.
-
- \i{Note: Breakpoints in interpreted code or byte-compiled code are
- not implemented. Function-end breakpoints are not implemented for
- compiled functions that use the known local return convention
- (e.g. for block-compiled or self-recursive functions.)}
-
-\end{defun}
-
-
-\begin{defun}{}{activate-breakpoint}{\args{\var{breakpoint}}}
-
- This function causes the system to invoke the \var{breakpoint}'s
- hook-function until the next call to \code{deactivate-breakpoint} or
- \code{delete-breakpoint}. The system invokes breakpoint hook
- functions in the opposite order that you activate them.
-\end{defun}
-
-
-\begin{defun}{}{deactivate-breakpoint}{\args{\var{breakpoint}}}
-
- This function stops the system from invoking the \var{breakpoint}'s
- hook-function.
-\end{defun}
-
-
-\begin{defun}{}{breakpoint-active-p}{\args{\var{breakpoint}}}
-
- This returns whether \var{breakpoint} is currently active.
-\end{defun}
-
-
-\begin{defun}{}{breakpoint-hook-function}{\args{\var{breakpoint}}}
-
- This function returns the \var{breakpoint}'s function the system
- calls when execution encounters \var{breakpoint}, and it is active.
- This is \code{SETF}'able.
-\end{defun}
-
-
-\begin{defun}{}{breakpoint-info}{\args{\var{breakpoint}}}
-
- This function returns \var{breakpoint}'s information supplied by the
- user. This is \code{SETF}'able.
-\end{defun}
-
-
-\begin{defun}{}{breakpoint-kind}{\args{\var{breakpoint}}}
-
- This function returns the \var{breakpoint}'s kind specification.
-\end{defun}
-
-
-\begin{defun}{}{breakpoint-what}{\args{\var{breakpoint}}}
-
- This function returns the \var{breakpoint}'s what specification.
-\end{defun}
-
-
-\begin{defun}{}{delete-breakpoint}{\args{\var{breakpoint}}}
-
- This function frees system storage and removes computational
- overhead associated with \var{breakpoint}. After calling this,
- \var{breakpoint} is useless and can never become active again.
-\end{defun}
-
-
-%%\f
-%%\node Code-locations, Debug-sources, Breakpoints, Debugger Programmer's Interface
-\section{Code-locations}
-
-Code-locations represent places in functions where the system has correct
-information about the function's environment and where interesting operations
-can occur---asking for a local variable's value, setting breakpoints,
-evaluating forms within the function's environment, etc.
-
-Sometimes the interface returns unknown code-locations. These
-represent places in functions, but there is no debug information
-associated with them. Some operations accept these since they may
-succeed even with missing debug data. These operations' argument is
-named \var{basic-code-location} indicating they take known and unknown
-code-locations. If an operation names its argument
-\var{code-location}, and you supply an unknown one, it will signal an
-error. For example, \code{frame-code-location} may return an unknown
-code-location if someone interrupted Lisp in the given frame. The
-system knows where execution will continue, but this place in the code
-may not be a place for which the compiler dumped debug information.
-
-\begin{defun}{}{code-location-debug-function}{\args{\var{basic-code-location}}}
-
- This function returns the debug-function representing information
- about the function corresponding to the code-location.
-\end{defun}
-
-
-\begin{defun}{}{code-location-debug-block}{\args{\var{basic-code-location}}}
-
- This function returns the debug-block containing code-location if it
- is available. Some debug policies inhibit debug-block information,
- and if none is available, then this signals a \code{no-debug-blocks}
- condition.
-\end{defun}
-
-
-\begin{defun}{}{code-location-top-level-form-offset}{%
- \args{\var{code-location}}}
-
- This function returns the number of top-level forms before the one
- containing \var{code-location} as seen by the compiler in some
- compilation unit. A compilation unit is not necessarily a single
- file, see the section on debug-sources.
-\end{defun}
-
-
-\begin{defun}{}{code-location-form-number}{\args{\var{code-location}}}
-
- This function returns the number of the form corresponding to
- \var{code-location}. The form number is derived by walking the
- subforms of a top-level form in depth-first order. While walking
- the top-level form, count one in depth-first order for each subform
- that is a cons. See \funref{form-number-translations}.
-\end{defun}
-
-
-\begin{defun}{}{code-location-debug-source}{\args{\var{code-location}}}
-
- This function returns \var{code-location}'s debug-source.
-\end{defun}
-
-
-\begin{defun}{}{code-location-unknown-p}{\args{\var{basic-code-location}}}
-
- This function returns whether \var{basic-code-location} is unknown.
- It returns \nil when the code-location is known.
-\end{defun}
-
-
-\begin{defun}{}{code-location=}{\args{\var{code-location1}
- \var{code-location2}}}
-
- This function returns whether the two code-locations are the same.
-\end{defun}
-
-
-%%\f
-%%\node Debug-sources, Source Translation Utilities, Code-locations, Debugger Programmer's Interface
-\section{Debug-sources}
-
-Debug-sources represent how to get back the source for some code. The
-source is either a file (\code{compile-file} or \code{load}), a
-lambda-expression (\code{compile}, \code{defun}, \code{defmacro}), or
-a stream (something particular to CMU Common Lisp,
-\code{compile-from-stream}).
-
-When compiling a source, the compiler counts each top-level form it
-processes, but when the compiler handles multiple files as one block
-compilation, the top-level form count continues past file boundaries.
-Therefore \code{code-location-top-level-form-offset} returns an offset
-that does not always start at zero for the code-location's
-debug-source. The offset into a particular source is
-\code{code-location-top-level-form-offset} minus
-\code{debug-source-root-number}.
-
-Inside a top-level form, a code-location's form number indicates the
-subform corresponding to the code-location.
-
-\begin{defun}{}{debug-source-from}{\args{\var{debug-source}}}
-
- This function returns an indication of the type of source. The
- following are the possible values:
- \begin{Lentry}
- \item[\kwd{file}] from a file (obtained by \code{compile-file} if
- compiled).
- \item[\kwd{lisp}] from Lisp (obtained by \code{compile} if
- compiled).
- \item[\kwd{stream}] from a non-file stream (CMU Common Lisp supports
- \code{compile-from-stream}).
- \end{Lentry}
-\end{defun}
-
-
-\begin{defun}{}{debug-source-name}{\args{\var{debug-source}}}
-
- This function returns the actual source in some sense represented by
- debug-source, which is related to \code{debug-source-from}:
- \begin{Lentry}
- \item[\kwd{file}] the pathname of the file.
- \item[\kwd{lisp}] a lambda-expression.
- \item[\kwd{stream}] some descriptive string that's otherwise
- useless.
-\end{Lentry}
-\end{defun}
-
-
-\begin{defun}{}{debug-source-created}{\args{\var{debug-source}}}
-
- This function returns the universal time someone created the source.
- This may be \nil{} if it is unavailable.
-\end{defun}
-
-
-\begin{defun}{}{debug-source-compiled}{\args{\var{debug-source}}}
-
- This function returns the time someone compiled the source. This is
- \nil if the source is uncompiled.
-\end{defun}
-
-
-\begin{defun}{}{debug-source-root-number}{\args{\var{debug-source}}}
-
- This returns the number of top-level forms processed by the compiler
- before compiling this source. If this source is uncompiled, this is
- zero. This may be zero even if the source is compiled since the
- first form in the first file compiled in one compilation, for
- example, must have a root number of zero---the compiler saw no other
- top-level forms before it.
-\end{defun}
-
-
-%%\node Source Translation Utilities, , Debug-sources, Debugger Programmer's Interface
-\section{Source Translation Utilities}
-
-These two functions provide a mechanism for converting the rather
-obscure (but highly compact) representation of source locations into an
-actual source form:
-
-\begin{defun}{}{debug-source-start-positions}{\args{\var{debug-source}}}
-
- This function returns the file position of each top-level form a
- vector if \var{debug-source} is from a \kwd{file}. If
- \code{debug-source-from} is \kwd{lisp} or \kwd{stream}, or the file
- is byte-compiled, then the result is \false.
-\end{defun}
-
-
-\begin{defun}{}{form-number-translations}{\args{\var{form}
- \var{tlf-number}}}
-
- This function returns a table mapping form numbers (see
- \code{code-location-form-number}) to source-paths. A source-path
- indicates a descent into the top-level-form \var{form}, going
- directly to the subform corresponding to a form number.
- \var{tlf-number} is the top-level-form number of \var{form}.
-\end{defun}
-
-
-\begin{defun}{}{source-path-context}{%
- \args{\var{form} \var{path} \var{context}}}
-
- This function returns the subform of \var{form} indicated by the
- source-path. \var{Form} is a top-level form, and \var{path} is a
- source-path into it. \var{Context} is the number of enclosing forms
- to return instead of directly returning the source-path form. When
- \var{context} is non-zero, the form returned contains a marker,
- \code{\#:****HERE****}, immediately before the form indicated by
- \var{path}.
-\end{defun}
-
-
-%%\f
-\twocolumn
-%%\node Function Index, Variable Index, Debugger Programmer's Interface, Top
-%%\unnumbered{Function Index}
-\cindex{Function Index}
-
-%%\printindex{fn}
-\printindex[funs]
-
-\twocolumn
-%%\node Variable Index, Type Index, Function Index, Top
-%%\unnumbered{Variable Index}
-\cindex{Variable Index}
-
-%%\printindex{vr}
-\printindex[vars]
-
-\twocolumn
-%%\node Type Index, Concept Index, Variable Index, Top
-%%\unnumbered{Type Index}
-\cindex{Type Index}
-
-%%\printindex{tp}
-\printindex[types]
-
-%%\node Concept Index, , Type Index, Top
-%%\unnumbered{Concept Index}
-\cindex{Concept Index}
-
-%%\printindex{cp}
-\onecolumn
-\printindex[concept]
-\end{document}
+++ /dev/null
-things from here which are invaluable for understanding current SBCL:
- object.tex
+++ /dev/null
-the function calling convention
-
-%ECX is used for a count of function argument words, represented as a
-fixnum, so it can also be thought of as a count of function argument
-bytes.
-
-The first three arguments are stored in registers. The remaining
-arguments are stored on the stack.
-
-The comments at the head of DEFINE-VOP (MORE-ARG) explain that
-;;; More args are stored contiguously on the stack, starting immediately at the
-;;; context pointer. The context pointer is not typed, so the lowtag is 0.
-
-?? Once we switch into more-arg arrangement, %ecx no longer seems to be
- used for argument count (judging from my walkthrough of kw arg parsing
- code while troubleshooting cold boot problems)
\ No newline at end of file
+++ /dev/null
-\part{System Architecture}% -*- Dictionary: int:design -*-
-
-\chapter{Package and File Structure}
-
-\section{RCS and build areas}
-
-The CMU CL sources are maintained using RCS in a hierarchical directory
-structure which supports:
-\begin{itemize}
-\item shared RCS config file across a build area,
-
-\item frozen sources for multiple releases, and
-
-\item separate system build areas for different architectures.
-\end{itemize}
-
-Since this organization maintains multiple copies of the source, it is somewhat
-space intensive. But it is easy to delete and later restore a copy of the
-source using RCS snapshots.
-
-There are three major subtrees of the root \verb|/afs/cs/project/clisp|:
-\begin{description}
-\item[rcs] holds the RCS source (suffix \verb|,v|) files.
-
-\item[src] holds ``checked out'' (but not locked) versions of the source files,
-and is subdivided by release. Each release directory in the source tree has a
-symbolic link named ``{\tt RCS}'' which points to the RCS subdirectory of the
-corresponding directory in the ``{\tt rcs} tree. At top-level in a source tree
-is the ``{\tt RCSconfig}'' file for that area. All subdirectories also have a
-symbolic link to this RCSconfig file, allowing the configuration for an area to
-be easily changed.
-
-\item[build] compiled object files are placed in this tree, which is subdivided
-by machine type and version. The CMU CL search-list mechanism is used to allow
-the source files to be located in a different tree than the object files. C
-programs are compiled by using the \verb|tools/dupsrcs| command to make
-symbolic links to the corresponding source tree.
-\end{description}
-
-On order to modify an file in RCS, it must be checked out with a lock to
-produce a writable working file. Each programmer checks out files into a
-personal ``play area'' subtree of \verb|clisp/hackers|. These tree duplicate
-the structure of source trees, but are normally empty except for files actively
-being worked on.
-
-See \verb|/afs/cs/project/clisp/pmax_mach/alpha/tools/| for
-various tools we use for RCS hacking:
-\begin{description}
-\item[rcs.lisp] Hemlock (editor) commands for RCS file manipulation
-
-\item[rcsupdate.c] Program to check out all files in a tree that have been
-modified since last checkout.
-
-\item[updates] Shell script to produce a single listing of all RCS log
- entries in a tree since a date.
-
-\item[snapshot-update.lisp] Lisp program to generate a shell script which
-generates a listing of updates since a particular RCS snapshot ({\tt RCSSNAP})
-file was created.
-\end{description}
-
-You can easily operate on all RCS files in a subtree using:
-\begin{verbatim}
-find . -follow -name '*,v' -exec <some command> {} \;
-\end{verbatim}
-
-\subsection{Configuration Management}
-
-config files are useful, especially in combinarion with ``{\tt snapshot}''. You
-can shapshot any particular version, giving an RCSconfig that designates that
-configuration. You can also use config files to specify the system as of a
-particular date. For example:
-\begin{verbatim}
-<3-jan-91
-\end{verbatim}
-in the the config file will cause the version as of that 3-jan-91 to be checked
-out, instead of the latest version.
-
-\subsection{RCS Branches}
-
-Branches and named revisions are used together to allow multiple paths of
-development to be supported. Each separate development has a branch, and each
-branch has a name. This project uses branches in two somewhat different cases
-of divergent development:
-\begin{itemize}
-\item For systems that we have imported from the outside, we generally assign a
-``{\tt cmu}'' branch for our local modifications. When a new release comes
-along, we check it in on the trunk, and then merge our branch back in.
-
-\item For the early development and debugging of major system changes, where
-the development and debugging is expected to take long enough that we wouldn't
-want the trunk to be in an inconsistent state for that long.
-\end{itemize}
-
-\section{Releases}
-
-We name releases according to the normal alpha, beta, default convention.
-Alpha releases are frequent, intended primarily for internal use, and are thus
-not subject to as high high documentation and configuration management
-standards. Alpha releases are designated by the date on which the system was
-built; the alpha releases for different systems may not be in exact
-correspondence, since they are built at different times.
-
-Beta and default releases are always based on a snapshot, ensuring that all
-systems are based on the same sources. A release name is an integer and a
-letter, like ``15d''. The integer is the name of the source tree which the
-system was built from, and the letter represents the release from that tree:
-``a'' is the first release, etc. Generally the numeric part increases when
-there are major system changes, whereas changes in the letter represent
-bug-fixes and minor enhancements.
-
-\section{Source Tree Structure}
-
-A source tree (and the master ``{\tt rcs}'' tree) has subdirectories for each
-major subsystem:
-\begin{description}
-\item[{\tt assembly/}] Holds the CMU CL source-file assembler, and has machine
-specific subdirectories holding assembly code for that architecture.
-
-\item[{\tt clx/}] The CLX interface to the X11 window system.
-
-\item[{\tt code/}] The Lisp code for the runtime system and standard CL
-utilities.
-
-\item[{\tt compiler/}] The Python compiler. Has architecture-specific
-subdirectories which hold backends for different machines. The {\tt generic}
-subdirectory holds code that is shared across most backends.
-
-\item[{\tt hemlock/}] The Hemlock editor.
-
-\item[{\tt lisp/}] The C runtime system code and low-level Lisp debugger.
-
-\item[{\tt pcl/}] CMU version of the PCL implementation of CLOS.
-
-\item[{\tt tools/}] System building command files and source management tools.
-\end{description}
-
-\f
-\section{Package structure}
-
-Goals: with the single exception of LISP, we want to be able to export from the
-package that the code lives in.
-
-\begin{description}
-\item[Mach, CLX...] --- These Implementation-dependent system-interface
-packages provide direct access to specific features available in the operating
-system environment, but hide details of how OS communication is done.
-
-\item[system] contains code that must know about the operating system
-environment: I/O, etc. Hides the operating system environment. Provides OS
-interface extensions such as {\tt print-directory}, etc.
-
-\item[kernel] hides state and types used for system integration: package
-system, error system, streams (?), reader, printer. Also, hides the VM, in
-that we don't export anything that reveals the VM interface. Contains code
-that needs to use the VM and SYSTEM interface, but is independent of OS and VM
-details. This code shouldn't need to be changed in any port of CMU CL, but
-won't work when plopped into an arbitrary CL. Uses SYSTEM, VM, EXTENSIONS. We
-export "hidden" symbols related to implementation of CL: setf-inverses,
-possibly some global variables.
-
-The boundary between KERNEL and VM is fuzzy, but this fuzziness reflects the
-fuzziness in the definition of the VM. We can make the VM large, and bring
-everything inside, or we make make it small. Obviously, we want the VM to be
-as small as possible, subject to efficiency constraints. Pretty much all of
-the code in KERNEL could be put in VM. The issue is more what VM hides from
-KERNEL: VM knows about everything.
-
-\item[lisp] Originally, this package had all the system code in it. The
-current ideal is that this package should have {\it no} code in it, and only
-exist to export the standard interface. Note that the name has been changed by
-x3j13 to common-lisp.
-
-\item[extensions] contains code that any random user could have written: list
-operations, syntactic sugar macros. Uses only LISP, so code in EXTENSIONS is
-pure CL. Exports everything defined within that is useful elsewhere. This
-package doesn't hide much, so it is relatively safe for users to use
-EXTENSIONS, since they aren't getting anything they couldn't have written
-themselves. Contrast this to KERNEL, which exports additional operations on
-CL's primitive data structures: PACKAGE-INTERNAL-SYMBOL-COUNT, etc. Although
-some of the functionality exported from KERNEL could have been defined in CL,
-the kernel implementation is much more efficient because it knows about
-implementation internals. Currently this package contains only extensions to
-CL, but in the ideal scheme of things, it should contain the implementations of
-all CL functions that are in KERNEL (the library.)
-
-\item[VM] hides information about the hardware and data structure
-representations. Contains all code that knows about this sort of thing: parts
-of the compiler, GC, etc. The bulk of the code is the compiler back-end.
-Exports useful things that are meaningful across all implementations, such as
-operations for examining compiled functions, system constants. Uses COMPILER
-and whatever else it wants. Actually, there are different {\it machine}{\tt
--VM} packages for each target implementation. VM is a nickname for whatever
-implementation we are currently targeting for.
-
-
-\item[compiler] hides the algorithms used to map Lisp semantics onto the
-operations supplied by the VM. Exports the mechanisms used for defining the
-VM. All the VM-independent code in the compiler, partially hiding the compiler
-intermediate representations. Uses KERNEL.
-
-\item[eval] holds code that does direct execution of the compiler's ICR. Uses
-KERNEL, COMPILER. Exports debugger interface to interpreted code.
-
-\item[debug-internals] presents a reasonable, unified interface to
-manipulation of the state of both compiled and interpreted code. (could be in
-KERNEL) Uses VM, INTERPRETER, EVAL, KERNEL.
-
-\item[debug] holds the standard debugger, and exports the debugger
-\end{description}
-
-\chapter{System Building}
-
-It's actually rather easy to build a CMU CL core with exactly what you want in
-it. But to do this you need two things: the source and a working CMU CL.
-
-Basically, you use the working copy of CMU CL to compile the sources,
-then run a process call ``genesis'' which builds a ``kernel'' core.
-You then load whatever you want into this kernel core, and save it.
-
-In the \verb|tools/| directory in the sources there are several files that
-compile everything, and build cores, etc. The first step is to compile the C
-startup code.
-
-{\bf Note:} {\it the various scripts mentioned below have hard-wired paths in
-them set up for our directory layout here at CMU. Anyone anywhere else will
-have to edit them before they will work.}
-
-\section{Compiling the C Startup Code}
-
-There is a circular dependancy between lisp/internals.h and lisp/lisp.map that
-causes bootstrapping problems. To the easiest way to get around this problem
-is to make a fake lisp.nm file that has nothing in it by a version number:
-
-\begin{verbatim}
- % echo "Map file for lisp version 0" > lisp.nm
-\end{verbatim}
-and then run genesis with NIL for the list of files:
-\begin{verbatim}
- * (load ".../compiler/generic/new-genesis") ; compile before loading
- * (lisp::genesis nil ".../lisp/lisp.nm" "/dev/null"
- ".../lisp/lisp.map" ".../lisp/lisp.h")
-\end{verbatim}
-It will generate
-a whole bunch of warnings about things being undefined, but ignore
-that, because it will also generate a correct lisp.h. You can then
-compile lisp producing a correct lisp.map:
-\begin{verbatim}
- % make
-\end{verbatim}
-and the use \verb|tools/do-worldbuild| and \verb|tools/mk-lisp| to build
-\verb|kernel.core| and \verb|lisp.core| (see section \ref[building-cores].)
-
-\section{Compiling the Lisp Code}
-
-The \verb|tools| directory contains various lisp and C-shell utilities for
-building CMU CL:
-\begin{description}
-\item[compile-all*] Will compile lisp files and build a kernel core. It has
-numerous command-line options to control what to compile and how. Try -help to
-see a description. It runs a separate Lisp process to compile each
-subsystem. Error output is generated in files with ``{\tt .log}'' extension in
-the root of the build area.
-
-\item[setup.lisp] Some lisp utilities used for compiling changed files in batch
-mode and collecting the error output Sort of a crude defsystem. Loads into the
-``user'' package. See {\tt with-compiler-log-file} and {\tt comf}.
-
-\item[{\it foo}com.lisp] Each system has a ``\verb|.lisp|'' file in
-\verb|tools/| which compiles that system.
-\end{description}
-
-\section{Building Core Images}
-\label{building-cores}
-Both the kernel and final core build are normally done using shell script
-drivers:
-\begin{description}
-\item[do-worldbuild*] Builds a kernel core for the current machine. The
-version to build is indicated by an optional argument, which defaults to
-``alpha''. The \verb|kernel.core| file is written either in the \verb|lisp/|
-directory in the build area, or in \verb|/usr/tmp/|. The directory which
-already contains \verb|kernel.core| is chosen. You can create a dummy version
-with e.g. ``touch'' to select the initial build location.
-
-\item[mk-lisp*] Builds a full core, with conditional loading of subsystems.
-The version is the first argument, which defaults to ``alpha''. Any additional
-arguments are added to the \verb|*features*| list, which controls system
-loading (among other things.) The \verb|lisp.core| file is written in the
-current working directory.
-\end{description}
-
-These scripts load Lisp command files. When \verb|tools/worldbuild.lisp| is
-loaded, it calls genesis with the correct arguments to build a kernel core.
-Similarly, \verb|worldload.lisp|
-builds a full core. Adding certain symbols to \verb|*features*| before
-loading worldload.lisp suppresses loading of different parts of the
-system. These symbols are:
-\begin{description}
-\item[:no-compiler] don't load the compiler.
-\item[:no-clx] don't load CLX.
-\item[:no-hemlock] don't load hemlock.
-\item[:no-pcl] don't load PCL.
-\item[:runtime] build a runtime code, implies all of the above, and then some.
-\end{description}
-
-Note: if you don't load the compiler, you can't (successfully) load the
-pretty-printer or pcl. And if you compiled hemlock with CLX loaded, you can't
-load it without CLX also being loaded.
+++ /dev/null
-% -*- Dictionary: design -*-
-\f
-\chapter{Copy propagation}
-
-File: {\tt copyprop}
-
-This phase is optional, but should be done whenever speed or space is more
-important than compile speed. We use global flow analysis to find the reaching
-definitions for each TN. This information is used here to eliminate
-unnecessary TNs, and is also used later on by loop invariant optimization.
-
-In some cases, VMR conversion will unnecessarily copy the value of a TN into
-another TN, since it may not be able to tell that the initial TN has the same
-value at the time the second TN is referenced. This can happen when ICR
-optimize is unable to eliminate a trivial variable binding, or when the user
-does a setq, or may also result from creation of expression evaluation
-temporaries during VMR conversion. Whatever the cause, we would like to avoid
-the unnecessary creation and assignment of these TNs.
-
-What we do is replace TN references whose only reaching definition is a Move
-VOP with a reference to the TN moved from, and then delete the Move VOP if the
-copy TN has no remaining references. There are several restrictions on copy
-propagation:
-\begin{itemize}
-\item The TNs must be ``ordinary'' TNs, not restricted or otherwise
-unusual. Extending the life of restricted (or wired) TNs can make register
-allocation impossible. Some other TN kinds have hidden references.
-
-\item We don't want to defeat source-level debugging by replacing named
-variables with anonymous temporaries.
-
-\item We can't delete moves that representation selected might want to change
-into a representation conversion, since we need the primitive types of both TNs
-to select a conversion.
-\end{itemize}
-
-Some cleverness reduces the cost of flow analysis. As for lifetime analysis,
-we only need to do flow analysis on global packed TNs. We can't do the real
-local TN assignment pass before this, since we allocate TNs afterward, so we do
-a pre-pass that marks the TNs that are local for our purposes. We don't care
-if block splitting eventually causes some of them to be considered global.
-
-Note also that we are really only are interested in knowing if there is a
-unique reaching definition, which we can mash into our flow analysis rules by
-doing an intersection. Then a definition only appears in the set when it is
-unique. We then propagate only definitions of TNs with only one write, which
-allows the TN to stand for the definition.
-
-\f
-\chapter{Representation selection}
-
-File: {\tt represent}
-
-Some types of object (such as {\tt single-float}) have multiple possible
-representations. Multiple representations are useful mainly when there is a
-particularly efficient non-descriptor representation. In this case, there is
-the normal descriptor representation, and an alternate non-descriptor
-representation.
-
-This possibility brings up two major issues:
-\begin{itemize}
-\item The compiler must decide which representation will be most efficient for
-any given value, and
-
-\item Representation conversion code must be inserted where the representation
-of a value is changed.
-\end{itemize}
-First, the representations for TNs are selected by examining all the TN
-references and attempting to minimize reference costs. Then representation
-conversion code is introduced.
-
-This phase is in effect a pre-pass to register allocation. The main reason for
-its existence is that representation conversions may be farily complex (e.g.
-involving memory allocation), and thus must be discovered before register
-allocation.
-
-
-VMR conversion leaves stubs for representation specific move operations.
-Representation selection recognizes {\tt move} by name. Argument and return
-value passing for call VOPs is controlled by the {\tt :move-arguments} option
-to {\tt define-vop}.
-
-Representation selection is also responsible for determining what functions use
-the number stack. If any representation is chosen which could involve packing
-into the {\tt non-descriptor-stack} SB, then we allocate the NFP register
-throughout the component. As an optimization, permit the decision of whether a
-number stack frame needs to be allocated to be made on a per-function basis.
-If a function doesn't use the number stack, and isn't in the same tail-set as
-any function that uses the number stack, then it doesn't need a number stack
-frame, even if other functions in the component do.
-
-\f
-\chapter{Lifetime analysis}
-
-File: {\tt life}
-
-This phase is a preliminary to Pack. It involves three passes:
- -- A pre-pass that computes the DEF and USE sets for live TN analysis, while
- also assigning local TN numbers, splitting blocks if necessary. \#\#\# But
-not really...
- -- A flow analysis pass that does backward flow analysis on the
- component to find the live TNs at each block boundary.
- -- A post-pass that finds the conflict set for each TN.
-
-\#|
-Exploit the fact that a single VOP can only exhaust LTN numbers when there are
-large more operands. Since more operand reference cannot be interleaved with
-temporary reference, the references all effectively occur at the same time.
-This means that we can assign all the more args and all the more results the
-same LTN number and the same lifetime info.
-|\#
-
-\f
-\section{Flow analysis}
-
-It seems we could use the global-conflicts structures during compute the
-inter-block lifetime information. The pre-pass creates all the
-global-conflicts for blocks that global TNs are referenced in. The flow
-analysis pass just adds always-live global-conflicts for the other blocks the
-TNs are live in. In addition to possibly being more efficient than SSets, this
-would directly result in the desired global-conflicts information, rather that
-having to create it from another representation.
-
-The DFO sorted per-TN global-conflicts thread suggests some kind of algorithm
-based on the manipulation of the sets of blocks each TN is live in (which is
-what we really want), rather than the set of TNs live in each block.
-
-If we sorted the per-TN global-conflicts in reverse DFO (which is just as good
-for determining conflicts between TNs), then it seems we could scan though the
-conflicts simultaneously with our flow-analysis scan through the blocks.
-
-The flow analysis step is the following:
- If a TN is always-live or read-before-written in a successor block, then we
- make it always-live in the current block unless there are already
- global-conflicts recorded for that TN in this block.
-
-The iteration terminates when we don't add any new global-conflicts during a
-pass.
-
-We may also want to promote TNs only read within a block to always-live when
-the TN is live in a successor. This should be easy enough as long as the
-global-conflicts structure contains this kind of info.
-
-The critical operation here is determining whether a given global TN has global
-conflicts in a given block. Note that since we scan the blocks in DFO, and the
-global-conflicts are sorted in DFO, if we give each global TN a pointer to the
-global-conflicts for the last block we checked the TN was in, then we can
-guarantee that the global-conflicts we are looking for are always at or after
-that pointer. If we need to insert a new structure, then the pointer will help
-us rapidly find the place to do the insertion.]
-
-\f
-\section{Conflict detection}
-
-[\#\#\# Environment, :more TNs.]
-
-This phase makes use of the results of lifetime analysis to find the set of TNs
-that have lifetimes overlapping with those of each TN. We also annotate call
-VOPs with information about the live TNs so that code generation knows which
-registers need to be saved.
-
-The basic action is a backward scan of each block, looking at each TN-Ref and
-maintaining a set of the currently live TNs. When we see a read, we check if
-the TN is in the live set. If not, we:
- -- Add the TN to the conflict set for every currently live TN,
- -- Union the set of currently live TNs with the conflict set for the TN, and
- -- Add the TN to the set of live TNs.
-
-When we see a write for a live TN, we just remove it from the live set. If we
-see a write to a dead TN, then we update the conflicts sets as for a read, but
-don't add the TN to the live set. We have to do this so that the bogus write
-doesn't clobber anything.
-
-[We don't consider always-live TNs at all in this process, since the conflict
-of always-live TNs with other TNs in the block is implicit in the
-global-conflicts structures.
-
-Before we do the scan on a block, we go through the global-conflicts structures
-of TNs that change liveness in the block, assigning the recorded LTN number to
-the TN's LTN number for the duration of processing of that block.]
-
-
-Efficiently computing and representing this information calls for some
-cleverness. It would be prohibitively expensive to represent the full conflict
-set for every TN with sparse sets, as is done at the block-level. Although it
-wouldn't cause non-linear behavior, it would require a complex linked structure
-containing tens of elements to be created for every TN. Fortunately we can
-improve on this if we take into account the fact that most TNs are "local" TNs:
-TNs which have all their uses in one block.
-
-First, many global TNs will be either live or dead for the entire duration of a
-given block. We can represent the conflict between global TNs live throughout
-the block and TNs local to the block by storing the set of always-live global
-TNs in the block. This reduces the number of global TNs that must be
-represented in the conflicts for local TNs.
-
-Second, we can represent conflicts within a block using bit-vectors. Each TN
-that changes liveness within a block is assigned a local TN number. Local
-conflicts are represented using a fixed-size bit-vector of 64 elements or so
-which has a 1 for the local TN number of every TN live at that time. The block
-has a simple-vector which maps from local TN numbers to TNs. Fixed-size
-vectors reduce the hassle of doing allocations and allow operations to be
-open-coded in a maximally tense fashion.
-
-We can represent the conflicts for a local TN by a single bit-vector indexed by
-the local TN numbers for that block, but in the global TN case, we need to be
-able to represent conflicts with arbitrary TNs. We could use a list-like
-sparse set representation, but then we would have to either special-case global
-TNs by using the sparse representation within the block, or convert the local
-conflicts bit-vector to the sparse representation at the block end. Instead,
-we give each global TN a list of the local conflicts bit-vectors for each block
-that the TN is live in. If the TN is always-live in a block, then we record
-that fact instead. This gives us a major reduction in the amount of work we
-have to do in lifetime analysis at the cost of some increase in the time to
-iterate over the set during Pack.
-
-Since we build the lists of local conflict vectors a block at a time, the
-blocks in the lists for each TN will be sorted by the block number. The
-structure also contains the local TN number for the TN in that block. These
-features allow pack to efficiently determine whether two arbitrary TNs
-conflict. You just scan the lists in order, skipping blocks that are in only
-one list by using the block numbers. When we find a block that both TNs are
-live in, we just check the local TN number of one TN in the local conflicts
-vector of the other.
-
-In order to do these optimizations, we must do a pre-pass that finds the
-always-live TNs and breaks blocks up into small enough pieces so that we don't
-run out of local TN numbers. If we can make a block arbitrarily small, then we
-can guarantee that an arbitrarily small number of TNs change liveness within
-the block. We must be prepared to make the arguments to unbounded arg count
-VOPs (such as function call) always-live even when they really aren't. This is
-enabled by a panic mode in the block splitter: if we discover that the block
-only contains one VOP and there are still too many TNs that aren't always-live,
-then we promote the arguments (which we'd better be able to do...).
-
-This is done during the pre-scan in lifetime analysis. We can do this because
-all TNs that change liveness within a block can be found by examining that
-block: the flow analysis only adds always-live TNs.
-
-
-When we are doing the conflict detection pass, we set the LTN number of global
-TNs. We can easily detect global TNs that have not been locally mapped because
-this slot is initially null for global TNs and we null it out after processing
-each block. We assign all Always-Live TNs to the same local number so that we
-don't need to treat references to them specially when making the scan.
-
-We also annotate call VOPs that do register saving with the TNs that are live
-during the call, and thus would need to be saved if they are packed in
-registers.
-
-We adjust the costs for TNs that need to be saved so that TNs costing more to
-save and restore than to reference get packed on the stack. We would also like
-more often saved TNs to get higher costs so that they are packed in more
-savable locations.
-
-\f
-\chapter{Packing}
-
-File: {\tt pack}
-
-\#|
-
-Add lifetime/pack support for pre-packed save TNs.
-
-Fix GTN/VMR conversion to use pre-packed save TNs for old-cont and return-PC.
-(Will prevent preference from passing location to save location from ever being
-honored?)
-
-We will need to make packing of passing locations smarter before we will be
-able to target the passing location on the stack in a tail call (when that is
-where the callee wants it.) Currently, we will almost always pack the passing
-location in a register without considering whether that is really a good idea.
-Maybe we should consider schemes that explicitly understand the parallel
-assignment semantics, and try to do the assignment with a minimum number of
-temporaries. We only need assignment temps for TNs that appear both as an
-actual argument value and as a formal parameter of the called function. This
-only happens in self-recursive functions.
-
-Could be a problem with lifetime analysis, though. The write by a move-arg VOP
-would look like a write in the current env, when it really isn't. If this is a
-problem, then we might want to make the result TN be an info arg rather than a
-real operand. But this would only be a problem in recursive calls, anyway.
-[This would prevent targeting, but targeting across passing locations rarely
-seems to work anyway.] [\#\#\# But the :ENVIRONMENT TN mechanism would get
-confused. Maybe put env explicitly in TN, and have it only always-live in that
-env, and normal in other envs (or blocks it is written in.) This would allow
-targeting into environment TNs.
-
-I guess we would also want the env/PC save TNs normal in the return block so
-that we can target them. We could do this by considering env TNs normal in
-read blocks with no successors.
-
-ENV TNs would be treated totally normally in non-env blocks, so we don't have
-to worry about lifetime analysis getting confused by variable initializations.
-Do some kind of TN costing to determine when it is more trouble than it is
-worth to allocate TNs in registers.
-
-Change pack ordering to be less pessimal. Pack TNs as they are seen in the LTN
-map in DFO, which at least in non-block compilations has an effect something
-like packing main trace TNs first, since control analysis tries to put the good
-code first. This could also reduce spilling, since it makes it less likely we
-will clog all registers with global TNs.
-
-If we pack a TN with a specified save location on the stack, pack in the
-specified location.
-
-Allow old-cont and return-pc to be kept in registers by adding a new "keep
-around" kind of TN. These are kind of like environment live, but are only
-always-live in blocks that they weren't referenced in. Lifetime analysis does
-a post-pass adding always-live conflicts for each "keep around" TN to those
-blocks with no conflict for that TN. The distinction between always-live and
-keep-around allows us to successfully target old-cont and return-pc to passing
-locations. MAKE-KEEP-AROUND-TN (ptype), PRE-PACK-SAVE-TN (tn scn offset).
-Environment needs a KEEP-AROUND-TNS slot so that conflict analysis can find
-them (no special casing is needed after then, they can be made with :NORMAL
-kind). VMR-component needs PRE-PACKED-SAVE-TNS so that conflict analysis or
-somebody can copy conflict info from the saved TN.
-
-
-
-Note that having block granularity in the conflict information doesn't mean
-that a localized packing scheme would have to do all moves at block boundaries
-(which would clash with the desire the have saving done as part of this
-mechanism.) All that it means is that if we want to do a move within the
-block, we would need to allocate both locations throughout that block (or
-something).
-
-
-
-
-
-Load TN pack:
-
-A location is out for load TN packing if:
-
-The location has TN live in it after the VOP for a result, or before the VOP
-for an argument, or
-
-The location is used earlier in the TN-ref list (after) the saved results ref
-or later in the TN-Ref list (before) the loaded argument's ref.
-
-To pack load TNs, we advance the live-tns to the interesting VOP, then
-repeatedly scan the vop-refs to find vop-local conflicts for each needed load
-TN. We insert move VOPs and change over the TN-Ref-TNs as we go so the TN-Refs
-will reflect conflicts with already packed load-TNs.
-
-If we fail to pack a load-TN in the desired SC, then we scan the Live-TNs for
-the SB, looking for a TN that can be packed in an unbounded SB. This TN must
-then be repacked in the unbounded SB. It is important the load-TNs are never
-packed in unbounded SBs, since that would invalidate the conflicts info,
-preventing us from repacking TNs in unbounded SBs. We can't repack in a finite
-SB, since there might have been load TNs packed in that SB which aren't
-represented in the original conflict structures.
-
-Is it permissible to "restrict" an operand to an unbounded SC? Not impossible
-to satisfy as long as a finite SC is also allowed. But in practice, no
-restriction would probably be as good.
-
-We assume all locations can be used when an sc is based on an unbounded sb.
-
-]
-
-
-TN-Refs are be convenient structures to build the target graph out of. If we
-allocated space in every TN-Ref, then there would certainly be enough to
-represent arbitrary target graphs. Would it be enough to allocate a single
-Target slot? If there is a target path though a given VOP, then the Target of
-the write ref would be the read, and vice-versa. To find all the TNs that
-target us, we look at the TN for the target of all our write refs.
-
-We separately chain together the read refs and the write refs for a TN,
-allowing easy determination of things such as whether a TN has only a single
-definition or has no reads. It would also allow easier traversal of the target
-graph.
-
-Represent per-location conflicts as vectors indexed by block number of
-per-block conflict info. To test whether a TN conflicts on a location, we
-would then have to iterate over the TNs global-conflicts, using the block
-number and LTN number to check for a conflict in that block. But since most
-TNs are local, this test actually isn't much more expensive than indexing into
-a bit-vector by GTN numbers.
-
-The big win of this scheme is that it is much cheaper to add conflicts into the
-conflict set for a location, since we never need to actually compute the
-conflict set in a list-like representation (which requires iterating over the
-LTN conflicts vectors and unioning in the always-live TNs). Instead, we just
-iterate over the global-conflicts for the TN, using BIT-IOR to combine the
-conflict set with the bit-vector for that block in that location, or marking
-that block/location combination as being always-live if the conflict is
-always-live.
-
-Generating the conflict set is inherently more costly, since although we
-believe the conflict set size to be roughly constant, it can easily contain
-tens of elements. We would have to generate these moderately large lists for
-all TNs, including local TNs. In contrast, the proposed scheme does work
-proportional to the number of blocks the TN is live in, which is small on
-average (1 for local TNs). This win exists independently from the win of not
-having to iterate over LTN conflict vectors.
-
-
-[\#\#\# Note that since we never do bitwise iteration over the LTN conflict
-vectors, part of the motivation for keeping these a small fixed size has been
-removed. But it would still be useful to keep the size fixed so that we can
-easily recycle the bit-vectors, and so that we could potentially have maximally
-tense special primitives for doing clear and bit-ior on these vectors.]
-
-This scheme is somewhat more space-intensive than having a per-location
-bit-vector. Each vector entry would be something like 150 bits rather than one
-bit, but this is mitigated by the number of blocks being 5-10x smaller than the
-number of TNs. This seems like an acceptable overhead, a small fraction of the
-total VMR representation.
-
-The space overhead could also be reduced by using something equivalent to a
-two-dimensional bit array, indexed first by LTN numbers, and then block numbers
-(instead of using a simple-vector of separate bit-vectors.) This would
-eliminate space wastage due to bit-vector overheads, which might be 50% or
-more, and would also make efficient zeroing of the vectors more
-straightforward. We would then want efficient operations for OR'ing LTN
-conflict vectors with rows in the array.
-
-This representation also opens a whole new range of allocation algorithms: ones
-that store allocate TNs in different locations within different portions of the
-program. This is because we can now represent a location being used to hold a
-certain TN within an arbitrary subset of the blocks the TN is referenced in.
-
-
-
-
-
-
-
-
-
-Pack goals:
-
-Pack should:
-
-Subject to resource constraints:
- -- Minimize use costs
- -- "Register allocation"
- Allocate as many values as possible in scarce "good" locations,
- attempting to minimize the aggregate use cost for the entire program.
- -- "Save optimization"
- Don't allocate values in registers when the save/restore costs exceed
- the expected gain for keeping the value in a register. (Similar to
- "opening costs" in RAOC.) [Really just a case of representation
- selection.]
-
- -- Minimize preference costs
- Eliminate as many moves as possible.
-
-
-"Register allocation" is basically an attempt to eliminate moves between
-registers and memory. "Save optimization" counterbalances "register
-allocation" to prevent it from becoming a pessimization, since saves can
-introduce register/memory moves.
-
-Preference optimization reduces the number of moves within an SC. Doing a good
-job of honoring preferences is important to the success of the compiler, since
-we have assumed in many places that moves will usually be optimized away.
-
-The scarcity-oriented aspect of "register allocation" is handled by a greedy
-algorithm in pack. We try to pack the "most important" TNs first, under the
-theory that earlier packing is more likely to succeed due to fewer constraints.
-
-The drawback of greedy algorithms is their inability to look ahead. Packing a
-TN may mess up later "register allocation" by precluding packing of TNs that
-are individually "less important", but more important in aggregate. Packing a
-TN may also prevent preferences from being honored.
-
-
-\f
-Initial packing:
-
-
-Pack all TNs restricted to a finite SC first, before packing any other TNs.
-
-One might suppose that Pack would have to treat TNs in different environments
-differently, but this is not the case. Pack simply assigns TNs to locations so
-that no two conflicting TNs are in the same location. In the process of
-implementing call semantics in conflict analysis, we cause TNs in different
-environments not to conflict. In the case of passing TNs, cross environment
-conflicts do exist, but this reflects reality, since the passing TNs are
-live in both the caller and the callee. Environment semantics has already been
-implemented at this point.
-
-This means that Pack can pack all TNs simultaneously, using one data structure
-to represent the conflicts for each location. So we have only one conflict set
-per SB location, rather than separating this information by environment
-environment.
-
-\f
-Load TN packing:
-
-We create load TNs as needed in a post-pass to the initial packing. After TNs
-are packed, it may be that some references to a TN will require it to be in a
-SC other than the one it was packed in. We create load-TNs and pack them on
-the fly during this post-pass.
-
-What we do is have an optional SC restriction associated with TN-refs. If we
-pack the TN in an SC which is different from the required SC for the reference,
-then we create a TN for each such reference, and pack it into the required SC.
-
-In many cases we will be able to pack the load TN with no hassle, but in
-general we may need to spill a TN that has already been packed. We choose a
-TN that isn't in use by the offending VOP, and then spill that TN onto the
-stack for the duration of that VOP. If the VOP is a conditional, then we must
-insert a new block interposed before the branch target so that the value TN
-value is restored regardless of which branch is taken.
-
-Instead of remembering lifetime information from conflict analysis, we rederive
-it. We scan each block backward while keeping track of which locations have
-live TNs in them. When we find a reference that needs a load TN packed, we try
-to pack it in an unused location. If we can't, we unpack the currently live TN
-with the lowest cost and force it into an unbounded SC.
-
-The per-location and per-TN conflict information used by pack doesn't
-need to be updated when we pack a load TN, since we are done using those data
-structures.
-
-We also don't need to create any TN-Refs for load TNs. [??? How do we keep
-track of load-tn lifetimes? It isn't really that hard, I guess. We just
-remember which load TNs we created at each VOP, killing them when we pass the
-loading (or saving) step. This suggests we could flush the Refs thread if we
-were willing to sacrifice some flexibility in explicit temporary lifetimes.
-Flushing the Refs would make creating the VMR representation easier.]
-
-The lifetime analysis done during load-TN packing doubles as a consistency
-check. If we see a read of a TN packed in a location which has a different TN
-currently live, then there is a packing bug. If any of the TNs recorded as
-being live at the block beginning are packed in a scarce SB, but aren't current
-in that location, then we also have a problem.
-
-The conflict structure for load TNs is fairly simple, the load TNs for
-arguments and results all conflict with each other, and don't conflict with
-much else. We just try packing in targeted locations before trying at random.
-
-
-\f
-\chapter{Code generation}
-
-This is fairly straightforward. We translate VOPs into instruction sequences
-on a per-block basis.
-
-After code generation, the VMR representation is gone. Everything is
-represented by the assembler data structures.
-
-\f
-\chapter{Assembly}
-
-In effect, we do much of the work of assembly when the compiler is compiled.
-
-The assembler makes one pass fixing up branch offsets, then squeezes out the
-space left by branch shortening and dumps out the code along with the load-time
-fixup information. The assembler also deals with dumping unboxed non-immediate
-constants and symbols. Boxed constants are created by explicit constructor
-code in the top-level form, while immediate constants are generated using
-inline code.
-
-[\#\#\# The basic output of the assembler is:
- A code vector
- A representation of the fixups along with indices into the code vector for
- the fixup locations
- A PC map translating PCs into source paths
-
-This information can then be used to build an output file or an in-core
-function object.
-]
-
-The assembler is table-driven and supports arbitrary instruction formats. As
-far as the assembler is concerned, an instruction is a bit sequence that is
-broken down into subsequences. Some of the subsequences are constant in value,
-while others can be determined at assemble or load time.
-
-Assemble Node Form*
- Allow instructions to be emitted during the evaluation of the Forms by
- defining Inst as a local macro. This macro caches various global
- information in local variables. Node tells the assembler what node
- ultimately caused this code to be generated. This is used to create the
- pc=>source map for the debugger.
-
-Assemble-Elsewhere Node Form*
- Similar to Assemble, but the current assembler location is changed to
- somewhere else. This is useful for generating error code and similar
- things. Assemble-Elsewhere may not be nested.
-
-Inst Name Arg*
- Emit the instruction Name with the specified arguments.
-
-Gen-Label
-Emit-Label (Label)
- Gen-Label returns a Label object, which describes a place in the code.
- Emit-Label marks the current position as being the location of Label.
-
-
-\f
-\chapter{Dumping}
-
-So far as input to the dumper/loader, how about having a list of Entry-Info
-structures in the VMR-Component? These structures contain all information
-needed to dump the associated function objects, and are only implicitly
-associated with the functional/XEP data structures. Load-time constants that
-reference these function objects should specify the Entry-Info, rather than the
-functional (or something). We would then need to maintain some sort of
-association so VMR conversion can find the appropriate Entry-Info.
-Alternatively, we could initially reference the functional, and then later
-clobber the reference to the Entry-Info.
-
-We have some kind of post-pass that runs after assembly, going through the
-functions and constants, annotating the VMR-Component for the benefit of the
-dumper:
- Resolve :Label load-time constants.
- Make the debug info.
- Make the entry-info structures.
-
-Fasl dumper and in-core loader are implementation (but not instruction set)
-dependent, so we want to give them a clear interface.
-
-open-fasl-file name => fasl-file
- Returns a "fasl-file" object representing all state needed by the dumper.
- We objectify the state, since the fasdumper should be reentrant. (but
- could fail to be at first.)
-
-close-fasl-file fasl-file abort-p
- Close the specified fasl-file.
-
-fasl-dump-component component code-vector length fixups fasl-file
- Dump the code, constants, etc. for component. Code-Vector is a vector
- holding the assembled code. Length is the number of elements of Vector
- that are actually in use. Fixups is a list of conses (offset . fixup)
- describing the locations and things that need to be fixed up at load time.
- If the component is a top-level component, then the top-level lambda will
- be called after the component is loaded.
-
-load-component component code-vector length fixups
- Like Fasl-Dump-Component, but directly installs the code in core, running
- any top-level code immediately. (???) but we need some way to glue
- together the componenents, since we don't have a fasl table.
-
-
-
-Dumping:
-
-Dump code for each component after compiling that component, but defer dumping
-of other stuff. We do the fixups on the code vectors, and accumulate them in
-the table.
-
-We have to grovel the constants for each component after compiling that
-component so that we can fix up load-time constants. Load-time constants are
-values needed my the code that are computed after code generation/assembly
-time. Since the code is fixed at this point, load-time constants are always
-represented as non-immediate constants in the constant pool. A load-time
-constant is distinguished by being a cons (Kind . What), instead of a Constant
-leaf. Kind is a keyword indicating how the constant is computed, and What is
-some context.
-
-Some interesting load-time constants:
-
- (:label . <label>)
- Is replaced with the byte offset of the label within the code-vector.
-
- (:code-vector . <component>)
- Is replaced by the component's code-vector.
-
- (:entry . <function>)
- (:closure-entry . <function>)
- Is replaced by the function-entry structure for the specified function.
- :Entry is how the top-level component gets a handle on the function
- definitions so that it can set them up.
-
-We also need to remember the starting offset for each entry, although these
-don't in general appear as explicit constants.
-
-We then dump out all the :Entry and :Closure-Entry objects, leaving any
-constant-pool pointers uninitialized. After dumping each :Entry, we dump some
-stuff to let genesis know that this is a function definition. Then we dump all
-the constant pools, fixing up any constant-pool pointers in the already-dumped
-function entry structures.
-
-The debug-info *is* a constant: the first constant in every constant pool. But
-the creation of this constant must be deferred until after the component is
-compiled, so we leave a (:debug-info) placeholder. [Or maybe this is
-implicitly added in by the dumper, being supplied in a VMR-component slot.]
-
-
- Work out details of the interface between the back-end and the
- assembler/dumper.
-
- Support for multiple assemblers concurrently loaded? (for byte code)
-
- We need various mechanisms for getting information out of the assembler.
-
- We can get entry PCs and similar things into function objects by making a
- Constant leaf, specifying that it goes in the closure, and then
- setting the value after assembly.
-
- We have an operation Label-Value which can be used to get the value of a
- label after assembly and before the assembler data structures are
- deallocated.
-
- The function map can be constructed without any special help from the
- assembler. Codegen just has to note the current label when the function
- changes from one block to the next, and then use the final value of these
- labels to make the function map.
-
- Probably we want to do the source map this way too. Although this will
- make zillions of spurious labels, we would have to effectively do that
- anyway.
-
- With both the function map and the source map, getting the locations right
- for uses of Elsewhere will be a bit tricky. Users of Elsewhere will need
- to know about how these maps are being built, since they must record the
- labels and corresponding information for the elsewhere range. It would be
- nice to have some cooperation from Elsewhere so that this isn't necessary,
- otherwise some VOP writer will break the rules, resulting in code that is
- nowhere.
-
- The Debug-Info and related structures are dumped by consing up the
- structure and making it be the value of a constant.
-
- Getting the code vector and fixups dumped may be a bit more interesting. I
- guess we want a Dump-Code-Vector function which dumps the code and fixups
- accumulated by the current assembly, returning a magic object that will
- become the code vector when it is dumped as a constant.
-]
+++ /dev/null
-\chapter{Compiler Overview} % -*- Dictionary: design -*-
-
-The structure of the compiler may be broadly characterized by describing the
-compilation phases and the data structures that they manipulate. The steps in
-the compilation are called phases rather than passes since they don't
-necessarily involve a full pass over the code. The data structure used to
-represent the code at some point is called an {\it intermediate
-representation.}
-
-Two major intermediate representations are used in the compiler:
-\begin{itemize}
-
-\item The Implicit Continuation Representation (ICR) represents the lisp-level
-semantics of the source code during the initial phases. Partial evaluation and
-semantic analysis are done on this representation. ICR is roughly equivalent
-to a subset of Common Lisp, but is represented as a flow-graph rather than a
-syntax tree. Phases which only manipulate ICR comprise the "front end". It
-would be possible to use a different back end such as one that directly
-generated code for a stack machine.
-
-\item The Virtual Machine Representation (VMR) represents the implementation of
-the source code on a virtual machine. The virtual machine may vary depending
-on the the target hardware, but VMR is sufficiently stylized that most of the
-phases which manipulate it are portable.
-\end{itemize}
-
-Each phase is briefly described here. The phases from ``local call analysis''
-to ``constraint propagation'' all interact; for maximum optimization, they
-are generally repeated until nothing new is discovered. The source files which
-primarily contain each phase are listed after ``Files: ''.
-\begin{description}
-
-\item[ICR conversion]
-Convert the source into ICR, doing macroexpansion and simple source-to-source
-transformation. All names are resolved at this time, so we don't have to worry
-about name conflicts later on. Files: {\tt ir1tran, srctran, typetran}
-
-\item[Local call analysis] Find calls to local functions and convert them to
-local calls to the correct entry point, doing keyword parsing, etc. Recognize
-once-called functions as lets. Create {\it external entry points} for
-entry-point functions. Files: {\tt locall}
-
-\item[Find components]
-Find flow graph components and compute depth-first ordering. Separate
-top-level code from run-time code, and determine which components are top-level
-components. Files: {\tt dfo}
-
-\item[ICR optimize] A grab-bag of all the non-flow ICR optimizations. Fold
-constant functions, propagate types and eliminate code that computes unused
-values. Special-case calls to some known global functions by replacing them
-with a computed function. Merge blocks and eliminate IF-IFs. Substitute let
-variables. Files: {\tt ir1opt, ir1tran, typetran, seqtran, vm/vm-tran}
-
-\item[Type constraint propagation]
-Use global flow analysis to propagate information about lexical variable
-types. Eliminate unnecessary type checks and tests. Files: {\tt constraint}
-
-\item[Type check generation]
-Emit explicit ICR code for any necessary type checks that are too complex to be
-easily generated on the fly by the back end. Files: {\tt checkgen}
-
-\item[Event driven operations]
-Various parts of ICR are incrementally recomputed, either eagerly on
-modification of the ICR, or lazily, when the relevant information is needed.
-\begin{itemize}
-\item Check that type assertions are satisfied, marking places where type
-checks need to be done.
-
-\item Locate let calls.
-
-\item Delete functions and variables with no references
-\end{itemize}
-Files: {\tt ir1util}, {\tt ir1opt}
-
-\item[ICR finalize]
-This phase is run after all components have been compiled. It scans the
-global variable references, looking for references to undefined variables
-and incompatible function redefinitions. Files: {\tt ir1final}, {\tt main}.
-
-\item[Environment analysis]
-Determine which distinct environments need to be allocated, and what
-context needed to be closed over by each environment. We detect non-local
-exits and set closure variables. We also emit cleanup code as funny
-function calls. This is the last pure ICR pass. Files: {\tt envanal}
-
-\item[Global TN allocation (GTN)]
-Iterate over all defined functions, determining calling conventions
-and assigning TNs to local variables. Files: {\tt gtn}
-
-\item[Local TN allocation (LTN)]
-Use type and policy information to determine which VMR translation to use
-for known functions, and then create TNs for expression evaluation
-temporaries. We also accumulate some random information needed by VMR
-conversion. Files: {\tt ltn}
-
-\item[Control analysis]
-Linearize the flow graph in a way that minimizes the number of branches. The
-block-level structure of the flow graph is basically frozen at this point.
-Files: {\tt control}
-
-\item[Stack analysis]
-Maintain stack discipline for unknown-values continuation in the presence
-of local exits. Files: {\tt stack}
-
-\item[Entry analysis]
-Collect some back-end information for each externally callable function.
-
-\item[VMR conversion] Convert ICR into VMR by translating nodes into VOPs.
-Emit type checks. Files: {\tt ir2tran, vmdef}
-
-\item[Copy propagation] Use flow analysis to eliminate unnecessary copying of
-TN values. Files: {\tt copyprop}
-
-\item[Representation selection]
-Look at all references to each TN to determine which representation has the
-lowest cost. Emit appropriate move and coerce VOPS for that representation.
-
-\item[Lifetime analysis]
-Do flow analysis to find the set of TNs whose lifetimes
-overlap with the lifetimes of each TN being packed. Annotate call VOPs with
-the TNs that need to be saved. Files: {\tt life}
-
-\item[Pack]
-Find a legal register allocation, attempting to minimize unnecessary moves.
-Files: {\tt pack}
-
-\item[Code generation]
-Call the VOP generators to emit assembly code. Files: {\tt codegen}
-
-\item[Pipeline reorganization] On some machines, move memory references
-backward in the code so that they can overlap with computation. On machines
-with delayed branch instructions, locate instructions that can be moved into
-delay slots. Files: {\tt assem-opt}
-
-\item[Assembly]
-Resolve branches and convert in to object code and fixup information.
-Files: {\tt assembler}
-
-\item[Dumping] Convert the compiled code into an object file or in-core
-function. Files: {\tt debug-dump}, {\tt dump}, {\tt vm/core}
-
-\end{description}
-
-\chapter{The Implicit Continuation Representation}
-
-The set of special forms recognized is exactly that specified in the Common
-Lisp manual. Everything that is described as a macro in CLTL is a macro.
-
-Large amounts of syntactic information are thrown away by the conversion to an
-anonymous flow graph representation. The elimination of names eliminates the
-need to represent most environment manipulation special forms. The explicit
-representation of control eliminates the need to represent BLOCK and GO, and
-makes flow analysis easy. The full Common Lisp LAMBDA is implemented with a
-simple fixed-arg lambda, which greatly simplifies later code.
-
-The elimination of syntactic information eliminates the need for most of the
-"beta transformation" optimizations in Rabbit. There are no progns, no
-tagbodys and no returns. There are no "close parens" which get in the way of
-determining which node receives a given value.
-
-In ICR, computation is represented by Nodes. These are the node types:
-\begin{description}
-\item[if] Represents all conditionals.
-
-\item[set] Represents a {\tt setq}.
-
-\item[ref] Represents a constant or variable reference.
-
-\item[combination] Represents a normal function call.
-
-\item[MV-combination] Represents a {\tt multiple-value-call}. This is used to
-implement all multiple value receiving forms except for {\tt
-multiple-value-prog1}, which is implicit.
-
-\item[bind]
-This represents the allocation and initialization of the variables in
-a lambda.
-
-\item[return]
-This collects the return value from a lambda and represents the
-control transfer on return.
-
-\item[entry] Marks the start of a dynamic extent that can have non-local exits
-to it. Dynamic state can be saved at this point for restoration on re-entry.
-
-\item[exit] Marks a potentially non-local exit. This node is interposed
-between the non-local uses of a continuation and the {\tt dest} so that code to
-do a non-local exit can be inserted if necessary.
-\end{description}
-
-Some slots are shared between all node types (via defstruct inheritance.) This
-information held in common between all nodes often makes it possible to avoid
-special-casing nodes on the basis of type. This shared information is
-primarily concerned with the order of evaluation and destinations and
-properties of results. This control and value flow is indicated in the node
-primarily by pointing to continuations.
-
-The {\tt continuation} structure represents information sufficiently related
-to the normal notion of a continuation that naming it so seems sensible.
-Basically, a continuation represents a place in the code, or alternatively the
-destination of an expression result and a transfer of control. These two
-notions are bound together for the same reasons that they are related in the
-standard functional continuation interpretation.
-
-A continuation may be deprived of either or both of its value or control
-significance. If the value of a continuation is unused due to evaluation for
-effect, then the continuation will have a null {\tt dest}. If the {\tt next}
-node for a continuation is deleted by some optimization, then {\tt next} will
-be {\tt :none}.
-
- [\#\#\# Continuation kinds...]
-
-The {\tt block} structure represents a basic block, in the the normal sense.
-Control transfers other than simple sequencing are represented by information
-in the block structure. The continuation for the last node in a block
-represents only the destination for the result.
-
-It is very difficult to reconstruct anything resembling the original source
-from ICR, so we record the original source form in each node. The location of
-the source form within the input is also recorded, allowing for interfaces such
-as "Edit Compiler Warnings". See section \ref{source-paths}.
-
-Forms such as special-bind and catch need to have cleanup code executed at all
-exit points from the form. We represent this constraint in ICR by annotating
-the code syntactically within the form with a Cleanup structure describing what
-needs to be cleaned up. Environment analysis determines the cleanup locations
-by watching for a change in the cleanup between two continuations. We can't
-emit cleanup code during ICR conversion, since we don't know which exits will
-be local until after ICR optimizations are done.
-
-Special binding is represented by a call to the funny function %Special-Bind.
-The first argument is the Global-Var structure for the variable bound and the
-second argument is the value to bind it to.
-
-Some subprimitives are implemented using a macro-like mechanism for translating
-%PRIMITIVE forms into arbitrary lisp code. Subprimitives special-cased by VMR
-conversion are represented by a call to the funny function %%Primitive. The
-corresponding Template structure is passed as the first argument.
-
-We check global function calls for syntactic legality with respect to any
-defined function type function. If the call is illegal or we are unable to
-tell if it is legal due to non-constant keywords, then we give a warning and
-mark the function reference as :notinline to force a full call and cause
-subsequent phases to ignore the call. If the call is legal and is to a known
-function, then we annotate the Combination node with the Function-Info
-structure that contains the compiler information for the function.
-
-\f
-\section{Tail sets}
-\#|
-Probably want to have a GTN-like function result equivalence class mechanism
-for ICR type inference. This would be like the return value propagation being
-done by Propagate-From-Calls, but more powerful, less hackish, and known to
-terminate. The ICR equivalence classes could probably be used by GTN, as well.
-
-What we do is have local call analysis eagerly maintain the equivalence classes
-of functions that return the same way by annotating functions with a Tail-Info
-structure shared between all functions whose value could be the value of this
-function. We don't require that the calls actually be tail-recursive, only
-that the call deliver its value to the result continuation. [\#\#\# Actually
-now done by ICR-OPTIMIZE-RETURN, which is currently making ICR optimize
-mandatory.]
-
-We can then use the Tail-Set during ICR type inference. It would have a type
-that is the union across all equivalent functions of the types of all the uses
-other than in local calls. This type would be recomputed during optimization
-of return nodes. When the type changes, we would propagate it to all calls to
-any of the equivalent functions. How do we know when and how to recompute the
-type for a tail-set? Recomputation is driven by type propagation on the result
-continuation.
-
-This is really special-casing of RETURN nodes. The return node has the type
-which is the union of all the non-call uses of the result. The tail-set is
-found though the lambda. We can then recompute the overall union by taking the
-union of the type per return node, rather than per-use.
-
-
-How do result type assertions work? We can't intersect the assertions across
-all functions in the equivalence class, since some of the call combinations may
-not happen (or even be possible). We can intersect the assertion of the result
-with the derived types for non-call uses.
-
-When we do a tail call, we obviously can't check that the returned value
-matches our assertion. Although in principle, we would like to be able to
-check all assertions, to preserve system integrity, we only need to check
-assertions that we depend on. We can afford to lose some assertion information
-as long as we entirely lose it, ignoring it for type inference as well as for
-type checking.
-
-Things will work out, since the caller will see the tail-info type as the
-derived type for the call, and will emit a type check if it needs a stronger
-result.
-
-A remaining question is whether we should intersect the assertion with
-per-RETURN derived types from the very beginning (i.e. before the type check
-pass). I think the answer is yes. We delay the type check pass so that we can
-get our best guess for the derived type before we decide whether a check is
-necessary. But with the function return type, we aren't committing to doing
-any type check when we intersect with the type assertion; the need to type
-check is still determined in the type check pass by examination of the result
-continuation.
-
-What is the relationship between the per-RETURN types and the types in the
-result continuation? The assertion is exactly the Continuation-Asserted-Type
-(note that the asserted type of result continuations will never change after
-ICR conversion). The per-RETURN derived type is different than the
-Continuation-Derived-Type, since it is intersected with the asserted type even
-before Type Check runs. Ignoring the Continuation-Derived-Type probably makes
-life simpler anyway, since this breaks the potential circularity of the
-Tail-Info-Type will affecting the Continuation-Derived-Type, which affects...
-
-When a given return has no non-call uses, we represent this by using
-*empty-type*. This consistent with the interpretation that a return type of
-NIL means the function can't return.
-
-\f
-\section{Hairy function representation}
-
-Non-fixed-arg functions are represented using Optional-Dispatch. An
-Optional-Dispatch has an entry-point function for each legal number of
-optionals, and one for when extra args are present. Each entry point function
-is a simple lambda. The entry point function for an optional is passed the
-arguments which were actually supplied; the entry point function is expected to
-default any remaining parameters and evaluate the actual function body.
-
-If no supplied-p arg is present, then we can do this fairly easily by having
-each entry point supply its default and call the next entry point, with the
-last entry point containing the body. If there are supplied-p args, then entry
-point function is replaced with a function that calls the original entry
-function with T's inserted at the position of all the supplied args with
-supplied-p parameters.
-
-We want to be a bit clever about how we handle arguments declared special when
-doing optional defaulting, or we will emit really gross code for special
-optionals. If we bound the arg specially over the entire entry-point function,
-then the entry point function would be caused to be non-tail-recursive. What
-we can do is only bind the variable specially around the evaluation of the
-default, and then read the special and store the final value of the special
-into a lexical variable which we then pass as the argument. In the common case
-where the default is a constant, we don't have to special-bind at all, since
-the computation of the default is not affected by and cannot affect any special
-bindings.
-
-Keyword and rest args are both implemented using a LEXPR-like "more args"
-convention. The More-Entry takes two arguments in addition to the fixed and
-optional arguments: the argument context and count. (ARG <context> <n>)
-accesses the N'th additional argument. Keyword args are implemented directly
-using this mechanism. Rest args are created by calling %Listify-Rest-Args with
-the context and count.
-
-The More-Entry parses the keyword arguments and passes the values to the main
-function as positional arguments. If a keyword default is not constant, then
-we pass a supplied-p parameter into the main entry and let it worry about
-defaulting the argument. Since the main entry accepts keywords in parsed form,
-we can parse keywords at compile time for calls to known functions. We keep
-around the original parsed lambda-list and related information so that people
-can figure out how to call the main entry.
-
-\f
-\section{ICR representation of non-local exits}
-
-All exits are initially represented by EXIT nodes:
-How about an Exit node:
- (defstruct (exit (:include node))
- value)
-The Exit node uses the continuation that is to receive the thrown Value.
-During optimization, if we discover that the Cont's home-lambda is the same is
-the exit node's, then we can delete the Exit node, substituting the Cont for
-all of the Value's uses.
-
-The successor block of an EXIT is the entry block in the entered environment.
-So we use the Exit node to mark the place where exit code is inserted. During
-environment analysis, we need only insert a single block containing the entry
-point stub.
-
-We ensure that all Exits that aren't for a NLX don't have any Value, so that
-local exits never require any value massaging.
-
-The Entry node marks the beginning of a block or tagbody:
- (defstruct (entry (:include node))
- (continuations nil :type list))
-
-It contains a list of all the continuations that the body could exit to. The
-Entry node is used as a marker for the the place to snapshot state, including
-the control stack pointer. Each lambda has a list of its Entries so
-that environment analysis can figure out which continuations are really being
-closed over. There is no reason for optimization to delete Entry nodes,
-since they are harmless in the degenerate case: we just emit no code (like a
-no-var let).
-
-
-We represent CATCH using the lexical exit mechanism. We do a transformation
-like this:
- (catch 'foo xxx) ==>
- (block \#:foo
- (%catch \#'(lambda () (return-from \#:foo (%unknown-values))) 'foo)
- (%within-cleanup :catch
- xxx))
-
-%CATCH just sets up the catch frame which points to the exit function. %Catch
-is an ordinary function as far as ICR is concerned. The fact that the catcher
-needs to be cleaned up is expressed by the Cleanup slots in the continuations
-in the body. %UNKNOWN-VALUES is a dummy function call which represents the
-fact that we don't know what values will be thrown.
-
-%WITHIN-CLEANUP is a special special form that instantiates its first argument
-as the current cleanup when converting the body. In reality, the lambda is
-also created by the special special form %ESCAPE-FUNCTION, which gives the
-lambda a special :ESCAPE kind so that the back end knows not to generate any
-code for it.
-
-
-We use a similar hack in Unwind-Protect to represent the fact that the cleanup
-forms can be invoked at arbitrarily random times.
- (unwind-protect p c) ==>
- (flet ((\#:cleanup () c))
- (block \#:return
- (multiple-value-bind
- (\#:next \#:start \#:count)
- (block \#:unwind
- (%unwind-protect \#'(lambda (x) (return-from \#:unwind x)))
- (%within-cleanup :unwind-protect
- (return-from \#:return p)))
- (\#:cleanup)
- (%continue-unwind \#:next \#:start \#:count))))
-
-We use the block \#:unwind to represent the entry to cleanup code in the case
-where we are non-locally unwound. Calling of the cleanup function in the
-drop-through case (or any local exit) is handled by cleanup generation. We
-make the cleanup a function so that cleanup generation can add calls at local
-exits from the protected form. \#:next, \#:start and \#:count are state used in
-the case where we are unwound. They indicate where to go after doing the
-cleanup and what values are being thrown. The cleanup encloses only the
-protected form. As in CATCH, the escape function is specially tagged as
-:ESCAPE. The cleanup function is tagged as :CLEANUP to inhibit let conversion
-(since references are added in environment analysis.)
-
-Notice that implementing these forms using closures over continuations
-eliminates any need to special-case ICR flow analysis. Obviously we don't
-really want to make heap-closures here. In reality these functions are
-special-cased by the back-end according to their KIND.
-
-\f
-\section{Block compilation}
-
-One of the properties of ICR is that supports "block compilation" by allowing
-arbitrarily large amounts of code to be converted at once, with actual
-compilation of the code being done at will.
-
-
-In order to preserve the normal semantics we must recognize that proclamations
-(possibly implicit) are scoped. A proclamation is in effect only from the time
-of appearance of the proclamation to the time it is contradicted. The current
-global environment at the end of a block is not necessarily the correct global
-environment for compilation of all the code within the block. We solve this
-problem by closing over the relevant information in the ICR at the time it is
-converted. For example, each functional variable reference is marked as
-inline, notinline or don't care. Similarly, each node contains a structure
-known as a Cookie which contains the appropriate settings of the compiler
-policy switches.
-
-We actually convert each form in the file separately, creating a separate
-"initial component" for each one. Later on, these components are merged as
-needed. The main reason for doing this is to cause EVAL-WHEN processing to be
-interleaved with reading.
-
-\f
-\section{Entry points}
-
-\#|
-
-Since we need to evaluate potentially arbitrary code in the XEP argument forms
-(for type checking), we can't leave the arguments in the wired passing
-locations. Instead, it seems better to give the XEP max-args fixed arguments,
-with the passing locations being the true passing locations. Instead of using
-%XEP-ARG, we reference the appropriate variable.
-
-Also, it might be a good idea to do argument count checking and dispatching
-with explicit conditional code in the XEP. This would simplify both the code
-that creates the XEP and the VMR conversion of XEPs. Also, argument count
-dispatching would automatically benefit from any cleverness in compilation of
-case-like forms (jump tables, etc). On the downside, this would push some
-assumptions about how arg dispatching is done into ICR. But then we are
-currently violating abstraction at least as badly in VMR conversion, which is
-also supposed to be implementation independent.
-|\#
-
-As a side-effect of finding which references to known functions can be
-converted to local calls, we find any references that cannot be converted.
-References that cannot be converted to a local call must evaluate to a
-"function object" (or function-entry) that can be called using the full call
-convention. A function that can be called from outside the component is called
-an "entry-point".
-
-Lots of stuff that happens at compile-time with local function calls must be
-done at run-time when an entry-point is called.
-
-It is desirable for optimization and other purposes if all the calls to every
-function were directly present in ICR as local calls. We cannot directly do
-this with entry-point functions, since we don't know where and how the
-entry-point will be called until run-time.
-
-What we do is represent all the calls possible from outside the component by
-local calls within the component. For each entry-point function, we create a
-corresponding lambda called the external entry point or XEP. This is a
-function which takes the number of arguments passed as the first argument,
-followed by arguments corresponding to each required or optional argument.
-
-If an optional argument is unsupplied, the value passed into the XEP is
-undefined. The XEP is responsible for doing argument count checking and
-dispatching.
-
-In the case of a fixed-arg lambda, we emit a call to the %VERIFY-ARGUMENT-COUNT
-funny function (conditional on policy), then call the real function on the
-passed arguments. Even in this simple case, we benefit several ways from
-having a separate XEP:
- -- The argument count checking is factored out, and only needs to be done in
- full calls.
- -- Argument type checking happens automatically as a consequence of passing
- the XEP arguments in a local call to the real function. This type checking
- is also only done in full calls.
- -- The real function may use a non-standard calling convention for the benefit
- of recursive or block-compiled calls. The XEP converts arguments/return
- values to/from the standard convention. This also requires little
- special-casing of XEPs.
-
-If the function has variable argument count (represented by an
-OPTIONAL-DISPATCH), then the XEP contains a COND which dispatches off of the
-argument count, calling the appropriate entry-point function (which then does
-defaulting). If there is a more entry (for keyword or rest args), then the XEP
-obtains the more arg context and count by calling the %MORE-ARG-CONTEXT funny
-function.
-
-All non-local-call references to functions are replaced with references to the
-corresponding XEP. ICR optimization may discover a local call that was
-previously a non-local reference. When we delete the reference to the XEP, we
-may find that it has no references. In this case, we can delete the XEP,
-causing the function to no longer be an entry-point.
-
-\f
\ No newline at end of file
+++ /dev/null
-\part{Compiler Organization}
-\include{compiler-overview}
-\include{front}
-\include{middle}
-\include{back}
-\include{interface}
+++ /dev/null
-% -*- Dictionary: design; Package: C -*-
-
-\#|
-\chapter{Debugger Information}
-\index{debugger information}
-\label{debug-info}
-
-Although the compiler's great freedom in choice of function call conventions
-and variable representations has major efficiency advantages, it also has
-unfortunate consequences for the debugger. The debug information that we need
-is even more elaborate than for conventional "compiled" languages, since we
-cannot even do a simple backtrace without some debug information. However,
-once having gone this far, it is not that difficult to go the extra distance,
-and provide full source level debugging of compiled code.
-
-Full debug information has a substantial space penalty, so we allow different
-levels of debug information to be specified. In the extreme case, we can
-totally omit debug information.
-
-\f
-\section{The Debug-Info Structure}
-\index{debug-info structure}
-
-The Debug-Info structure directly represents information about the
-source code, and points to other structures that describe the layout of
-run-time data structures.
-
-
-Make some sort of minimal debug-info format that would support at least the
-common cases of level 1 (since that is what we would release), and perhaps
-level 0. Actually, it seems it wouldn't be hard to crunch nearly all of the
-debug-function structure and debug-info function map into a single byte-vector.
-We could have an uncrunch function that restored the current format. This
-would be used by the debugger, and also could be used by purify to delete parts
-of the debug-info even when the compiler dumps it in crunched form.
-[Note that this isn't terribly important if purify is smart about
-debug-info...]
-|\#
-
-\f
-Compiled source map representation:
-
-[\#\#\# store in debug-function PC at which env is properly initialized, i.e.
-args (and return-pc, etc.) in internal locations. This is where a
-:function-start breakpoint would break.]
-
-[\#\#\# Note that that we can easily cache the form-number => source-path or
-form-number => form translation using a vector indexed by form numbers that we
-build during a walk.]
-
-
-
-
-Instead of using source paths in the debug-info, use "form numbers". The form
-number of a form is the number of forms that we walk to reach that form when
-doing a pre-order walk of the source form. [Might want to use a post-order
-walk, as that would more closely approximate evaluation order.]
-
-
-We probably want to continue using source-paths in the compiler, since they are
-quick to compute and to get you to a particular form. [\#\#\# But actually, I
-guess we don't have to precompute the source paths and annotate nodes with
-them: instead we could annotate the nodes with the actual original source form.
-Then if we wanted to find the location of that form, we could walk the root
-source form, looking that original form. But we might still need to enter all
-the forms in a hashtable so that we can tell during IR1 conversion that a given
-form appeared in the original source.]
-
-
-Note that form numbers have an interesting property: it is quite efficient to
-determine whether an arbitrary form is a subform of some other form, since the
-form number of B will be > than A's number and < A's next sibling's number iff
-B is a subform of A.
-
-This should be quite useful for doing the source=>pc mapping in the debugger,
-since that problem reduces to finding the subset of the known locations that
-are for subforms of the specified form.
-
-
-Assume a byte vector with a standard variable-length integer format, something
-like this:
- 0..253 => the integer
- 254 => read next two bytes for integer
- 255 => read next four bytes for integer
-
-Then a compiled debug block is just a sequence of variable-length integers in a
-particular order, something like this:
- number of successors
- ...offsets of each successor in the function's blocks vector...
- first PC
- [offset of first top-level form (in forms) (only if not component default)]
- form number of first source form
- first live mask (length in bytes determined by number of VARIABLES)
- ...more <PC, top-level form offset, form-number, live-set> tuples...
-
-We determine the number of locations recorded in a block by the finding the
-start of the next compiled debug block in the blocks vector.
-
-[\#\#\# Actually, only need 2 bits for number of successors {0,1,2}. We might
-want to use other bits in the first byte to indicate the kind of location.]
-[\#\#\# We could support local packing by having a general concept of "alternate
-locations" instead of just regular and save locations. The location would have
-a bit indicating that there are alternate locations, in which case we read the
-number of alternate locations and then that many more SC-OFFSETs. In the
-debug-block, we would have a second bit mask with bits set for TNs that are in
-an alternate location. We then read a number for each such TN, with the value
-being interpreted as an index into the Location's alternate locations.]
-
-
-
-It looks like using structures for the compiled-location-info is too bulky.
-Instead we need some packed binary representation.
-
-First, let's represent a SC/offset pair with an "SC-Offset", which is an
-integer with the SC in the low 5 bits and the offset in the remaining bits:
- ----------------------------------------------------
- | Offset (as many bits as necessary) | SC (5 bits) |
- ----------------------------------------------------
-Probably the result should be constrained to fit in a fixnum, since it will be
-more efficient and gives more than enough possible offsets.
-
-We can the represent a compiled location like this:
- single byte of boolean flags:
- uninterned name
- packaged name
- environment-live
- has distinct save location
- has ID (name not unique in this fun)
- name length in bytes (as var-length integer)
- ...name bytes...
- [if packaged, var-length integer that is package name length]
- ...package name bytes...]
- [If has ID, ID as var-length integer]
- SC-Offset of primary location (as var-length integer)
- [If has save SC, SC-Offset of save location (as var-length integer)]
-
-
-\f
-
-But for a whizzy breakpoint facility, we would need a good source=>code map.
-Dumping a complete code=>source map might be as good a way as any to represent
-this, due to the one-to-many relationship between source and code locations.
-
-We might be able to get away with just storing the source locations for the
-beginnings of blocks and maintaining a mapping from code ranges to blocks.
-This would be fine both for the profiler and for the "where am I running now"
-indication. Users might also be convinced that it was most interesting to
-break at block starts, but I don't really know how easily people could develop
-an understanding of basic blocks.
-
-It could also be a bit tricky to map an arbitrary user-designated source
-location to some "closest" source location actually in the debug info.
-This problem probably exists to some degree even with a full source map, since
-some forms will never appear as the source of any node. It seems you might
-have to negotiate with the user. He would mouse something, and then you would
-highlight some source form that has a common prefix (i.e. is a prefix of the
-user path, or vice-versa.) If they aren't happy with the result, they could
-try something else. In some cases, the designated path might be a prefix of
-several paths. This ambiguity might be resolved by picking the shortest path
-or letting the user choose.
-
-At the primitive level, I guess what this means is that the structure of source
-locations (i.e. source paths) must be known, and the source=>code operation
-should return a list of <source,code> pairs, rather than just a list of code
-locations. This allows the debugger to resolve the ambiguity however it wants.
-
-I guess the formal definition of which source paths we would return is:
- All source paths in the debug info that have a maximal common prefix with
- the specified path. i.e. if several paths have the complete specified path
- as a prefix, we return them all. Otherwise, all paths with an equally
- large common prefix are returned: if the path with the most in common
- matches only the first three elements, then we return all paths that match
- in the first three elements. As a degenerate case (which probably
- shouldn't happen), if there is no path with anything in common, then we
- return *all* of the paths.
-
-
-
-In the DEBUG-SOURCE structure we may ultimately want a vector of the start
-positions of each source form, since that would make it easier for the debugger
-to locate the source. It could just open the file, FILE-POSITION to the form,
-do a READ, then loop down the source path. Of course, it could read each form
-starting from the beginning, but that might be too slow.
-
-
-Do XEPs really need Debug-Functions? The only time that we will commonly end
-up in the debugger on an XEP is when an argument type check fails. But I
-suppose it would be nice to be able to print the arguments passed...
-
-
-Note that assembler-level code motion such as pipeline reorganization can cause
-problems with our PC maps. The assembler needs to know that debug info markers
-are different from real labels anyway, so I suppose it could inhibit motion
-across debug markers conditional on policy. It seems unworthwhile to remember
-the node for each individual instruction.
-
-
-For tracing block-compiled calls:
- Info about return value passing locations?
- Info about where all the returns are?
-
-We definitely need the return-value passing locations for debug-return. The
-question is what the interface should be. We don't really want to have a
-visible debug-function-return-locations operation, since there are various
-value passing conventions, and we want to paper over the differences.
-
-
-Probably should be a compiler option to initialize stack frame to a special
-uninitialized object (some random immediate type). This would aid debugging,
-and would also help GC problems. For the latter reason especially, this should
-be locally-turn-onable (off of policy? the new debug-info quality?).
-
-
-What about the interface between the evaluator and the debugger? (i.e. what
-happens on an error, etc.) Compiler error handling should be integrated with
-run-time error handling. Ideally the error messages should look the same.
-Practically, in some cases the run-time errors will have less information. But
-the error should look the same to the debugger (or at least similar).
-
-
-\f
-;;;; Debugger interface:
-
-How does the debugger interface to the "evaluator" (where the evaluator means
-all of native code, byte-code and interpreted IR1)? It seems that it would be
-much more straightforward to have a consistent user interface to debugging
-all code representations if there was a uniform debugger interface to the
-underlying stuff, and vice-versa.
-
-Of course, some operations might not be supported by some representations, etc.
-For example, fine-control stepping might not be available in native code.
-In other cases, we might reduce an operation to the lowest common denominator,
-for example fetching lexical variables by string and admitting the possibility
-of ambiguous matches. [Actually, it would probably be a good idea to store the
-package if we are going to allow variables to be closed over.]
-
-Some objects we would need:
-Location:
- The constant information about the place where a value is stored,
- everything but which particular frame it is in. Operations:
- location name, type, etc.
- location-value frame location (setf'able)
- monitor-location location function
- Function is called whenever location is set with the location,
- frame and old value. If active values aren't supported, then we
- dummy the effect using breakpoints, in which case the change won't
- be noticed until the end of the block (and intermediate changes
- will be lost.)
-debug info:
- All the debug information for a component.
-Frame:
- frame-changed-locations frame => location*
- Return a list of the locations in frame that were changed since the
- last time this function was called. Or something. This is for
- displaying interesting state changes at breakpoints.
- save-frame-state frame => frame-state
- restore-frame-state frame frame-state
- These operations allow the debugger to back up evaluation, modulo
- side-effects and non-local control transfers. This copies and
- restores all variables, temporaries, etc, local to the frame, and
- also the current PC and dynamic environment (current catch, etc.)
-
- At the time of the save, the frame must be for the running function
- (not waiting for a call to return.) When we restore, the frame
- becomes current again, effectively exiting from any frames on top.
- (Of course, frame must not already be exited.)
-
-Thread:
- Representation of which stack to use, etc.
-Block:
- What successors the block has, what calls there are in the block.
- (Don't need to know where calls are as long as we know called function,
- since can breakpoint at the function.) Whether code in this block is
- wildly out of order due to being the result of loop-invariant
- optimization, etc. Operations:
- block-successors block => code-location*
- block-forms block => (source-location code-location)*
- Return the corresponding source locations and code locations for
- all forms (and form fragments) in the block.
-
-\f
-Variable maps:
-
-There are about five things that the debugger might want to know about a
-variable:
-
- Name
- Although a lexical variable's name is "really" a symbol (package and
- all), in practice it doesn't seem worthwhile to require all the symbols
- for local variable names to be retained. There is much less VM and GC
- overhead for a constant string than for a symbol. (Also it is useful
- to be able to access gensyms in the debugger, even though they are
- theoretically ineffable).
-
- ID
- Which variable with the specified name is this? It is possible to have
- multiple variables with the same name in a given function. The ID is
- something that makes Name unique, probably a small integer. When
- variables aren't unique, we could make this be part of the name, e.g.
- "FOO\#1", "FOO\#2". But there are advantages to keeping this separate,
- since in many cases lifetime information can be used to disambiguate,
- making qualification unnecessary.
-
- SC
- When unboxed representations are in use, we must have type information
- to properly read and write a location. We only need to know the
- SC for this, which would be amenable to a space-saving
- numeric encoding.
-
- Location
- Simple: the offset in SC. [Actually, we need the save location too.]
-
- Lifetime
- In what parts of the program does this variable hold a meaningful
- value? It seems prohibitive to record precise lifetime information,
- both in space and compiler effort, so we will have to settle for some
- sort of approximation.
-
- The finest granularity at which it is easy to determine liveness is the
- the block: we can regard the variable lifetime as the set of blocks
- that the variable is live in. Of course, the variable may be dead (and
- thus contain meaningless garbage) during arbitrarily large portions of
- the block.
-
- Note that this subsumes the notion of which function a variable belongs
- to. A given block is only in one function, so the function is
- implicit.
-
-
-The variable map should represent this information space-efficiently and with
-adequate computational efficiency.
-
-The SC and ID can be represented as small integers. Although the ID can in
-principle be arbitrarily large, it should be <100 in practice. The location
-can be represented by just the offset (a moderately small integer), since the
-SB is implicit in the SC.
-
-The lifetime info can be represented either as a bit-vector indexed by block
-numbers, or by a list of block numbers. Which is more compact depends both on
-the size of the component and on the number of blocks the variable is live in.
-In the limit of large component size, the sparse representation will be more
-compact, but it isn't clear where this crossover occurs. Of course, it would
-be possible to use both representations, choosing the more compact one on a
-per-variable basis. Another interesting special case is when the variable is
-live in only one block: this may be common enough to be worth picking off,
-although it is probably rarer for named variables than for TNs in general.
-
-If we dump the type, then a normal list-style type descriptor is fine: the
-space overhead is small, since the shareability is high.
-
-We could probably save some space by cleverly representing the var-info as
-parallel vectors of different types, but this would be more painful in use.
-It seems better to just use a structure, encoding the unboxed fields in a
-fixnum. This way, we can pass around the structure in the debugger, perhaps
-even exporting it from the the low-level debugger interface.
-
-[\#\#\# We need the save location too. This probably means that we need two slots
-of bits, since we need the save offset and save SC. Actually, we could let the
-save SC be implied by the normal SC, since at least currently, we always choose
-the same save SC for a given SC. But even so, we probably can't fit all that
-stuff in one fixnum without squeezing a lot, so we might as well split and
-record both SCs.
-
-In a localized packing scheme, we would have to dump a different var-info
-whenever either the main location or the save location changes. As a practical
-matter, the save location is less likely to change than the main location, and
-should never change without the main location changing.
-
-One can conceive of localized packing schemes that do saving as a special case
-of localized packing. If we did this, then the concept of a save location
-might be eliminated, but this would require major changes in the IR2
-representation for call and/or lifetime info. Probably we will want saving to
-continue to be somewhat magical.]
-
-
-How about:
-
-(defstruct var-info
- ;;
- ;; This variable's name. (symbol-name of the symbol)
- (name nil :type simple-string)
- ;;
- ;; The SC, ID and offset, encoded as bit-fields.
- (bits nil :type fixnum)
- ;;
- ;; The set of blocks this variable is live in. If a bit-vector, then it has
- ;; a 1 when indexed by the number of a block that it is live in. If an
- ;; I-vector, then it lists the live block numbers. If a fixnum, then that is
- ;; the number of the sole live block.
- (lifetime nil :type (or vector fixnum))
- ;;
- ;; The variable's type, represented as list-style type descriptor.
- type)
-
-Then the debug-info holds a simple-vector of all the var-info structures for
-that component. We might as well make it sorted alphabetically by name, so
-that we can binary-search to find the variable corresponding to a particular
-name.
-
-We need to be able to translate PCs to block numbers. This can be done by an
-I-Vector in the component that contains the start location of each block. The
-block number is the index at which we find the correct PC range. This requires
-that we use an emit-order block numbering distinct from the IR2-Block-Number,
-but that isn't any big deal. This seems space-expensive, but it isn't too bad,
-since it would only be a fraction of the code size if the average block length
-is a few words or more.
-
-An advantage of our per-block lifetime representation is that it directly
-supports keeping a variable in different locations when in different blocks,
-i.e. multi-location packing. We use a different var-info for each different
-packing, since the SC and offset are potentially different. The Name and ID
-are the same, representing the fact that it is the same variable. It is here
-that the ID is most significant, since the debugger could otherwise make
-same-name variables unique all by itself.
-
-
-
-Stack parsing:
-
-[\#\#\# Probably not worth trying to make the stack parseable from the bottom up.
-There are too many complications when we start having variable sized stuff on
-the stack. It seems more profitable to work on making top-down parsing robust.
-Since we are now planning to wire the bottom-up linkage info, scanning from the
-bottom to find the top frame shouldn't be too inefficient, even when there was
-a runaway recursion. If we somehow jump into hyperspace, then the debugger may
-get confused, but we can debug this sort of low-level system lossage using
-ADB.]
-
-
-There are currently three relevant context pointers:
- -- The PC. The current PC is wired (implicit in the machine). A saved
- PC (RETURN-PC) may be anywhere in the current frame.
- -- The current stack context (CONT). The current CONT is wired. A saved
- CONT (OLD-CONT) may be anywhere in the current frame.
- -- The current code object (ENV). The current ENV is wired. When saved,
- this is extra-difficult to locate, since it is saved by the caller, and is
- thus at an unknown offset in OLD-CONT, rather than anywhere in the current
- frame.
-
-We must have all of these to parse the stack.
-
-With the proposed Debug-Function, we parse the stack (starting at the top) like
-this:
- 1] Use ENV to locate the current Debug-Info
- 2] Use the Debug-Info and PC to determine the current Debug-Function.
- 3] Use the Debug-Function to find the OLD-CONT and RETURN-PC.
- 4] Find the old ENV by searching up the stack for a saved code object
- containing the RETURN-PC.
- 5] Assign old ENV to ENV, OLD-CONT to CONT, RETURN-PC to PC and goto 1.
-
-If we changed the function representation so that the code and environment were
-a single object, then the location of the old ENV would be simplified. But we
-still need to represent ENV as separate from PC, since interrupts and errors
-can happen when the current PC isn't positioned at a valid return PC.
-
-It seems like it might be a good idea to save OLD-CONT, RETURN-PC and ENV at
-the beginning of the frame (before any stack arguments). Then we wouldn't have
-to search to locate ENV, and we also have a hope of parsing the stack even if
-it is damaged. As long as we can locate the start of some frame, we can trace
-the stack above that frame. We can recognize a probable frame start by
-scanning the stack for a code object (presumably a saved ENV).
-
- Probably we want some fairly general
-mechanism for specifying that a TN should be considered to be live for the
-duration of a specified environment. It would be somewhat easier to specify
-that the TN is live for all time, but this would become very space-inefficient
-in large block compilations.
-
-This mechanism could be quite useful for other debugger-related things. For
-example, when debuggability is important, we could make the TNs holding
-arguments live for the entire environment. This would guarantee that a
-backtrace would always get the right value (modulo setqs).
-
-Note that in this context, "environment" means the Environment structure (one
-per non-let function). At least according to current plans, even when we do
-inter-routine register allocation, the different functions will have different
-environments: we just "equate" the environments. So the number of live
-per-environment TNs is bounded by the size of a "function", and doesn't blow up
-in block compilation.
-
-The implementation is simple: per-environment TNs are flagged by the
-:Environment kind. :Environment TNs are treated the same as :Normal TNs by
-everyone except for lifetime/conflict analysis. An environment's TNs are also
-stashed in a list in the IR2-Environment structure. During during the conflict
-analysis post-pass, we look at each block's environment, and make all the
-environment's TNs always-live in that block.
-
-We can implement the "fixed save location" concept needed for lazy frame
-creation by allocating the save TNs as wired TNs at IR2 conversion time. We
-would use the new "environment lifetime" concept to specify the lifetimes of
-the save locations. There isn't any run-time overhead if we never get around
-to using the save TNs. [Pack would also have to notice TNs with pre-allocated
-save TNs, packing the original TN in the stack location if its FSC is the
-stack.]
-
-
-We want a standard (recognizable) format for an "escape" frame. We must make
-an escape frame whenever we start running another function without the current
-function getting a chance to save its registers. This may be due either to a
-truly asynchronous event such as a software interrupt, or due to an "escape"
-from a miscop. An escape frame marks a brief conversion to a callee-saves
-convention.
-
-Whenever a miscop saves registers, it should make an escape frame. This
-ensures that the "current" register contents can always be located by the
-debugger. In this case, it may be desirable to be able to indicate that only
-partial saving has been done. For example, we don't want to have to save all
-the FP registers just so that we can use a couple extra general registers.
-
-When when the debugger see an escape frame, it knows that register values are
-located in the escape frame's "register save" area, rather than in the normal
-save locations.
-
-It would be nice if there was a better solution to this internal error concept.
-One problem is that it seems there is a substantial space penalty for emitting
-all that error code, especially now that we don't share error code between
-errors because we want to preserve the source context in the PC. But this
-probably isn't really all that bad when considered as a fraction of the code.
-For example, the check part of a type check is 12 bytes, whereas the error part
-is usually only 6. In this case, we could never reduce the space overhead for
-type checks by more than 1/3, thus the total code size reduction would be
-small. This will be made even less important when we do type check
-optimizations to reduce the number of type checks.
-
-Probably we should stick to the same general internal error mechanism, but make
-it interact with the debugger better by allocating linkage registers and
-allowing proceedable errors. We could support shared error calls and
-non-proceedable errors when space is more important than debuggability, but
-this is probably more complexity than is worthwhile.
-
-We jump or trap to a routine that saves the context (allocating at most the
-return PC register). We then encode the error and context in the code
-immediately following the jump/trap. (On the MIPS, the error code can be
-encoded in the trap itself.) The error arguments would be encoded as
-SC-offsets relative to the saved context. This could solve both the
-arg-trashing problem and save space, since we could encode the SC-offsets more
-tersely than the corresponding move instructions.
+++ /dev/null
-\documentstyle[cmu-titlepage]{report} % -*- Dictionary: design -*-
-\title{Design of CMU Common Lisp}
-\author{Robert A. MacLachlan (ed)}
-\trnumber{CMU-CS-91-???}
-\abstract{This report documents internal details of the CMU Common Lisp
-compiler and run-time system. CMU Common Lisp is a public domain
-implementation of Common Lisp that runs on various Unix workstations.}
-
-\begin{document}
-\maketitle
-\tableofcontents
-\include{architecture}
-\include{compiler}
-\include{retargeting}
-\include{run-time}
-\appendix
-\include{glossary}
-\end{document}
+++ /dev/null
-\chapter{The Type System}
-
-\chapter{The Info Database}
+++ /dev/null
-Look at primtype.lisp and objdef.lisp (and early-objdef.lisp) for more
-up-to-date definitions of various tags. (For example, the simple
-string tag has changed since object.tex was written.)
-
-The string format has changed. According to "object.tex", string length is
-stored in the 24 bits of the string header. Instead, those 24 bits
-are set to zero, and string length is encoded in the same way as the
-other specialized simple-array counts, as a fixnum following the
-header.
-
-The number of slots for objects has changed since object.tex was
-written. The only reliable source for current slot definitions seems
-to be the primitive object data maintained by the compiler itself. See
-primtype.lisp and objdef.lisp, or look at the genesis code which reads
-this data to generate the various slot offsets in the C header file.
-
-The meaning of the function-self slot has changed in the X86 port:
-it points directly to the code to be executed.
-
-Nothing about FDEFN objects seems to be documented. FDEFN objects
-replace the simple SYMBOL-FUNCTION slot with a much more complicated
-mechanism, which I [WHN] dislike and would like to get rid of, but
-haven't [yet?].
+++ /dev/null
-\chapter{Fasload File Format}% -*- Dictionary: design -*-
-\section{General}
-
-The purpose of Fasload files is to allow concise storage and rapid
-loading of Lisp data, particularly function definitions. The intent
-is that loading a Fasload file has the same effect as loading the
-ASCII file from which the Fasload file was compiled, but accomplishes
-the tasks more efficiently. One noticeable difference, of course, is
-that function definitions may be in compiled form rather than
-S-expression form. Another is that Fasload files may specify in what
-parts of memory the Lisp data should be allocated. For example,
-constant lists used by compiled code may be regarded as read-only.
-
-In some Lisp implementations, Fasload file formats are designed to
-allow sharing of code parts of the file, possibly by direct mapping
-of pages of the file into the address space of a process. This
-technique produces great performance improvements in a paged
-time-sharing system. Since the Mach project is to produce a
-distributed personal-computer network system rather than a
-time-sharing system, efficiencies of this type are explicitly {\it not}
-a goal for the CMU Common Lisp Fasload file format.
-
-On the other hand, CMU Common Lisp is intended to be portable, as it will
-eventually run on a variety of machines. Therefore an explicit goal
-is that Fasload files shall be transportable among various
-implementations, to permit efficient distribution of programs in
-compiled form. The representations of data objects in Fasload files
-shall be relatively independent of such considerations as word
-length, number of type bits, and so on. If two implementations
-interpret the same macrocode (compiled code format), then Fasload
-files should be completely compatible. If they do not, then files
-not containing compiled code (so-called "Fasdump" data files) should
-still be compatible. While this may lead to a format which is not
-maximally efficient for a particular implementation, the sacrifice of
-a small amount of performance is deemed a worthwhile price to pay to
-achieve portability.
-
-The primary assumption about data format compatibility is that all
-implementations can support I/O on finite streams of eight-bit bytes.
-By "finite" we mean that a definite end-of-file point can be detected
-irrespective of the content of the data stream. A Fasload file will
-be regarded as such a byte stream.
-
-\section{Strategy}
-
-A Fasload file may be regarded as a human-readable prefix followed by
-code in a funny little language. When interpreted, this code will
-cause the construction of the encoded data structures. The virtual
-machine which interprets this code has a {\it stack} and a {\it table},
-both initially empty. The table may be thought of as an expandable
-register file; it is used to remember quantities which are needed
-more than once. The elements of both the stack and the table are
-Lisp data objects. Operators of the funny language may take as
-operands following bytes of the data stream, or items popped from the
-stack. Results may be pushed back onto the stack or pushed onto the
-table. The table is an indexable stack that is never popped; it is
-indexed relative to the base, not the top, so that an item once
-pushed always has the same index.
-
-More precisely, a Fasload file has the following macroscopic
-organization. It is a sequence of zero or more groups concatenated
-together. End-of-file must occur at the end of the last group. Each
-group begins with a series of seven-bit ASCII characters terminated
-by one or more bytes of all ones \verb|#xFF|; this is called the
-{\it header}. Following the bytes which terminate the header is the
-{\it body}, a stream of bytes in the funny binary language. The body
-of necessity begins with a byte other than \verb|#xFF|. The body is
-terminated by the operation {\tt FOP-END-GROUP}.
-
-The first nine characters of the header must be "{\tt FASL FILE}" in
-upper-case letters. The rest may be any ASCII text, but by
-convention it is formatted in a certain way. The header is divided
-into lines, which are grouped into paragraphs. A paragraph begins
-with a line which does {\it not} begin with a space or tab character,
-and contains all lines up to, but not including, the next such line.
-The first word of a paragraph, defined to be all characters up to but
-not including the first space, tab, or end-of-line character, is the
-{\it name} of the paragraph. A Fasload file header might look something like
-this:
-\begin{verbatim}
-FASL FILE >SteelesPerq>User>Guy>IoHacks>Pretty-Print.Slisp
-Package Pretty-Print
-Compiled 31-Mar-1988 09:01:32 by some random luser
-Compiler Version 1.6, Lisp Version 3.0.
-Functions: INITIALIZE DRIVER HACK HACK1 MUNGE MUNGE1 GAZORCH
- MINGLE MUDDLE PERTURB OVERDRIVE GOBBLE-KEYBOARD
- FRY-USER DROP-DEAD HELP CLEAR-MICROCODE
- %AOS-TRIANGLE %HARASS-READTABLE-MAYBE
-Macros: PUSH POP FROB TWIDDLE
-\end{verbatim}
-{\it one or more bytes of \verb|#xFF|}
-
-The particular paragraph names and contents shown here are only intended as
-suggestions.
-
-\section{Fasload Language}
-
-Each operation in the binary Fasload language is an eight-bit
-(one-byte) opcode. Each has a name beginning with "{\tt FOP-}". In
-the following descriptions, the name is followed by operand
-descriptors. Each descriptor denotes operands that follow the opcode
-in the input stream. A quantity in parentheses indicates the number
-of bytes of data from the stream making up the operand. Operands
-which implicitly come from the stack are noted in the text. The
-notation "$\Rightarrow$ stack" means that the result is pushed onto the
-stack; "$\Rightarrow$ table" similarly means that the result is added to the
-table. A construction like "{\it n}(1) {\it value}({\it n})" means that
-first a single byte {\it n} is read from the input stream, and this
-byte specifies how many bytes to read as the operand named {\it value}.
-All numeric values are unsigned binary integers unless otherwise
-specified. Values described as "signed" are in two's-complement form
-unless otherwise specified. When an integer read from the stream
-occupies more than one byte, the first byte read is the least
-significant byte, and the last byte read is the most significant (and
-contains the sign bit as its high-order bit if the entire integer is
-signed).
-
-Some of the operations are not necessary, but are rather special
-cases of or combinations of others. These are included to reduce the
-size of the file or to speed up important cases. As an example,
-nearly all strings are less than 256 bytes long, and so a special
-form of string operation might take a one-byte length rather than a
-four-byte length. As another example, some implementations may
-choose to store bits in an array in a left-to-right format within
-each word, rather than right-to-left. The Fasload file format may
-support both formats, with one being significantly more efficient
-than the other for a given implementation. The compiler for any
-implementation may generate the more efficient form for that
-implementation, and yet compatibility can be maintained by requiring
-all implementations to support both formats in Fasload files.
-
-Measurements are to be made to determine which operation codes are
-worthwhile; little-used operations may be discarded and new ones
-added. After a point the definition will be "frozen", meaning that
-existing operations may not be deleted (though new ones may be added;
-some operations codes will be reserved for that purpose).
-
-\begin{description}
-\item[0:] \hspace{2em} {\tt FOP-NOP} \\
-No operation. (This is included because it is recognized
-that some implementations may benefit from alignment of operands to some
-operations, for example to 32-bit boundaries. This operation can be used
-to pad the instruction stream to a desired boundary.)
-
-\item[1:] \hspace{2em} {\tt FOP-POP} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
-One item is popped from the stack and added to the table.
-
-\item[2:] \hspace{2em} {\tt FOP-PUSH} \hspace{2em} {\it index}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Item number {\it index} of the table is pushed onto the stack.
-The first element of the table is item number zero.
-
-\item[3:] \hspace{2em} {\tt FOP-BYTE-PUSH} \hspace{2em} {\it index}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Item number {\it index} of the table is pushed onto the stack.
-The first element of the table is item number zero.
-
-\item[4:] \hspace{2em} {\tt FOP-EMPTY-LIST} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The empty list ({\tt ()}) is pushed onto the stack.
-
-\item[5:] \hspace{2em} {\tt FOP-TRUTH} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The standard truth value ({\tt T}) is pushed onto the stack.
-
-\item[6:] \hspace{2em} {\tt FOP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-The four-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the default package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[7:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-The one-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the default package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[8:] \hspace{2em} {\tt FOP-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
-\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-The four-byte {\it index} specifies a package stored in the table.
-The four-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[9:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
-\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \& table\\
-The four-byte {\it index} specifies a package stored in the table.
-The one-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[10:] \hspace{2em} {\tt FOP-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
-\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-The one-byte {\it index} specifies a package stored in the table.
-The four-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[11:]\hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
-\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \& table\\
-The one-byte {\it index} specifies a package stored in the table.
-The one-byte operand {\it n} specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-\item[12:] \hspace{2em} {\tt FOP-UNINTERNED-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-Like {\tt FOP-SYMBOL-SAVE}, except that it creates an uninterned symbol.
-
-\item[13:] \hspace{2em} {\tt FOP-UNINTERNED-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
-\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
-\& table\\
-Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates an uninterned symbol.
-
-\item[14:] \hspace{2em} {\tt FOP-PACKAGE} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
-An item is popped from the stack; it must be a symbol. The package of
-that name is located and pushed onto the table.
-
-\item[15:] \hspace{2em} {\tt FOP-LIST} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The unsigned operand {\it length} specifies a number of
-operands to be popped from the stack. These are made into a list
-of that length, and the list is pushed onto the stack.
-The first item popped from the stack becomes the last element of
-the list, and so on. Hence an iterative loop can start with
-the empty list and perform "pop an item and cons it onto the list"
-{\it length} times.
-(Lists of length greater than 255 can be made by using {\tt FOP-LIST*}
-repeatedly.)
-
-\item[16:] \hspace{2em} {\tt FOP-LIST*} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-This is like {\tt FOP-LIST} except that the constructed list is terminated
-not by {\tt ()} (the empty list), but by an item popped from the stack
-before any others are. Therefore {\it length}+1 items are popped in all.
-Hence an iterative loop can start with
-a popped item and perform "pop an item and cons it onto the list"
-{\it length}+1 times.
-
-\item[17-24:] \hspace{2em} {\tt FOP-LIST-1}, {\tt FOP-LIST-2}, ..., {\tt FOP-LIST-8} \\
-{\tt FOP-LIST-{\it k}} is like {\tt FOP-LIST} with a byte containing {\it k}
-following it. These exist purely to reduce the size of Fasload files.
-Measurements need to be made to determine the useful values of {\it k}.
-
-\item[25-32:] \hspace{2em} {\tt FOP-LIST*-1}, {\tt FOP-LIST*-2}, ..., {\tt FOP-LIST*-8} \\
-{\tt FOP-LIST*-{\it k}} is like {\tt FOP-LIST*} with a byte containing {\it k}
-following it. These exist purely to reduce the size of Fasload files.
-Measurements need to be made to determine the useful values of {\it k}.
-
-\item[33:] \hspace{2em} {\tt FOP-INTEGER} \hspace{2em} {\it n}(4) \hspace{2em} {\it value}({\it n}) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \\
-A four-byte unsigned operand specifies the number of following
-bytes. These bytes define the value of a signed integer in two's-complement
-form. The first byte of the value is the least significant byte.
-
-\item[34:] \hspace{2em} {\tt FOP-SMALL-INTEGER} \hspace{2em} {\it n}(1) \hspace{2em} {\it value}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A one-byte unsigned operand specifies the number of following
-bytes. These bytes define the value of a signed integer in two's-complement
-form. The first byte of the value is the least significant byte.
-
-\item[35:] \hspace{2em} {\tt FOP-WORD-INTEGER} \hspace{2em} {\it value}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A four-byte signed integer (in the range $-2^{31}$ to $2^{31}-1$) follows the
-operation code. A LISP integer (fixnum or bignum) with that value
-is constructed and pushed onto the stack.
-
-\item[36:] \hspace{2em} {\tt FOP-BYTE-INTEGER} \hspace{2em} {\it value}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A one-byte signed integer (in the range -128 to 127) follows the
-operation code. A LISP integer (fixnum or bignum) with that value
-is constructed and pushed onto the stack.
-
-\item[37:] \hspace{2em} {\tt FOP-STRING} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length of a string to
-construct. The characters of the string follow, one per byte.
-The constructed string is pushed onto the stack.
-
-\item[38:] \hspace{2em} {\tt FOP-SMALL-STRING} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The one-byte operand {\it n} specifies the length of a string to
-construct. The characters of the string follow, one per byte.
-The constructed string is pushed onto the stack.
-
-\item[39:] \hspace{2em} {\tt FOP-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length of a vector of LISP objects
-to construct. The elements of the vector are popped off the stack;
-the first one popped becomes the last element of the vector.
-The constructed vector is pushed onto the stack.
-
-\item[40:] \hspace{2em} {\tt FOP-SMALL-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The one-byte operand {\it n} specifies the length of a vector of LISP objects
-to construct. The elements of the vector are popped off the stack;
-the first one popped becomes the last element of the vector.
-The constructed vector is pushed onto the stack.
-
-\item[41:] \hspace{2em} {\tt FOP-UNIFORM-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length of a vector of LISP objects
-to construct. A single item is popped from the stack and used to initialize
-all elements of the vector. The constructed vector is pushed onto the stack.
-
-\item[42:] \hspace{2em} {\tt FOP-SMALL-UNIFORM-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The one-byte operand {\it n} specifies the length of a vector of LISP objects
-to construct. A single item is popped from the stack and used to initialize
-all elements of the vector. The constructed vector is pushed onto the stack.
-
-\item[43:] \hspace{2em} {\tt FOP-INT-VECTOR} \hspace{2em} {\it len}(4) \hspace{2em}
-{\it size}(1) \hspace{2em} {\it data}($\left\lceil len*count/8\right\rceil$)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length of a vector of
-unsigned integers to be constructed. Each integer is {\it size}
-bits long, and is packed according to the machine's native byte ordering.
-{\it size} must be a directly supported i-vector element size. Currently
-supported values are 1,2,4,8,16 and 32.
-
-\item[44:] \hspace{2em} {\tt FOP-UNIFORM-INT-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} {\it size}(1) \hspace{2em}
-{\it value}(@ceiling<{\it size}/8>) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length of a vector of unsigned
-integers to construct.
-Each integer is {\it size} bits big, and is initialized to the value
-of the operand {\it value}.
-The constructed vector is pushed onto the stack.
-
-\item[45:] Unused
-
-\item[46:] \hspace{2em} {\tt FOP-SINGLE-FLOAT} \hspace{2em} {\it data}(4) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \\
-The {\it data} bytes are read as an integer, then turned into an IEEE single
-float (as though by {\tt make-single-float}).
-
-\item[47:] \hspace{2em} {\tt FOP-DOUBLE-FLOAT} \hspace{2em} {\it data}(8) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \\
-The {\it data} bytes are read as an integer, then turned into an IEEE double
-float (as though by {\tt make-double-float}).
-
-\item[48:] \hspace{2em} {\tt FOP-STRUCT} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The four-byte operand {\it n} specifies the length structure to construct. The
-elements of the vector are popped off the stack; the first one popped becomes
-the last element of the structure. The constructed vector is pushed onto the
-stack.
-
-\item[49:] \hspace{2em} {\tt FOP-SMALL-STRUCT} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The one-byte operand {\it n} specifies the length structure to construct. The
-elements of the vector are popped off the stack; the first one popped becomes
-the last element of the structure. The constructed vector is pushed onto the
-stack.
-
-\item[50-52:] Unused
-
-\item[53:] \hspace{2em} {\tt FOP-EVAL} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
-Push the result back onto the stack.
-
-\item[54:] \hspace{2em} {\tt FOP-EVAL-FOR-EFFECT} \\
-Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
-The result is ignored.
-
-\item[55:] \hspace{2em} {\tt FOP-FUNCALL} \hspace{2em} {\it nargs}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Pop {\it nargs}+1 items from the stack and apply the last one popped
-as a function to
-all the rest as arguments (the first one popped being the last argument).
-Push the result back onto the stack.
-
-\item[56:] \hspace{2em} {\tt FOP-FUNCALL-FOR-EFFECT} \hspace{2em} {\it nargs}(1) \\
-Pop {\it nargs}+1 items from the stack and apply the last one popped
-as a function to
-all the rest as arguments (the first one popped being the last argument).
-The result is ignored.
-
-\item[57:] \hspace{2em} {\tt FOP-CODE-FORMAT} \hspace{2em} {\it implementation}(1)
-\hspace{2em} {\it version}(1) \\
-This FOP specifiers the code format for following code objects. The operations
-{\tt FOP-CODE} and its relatives may not occur in a group until after {\tt
-FOP-CODE-FORMAT} has appeared; there is no default format. The {\it
-implementation} is an integer indicating the target hardware and environment.
-See {\tt compiler/generic/vm-macs.lisp} for the currently defined
-implementations. {\it version} for an implementation is increased whenever
-there is a change that renders old fasl files unusable.
-
-\item[58:] \hspace{2em} {\tt FOP-CODE} \hspace{2em} {\it nitems}(4) \hspace{2em} {\it size}(4) \hspace{2em}
-{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A compiled function is constructed and pushed onto the stack.
-This object is in the format specified by the most recent
-occurrence of {\tt FOP-CODE-FORMAT}.
-The operand {\it nitems} specifies a number of items to pop off
-the stack to use in the "boxed storage" section. The operand {\it code}
-is a string of bytes constituting the compiled executable code.
-
-\item[59:] \hspace{2em} {\tt FOP-SMALL-CODE} \hspace{2em} {\it nitems}(1) \hspace{2em} {\it size}(2) \hspace{2em}
-{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A compiled function is constructed and pushed onto the stack.
-This object is in the format specified by the most recent
-occurrence of {\tt FOP-CODE-FORMAT}.
-The operand {\it nitems} specifies a number of items to pop off
-the stack to use in the "boxed storage" section. The operand {\it code}
-is a string of bytes constituting the compiled executable code.
-
-\item[60-61:] Unused
-
-\item[62:] \hspace{2em} {\tt FOP-VERIFY-TABLE-SIZE} \hspace{2em} {\it size}(4) \\
-If the current size of the table is not equal to {\it size},
-then an inconsistency has been detected. This operation
-is inserted into a Fasload file purely for error-checking purposes.
-It is good practice for a compiler to output this at least at the
-end of every group, if not more often.
-
-\item[63:] \hspace{2em} {\tt FOP-VERIFY-EMPTY-STACK} \\
-If the stack is not currently empty,
-then an inconsistency has been detected. This operation
-is inserted into a Fasload file purely for error-checking purposes.
-It is good practice for a compiler to output this at least at the
-end of every group, if not more often.
-
-\item[64:] \hspace{2em} {\tt FOP-END-GROUP} \\
-This is the last operation of a group. If this is not the
-last byte of the file, then a new group follows; the next
-nine bytes must be "{\tt FASL FILE}".
-
-\item[65:] \hspace{2em} {\tt FOP-POP-FOR-EFFECT} \hspace{2em} stack \hspace{2em} $\Rightarrow$ \hspace{2em} \\
-One item is popped from the stack.
-
-\item[66:] \hspace{2em} {\tt FOP-MISC-TRAP} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-A trap object is pushed onto the stack.
-
-\item[67:] Unused
-
-\item[68:] \hspace{2em} {\tt FOP-CHARACTER} \hspace{2em} {\it character}(3) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-The three bytes are read as an integer then converted to a character. This FOP
-is currently rather useless, as extended characters are not supported.
-
-\item[69:] \hspace{2em} {\tt FOP-SHORT-CHARACTER} \hspace{2em} {\it character}(1) \hspace{2em}
-$\Rightarrow$ \hspace{2em} stack \\
-The one byte specifies the code of a Common Lisp character object. A character
-is constructed and pushed onto the stack.
-
-\item[70:] \hspace{2em} {\tt FOP-RATIO} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Creates a ratio from two integers popped from the stack.
-The denominator is popped first, the numerator second.
-
-\item[71:] \hspace{2em} {\tt FOP-COMPLEX} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
-Creates a complex number from two numbers popped from the stack.
-The imaginary part is popped first, the real part second.
-
-\item[72-73:] Unused
-
-\item[74:] \hspace{2em} {\tt FOP-FSET} \hspace{2em} \\
-Except in the cold loader (Genesis), this is a no-op with two stack arguments.
-In the initial core this is used to make DEFUN functions defined at cold-load
-time so that global functions can be called before top-level forms are run
-(which normally installs definitions.) Genesis pops the top two things off of
-the stack and effectively does (SETF SYMBOL-FUNCTION).
-
-\item[75:] \hspace{2em} {\tt FOP-LISP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the LISP
-package.
-
-\item[76:] \hspace{2em} {\tt FOP-LISP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
-\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
-\& table\\
-Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the LISP
-package.
-
-\item[77:] \hspace{2em} {\tt FOP-KEYWORD-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
-Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the
-KEYWORD package.
-
-\item[78:] \hspace{2em} {\tt FOP-KEYWORD-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
-\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
-\& table\\
-Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the
-KEYWORD package.
-
-\item[79-80:] Unused
-
-\item[81:] \hspace{2em} {\tt FOP-NORMAL-LOAD}\\
-This FOP is used in conjunction with the cold loader (Genesis) to read
-top-level package manipulation forms. These forms are to be read as though by
-the normal loaded, so that they can be evaluated at cold load time, instead of
-being dumped into the initial core image. A no-op in normal loading.
-
-\item[82:] \hspace{2em} {\tt FOP-MAYBE-COLD-LOAD}\\
-Undoes the effect of {\tt FOP-NORMAL-LOAD}.
-
-\item[83:] \hspace{2em} {\tt FOP-ARRAY} \hspace{2em} {\it rank}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-This operation creates a simple array header (used for simple-arrays with rank
-/= 1). The data vector is popped off of the stack, and then {\it rank}
-dimensions are popped off of the stack (the highest dimensions is on top.)
-
-\item[84-139:] Unused
-
-\item[140:] \hspace{2em} {\tt FOP-ALTER-CODE} \hspace{2em} {\it index}(4)\\
-This operation modifies the constants part of a code object (necessary for
-creating certain circular function references.) It pops the new value and code
-object are off of the stack, storing the new value at the specified index.
-
-\item[141:] \hspace{2em} {\tt FOP-BYTE-ALTER-CODE} \hspace{2em} {\it index}(1)\\
-Like {\tt FOP-ALTER-CODE}, but has only a one byte offset.
-
-\item[142:] \hspace{2em} {\tt FOP-FUNCTION-ENTRY} \hspace{2em} {\it index}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-Initializes a function-entry header inside of a pre-existing code object, and
-returns the corresponding function descriptor. {\it index} is the byte offset
-inside of the code object where the header should be plunked down. The stack
-arguments to this operation are the code object, function name, function debug
-arglist and function type.
-
-\item[143:] Unused
-
-\item[144:] \hspace{2em} {\tt FOP-ASSEMBLER-CODE} \hspace{2em} {\it length}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-This operation creates a code object holding assembly routines. {\it length}
-bytes of code are read and placed in the code object, and the code object
-descriptor is pushed on the stack. This FOP is only recognized by the cold
-loader (Genesis.)
-
-\item[145:] \hspace{2em} {\tt FOP-ASSEMBLER-ROUTINE} \hspace{2em} {\it offset}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-This operation records an entry point into an assembler code object (for use
-with {\tt FOP-ASSEMBLER-FIXUP}). The routine name (a symbol) is on stack top.
-The code object is underneath. The entry point is defined at {\it offset}
-bytes inside the code area of the code object, and the code object is left on
-stack top (allowing multiple uses of this FOP to be chained.) This FOP is only
-recognized by the cold loader (Genesis.)
-
-\item[146:] Unused
-
-\item[147:] \hspace{2em} {\tt FOP-FOREIGN-FIXUP} \hspace{2em} {\it len}(1)
-\hspace{2em} {\it name}({\it len})
-\hspace{2em} {\it offset}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-This operation resolves a reference to a foreign (C) symbol. {\it len} bytes
-are read and interpreted as the symbol {\it name}. First the {\it kind} and the
-code-object to patch are popped from the stack. The kind is a target-dependent
-symbol indicating the instruction format of the patch target (at {\it offset}
-bytes from the start of the code area.) The code object is left on
-stack top (allowing multiple uses of this FOP to be chained.)
-
-\item[148:] \hspace{2em} {\tt FOP-ASSEMBLER-FIXUP} \hspace{2em} {\it offset}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-This operation resolves a reference to an assembler routine. The stack args
-are ({\it routine-name}, {\it kind} and {\it code-object}). The kind is a
-target-dependent symbol indicating the instruction format of the patch target
-(at {\it offset} bytes from the start of the code area.) The code object is
-left on stack top (allowing multiple uses of this FOP to be chained.)
-
-\item[149-199:] Unused
-
-\item[200:] \hspace{2em} {\tt FOP-RPLACA} \hspace{2em} {\it table-idx}(4)
-\hspace{2em} {\it cdr-offset}(4)\\
-
-\item[201:] \hspace{2em} {\tt FOP-RPLACD} \hspace{2em} {\it table-idx}(4)
-\hspace{2em} {\it cdr-offset}(4)\\
-These operations destructively modify a list entered in the table. {\it
-table-idx} is the table entry holding the list, and {\it cdr-offset} designates
-the cons in the list to modify (like the argument to {\tt nthcdr}.) The new
-value is popped off of the stack, and stored in the {\tt car} or {\tt cdr},
-respectively.
-
-\item[202:] \hspace{2em} {\tt FOP-SVSET} \hspace{2em} {\it table-idx}(4)
-\hspace{2em} {\it vector-idx}(4)\\
-Destructively modifies a {\tt simple-vector} entered in the table. Pops the
-new value off of the stack, and stores it in the {\it vector-idx} element of
-the contents of the table entry {\it table-idx.}
-
-\item[203:] \hspace{2em} {\tt FOP-NTHCDR} \hspace{2em} {\it cdr-offset}(4)
-\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
-Does {\tt nthcdr} on the top-of stack, leaving the result there.
-
-\item[204:] \hspace{2em} {\tt FOP-STRUCTSET} \hspace{2em} {\it table-idx}(4)
-\hspace{2em} {\it vector-idx}(4)\\
-Like {\tt FOP-SVSET}, except it alters structure slots.
-
-\item[255:] \hspace{2em} {\tt FOP-END-HEADER} \\ Indicates the end of a group header,
-as described above.
-\end{description}
+++ /dev/null
-\chapter{ICR conversion} % -*- Dictionary: design -*-
-
-
-\f
-\section{Canonical forms}
-
-\#|
-
-Would be useful to have a Freeze-Type proclamation. Its primary use would to
-be say that the indicated type won't acquire any new subtypes in the future.
-This allows better open-coding of structure type predicates, since the possible
-types that would satisfy the predicate will be constant at compile time, and
-thus can be compiled as a skip-chain of EQ tests.
-
-Of course, this is only a big win when the subtypes are few: the most important
-case is when there are none. If the closure of the subtypes is much larger
-than the average number of supertypes of an inferior, then it is better to grab
-the list of superiors out of the object's type, and test for membership in that
-list.
-
-Should type-specific numeric equality be done by EQL rather than =? i.e.
-should = on two fixnums become EQL and then convert to EQL/FIXNUM?
-Currently we transform EQL into =, which is complicated, since we have to prove
-the operands are the class of numeric type before we do it. Also, when EQL
-sees one operand is a FIXNUM, it transforms to EQ, but the generator for EQ
-isn't expecting numbers, so it doesn't use an immediate compare.
-
-
-Array hackery:
-
-
-Array type tests are transformed to %array-typep, separation of the
-implementation-dependent array-type handling. This way we can transform
-STRINGP to:
- (or (simple-string-p x)
- (and (complex-array-p x)
- (= (array-rank x) 1)
- (simple-string-p (%array-data x))))
-
-In addition to the similar bit-vector-p, we also handle vectorp and any type
-tests on which the dimension isn't wild.
-[Note that we will want to expand into frobs compatible with those that
-array references expand into so that the same optimizations will work on both.]
-
-These changes combine to convert hairy type checks into hairy typep's, and then
-convert hairyp typeps into simple typeps.
-
-
-Do we really need non-VOP templates? It seems that we could get the desired
-effect through implementation-dependent ICR transforms. The main risk would be
-of obscuring the type semantics of the code. We could fairly easily retain all
-the type information present at the time the tranform is run, but if we
-discover new type information, then it won't be propagated unless the VM also
-supplies type inference methods for its internal frobs (precluding the use of
-%PRIMITIVE, since primitives don't have derive-type methods.)
-
-I guess one possibility would be to have the call still considered "known" even
-though it has been transformed. But this doesn't work, since we start doing
-LET optimizations that trash the arglist once the call has been transformed
-(and indeed we want to.)
-
-Actually, I guess the overhead for providing type inference methods for the
-internal frobs isn't that great, since we can usually borrow the inference
-method for a Common Lisp function. For example, in our AREF case:
- (aref x y)
-==>
- (let ((\#:len (array-dimension x 0)))
- (%unchecked-aref x (%check-in-bounds y \#:len)))
-
-Now in this case, if we made %UNCHECKED-AREF have the same derive-type method
-as AREF, then if we discovered something new about X's element type, we could
-derive a new type for the entire expression.
-
-Actually, it seems that baring this detail at the ICR level is beneficial,
-since it admits the possibly of optimizing away the bounds check using type
-information. If we discover X's dimensions, then \#:LEN becomes a constant that
-can be substituted. Then %CHECK-IN-BOUNDS can notice that the bound is
-constant and check it against the type for Y. If Y is known to be in range,
-then we can optimize away the bounds check.
-
-Actually in this particular case, the best thing to do would be if we
-discovered the bound is constant, then replace the bounds check with an
-implicit type check. This way all the type check optimization mechanisms would
-be brought into the act.
-
-So we actually want to do the bounds-check expansion as soon as possible,
-rather than later than possible: it should be a source-transform, enabled by
-the fast-safe policy.
-
-With multi-dimensional arrays we probably want to explicitly do the index
-computation: this way portions of the index computation can become loop
-invariants. In a scan in row-major order, the inner loop wouldn't have to do
-any multiplication: it would only do an addition. We would use normal
-fixnum arithmetic, counting on * to cleverly handle multiplication by a
-constant, and appropriate inline expansion.
-
-Note that in a source transform, we can't make any assumptions the type of the
-array. If it turns out to be a complex array without declared dimensions, then
-the calls to ARRAY-DIMENSION will have to turn into a VOP that can be affected.
-But if it is simple, then the VOP is unaffected, and if we know the bounds, it
-is constant. Similarly, we would have %ARRAY-DATA and %ARRAY-DISPLACEMENT
-operations. %ARRAY-DISPLACEMENT would optimize to 0 if we discover the array
-is simple. [This is somewhat inefficient when the array isn't eventually
-discovered to be simple, since finding the data and finding the displacement
-duplicate each other. We could make %ARRAY-DATA return both as MVs, and then
-optimize to (VALUES (%SIMPLE-ARRAY-DATA x) 0), but this would require
-optimization of trivial VALUES uses.]
-
-Also need (THE (ARRAY * * * ...) x) to assert correct rank.
-
-|\#
-
-A bunch of functions have source transforms that convert them into the
-canonical form that later parts of the compiler want to see. It is not legal
-to rely on the canonical form since source transforms can be inhibited by a
-Notinline declaration. This shouldn't be a problem, since everyone should keep
-their hands off of Notinline calls.
-
-Some transformations:
-
-Endp ==> (NULL (THE LIST ...))
-(NOT xxx) or (NULL xxx) => (IF xxx NIL T)
-
-(typep x '<simple type>) => (<simple predicate> x)
-(typep x '<complex type>) => ...composition of simpler operations...
-TYPEP of AND, OR and NOT types turned into conditionals over multiple TYPEP
-calls. This makes hairy TYPEP calls more digestible to type constraint
-propagation, and also means that the TYPEP code generators don't have to deal
-with these cases. [\#\#\# In the case of union types we may want to do something
-to preserve information for type constraint propagation.]
-
-
- (apply \#'foo a b c)
-==>
- (multiple-value-call \#'foo (values a) (values b) (values-list c))
-
-This way only MV-CALL needs to know how to do calls with unknown numbers of
-arguments. It should be nearly as efficient as a special-case VMR-Convert
-method could be.
-
-
-Make-String => Make-Array
-N-arg predicates associated into two-arg versions.
-Associate N-arg arithmetic ops.
-Expand CxxxR and FIRST...nTH
-Zerop, Plusp, Minusp, 1+, 1-, Min, Max, Rem, Mod
-(Values x), (Identity x) => (Prog1 x)
-
-All specialized aref functions => (aref (the xxx) ...)
-
-Convert (ldb (byte ...) ...) into internal frob that takes size and position as
-separate args. Other byte functions also...
-
-Change for-value primitive predicates into (if <pred> t nil). This isn't
-particularly useful during ICR phases, but makes life easy for VMR conversion.
-
-This last can't be a source transformation, since a source transform can't tell
-where the form appears. Instead, ICR conversion special-cases calls to known
-functions with the Predicate attribute by doing the conversion when the
-destination of the result isn't an IF. It isn't critical that this never be
-done for predicates that we ultimately discover to deliver their value to an
-IF, since IF optimizations will flush unnecessary IFs in a predicate.
-
-\f
-\section{Inline functions}
-
-[\#\#\# Inline expansion is especially powerful in the presence of good lisp-level
-optimization ("partial evaluation"). Many "optimizations" usually done in Lisp
-compilers by special-case source-to-source transforms can be had simply by
-making the source of the general case function available for inline expansion.
-This is especially helpful in Common Lisp, which has many commonly used
-functions with simple special cases but bad general cases (list and sequence
-functions, for example.)
-
-Inline expansion of recursive functions is allowed, and is not as silly as it
-sounds. When expanded in a specific context, much of the overhead of the
-recursive calls may be eliminated (especially if there are many keyword
-arguments, etc.)
-
-[Also have MAYBE-INLINE]
-]
-
-We only record a function's inline expansion in the global environment when the
-function is in the null lexical environment, since it the expansion must be
-represented as source.
-
-We do inline expansion of functions locally defined by FLET or LABELS even when
-the environment is not null. Since the appearances of the local function must
-be nested within the desired environment, it is possible to expand local
-functions inline even when they use the environment. We just stash the source
-form and environments in the Functional for the local function. When we
-convert a call to it, we just reconvert the source in the saved environment.
-
-An interesting alternative to the inline/full-call dichotomy is "semi-inline"
-coding. Whenever we have an inline expansion for a function, we can expand it
-only once per block compilation, and then use local call to call this copied
-version. This should get most of the speed advantage of real inline coding
-with much less code bloat. This is especially attractive for simple system
-functions such as Read-Char.
-
-The main place where true inline expansion would still be worth doing is where
-large amounts of the function could be optimized away by constant folding or
-other optimizations that depend on the exact arguments to the call.
-
-
-\f
-\section{Compilation policy}
-
-We want more sophisticated control of compilation safety than is offered in CL,
-so that we can emit only those type checks that are likely to discover
-something (i.e. external interfaces.)
-
-\#|
-
-\f
-\section{Notes}
-
-Generalized back-end notion provides dynamic retargeting? (for byte code)
-
-The current node type annotations seem to be somewhat unsatisfactory, since we
-lose information when we do a THE on a continuation that already has uses, or
-when we convert a let where the actual result continuation has other uses.
-
-But the case with THE isn't really all that bad, since the test of whether
-there are any uses happens before conversion of the argument, thus THE loses
-information only when there are uses outside of the declared form. The LET
-case may not be a big deal either.
-
-Note also that losing user assertions isn't really all that bad, since it won't
-damage system integrity. At worst, it will cause a bug to go undetected. More
-likely, it will just cause the error to be signaled in a different place (and
-possibly in a less informative way). Of course, there is an efficiency hit for
-losing type information, but if it only happens in strange cases, then this
-isn't a big deal.
-
-\f
-\chapter{Local call analysis}
-
-All calls to local functions (known named functions and LETs) are resolved to
-the exact LAMBDA node which is to be called. If the call is syntactically
-illegal, then we emit a warning and mark the reference as :notinline, forcing
-the call to be a full call. We don't even think about converting APPLY calls;
-APPLY is not special-cased at all in ICR. We also take care not to convert
-calls in the top-level component, which would join it to normal code. Calls to
-functions with rest args and calls with non-constant keywords are also not
-converted.
-
-We also convert MV-Calls that look like MULTIPLE-VALUE-BIND to local calls,
-since we know that they can be open-coded. We replace the optional dispatch
-with a call to the last optional entry point, letting MV-Call magically default
-the unsupplied values to NIL.
-
-When ICR optimizations discover a possible new local call, they explicitly
-invoke local call analysis on the code that needs to be reanalyzed.
-
-[\#\#\# Let conversion. What is means to be a let. Argument type checking done
-by caller. Significance of local call is that all callers are known, so
-special call conventions may be used.]
-A lambda called in only one place is called a "let" call, since a Let would
-turn into one.
-
-In addition to enabling various ICR optimizations, the let/non-let distinction
-has important environment significance. We treat the code in function and all
-of the lets called by that function as being in the same environment. This
-allows exits from lets to be treated as local exits, and makes life easy for
-environment analysis.
-
-Since we will let-convert any function with only one call, we must be careful
-about cleanups. It is possible that a lexical exit from the let function may
-have to clean up dynamic bindings not lexically apparent at the exit point. We
-handle this by annotating lets with any cleanup in effect at the call site.
-The cleanup for continuations with no immediately enclosing cleanup is the
-lambda that the continuation is in. In this case, we look at the lambda to see
-if any cleanups need to be done.
-
-Let conversion is disabled for entry-point functions, since otherwise we might
-convert the call from the XEP to the entry point into a let. Then later on, we
-might want to convert a non-local reference into a local call, and not be able
-to, since once a function has been converted to a let, we can't convert it
-back.
-
-
-A function's return node may also be deleted if it is unreachable, which can
-happen if the function never returns normally. Such functions are not lets.
-
-\f
-\chapter{Find components}
-
-This is a post-pass to ICR conversion that massages the flow graph into the
-shape subsequent phases expect. Things done:
- Compute the depth-first ordering for the flow graph.
- Find the components (disconnected parts) of the flow graph.
-
-This pass need only be redone when newly converted code has been added to the
-flow graph. The reanalyze flag in the component structure should be set by
-people who mess things up.
-
-We create the initial DFO using a variant of the basic algorithm. The initial
-DFO computation breaks the ICR up into components, which are parts that can be
-compiled independently. This is done to increase the efficiency of large block
-compilations. In addition to improving locality of reference and reducing the
-size of flow analysis problems, this allows back-end data structures to be
-reclaimed after the compilation of each component.
-
-ICR optimization can change the connectivity of the flow graph by discovering
-new calls or eliminating dead code. Initial DFO determination splits up the
-flow graph into separate components, but does so conservatively, ensuring that
-parts that might become joined (due to local call conversion) are joined from
-the start. Initial DFO computation also guarantees that all code which shares
-a lexical environment is in the same component so that environment analysis
-needs to operate only on a single component at a time.
-
-[This can get a bit hairy, since code seemingly reachable from the
-environment entry may be reachable from a NLX into that environment. Also,
-function references must be considered as links joining components even though
-the flow graph doesn't represent these.]
-
-After initial DFO determination, components are neither split nor joined. The
-standard DFO computation doesn't attempt to split components that have been
-disconnected.
-
-\f
-\chapter{ICR optimize}
-
-{\bf Somewhere describe basic ICR utilities: continuation-type,
-constant-continuation-p, etc. Perhaps group by type in ICR description?}
-
-We are conservative about doing variable-for-variable substitution in ICR
-optimization, since if we substitute a variable with a less restrictive type,
-then we may prevent use of a "good" representation within the scope of the
-inner binding.
-
-Note that variable-variable substitutions aren't really crucial in ICR, since
-they don't create opportunities for new optimizations (unlike substitution of
-constants and functions). A spurious variable-variable binding will show up as
-a Move operation in VMR. This can be optimized away by reaching-definitions
-and also by targeting. [\#\#\# But actually, some optimizers do see if operands
-are the same variable.]
-
-\#|
-
-The IF-IF optimization can be modeled as a value driven optimization, since
-adding a use definitely is cause for marking the continuation for
-reoptimization. [When do we add uses? Let conversion is the only obvious
-time.] I guess IF-IF conversion could also be triggered by a non-immediate use
-of the test continuation becoming immediate, but to allow this to happen would
-require Delete-Block (or somebody) to mark block-starts as needing to be
-reoptimized when a predecessor changes. It's not clear how important it is
-that IF-IF conversion happen under all possible circumstances, as long as it
-happens to the obvious cases.
-
-[\#\#\# It isn't totally true that code flushing never enables other worthwhile
-optimizations. Deleting a functional reference can cause a function to cease
-being an XEP, or even trigger let conversion. It seems we still want to flush
-code during ICR optimize, but maybe we want to interleave it more intimately
-with the optimization pass.
-
-Ref-flushing works just as well forward as backward, so it could be done in the
-forward pass. Call flushing doesn't work so well, but we could scan the block
-backward looking for any new flushable stuff if we flushed a call on the
-forward pass.
-
-When we delete a variable due to lack of references, we leave the variable
-in the lambda-list so that positional references still work. The initial value
-continuation is flushed, though (replaced with NIL) allowing the initial value
-for to be deleted (modulo side-effects.)
-
-Note that we can delete vars with no refs even when they have sets. I guess
-when there are no refs, we should also flush all sets, allowing the value
-expressions to be flushed as well.
-
-Squeeze out single-reference unset let variables by changing the dest of the
-initial value continuation to be the node that receives the ref. This can be
-done regardless of what the initial value form is, since we aren't actually
-moving the evaluation. Instead, we are in effect using the continuation's
-locations in place of the temporary variable.
-
-Doing this is of course, a wild violation of stack discipline, since the ref
-might be inside a loop, etc. But with the VMR back-end, we only need to
-preserve stack discipline for unknown-value continuations; this ICR
-transformation must be already be inhibited when the DEST of the REF is a
-multiple-values receiver (EXIT, RETURN or MV-COMBINATION), since we must
-preserve the single-value semantics of the let-binding in this case.
-
-The REF and variable must be deleted as part of this operation, since the ICR
-would otherwise be left in an inconsistent state; we can't wait for the REF to
-be deleted due to bing unused, since we have grabbed the arg continuation and
-substituted it into the old DEST.
-
-The big reason for doing this transformation is that in macros such as INCF and
-PSETQ, temporaries are squeezed out, and the new value expression is evaluated
-directly to the setter, allowing any result type assertion to be applied to the
-expression evaluation. Unlike in the case of substitution, there is no point
-in inhibiting this transformation when the initial value type is weaker than
-the variable type. Instead, we intersect the asserted type for the old REF's
-CONT with the type assertion on the initial value continuation. Note that the
-variable's type has already been asserted on the initial-value continuation.
-
-Of course, this transformation also simplifies the ICR even when it doesn't
-discover interesting type assertions, so it makes sense to do it whenever
-possible. This reduces the demands placed on register allocation, etc.
-
-|\#
-
-There are three dead-code flushing rules:
- 1] Refs with no DEST may be flushed.
- 2] Known calls with no dest that are flushable may be flushed. We null the
- DEST in all the args.
- 3] If a lambda-var has no refs, then it may be deleted. The flushed argument
- continuations have their DEST nulled.
-
-These optimizations all enable one another. We scan blocks backward, looking
-for nodes whose CONT has no DEST, then type-dispatching off of the node. If we
-delete a ref, then we check to see if it is a lambda-var with no refs. When we
-flush an argument, we mark the blocks for all uses of the CONT as needing to be
-reoptimized.
-
-\f
-\section{Goals for ICR optimizations}
-
-\#|
-
-When an optimization is disabled, code should still be correct and not
-ridiculously inefficient. Phases shouldn't be made mandatory when they have
-lots of non-required stuff jammed into them.
-
-|\#
-
-This pass is optional, but is desirable if anything is more important than
-compilation speed.
-
-This phase is a grab-bag of optimizations that concern themselves with the flow
-of values through the code representation. The main things done are type
-inference, constant folding and dead expression elimination. This phase can be
-understood as a walk of the expression tree that propagates assertions down the
-tree and propagates derived information up the tree. The main complication is
-that there isn't any expression tree, since ICR is flow-graph based.
-
-We repeat this pass until we don't discover anything new. This is a bit of
-feat, since we dispatch to arbitrary functions which may do arbitrary things,
-making it hard to tell if anything really happened. Even if we solve this
-problem by requiring people to flag when they changed or by checking to see if
-they changed something, there are serious efficiency problems due to massive
-redundant computation, since in many cases the only way to tell if anything
-changed is to recompute the value and see if it is different from the old one.
-
-We solve this problem by requiring that optimizations for a node only depend on
-the properties of the CONT and the continuations that have the node as their
-DEST. If the continuations haven't changed since the last pass, then we don't
-attempt to re-optimize the node, since we know nothing interesting will happen.
-
-We keep track of which continuations have changed by a REOPTIMIZE flag that is
-set whenever something about the continuation's value changes.
-
-When doing the bottom up pass, we dispatch to type specific code that knows how
-to tell when a node needs to be reoptimized and does the optimization. These
-node types are special-cased: COMBINATION, IF, RETURN, EXIT, SET.
-
-The REOPTIMIZE flag in the COMBINATION-FUN is used to detect when the function
-information might have changed, so that we know when where are new assertions
-that could be propagated from the function type to the arguments.
-
-When we discover something about a leaf, or substitute for leaf, we reoptimize
-the CONT for all the REF and SET nodes.
-
-We have flags in each block that indicate when any nodes or continuations in
-the block need to be re-optimized, so we don't have to scan blocks where there
-is no chance of anything happening.
-
-It is important for efficiency purposes that optimizers never say that they did
-something when they didn't, but this by itself doesn't guarantee timely
-termination. I believe that with the type system implemented, type inference
-will converge in finite time, but as a practical matter, it can take far too
-long to discover not much. For this reason, ICR optimization is terminated
-after three consecutive passes that don't add or delete code. This premature
-termination only happens 2% of the time.
-
-\f
-\section{Flow graph simplification}
-
-Things done:
- Delete blocks with no predecessors.
- Merge blocks that can be merged.
- Convert local calls to Let calls.
- Eliminate degenerate IFs.
-
-We take care not to merge blocks that are in different functions or have
-different cleanups. This guarantees that non-local exits are always at block
-ends and that cleanup code never needs to be inserted within a block.
-
-We eliminate IFs with identical consequent and alternative. This would most
-likely happen if both the consequent and alternative were optimized away.
-
-[Could also be done if the consequent and alternative were different blocks,
-but computed the same value. This could be done by a sort of cross-jumping
-optimization that looked at the predecessors for a block and merged code shared
-between predecessors. IFs with identical branches would eventually be left
-with nothing in their branches.]
-
-We eliminate IF-IF constructs:
- (IF (IF A B C) D E) ==>
- (IF A (IF B D E) (IF C D E))
-
-In reality, what we do is replicate blocks containing only an IF node where the
-predicate continuation is the block start. We make one copy of the IF node for
-each use, leaving the consequent and alternative the same. If you look at the
-flow graph representation, you will see that this is really the same thing as
-the above source to source transformation.
-
-\f
-\section{Forward ICR optimizations}
-
-In the forward pass, we scan the code in forward depth-first order. We
-examine each call to a known function, and:
-
-\begin{itemize}
-\item Eliminate any bindings for unused variables.
-
-\item Do top-down type assertion propagation. In local calls, we propagate
-asserted and derived types between the call and the called lambda.
-
-\item
- Replace calls of foldable functions with constant arguments with the
- result. We don't have to actually delete the call node, since Top-Down
- optimize will delete it now that its value is unused.
-
-\item
- Run any Optimizer for the current function. The optimizer does arbitrary
- transformations by hacking directly on the IR. This is useful primarily
- for arithmetic simplification and similar things that may need to examine
- and modify calls other than the current call. The optimizer is responsible
- for recording any changes that it makes. An optimizer can inhibit further
- optimization of the node during the current pass by returning true. This
- is useful when deleting the node.
-
-\item
- Do ICR transformations, replacing a global function call with equivalent
- inline lisp code.
-
-\item
- Do bottom-up type propagation/inferencing. For some functions such as
- Coerce we will dispatch to a function to find the result type. The
- Derive-Type function just returns a type structure, and we check if it is
- different from the old type in order to see if there was a change.
-
-\item
- Eliminate IFs with predicates known to be true or false.
-
-\item
- Substitute the value for unset let variables that are bound to constants,
- unset lambda variables or functionals.
-
-\item
- Propagate types from local call args to var refs.
-\end{itemize}
-
-We use type info from the function continuation to find result types for
-functions that don't have a derive-type method.
-
-
-ICR transformation:
-
-ICR transformation does "source to source" transformations on known global
-functions, taking advantage of semantic information such as argument types and
-constant arguments. Transformation is optional, but should be done if speed or
-space is more important than compilation speed. Transformations which increase
-space should pass when space is more important than speed.
-
-A transform is actually an inline function call where the function is computed
-at compile time. The transform gets to peek at the continuations for the
-arguments, and computes a function using the information gained. Transforms
-should be cautious about directly using the values of constant continuations,
-since the compiler must preserve eqlness of named constants, and it will have a
-hard time if transforms go around randomly copying constants.
-
-The lambda that the transform computes replaces the original function variable
-reference as the function for the call. This lets the compiler worry about
-evaluating each argument once in the right order. We want to be careful to
-preserve type information when we do a transform, since it may be less than
-obvious what the transformed code does.
-
-There can be any number of transforms for a function. Each transform is
-associated with a function type that the call must be compatible with. A
-transform is only invoked if the call has the right type. This provides a way
-to deal with the common case of a transform that only applies when the
-arguments are of certain types and some arguments are not specified. We always
-use the derived type when determining whether a transform is applicable. Type
-check is responsible for setting the derived type to the intersection of the
-asserted and derived types.
-
-If the code in the expansion has insufficient explicit or implicit argument
-type checking, then it should cause checks to be generated by making
-declarations.
-
-A transformation may decide to pass if it doesn't like what it sees when it
-looks at the args. The Give-Up function unwinds out of the transform and deals
-with complaining about inefficiency if speed is more important than brevity.
-The format args for the message are arguments to Give-Up. If a transform can't
-be done, we just record the message where ICR finalize can find it. note. We
-can't complain immediately, since it might get transformed later on.
-
-\f
-\section{Backward ICR optimizations}
-
-In the backward pass, we scan each block in reverse order, and
-eliminate any effectless nodes with unused values. In ICR this is the
-only way that code is deleted other than the elimination of unreachable blocks.
-
-\f
-\chapter{Type checking}
-
-[\#\#\# Somehow split this section up into three parts:
- -- Conceptual: how we know a check is necessary, and who is responsible for
- doing checks.
- -- Incremental: intersection of derived and asserted types, checking for
- non-subtype relationship.
- -- Check generation phase.
-]
-
-
-We need to do a pretty good job of guessing when a type check will ultimately
-need to be done. Generic arithmetic, for example: In the absence of
-declarations, we will use use the safe variant, but if we don't know this, we
-will generate a check for NUMBER anyway. We need to look at the fast-safe
-templates and guess if any of them could apply.
-
-We compute a function type from the VOP arguments
-and assertions on those arguments. This can be used with Valid-Function-Use
-to see which templates do or might apply to a particular call. If we guess
-that a safe implementation will be used, then we mark the continuation so as to
-force a safe implementation to be chosen. [This will happen if ICR optimize
-doesn't run to completion, so the icr optimization after type check generation
-can discover new type information. Since we won't redo type check at that
-point, there could be a call that has applicable unsafe templates, but isn't
-type checkable.]
-
-[\#\#\# A better and more general optimization of structure type checks: in type
-check conversion, we look at the *original derived* type of the continuation:
-if the difference between the proven type and the asserted type is a simple
-type check, then check for the negation of the difference. e.g. if we want a
-FOO and we know we've got (OR FOO NULL), then test for (NOT NULL). This is a
-very important optimization for linked lists of structures, but can also apply
-in other situations.]
-
-If after ICR phases, we have a continuation with check-type set in a context
-where it seems likely a check will be emitted, and the type is too
-hairy to be easily checked (i.e. no CHECK-xxx VOP), then we do a transformation
-on the ICR equivalent to:
- (... (the hair <foo>) ...)
-==>
- (... (funcall \#'(lambda (\#:val)
- (if (typep \#:val 'hair)
- \#:val
- (%type-check-error \#:val 'hair)))
- <foo>)
- ...)
-This way, we guarantee that VMR conversion never has to emit type checks for
-hairy types.
-
-[Actually, we need to do a MV-bind and several type checks when there is a MV
-continuation. And some values types are just too hairy to check. We really
-can't check any assertion for a non-fixed number of values, since there isn't
-any efficient way to bind arbitrary numbers of values. (could be done with
-MV-call of a more-arg function, I guess...)
-]
-
-[Perhaps only use CHECK-xxx VOPs for types equivalent to a ptype? Exceptions
-for CONS and SYMBOL? Anyway, no point in going to trouble to implement and
-emit rarely used CHECK-xxx vops.]
-
-One potential lose in converting a type check to explicit conditionals rather
-than to a CHECK-xxx VOP is that VMR code motion optimizations won't be able to
-do anything. This shouldn't be much of an issue, though, since type constraint
-propagation has already done global optimization of type checks.
-
-
-This phase is optional, but should be done if anything is more important than
-compile speed.
-
-Type check is responsible for reconciling the continuation asserted and derived
-types, emitting type checks if appropriate. If the derived type is a subtype
-of the asserted type, then we don't need to do anything.
-
-If there is no intersection between the asserted and derived types, then there
-is a manifest type error. We print a warning message, indicating that
-something is almost surely wrong. This will inhibit any transforms or
-generators that care about their argument types, yet also inhibits further
-error messages, since NIL is a subtype of every type.
-
-If the intersection is not null, then we set the derived type to the
-intersection of the asserted and derived types and set the Type-Check flag in
-the continuation. We always set the flag when we can't prove that the type
-assertion is satisfied, regardless of whether we will ultimately actually emit
-a type check or not. This is so other phases such as type constraint
-propagation can use the Type-Check flag to detect an interesting type
-assertion, instead of having to duplicate much of the work in this phase.
-[\#\#\# 7 extremely random values for CONTINUATION-TYPE-CHECK.]
-
-Type checks are generated on the fly during VMR conversion. When VMR
-conversion generates the check, it prints an efficiency note if speed is
-important. We don't flame now since type constraint progpagation may decide
-that the check is unnecessary. [\#\#\# Not done now, maybe never.]
-
-In local function call, it is the caller that is in effect responsible for
-checking argument types. This happens in the same way as any other type check,
-since ICR optimize propagates the declared argument types to the type
-assertions for the argument continuations in all the calls.
-
-Since the types of arguments to entry points are unknown at compile time, we
-want to do runtime checks to ensure that the incoming arguments are of the
-correct type. This happens without any special effort on the part of type
-check, since the XEP is represented as a local call with unknown type
-arguments. These arguments will be marked as needing to be checked.
-
-\f
-\chapter{Constraint propagation}
-
-\#|
-New lambda-var-slot:
-
-constraints: a list of all the constraints on this var for either X or Y.
-
-How to maintain consistency? Does it really matter if there are constraints
-with deleted vars lying around? Note that whatever mechanism we use for
-getting the constraints in the first place should tend to keep them up to date.
-Probably we would define optimizers for the interesting relations that look at
-their CONT's dest and annotate it if it is an IF.
-
-But maybe it is more trouble then it is worth trying to build up the set of
-constraints during ICR optimize (maintaining consistency in the process).
-Since ICR optimize iterates a bunch of times before it converges, we would be
-wasting time recomputing the constraints, when nobody uses them till constraint
-propagation runs.
-
-It seems that the only possible win is if we re-ran constraint propagation
-(which we might want to do.) In that case, we wouldn't have to recompute all
-the constraints from scratch. But it seems that we could do this just as well
-by having ICR optimize invalidate the affected parts of the constraint
-annotation, rather than trying to keep them up to date. This also fits better
-with the optional nature of constraint propagation, since we don't want ICR
-optimize to commit to doing a lot of the work of constraint propagation.
-
-For example, we might have a per-block flag indicating that something happened
-in that block since the last time constraint propagation ran. We might have
-different flags to represent the distinction between discovering a new type
-assertion inside the block and discovering something new about an if
-predicate, since the latter would be cheaper to update and probably is more
-common.
-
-It's fairly easy to see how we can build these sets of restrictions and
-propagate them using flow analysis, but actually using this information seems
-a bit more ad-hoc.
-
-Probably the biggest thing we do is look at all the refs. If have proven that
-the value is EQ (EQL for a number) to some other leaf (constant or lambda-var),
-then we can substitute for that reference. In some cases, we will want to do
-special stuff depending on the DEST. If the dest is an IF and we proved (not
-null), then we can substitute T. And if the dest is some relation on the same
-two lambda-vars, then we want to see if we can show that relation is definitely
-true or false.
-
-Otherwise, we can do our best to invert the set of restrictions into a type.
-Since types hold only constant info, we have to ignore any constraints between
-two vars. We can make some use of negated type restrictions by using
-TYPE-DIFFERENCE to remove the type from the ref types. If our inferred type is
-as good as the type assertion, then the continuation's type-check flag will be
-cleared.
-
-It really isn't much of a problem that we don't infer union types on joins,
-since union types are relatively easy to derive without using flow information.
-The normal bottom-up type inference done by ICR optimize does this for us: it
-annotates everything with the union of all of the things it might possibly be.
-Then constraint propagation subtracts out those types that can't be in effect
-because of predicates or checks.
-
-
-
-This phase is optional, but is desirable if anything is more important than
-compilation speed. We use an algorithm similar to available expressions to
-propagate variable type information that has been discovered by implicit or
-explicit type tests, or by type inference.
-
-We must do a pre-pass which locates set closure variables, since we cannot do
-flow analysis on such variables. We set a flag in each set closure variable so
-that we can quickly tell that it is losing when we see it again. Although this
-may seem to be wastefully redundant with environment analysis, the overlap
-isn't really that great, and the cost should be small compared to that of the
-flow analysis that we are preparing to do. [Or we could punt on set
-variables...]
-
-A type constraint is a structure that includes sset-element and has the type
-and variable.
-[\#\#\# Also a not-p flag indicating whether the sense is negated.]
- Each variable has a list of its type constraints. We create a
-type constraint when we see a type test or check. If there is already a
-constraint for the same variable and type, then we just re-use it. If there is
-already a weaker constraint, then we generate both the weak constraints and the
-strong constraint so that the weak constraints won't be lost even if the strong
-one is unavailable.
-
-We find all the distinct type constraints for each variable during the pre-pass
-over the lambda nesting. Each constraint has a list of the weaker constraints
-so that we can easily generate them.
-
-Every block generates all the type constraints in it, but a constraint is
-available in a successor only if it is available in all predecessors. We
-determine the actual type constraint for a variable at a block by intersecting
-all the available type constraints for that variable.
-
-This isn't maximally tense when there are constraints that are not
-hierarchically related, e.g. (or a b) (or b c). If these constraints were
-available from two predecessors, then we could infer that we have an (or a b c)
-constraint, but the above algorithm would come up with none. This probably
-isn't a big problem.
-
-[\#\#\# Do we want to deal with (if (eq <var> '<foo>) ...) indicating singleton
-member type?]
-
-We detect explicit type tests by looking at type test annotation in the IF
-node. If there is a type check, the OUT sets are stored in the node, with
-different sets for the consequent and alternative. Implicit type checks are
-located by finding Ref nodes whose Cont has the Type-Check flag set. We don't
-actually represent the GEN sets, we just initialize OUT to it, and then form
-the union in place.
-
-When we do the post-pass, we clear the Type-Check flags in the continuations
-for Refs when we discover that the available constraints satisfy the asserted
-type. Any explicit uses of typep should be cleaned up by the ICR optimizer for
-typep. We can also set the derived type for Refs to the intersection of the
-available type assertions. If we discover anything, we should consider redoing
-ICR optimization, since better type information might enable more
-optimizations.
-
-
-\chapter{ICR finalize} % -*- Dictionary: design -*-
-
-This pass looks for interesting things in the ICR so that we can forget about
-them. Used and not defined things are flamed about.
-
-We postpone these checks until now because the ICR optimizations may discover
-errors that are not initially obvious. We also emit efficiency notes about
-optimizations that we were unable to do. We can't emit the notes immediately,
-since we don't know for sure whether a repeated attempt at optimization will
-succeed.
-
-We examine all references to unknown global function variables and update the
-approximate type accordingly. We also record the names of the unknown
-functions so that they can be flamed about if they are never defined. Unknown
-normal variables are flamed about on the fly during ICR conversion, so we
-ignore them here.
-
-We check each newly defined global function for compatibility with previously
-recorded type information. If there is no :defined or :declared type, then we
-check for compatibility with any approximate function type inferred from
-previous uses.
-\f
-\chapter{Environment analysis}
-\#|
-
-A related change would be to annotate ICR with information about tail-recursion
-relations. What we would do is add a slot to the node structure that points to
-the corresponding Tail-Info when a node is in a TR position. This annotation
-would be made in a final ICR pass that runs after cleanup code is generated
-(part of environment analysis). When true, the node is in a true TR position
-(modulo return-convention incompatibility). When we determine return
-conventions, we null out the tail-p slots in XEP calls or known calls where we
-decided not to preserve tail-recursion.
-
-
-In this phase, we also check for changes in the dynamic binding environment
-that require cleanup code to be generated. We just check for changes in the
-Continuation-Cleanup on local control transfers. If it changes from
-an inner dynamic context to an outer one that is in the same environment, then
-we emit code to clean up the dynamic bindings between the old and new
-continuation. We represent the result of cleanup detection to the back end by
-interposing a new block containing a call to a funny function. Local exits
-from CATCH or UNWIND-PROTECT are detected in the same way.
-
-
-|\#
-
-The primary activity in environment analysis is the annotation of ICR with
-environment structures describing where variables are allocated and what values
-the environment closes over.
-
-Each lambda points to the environment where its variables are allocated, and
-the environments point back. We always allocate the environment at the Bind
-node for the sole non-let lambda in the environment, so there is a close
-relationship between environments and functions. Each "real function" (i.e.
-not a LET) has a corresponding environment.
-
-We attempt to share the same environment among as many lambdas as possible so
-that unnecessary environment manipulation is not done. During environment
-analysis the only optimization of this sort is realizing that a Let (a lambda
-with no Return node) cannot need its own environment, since there is no way
-that it can return and discover that its old values have been clobbered.
-
-When the function is called, values from other environments may need to be made
-available in the function's environment. These values are said to be "closed
-over".
-
-Even if a value is not referenced in a given environment, it may need to be
-closed over in that environment so that it can be passed to a called function
-that does reference the value. When we discover that a value must be closed
-over by a function, we must close over the value in all the environments where
-that function is referenced. This applies to all references, not just local
-calls, since at other references we must have the values on hand so that we can
-build a closure. This propagation must be applied recursively, since the value
-must also be available in *those* functions' callers.
-
-If a closure reference is known to be "safe" (not an upward funarg), then the
-closure structure may be allocated on the stack.
-
-Closure analysis deals only with closures over values, while Common Lisp
-requires closures over variables. The difference only becomes significant when
-variables are set. If a variable is not set, then we can freely make copies of
-it without keeping track of where they are. When a variable is set, we must
-maintain a single value cell, or at least the illusion thereof. We achieve
-this by creating a heap-allocated "value cell" structure for each set variable
-that is closed over. The pointer to this value cell is passed around as the
-"value" corresponding to that variable. References to the variable must
-explicitly indirect through the value cell.
-
-When we are scanning over the lambdas in the component, we also check for bound
-but not referenced variables.
-
-Environment analysis emits cleanup code for local exits and markers for
-non-local exits.
-
-A non-local exit is a control transfer from one environment to another. In a
-non-local exit, we must close over the continuation that we transfer to so that
-the exiting function can find its way back. We indicate the need to close a
-continuation by placing the continuation structure in the closure and also
-pushing it on a list in the environment structure for the target of the exit.
-[\#\#\# To be safe, we would treat the continuation as a set closure variable so
-that we could invalidate it when we leave the dynamic extent of the exit point.
-Transferring control to a meaningless stack pointer would be apt to cause
-horrible death.]
-
-Each local control transfer may require dynamic state such as special bindings
-to be undone. We represent cleanup actions by funny function calls in a new
-block linked in as an implicit MV-PROG1.
-
+++ /dev/null
-\chapter{Glossary}% -*- Dictionary: int:design -*-
-
-% Note: in an entry, any word that is also defined should be \it
-% should entries have page references as well?
-
-\begin{description}
-\item[assert (a type)]
-In Python, all type checking is done via a general type assertion
-mechanism. Explicit declarations and implicit assertions (e.g. the arg to
-+ is a number) are recorded in the front-end (implicit continuation)
-representation. Type assertions (and thus type-checking) are "unbundled"
-from the operations that are affected by the assertion. This has two major
-advantages:
-\begin{itemize}
-\item Code that implements operations need not concern itself with checking
-operand types.
-
-\item Run-time type checks can be eliminated when the compiler can prove that
-the assertion will always be satisfied.
-\end{itemize}
-See also {\it restrict}.
-
-\item[back end] The back end is the part of the compiler that operates on the
-{\it virtual machine} intermediate representation. Also included are the
-compiler phases involved in the conversion from the {\it front end}
-representation (or {\it ICR}).
-
-\item[bind node] This is a node type the that marks the start of a {\it lambda}
-body in {\it ICR}. This serves as a placeholder for environment manipulation
-code.
-
-\item[IR1] The first intermediate representation, also known as {\it ICR}, or
-the Implicit Continuation Represenation.
-
-\item[IR2] The second intermediate representation, also known as {\it VMR}, or
-the Virtual Machine Representation.
-
-\item[basic block] A basic block (or simply "block") has the pretty much the
-usual meaning of representing a straight-line sequence of code. However, the
-code sequence ultimately generated for a block might contain internal branches
-that were hidden inside the implementation of a particular operation. The type
-of a block is actually {\tt cblock}. The {\tt block-info} slot holds an
-{\tt VMR-block} containing backend information.
-
-\item[block compilation] Block compilation is a term commonly used to describe
-the compile-time resolution of function names. This enables many
-optimizations.
-
-\item[call graph]
-Each node in the call graph is a function (represented by a {\it flow graph}.)
-The arcs in the call graph represent a possible call from one function to
-another. See also {\it tail set}.
-
-\item[cleanup]
-A cleanup is the part of the implicit continuation representation that
-retains information scoping relationships. For indefinite extent bindings
-(variables and functions), we can abandon scoping information after ICR
-conversion, recovering the lifetime information using flow analysis. But
-dynamic bindings (special values, catch, unwind protect, etc.) must be
-removed at a precise time (whenever the scope is exited.) Cleanup
-structures form a hierarchy that represents the static nesting of dynamic
-binding structures. When the compiler does a control transfer, it can use
-the cleanup information to determine what cleanup code needs to be emitted.
-
-\item[closure variable]
-A closure variable is any lexical variable that has references outside of
-its {\it home environment}. See also {\it indirect value cell}.
-
-\item[closed continuation] A closed continuation represents a {\tt tagbody} tag
-or {\tt block} name that is closed over. These two cases are mostly
-indistinguishable in {\it ICR}.
-
-\item[home] Home is a term used to describe various back-pointers. A lambda
-variable's "home" is the lambda that the variable belongs to. A lambda's "home
-environment" is the environment in which that lambda's variables are allocated.
-
-\item[indirect value cell]
-Any closure variable that has assignments ({\tt setq}s) will be allocated in an
-indirect value cell. This is necessary to ensure that all references to
-the variable will see assigned values, since the compiler normally freely
-copies values when creating a closure.
-
-\item[set variable] Any variable that is assigned to is called a "set
-variable". Several optimizations must special-case set variables, and set
-closure variables must have an {\it indirect value cell}.
-
-\item[code generator] The code generator for a {\it VOP} is a potentially
-arbitrary list code fragment which is responsible for emitting assembly code to
-implement that VOP.
-
-\item[constant pool] The part of a compiled code object that holds pointers to
-non-immediate constants.
-
-\item[constant TN]
-A constant TN is the {\it VMR} of a compile-time constant value. A
-constant may be immediate, or may be allocated in the {\it constant pool}.
-
-\item[constant leaf]
-A constant {\it leaf} is the {\it ICR} of a compile-time constant value.
-
-\item[combination]
-A combination {\it node} is the {\it ICR} of any fixed-argument function
-call (not {\tt apply} or {\tt multiple-value-call}.)
-
-\item[top-level component]
-A top-level component is any component whose only entry points are top-level
-lambdas.
-
-\item[top-level lambda]
-A top-level lambda represents the execution of the outermost form on which
-the compiler was invoked. In the case of {\tt compile-file}, this is often a
-truly top-level form in the source file, but the compiler can recursively
-descend into some forms ({\tt eval-when}, etc.) breaking them into separate
-compilations.
-
-\item[component] A component is basically a sequence of blocks. Each component
-is compiled into a separate code object. With {\it block compilation} or {\it
-local functions}, a component will contain the code for more than one function.
-This is called a component because it represents a connected portion of the
-call graph. Normally the blocks are in depth-first order ({\it DFO}).
-
-\item[component, initial] During ICR conversion, blocks are temporarily
-assigned to initial components. The "flow graph canonicalization" phase
-determines the true component structure.
-
-\item[component, head and tail]
-The head and tail of a component are dummy blocks that mark the start and
-end of the {\it DFO} sequence. The component head and tail double as the root
-and finish node of the component's flow graph.
-
-\item[local function (call)]
-A local function call is a call to a function known at compile time to be
-in the same {\it component}. Local call allows compile time resolution of the
-target address and calling conventions. See {\it block compilation}.
-
-\item[conflict (of TNs, set)]
-Register allocation terminology. Two TNs conflict if they could ever be
-live simultaneously. The conflict set of a TN is all TNs that it conflicts
-with.
-
-\item[continuation]
-The ICR data structure which represents both:
-\begin{itemize}
-\item The receiving of a value (or multiple values), and
-
-\item A control location in the flow graph.
-\end{itemize}
-In the Implicit Continuation Representation, the environment is implicit in the
-continuation's BLOCK (hence the name.) The ICR continuation is very similar to
-a CPS continuation in its use, but its representation doesn't much resemble (is
-not interchangeable with) a lambda.
-
-\item[cont] A slot in the {\it node} holding the {\it continuation} which
-receives the node's value(s). Unless the node ends a {\it block}, this also
-implicitly indicates which node should be evaluated next.
-
-\item[cost] Approximations of the run-time costs of operations are widely used
-in the back end. By convention, the unit is generally machine cycles, but the
-values are only used for comparison between alternatives. For example, the
-VOP cost is used to determine the preferred order in which to try possible
-implementations.
-
-\item[CSP, CFP] See {\it control stack pointer} and {\it control frame
-pointer}.
-
-\item[Control stack] The main call stack, which holds function stack frames.
-All words on the control stack are tagged {\it descriptors}. In all ports done
-so far, the control stack grows from low memory to high memory. The most
-recent call frames are considered to be ``on top'' of earlier call frames.
-
-\item[Control stack pointer] The allocation pointer for the {\it control
-stack}. Generally this points to the first free word at the top of the stack.
-
-\item[Control frame pointer] The pointer to the base of the {\it control stack}
-frame for a particular function invocation. The CFP for the running function
-must be in a register.
-
-\item[Number stack] The auxiliary stack used to hold any {\it non-descriptor}
-(untagged) objects. This is generally the same as the C call stack, and thus
-typically grows down.
-
-\item[Number stack pointer] The allocation pointer for the {\it number stack}.
-This is typically the C stack pointer, and is thus kept in a register.
-
-\item[NSP, NFP] See {\it number stack pointer}, {\it number frame pointer}.
-
-\item[Number frame pointer] The pointer to the base of the {\it number stack}
-frame for a particular function invocation. Functions that don't use the
-number stack won't have an NFP, but if an NFP is allocated, it is always
-allocated in a particular register. If there is no variable-size data on the
-number stack, then the NFP will generally be identical to the NSP.
-
-\item[Lisp return address] The name of the {\it descriptor} encoding the
-"return pc" for a function call.
-
-\item[LRA] See {\it lisp return address}. Also, the name of the register where
-the LRA is passed.
-
-
-\item[Code pointer] A pointer to the header of a code object. The code pointer
-for the currently running function is stored in the {\tt code} register.
-
-\item[Interior pointer] A pointer into the inside of some heap-allocated
-object. Interior pointers confuse the garbage collector, so their use is
-highly constrained. Typically there is a single register dedicated to holding
-interior pointers.
-
-\item[dest]
-A slot in the {\it continuation} which points the the node that receives this
-value. Null if this value is not received by anyone.
-
-\item[DFN, DFO] See {\it Depth First Number}, {\it Depth First Order}.
-
-\item[Depth first number] Blocks are numbered according to their appearance in
-the depth-first ordering (the {\tt block-number} slot.) The numbering actually
-increases from the component tail, so earlier blocks have larger numbers.
-
-\item[Depth first order] This is a linearization of the flow graph, obtained by
-a depth-first walk. Iterative flow analysis algorithms work better when blocks
-are processed in DFO (or reverse DFO.)
-
-
-\item[Object] In low-level design discussions, an object is one of the
-following:
-\begin{itemize}
-\item a single word containing immediate data (characters, fixnums, etc)
-\item a single word pointing to an object (structures, conses, etc.)
-\end{itemize}
-These are tagged with three low-tag bits as described in the section
-\ref{tagging} This is synonymous with {\it descriptor}.
-In other parts of the documentation, may be used more loosely to refer to a
-{\it lisp object}.
-
-\item[Lisp object]
-A Lisp object is a high-level object discussed as a data type in the Common
-Lisp definition.
-
-\item[Data-block]
-A data-block is a dual-word aligned block of memory that either manifests a
-Lisp object (vectors, code, symbols, etc.) or helps manage a Lisp object on
-the heap (array header, function header, etc.).
-
-\item[Descriptor]
-A descriptor is a tagged, single-word object. It either contains immediate
-data or a pointer to data. This is synonymous with {\it object}. Storage
-locations that must contain descriptors are referred to as descriptor
-locations.
-
-\item[Pointer descriptor]
-A descriptor that points to a {\it data block} in memory (i.e. not an immediate
-object.)
-
-\item[Immediate descriptor]
-A descriptor that encodes the object value in the descriptor itself; used for
-characters, fixnums, etc.
-
-\item[Word]
-A word is a 32-bit quantity.
-
-\item[Non-descriptor]
-Any chunk of bits that isn't a valid tagged descriptor. For example, a
-double-float on the number stack. Storage locations that are not scanned by
-the garbage collector (and thus cannot contain {\it pointer descriptors}) are
-called non-descriptor locations. {\it Immediate descriptors} can be stored in
-non-descriptor locations.
-
-
-\item[Entry point] An entry point is a function that may be subject to
-``unpredictable'' control transfers. All entry points are linked to the root
-of the flow graph (the component head.) The only functions that aren't entry
-points are {\it let} functions. When complex lambda-list syntax is used,
-multiple entry points may be created for a single lisp-level function.
-See {\it external entry point}.
-
-\item[External entry point] A function that serves as a ``trampoline'' to
-intercept function calls coming in from outside of the component. The XEP does
-argument syntax and type checking, and may also translate the arguments and
-return values for a locally specialized calling calling convention.
-
-\item[XEP] An {\it external entry point}.
-
-\item[lexical environment] A lexical environment is a structure that is used
-during VMR conversion to represent all lexically scoped bindings (variables,
-functions, declarations, etc.) Each {\tt node} is annotated with its lexical
-environment, primarily for use by the debugger and other user interfaces. This
-structure is also the environment object passed to {\tt macroexpand}.
-
-\item[environment] The environment is part of the ICR, created during
-environment analysis. Environment analysis apportions code to disjoint
-environments, with all code in the same environment sharing the same stack
-frame. Each environment has a ``{\it real}'' function that allocates it, and
-some collection {\tt let} functions. Although environment analysis is the
-last ICR phase, in earlier phases, code is sometimes said to be ``in the
-same/different environment(s)''. This means that the code will definitely be
-in the same environment (because it is in the same real function), or that is
-might not be in the same environment, because it is not in the same function.
-
-\item[fixup] Some sort of back-patching annotation. The main sort encountered
-are load-time {\it assembler fixups}, which are a linkage annotation mechanism.
-
-\item[flow graph] A flow graph is a directed graph of basic blocks, where each
-arc represents a possible control transfer. The flow graph is the basic data
-structure used to represent code, and provides direct support for data flow
-analysis. See component and ICR.
-
-\item[foldable] An attribute of {\it known functions}. A function is foldable
-if calls may be constant folded whenever the arguments are compile-time
-constant. Generally this means that it is a pure function with no side
-effects.
-
-
-FSC
-full call
-function attribute
-function
- "real" (allocates environment)
- meaning function-entry
- more vague (any lambda?)
-funny function
-GEN (kill and...)
-global TN, conflicts, preference
-GTN (number)
-IR ICR VMR ICR conversion, VMR conversion (translation)
-inline expansion, call
-kill (to make dead)
-known function
-LAMBDA
-leaf
-let call
-lifetime analysis, live (tn, variable)
-load tn
-LOCS (passing, return locations)
-local call
-local TN, conflicts, (or just used in one block)
-location (selection)
-LTN (number)
-main entry
-mess-up (for cleanup)
-more arg (entry)
-MV
-non-local exit
-non-packed SC, TN
-non-set variable
-operand (to vop)
-optimizer (in icr optimize)
-optional-dispatch
-pack, packing, packed
-pass (in a transform)
-passing
- locations (value)
- conventions (known, unknown)
-policy (safe, fast, small, ...)
-predecessor block
-primitive-type
-reaching definition
-REF
-representation
- selection
- for value
-result continuation (for function)
-result type assertion (for template) (or is it restriction)
-restrict
- a TN to finite SBs
- a template operand to a primitive type (boxed...)
- a tn-ref to particular SCs
-
-return (node, vops)
-safe, safety
-saving (of registers, costs)
-SB
-SC (restriction)
-semi-inline
-side-effect
- in ICR
- in VMR
-sparse set
-splitting (of VMR blocks)
-SSET
-SUBPRIMITIVE
-successor block
-tail recursion
- tail recursive
- tail recursive loop
- user tail recursion
-
-template
-TN
-TNBIND
-TN-REF
-transform (source, ICR)
-type
- assertion
- inference
- top-down, bottom-up
- assertion propagation
- derived, asserted
- descriptor, specifier, intersection, union, member type
- check
-type-check (in continuation)
-UNBOXED (boxed) descriptor
-unknown values continuation
-unset variable
-unwind-block, unwinding
-used value (dest)
-value passing
-VAR
-VM
-VOP
-XEP
-
-\end{description}
+++ /dev/null
-\chapter{User Interface}
-
-\section{Error Message Utilities}
-
-\section{Source Paths}
-
+++ /dev/null
-
-\f
-;;;; Terminology.
-
-OBJECT
- An object is one of the following:
- a single word containing immediate data (characters, fixnums, etc)
- a single word pointing to an object (structures, conses, etc.)
- These are tagged with three low-tag bits as described in the section
- "Tagging". This is synonymous with DESCRIPTOR.
-
-LISP OBJECT
- A Lisp object is a high-level object discussed as a data type in Common
- Lisp: The Language.
-
-DATA-BLOCK
- A data-block is a dual-word aligned block of memory that either manifests a
- Lisp object (vectors, code, symbols, etc.) or helps manage a Lisp object on
- the heap (array header, function header, etc.).
-
-DESCRIPTOR
- A descriptor is a tagged, single-word object. It either contains immediate
- data or a pointer to data. This is synonymous with OBJECT.
-
-WORD
- A word is a 32-bit quantity.
-
-
-\f
-;;;; Tagging.
-
-The following is a key of the three bit low-tagging scheme:
- 000 even fixnum
- 001 function pointer
- 010 other-immediate (header-words, characters, symbol-value trap value, etc.)
- 011 list pointer
- 100 odd fixnum
- 101 structure pointer
- 110 unused
- 111 other-pointer to data-blocks (other than conses, structures,
- and functions)
-
-This taging scheme forces a dual-word alignment of data-blocks on the heap, but
-this can be pretty negligible:
- RATIOS and COMPLEX must have a header-word anyway since they are not a
- major type. This wastes one word for these infrequent data-blocks since
- they require two words for the data.
- BIGNUMS must have a header-word and probably contain only one other word
- anyway, so we probably don't waste any words here. Most bignums just
- barely overflow fixnums, that is by a bit or two.
- Single and double FLOATS?
- no waste
- one word wasted
- SYMBOLS are dual-word aligned with the header-word.
- Everything else is vector-like including code, so these probably take up
- so many words that one extra one doesn't matter.
-
-
-\f
-;;;; GC Comments.
-
-Data-Blocks comprise only descriptors, or they contain immediate data and raw
-bits interpreted by the system. GC must skip the latter when scanning the
-heap, so it does not look at a word of raw bits and interpret it as a pointer
-descriptor. These data-blocks require headers for GC as well as for operations
-that need to know how to interpret the raw bits. When GC is scanning, and it
-sees a header-word, then it can determine how to skip that data-block if
-necessary. Header-Words are tagged as other-immediates. See the sections
-"Other-Immediates" and "Data-Blocks and Header-Words" for comments on
-distinguishing header-words from other-immediate data. This distinction is
-necessary since we scan through data-blocks containing only descriptors just as
-we scan through the heap looking for header-words introducing data-blocks.
-
-Data-Blocks containing only descriptors do not require header-words for GC
-since the entire data-block can be scanned by GC a word at a time, taking
-whatever action is necessary or appropriate for the data in that slot. For
-example, a cons is referenced by a descriptor with a specific tag, and the
-system always knows the size of this data-block. When GC encounters a pointer
-to a cons, it can transport it into the new space, and when scanning, it can
-simply scan the two words manifesting the cons interpreting each word as a
-descriptor. Actually there is no cons tag, but a list tag, so we make sure the
-cons is not nil when appropriate. A header may still be desired if the pointer
-to the data-block does not contain enough information to adequately maintain
-the data-block. An example of this is a simple-vector containing only
-descriptor slots, and we attach a header-word because the descriptor pointing
-to the vector lacks necessary information -- the type of the vector's elements,
-its length, etc.
-
-There is no need for a major tag for GC forwarding pointers. Since the tag
-bits are in the low end of the word, a range check on the start and end of old
-space tells you if you need to move the thing. This is all GC overhead.
-
-
-\f
-;;;; Structures.
-
-Structures comprise a word for each slot in the definition in addition to one
-word, a type slot which is a pointer descriptor. This points to a structure
-describing the data-block as a structure, a defstruct-descriptor object. When
-operating on a structure, doing a structure test can be done by simply checking
-the tag bits on the pointer descriptor referencing it. As described in section
-"GC Comments", data-blocks such as those representing structures may avoid
-having a header-word since they are GC-scanable without any problem. This
-saves two words for every structure instance.
-
-
-\f
-;;;; Fixnums.
-
-A fixnum has one of the following formats in 32 bits:
- -------------------------------------------------------
- | 30 bit 2's complement even integer | 0 0 0 |
- -------------------------------------------------------
-or
- -------------------------------------------------------
- | 30 bit 2's complement odd integer | 1 0 0 |
- -------------------------------------------------------
-
-Effectively, there is one tag for immediate integers, two zeros. This buys one
-more bit for fixnums, and now when these numbers index into simple-vectors or
-offset into memory, they point to word boundaries on 32-bit, byte-addressable
-machines. That is, no shifting need occur to use the number directly as an
-offset.
-
-This format has another advantage on byte-addressable machines when fixnums are
-offsets into vector-like data-blocks, including structures. Even though we
-previously mentioned data-blocks are dual-word aligned, most indexing and slot
-accessing is word aligned, and so are fixnums with effectively two tag bits.
-
-Two tags also allow better usage of special instructions on some machines that
-can deal with two low-tag bits but not three.
-
-Since the two bits are zeros, we avoid having to mask them off before using the
-words for arithmetic, but division and multiplication require special shifting.
-
-
-\f
-;;;; Other-immediates.
-
-An other-immediate has the following format:
- ----------------------------------------------------------------
- | Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
- ----------------------------------------------------------------
-
-The system uses eight bits of type when checking types and defining system
-constants. This allows allows for 32 distinct other-immediate objects given
-the three low-tag bits tied down.
-
-The system uses this format for characters, SYMBOL-VALUE unbound trap value,
-and header-words for data-blocks on the heap. The type codes are laid out to
-facilitate range checks for common subtypes; for example, all numbers will have
-contiguous type codes which are distinct from the contiguous array type codes.
-See section "Data-Blocks and Other-immediates Typing" for details.
-
-
-\f
-;;;; Data-Blocks and Header-Word Format.
-
-Pointers to data-blocks have the following format:
- ----------------------------------------------------------------
- | Dual-word address of data-block (29 bits) | 1 1 1 |
- ----------------------------------------------------------------
-
-The word pointed to by the above descriptor is a header-word, and it has the
-same format as an other-immediate:
- ----------------------------------------------------------------
- | Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
- ----------------------------------------------------------------
-
-This is convenient for scanning the heap when GC'ing, but it does mean that
-whenever GC encounters an other-immediate word, it has to do a range check on
-the low byte to see if it is a header-word or just a character (for example).
-This is easily acceptable performance hit for scanning.
-
-The system interprets the data portion of the header-word for non-vector
-data-blocks as the word length excluding the header-word. For example, the
-data field of the header for ratio and complex numbers is two, one word each
-for the numerator and denominator or for the real and imaginary parts.
-
-For vectors and data-blocks representing Lisp objects stored like vectors, the
-system ignores the data portion of the header-word:
- ----------------------------------------------------------------
- | Unused Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
- ----------------------------------------------------------------
- | Element Length of Vector (30 bits) | 0 0 |
- ----------------------------------------------------------------
-
-Using a separate word allows for much larger vectors, and it allows LENGTH to
-simply access a single word without masking or shifting. Similarly, the header
-for complex arrays and vectors has a second word, following the header-word,
-the system uses for the fill pointer, so computing the length of any array is
-the same code sequence.
-
-
-\f
-;;;; Data-Blocks and Other-immediates Typing.
-
-These are the other-immediate types. We specify them including all low eight
-bits, including the other-immediate tag, so we can think of the type bits as
-one type -- not an other-immediate major type and a subtype. Also, fetching a
-byte and comparing it against a constant is more efficient than wasting even a
-small amount of time shifting out the other-immediate tag to compare against a
-five bit constant.
-
- Number (< 30)
-00000 010 bignum 10
-00000 010 ratio 14
-00000 010 single-float 18
-00000 010 double-float 22
-00000 010 complex 26
-
- Array (>= 30 code 86)
- Simple-Array (>= 20 code 70)
-00000 010 simple-array 30
- Vector (>= 34 code 82)
-00000 010 simple-string 34
-00000 010 simple-bit-vector 38
-00000 010 simple-vector 42
-00000 010 (simple-array (unsigned-byte 2) (*)) 46
-00000 010 (simple-array (unsigned-byte 4) (*)) 50
-00000 010 (simple-array (unsigned-byte 8) (*)) 54
-00000 010 (simple-array (unsigned-byte 16) (*)) 58
-00000 010 (simple-array (unsigned-byte 32) (*)) 62
-00000 010 (simple-array single-float (*)) 66
-00000 010 (simple-array double-float (*)) 70
-00000 010 complex-string 74
-00000 010 complex-bit-vector 78
-00000 010 (array * (*)) -- general complex vector. 82
-00000 010 complex-array 86
-
-00000 010 code-header-type 90
-00000 010 function-header-type 94
-00000 010 closure-header-type 98
-00000 010 funcallable-instance-header-type 102
-00000 010 unused-function-header-1-type 106
-00000 010 unused-function-header-2-type 110
-00000 010 unused-function-header-3-type 114
-00000 010 closure-function-header-type 118
-00000 010 return-pc-header-type 122
-00000 010 value-cell-header-type 126
-00000 010 symbol-header-type 130
-00000 010 base-character-type 134
-00000 010 system-area-pointer-type (header type) 138
-00000 010 unbound-marker 142
-00000 010 weak-pointer-type 146
-
-
-\f
-;;;; Strings.
-
-All strings in the system are C-null terminated. This saves copying the bytes
-when calling out to C. The only time this wastes memory is when the string
-contains a multiple of eight characters, and then the system allocates two more
-words (since Lisp objects are dual-word aligned) to hold the C-null byte.
-Since the system will make heavy use of C routines for systems calls and
-libraries that save reimplementation of higher level operating system
-functionality (such as pathname resolution or current directory computation),
-saving on copying strings for C should make C call out more efficient.
-
-The length word in a string header, see section "Data-Blocks and Header-Word
-Format", counts only the characters truly in the Common Lisp string.
-Allocation and GC will have to know to handle the extra C-null byte, and GC
-already has to deal with rounding up various objects to dual-word alignment.
-
-
-\f
-;;;; Symbols and NIL.
-
-Symbol data-block has the following format:
- -------------------------------------------------------
- | 5 (data-block words) | Symbol Type (8 bits) |
- -------------------------------------------------------
- | Value Descriptor |
- -------------------------------------------------------
- | Function Pointer |
- -------------------------------------------------------
- | Raw Function Address |
- -------------------------------------------------------
- | Setf Function |
- -------------------------------------------------------
- | Property List |
- -------------------------------------------------------
- | Print Name |
- -------------------------------------------------------
- | Package |
- -------------------------------------------------------
-
-Most of these slots are self-explanatory given what symbols must do in Common
-Lisp, but a couple require comments. We added the Raw Function Address slot to
-speed up named call which is the most common calling convention. This is a
-non-descriptor slot, but since objects are dual word aligned, the value
-inherently has fixnum low-tag bits. The GC method for symbols must know to
-update this slot. The Setf Function slot is currently unused, but we had an
-extra slot due to adding Raw Function Address since objects must be dual-word
-aligned.
-
-The issues with nil are that we want it to act like a symbol, and we need list
-operations such as CAR and CDR to be fast on it. CMU Common Lisp solves this
-by putting nil as the first object in static space, where other global values
-reside, so it has a known address in the system:
- ------------------------------------------------------- <-- start static
- | 0 | space
- -------------------------------------------------------
- | 5 (data-block words) | Symbol Type (8 bits) |
- ------------------------------------------------------- <-- nil
- | Value/CAR |
- -------------------------------------------------------
- | Definition/CDR |
- -------------------------------------------------------
- | Raw Function Address |
- -------------------------------------------------------
- | Setf Function |
- -------------------------------------------------------
- | Property List |
- -------------------------------------------------------
- | Print Name |
- -------------------------------------------------------
- | Package |
- -------------------------------------------------------
- | ... |
- -------------------------------------------------------
-In addition, we make the list typed pointer to nil actually point past the
-header word of the nil symbol data-block. This has usefulness explained below.
-The value and definition of nil are nil. Therefore, any reference to nil used
-as a list has quick list type checking, and CAR and CDR can go right through
-the first and second words as if nil were a cons object.
-
-When there is a reference to nil used as a symbol, the system adds offsets to
-the address the same as it does for any symbol. This works due to a
-combination of nil pointing past the symbol header-word and the chosen list and
-other-pointer type tags. The list type tag is four less than the other-pointer
-type tag, but nil points four additional bytes into its symbol data-block.
-
-
-\f
-;;;; Array Headers.
-
-The array-header data-block has the following format:
- ----------------------------------------------------------------
- | Header Len (24 bits) = Array Rank +5 | Array Type (8 bits) |
- ----------------------------------------------------------------
- | Fill Pointer (30 bits) | 0 0 |
- ----------------------------------------------------------------
- | Available Elements (30 bits) | 0 0 |
- ----------------------------------------------------------------
- | Data Vector (29 bits) | 1 1 1 |
- ----------------------------------------------------------------
- | Displacement (30 bits) | 0 0 |
- ----------------------------------------------------------------
- | Displacedp (29 bits) -- t or nil | 1 1 1 |
- ----------------------------------------------------------------
- | Range of First Index (30 bits) | 0 0 |
- ----------------------------------------------------------------
- .
- .
- .
-
-The array type in the header-word is one of the eight-bit patterns from section
-"Data-Blocks and Other-immediates Typing", indicating that this is a complex
-string, complex vector, complex bit-vector, or a multi-dimensional array. The
-data portion of the other-immediate word is the length of the array header
-data-block. Due to its format, its length is always five greater than the
-array's number of dimensions. The following words have the following
-interpretations and types:
- Fill Pointer
- This is a fixnum indicating the number of elements in the data vector
- actually in use. This is the logical length of the array, and it is
- typically the same value as the next slot. This is the second word, so
- LENGTH of any array, with or without an array header, is just four bytes
- off the pointer to it.
- Available Elements
- This is a fixnum indicating the number of elements for which there is
- space in the data vector. This is greater than or equal to the logical
- length of the array when it is a vector having a fill pointer.
- Data Vector
- This is a pointer descriptor referencing the actual data of the array.
- This a data-block whose first word is a header-word with an array type as
- described in sections "Data-Blocks and Header-Word Format" and
- "Data-Blocks and Other-immediates Typing"
- Displacement
- This is a fixnum added to the computed row-major index for any array.
- This is typically zero.
- Displacedp
- This is either t or nil. This is separate from the displacement slot, so
- most array accesses can simply add in the displacement slot. The rare
- need to know if an array is displaced costs one extra word in array
- headers which probably aren't very frequent anyway.
- Range of First Index
- This is a fixnum indicating the number of elements in the first dimension
- of the array. Legal index values are zero to one less than this number
- inclusively. IF the array is zero-dimensional, this slot is
- non-existent.
- ... (remaining slots)
- There is an additional slot in the header for each dimension of the
- array. These are the same as the Range of First Index slot.
-
-
-\f
-;;;; Bignums.
-
-Bignum data-blocks have the following format:
- -------------------------------------------------------
- | Length (24 bits) | Bignum Type (8 bits) |
- -------------------------------------------------------
- | least significant bits |
- -------------------------------------------------------
- .
- .
- .
-
-The elements contain the two's complement representation of the integer with
-the least significant bits in the first element or closer to the header. The
-sign information is in the high end of the last element.
-
-
-
-\f
-;;;; Code Data-Blocks.
-
-A code data-block is the run-time representation of a "component". A component
-is a connected portion of a program's flow graph that is compiled as a single
-unit, and it contains code for many functions. Some of these functions are
-callable from outside of the component, and these are termed "entry points".
-
-Each entry point has an associated user-visible function data-block (of type
-FUNCTION). The full call convention provides for calling an entry point
-specified by a function object.
-
-Although all of the function data-blocks for a component's entry points appear
-to the user as distinct objects, the system keeps all of the code in a single
-code data-block. The user-visible function object is actually a pointer into
-the middle of a code data-block. This allows any control transfer within a
-component to be done using a relative branch.
-
-Besides a function object, there are other kinds of references into the middle
-of a code data-block. Control transfer into a function also occurs at the
-return-PC for a call. The system represents a return-PC somewhat similarly to
-a function, so GC can also recognize a return-PC as a reference to a code
-data-block.
-
-It is incorrect to think of a code data-block as a concatenation of "function
-data-blocks". Code for a function is not emitted in any particular order with
-respect to that function's function-header (if any). The code following a
-function-header may only be a branch to some other location where the
-function's "real" definition is.
-
-
-The following are the three kinds of pointers to code data-blocks:
- Code pointer (labeled A below):
- A code pointer is a descriptor, with other-pointer low-tag bits, pointing
- to the beginning of the code data-block. The code pointer for the
- currently running function is always kept in a register (CODE). In
- addition to allowing loading of non-immediate constants, this also serves
- to represent the currently running function to the debugger.
- Return-PC (labeled B below):
- The return-PC is a descriptor, with other-pointer low-tag bits, pointing
- to a location for a function call. Note that this location contains no
- descriptors other than the one word of immediate data, so GC can treat
- return-PC locations the same as instructions.
- Function (labeled C below):
- A function is a descriptor, with function low-tag bits, that is user
- callable. When a function header is referenced from a closure or from
- the function header's self-pointer, the pointer has other-pointer low-tag
- bits, instead of function low-tag bits. This ensures that the internal
- function data-block associated with a closure appears to be uncallable
- (although users should never see such an object anyway).
-
- Information about functions that is only useful for entry points is kept
- in some descriptors following the function's self-pointer descriptor.
- All of these together with the function's header-word are known as the
- "function header". GC must be able to locate the function header. We
- provide for this by chaining together the function headers in a NIL
- terminated list kept in a known slot in the code data-block.
-
-
-A code data-block has the following format:
- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- A
- | Header-Word count (24 bits) | %Code-Type (8 bits) |
- ----------------------------------------------------------------
- | Number of code words (fixnum tag) |
- ----------------------------------------------------------------
- | Pointer to first function header (other-pointer tag) |
- ----------------------------------------------------------------
- | Debug information (structure tag) |
- ----------------------------------------------------------------
- | First constant (a descriptor) |
- ----------------------------------------------------------------
- | ... |
- ----------------------------------------------------------------
- | Last constant (and last word of code header) |
- ----------------------------------------------------------------
- | Some instructions (non-descriptor) |
- ----------------------------------------------------------------
- | (pad to dual-word boundary if necessary) |
- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- B
- | Word offset from code header (24) | %Return-PC-Type (8) |
- ----------------------------------------------------------------
- | First instruction after return |
- ----------------------------------------------------------------
- | ... more code and return-PC header-words |
- ----------------------------------------------------------------
- | (pad to dual-word boundary if necessary) |
- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- C
- | Offset from code header (24) | %Function-Header-Type (8) |
- ----------------------------------------------------------------
- | Self-pointer back to previous word (with other-pointer tag) |
- ----------------------------------------------------------------
- | Pointer to next function (other-pointer low-tag) or NIL |
- ----------------------------------------------------------------
- | Function name (a string or a symbol) |
- ----------------------------------------------------------------
- | Function debug arglist (a string) |
- ----------------------------------------------------------------
- | Function type (a list-style function type specifier) |
- ----------------------------------------------------------------
- | Start of instructions for function (non-descriptor) |
- ----------------------------------------------------------------
- | More function headers and instructions and return PCs, |
- | until we reach the total size of header-words + code |
- | words. |
- ----------------------------------------------------------------
-
-
-The following are detailed slot descriptions:
- Code data-block header-word:
- The immediate data in the code data-block's header-word is the number of
- leading descriptors in the code data-block, the fixed overhead words plus
- the number of constants. The first non-descriptor word, some code,
- appears at this word offset from the header.
- Number of code words:
- The total number of non-header-words in the code data-block. The total
- word size of the code data-block is the sum of this slot and the
- immediate header-word data of the previous slot. The system accesses
- this slot with the system constant, %Code-Code-Size-Slot, offset from the
- header-word.
- Pointer to first function header:
- A NIL-terminated list of the function headers for all entry points to
- this component. The system accesses this slot with the system constant,
- %Code-Entry-Points-Slot, offset from the header-word.
- Debug information:
- The DEBUG-INFO structure describing this component. All information that
- the debugger wants to get from a running function is kept in this
- structure. Since there are many functions, the current PC is used to
- locate the appropriate debug information. The system keeps the debug
- information separate from the function data-block, since the currently
- running function may not be an entry point. There is no way to recover
- the function object for the currently running function, since this
- data-block may not exist. The system accesses this slot with the system
- constant, %Code-Debug-Info-Slot, offset from the header-word.
- First constant ... last constant:
- These are the constants referenced by the component, if there are any.
- The system accesses the first constant slot with the system constant,
- %Code-Constants-Offset, offset from the header-word.
-
- Return-PC header word:
- The immediate header-word data is the word offset from the enclosing code
- data-block's header-word to this word. This allows GC and the debugger
- to easily recover the code data-block from a return-PC. The code at the
- return point restores the current code pointer using a subtract immediate
- of the offset, which is known at compile time.
-
- Function entry point header-word:
- The immediate header-word data is the word offset from the enclosing code
- data-block's header-word to this word. This is the same as for the
- retrun-PC header-word.
- Self-pointer back to header-word:
- In a non-closure function, this self-pointer to the previous header-word
- allows the call sequence to always indirect through the second word in a
- user callable function. See section "Closure Format". With a closure,
- indirecting through the second word gets you a function header-word. The
- system ignores this slot in the function header for a closure, since it
- has already indirected once, and this slot could be some random thing
- that causes an error if you jump to it. This pointer has an
- other-pointer tag instead of a function pointer tag, indicating it is not
- a user callable Lisp object. The system accesses this slot with the
- system constant, %Function-Code-Slot, offset from the function
- header-word.
- Pointer to next function:
- This is the next link in the thread of entry point functions found in
- this component. This value is NIL when the current header is the last
- entry point in the component. The system accesses this slot with the
- system constant, %Function-Header-Next-Slot, offset from the function
- header-word.
- Function name:
- This function's name (for printing). If the user defined this function
- with DEFUN, then this is the defined symbol, otherwise it is a
- descriptive string. The system accesses this slot with the system
- constant, %Function-Header-Name-Slot, offset from the function
- header-word.
- Function debug arglist:
- A printed string representing the function's argument list, for human
- readability. If it is a macroexpansion function, then this is the
- original DEFMACRO arglist, not the actual expander function arglist. The
- system accesses this slot with the system constant,
- %Function-Header-Debug-Arglist-Slot, offset from the function
- header-word.
- Function type:
- A list-style function type specifier representing the argument signature
- and return types for this function. For example,
- (FUNCTION (FIXNUM FIXNUM FIXNUM) FIXNUM)
- or
- (FUNCTION (STRING &KEY (:START UNSIGNED-BYTE)) STRING)
- This information is intended for machine readablilty, such as by the
- compiler. The system accesses this slot with the system constant,
- %Function-Header-Type-Slot, offset from the function header-word.
-
-
-\f
-;;;; Closure Format.
-
-A closure data-block has the following format:
- ----------------------------------------------------------------
- | Word size (24 bits) | %Closure-Type (8 bits) |
- ----------------------------------------------------------------
- | Pointer to function header (other-pointer low-tag) |
- ----------------------------------------------------------------
- | . |
- | Environment information |
- | . |
- ----------------------------------------------------------------
-
-
-A closure descriptor has function low-tag bits. This means that a descriptor
-with function low-tag bits may point to either a function header or to a
-closure. The idea is that any callable Lisp object has function low-tag bits.
-Insofar as call is concerned, we make the format of closures and non-closure
-functions compatible. This is the reason for the self-pointer in a function
-header. Whenever you have a callable object, you just jump through the second
-word, offset some bytes, and go.
-
-
-\f
-;;;; Function call.
-
-Due to alignment requirements and low-tag codes, it is not possible to use a
-hardware call instruction to compute the return-PC. Instead the return-PC
-for a call is computed by doing an add-immediate to the start of the code
-data-block.
-
-An advantage of using a single data-block to represent both the descriptor and
-non-descriptor parts of a function is that both can be represented by a
-single pointer. This reduces the number of memory accesses that have to be
-done in a full call. For example, since the constant pool is implicit in a
-return-PC, a call need only save the return-PC, rather than saving both the
-return PC and the constant pool.
-
-
-\f
-;;;; Memory Layout.
-
-CMU Common Lisp has four spaces, read-only, static, dynamic-0, and dynamic-1.
-Read-only contains objects that the system never modifies, moves, or reclaims.
-Static space contains some global objects necessary for the system's runtime or
-performance (since they are located at a known offset at a know address), and
-the system never moves or reclaims these. However, GC does need to scan static
-space for references to moved objects. Dynamic-0 and dynamic-1 are the two
-heap areas for stop-and-copy GC algorithms.
-
-What global objects are at the head of static space???
- NIL
- eval::*top-of-stack*
- lisp::*current-catch-block*
- lisp::*current-unwind-protect*
- FLAGS (RT only)
- BSP (RT only)
- HEAP (RT only)
-
-In addition to the above spaces, the system has a control stack, binding stack,
-and a number stack. The binding stack contains pairs of descriptors, a symbol
-and its previous value. The number stack is the same as the C stack, and the
-system uses it for non-Lisp objects such as raw system pointers, saving
-non-Lisp registers, parts of bignum computations, etc.
-
-
-\f
-;;;; System Pointers.
-
-The system pointers reference raw allocated memory, data returned by foreign
-function calls, etc. The system uses these when you need a pointer to a
-non-Lisp block of memory, using an other-pointer. This provides the greatest
-flexibility by relieving contraints placed by having more direct references
-that require descriptor type tags.
-
-A system area pointer data-block has the following format:
- -------------------------------------------------------
- | 1 (data-block words) | SAP Type (8 bits) |
- -------------------------------------------------------
- | system area pointer |
- -------------------------------------------------------
-
-"SAP" means "system area pointer", and much of our code contains this naming
-scheme. We don't currently restrict system pointers to one area of memory, but
-if they do point onto the heap, it is up to the user to prevent being screwed
-by GC or whatever.
+++ /dev/null
-% -*- Dictionary: design; Package: C -*-
-
-May be worth having a byte-code representation for interpreted code. This way,
-an entire system could be compiled into byte-code for debugging (the
-"check-out" compiler?).
-
-Given our current inclination for using a stack machine to interpret IR1, it
-would be straightforward to layer a byte-code interpreter on top of this.
-
-
-Interpreter:
-
-Instead of having no interpreter, or a more-or-less conventional interpreter,
-or byte-code interpreter, how about directly executing IR1?
-
-We run through the IR1 passes, possibly skipping optional ones, until we get
-through environment analysis. Then we run a post-pass that annotates IR1 with
-information about where values are kept, i.e. the stack slot.
-
-We can lazily convert functions by having FUNCTION make an interpreted function
-object that holds the code (really a closure over the interpreter). The first
-time that we try to call the function, we do the conversion and processing.
-Also, we can easily keep track of which interpreted functions we have expanded
-macros in, so that macro redefinition automatically invalidates the old
-expansion, causing lazy reconversion.
-
-Probably the interpreter will want to represent MVs by a recognizable structure
-that is always heap-allocated. This way, we can punt the stack issues involved
-in trying to spread MVs. So a continuation value can always be kept in a
-single cell.
-
-The compiler can have some special frobs for making the interpreter efficient,
-such as a call operation that extracts arguments from the stack
-slots designated by a continuation list. Perhaps
- (values-mapcar fun . lists)
-<==>
- (values-list (mapcar fun . lists))
-This would be used with MV-CALL.
-
-
-This scheme seems to provide nearly all of the advantages of both the compiler
-and conventional interpretation. The only significant disadvantage with
-respect to a conventional interpreter is that there is the one-time overhead of
-conversion, but doing this lazily should make this quite acceptable.
-
-With respect to a conventional interpreter, we have major advantages:
- + Full syntax checking: safety comparable to compiled code.
- + Semantics similar to compiled code due to code sharing. Similar diagnostic
- messages, etc. Reduction of error-prone code duplication.
- + Potential for full type checking according to declarations (would require
- running IR1 optimize?)
- + Simplifies debugger interface, since interpreted code can look more like
- compiled code: source paths, edit definition, etc.
-
-For all non-run-time symbol annotations (anything other than SYMBOL-FUNCTION
-and SYMBOL-VALUE), we use the compiler's global database. MACRO-FUNCTION will
-use INFO, rather than vice-versa.
-
-When doing the IR1 phases for the interpreter, we probably want to suppress
-optimizations that change user-visible function calls:
- -- Don't do local call conversion of any named functions (even lexical ones).
- This is so that a call will appear on the stack that looks like the call in
- the original source. The keyword and optional argument transformations
- done by local call mangle things quite a bit. Also, note local-call
- converting prevents unreferenced arguments from being deleted, which is
- another non-obvious transformation.
- -- Don't run source-transforms, IR1 transforms and IR1 optimizers. This way,
- TRACE and BACKTRACE will show calls with the original arguments, rather
- than the "optimized" form, etc. Also, for the interpreter it will
- actually be faster to call the original function (which is compiled) than
- to "inline expand" it. Also, this allows implementation-dependent
- transforms to expand into %PRIMITIVE uses.
-
-There are some problems with stepping, due to our non-syntactic IR1
-representation. The source path information is the key that makes this
-conceivable. We can skip over the stepping of a subform by quietly evaluating
-nodes whose source path lies within the form being skipped.
-
-One problem with determining what value has been returned by a form. With a
-function call, it is theoretically possible to precisely determine this, since
-if we complete evaluation of the arguments, then we arrive at the Combination
-node whose value is synonymous with the value of the form. We can even detect
-this case, since the Node-Source will be EQ to the form. And we can also
-detect when we unwind out of the evaluation, since we will leave the form
-without having ever reached this node.
-
-But with macros and special-forms, there is no node whose value is the value of
-the form, and no node whose source is the macro call or special form. We can
-still detect when we leave the form, but we can't be sure whether this was a
-normal evaluation result or an explicit RETURN-FROM.
-
-But does this really matter? It seems that we can print the value returned (if
-any), then just print the next form to step. In the rare case where we did
-unwind, the user should be able to figure it out.
-
-[We can look at this as a side-effect of CPS: there isn't any difference
-between a "normal" return and a non-local one.]
-
-[Note that in any control transfer (normal or otherwise), the stepper may need
-to unwind out of an arbitrary number of levels of stepping. This is because a
-form in a TR position may yield its to a node arbitrarily far our.]
-
-Another problem is with deciding what form is being stepped. When we start
-evaluating a node, we dive into code that is nested somewhere down inside that
-form. So we actually have to do a loop of asking questions before we do any
-evaluation. But what do we ask about?
-
-If we ask about the outermost enclosing form that is a subform of the the last
-form that the user said to execute, then we might offer a form that isn't
-really evaluated, such as a LET binding list.
-
-But once again, is this really a problem? It is certainly different from a
-conventional stepper, but a pretty good argument could be made that it is
-superior. Haven't you ever wanted to skip the evaluation of all the
-LET bindings, but not the body? Wouldn't it be useful to be able to skip the
-DO step forms?
-
-All of this assumes that nobody ever wants to step through the guts of a
-macroexpansion. This seems reasonable, since steppers are for weenies, and
-weenies don't define macros (hence don't debug them). But there are probably
-some weenies who don't know that they shouldn't be writing macros.
-
-We could handle this by finding the "source paths" in the expansion of each
-macro by sticking some special frob in the source path marking the place where
-the expansion happened. When we hit code again that is in the source, then we
-revert to the normal source path. Something along these lines might be a good
-idea anyway (for compiler error messages, for example).
-
-The source path hack isn't guaranteed to work quite so well in generated code,
-though, since macros return stuff that isn't freshly consed. But we could
-probably arrange to win as long as any given expansion doesn't return two EQ
-forms.
-
-It might be nice to have a command that skipped stepping of the form, but
-printed the results of each outermost enclosed evaluated subform, i.e. if you
-used this on the DO step-list, it would print the result of each new-value
-form. I think this is implementable. I guess what you would do is print each
-value delivered to a DEST whose source form is the current or an enclosing
-form. Along with the value, you would print the source form for the node that
-is computing the value.
-
-The stepper can also have a "back" command that "unskips" or "unsteps". This
-would allow the evaluation of forms that are pure (modulo lexical variable
-setting) to be undone. This is useful, since in stepping it is common that you
-skip a form that you shouldn't have, or get confused and want to restart at
-some earlier point.
-
-What we would do is remember the current node and the values of all local
-variables. heap before doing each step or skip action. We can then back up
-the state of all lexical variables and the "program counter". To make this
-work right with set closure variables, we would copy the cell's value, rather
-than the value cell itself.
-
-[To be fair, note that this could easily be done with our current interpreter:
-the stepper could copy the environment alists.]
-
-We can't back up the "program counter" when a control transfer leaves the
-current function, since this state is implicitly represented in the
-interpreter's state, and is discarded when we exit. We probably want to ask
-for confirmation before leaving the function to give users a chance to "unskip"
-the forms in a TR position.
-
-Another question is whether the conventional stepper is really a good thing to
-imitate... How about an editor-based mouse-driven interface? Instead of
-"skipping" and "stepping", you would just designate the next form that you
-wanted to stop at. Instead of displaying return values, you replace the source
-text with the printed representation of the value.
-
-It would show the "program counter" by highlighting the *innermost* form that
-we are about to evaluate, i.e. the source form for the node that we are stopped
-at. It would probably also be useful to display the start of the form that was
-used to designate the next stopping point, although I guess this could be
-implied by the mouse position.
-
-
-Such an interface would be a little harder to implement than a dumb stepper,
-but it would be much easier to use. [It would be impossible for an evalhook
-stepper to do this.]
-
-
-%PRIMITIVE usage:
-
-Note: %PRIMITIVE can only be used in compiled code. It is a trapdoor into the
-compiler, not a general syntax for accessing "sub-primitives". It's main use
-is in implementation-dependent compiler transforms. It saves us the effort of
-defining a "phony function" (that is not really defined), and also allows
-direct communication with the code generator through codegen-info arguments.
-
-Some primitives may be exported from the VM so that %PRIMITIVE can be used to
-make it explicit that an escape routine or interpreter stub is assuming an
-operation is implemented by the compiler.
+++ /dev/null
-\chapter{Memory Management}
-\section{Stacks and Globals}
-\section{Heap Layout}
-\section{Garbage Collection}
-
-\chapter{Interface to C and Assembler}
-
-\chapter{Low-level debugging}
-
-\chapter{Core File Format}
+++ /dev/null
-% -*- Dictionary: design -*-
-
-\f
-\chapter{Virtual Machine Representation Introduction}
-
-\f
-\chapter{Global TN assignment}
-
-[\#\#\# Rename this phase so as not to be confused with the local/global TN
-representation.]
-
-The basic mechanism for closing over values is to pass the values as additional
-implicit arguments in the function call. This technique is only applicable
-when:
- -- the calling function knows which values the called function wants to close
- over, and
- -- the values to be closed over are available in the calling environment.
-
-The first condition is always true of local function calls. Environment
-analysis can guarantee that the second condition holds by closing over any
-needed values in the calling environment.
-
-If the function that closes over values may be called in an environment where
-the closed over values are not available, then we must store the values in a
-"closure" so that they are always accessible. Closures are called using the
-"full call" convention. When a closure is called, control is transferred to
-the "external entry point", which fetches the values out of the closure and
-then does a local call to the real function, passing the closure values as
-implicit arguments.
-
-In this scheme there is no such thing as a "heap closure variable" in code,
-since the closure values are moved into TNs by the external entry point. There
-is some potential for pessimization here, since we may end up moving the values
-from the closure into a stack memory location, but the advantages are also
-substantial. Simplicity is gained by always representing closure values the
-same way, and functions with closure references may still be called locally
-without allocating a closure. All the TN based VMR optimizations will apply
-to closure variables, since closure variables are represented in the same way
-as all other variables in VMR. Closure values will be allocated in registers
-where appropriate.
-
-Closures are created at the point where the function is referenced, eliminating
-the need to be able to close over closures. This lazy creation of closures has
-the additional advantage that when a closure reference is conditionally not
-done, then the closure consing will never be done at all. The corresponding
-disadvantage is that a closure over the same values may be created multiple
-times if there are multiple references. Note however, that VMR loop and common
-subexpression optimizations can eliminate redundant closure consing. In any
-case, multiple closures over the same variables doesn't seem to be that common.
-
-\#|
-Having the Tail-Info would also make return convention determination trivial.
-We could just look at the type, checking to see if it represents a fixed number
-of values. To determine if the standard return convention is necessary to
-preserve tail-recursion, we just iterate over the equivalent functions, looking
-for XEPs and uses in full calls.
-|\#
-
-The Global TN Assignment pass (GTN) can be considered a post-pass to
-environment analysis. This phase assigns the TNs used to hold local lexical
-variables and pass arguments and return values and determines the value-passing
-strategy used in local calls.
-
-To assign return locations, we look at the function's tail-set.
-
-If the result continuation for an entry point is used as the continuation for a
-full call, then we may need to constrain the continuation's values passing
-convention to the standard one. This is not necessary when the call is known
-not to be part of a tail-recursive loop (due to being a known function).
-
-Once we have figured out where we must use the standard value passing strategy,
-we can use a more flexible strategy to determine the return locations for local
-functions. We determine the possible numbers of return values from each
-function by examining the uses of all the result continuations in the
-equivalence class of the result continuation.
-
-If the tail-set type is for a fixed number of
-values, then we return that fixed number of values from all the functions whose
-result continuations are equated. If the number of values is not fixed, then
-we must use the unknown-values convention, although we are not forced to use
-the standard locations. We assign the result TNs at this time.
-
-We also use the tail-sets to see what convention we want to use. What we do is
-use the full convention for any function that has a XEP its tail-set, even if
-we aren't required to do so by a tail-recursive full call, as long as there are
-no non-tail-recursive local calls in the set. This prevents us from
-gratuitously using a non-standard convention when there is no reason to.
-
-\f
-\chapter{Local TN assignment}
-
-[Want a different name for this so as not to be confused with the different
-local/global TN representations. The really interesting stuff in this phase is
-operation selection, values representation selection, return strategy, etc.
-Maybe this phase should be conceptually lumped with GTN as "implementation
-selection", since GTN determines call strategies and locations.]
-
-\#|
-
-[\#\#\# I guess I believe that it is OK for VMR conversion to dick the ICR flow
-graph. An alternative would be to give VMR its very own flow graph, but that
-seems like overkill.
-
-In particular, it would be very nice if a TR local call looked exactly like a
-jump in VMR. This would allow loop optimizations to be done on loops written
-as recursions. In addition to making the call block transfer to the head of
-the function rather than to the return, we would also have to do something
-about skipping the part of the function prolog that moves arguments from the
-passing locations, since in a TR call they are already in the right frame.
-
-
-In addition to directly indicating whether a call should be coded with a TR
-variant, the Tail-P annotation flags non-call nodes that can directly return
-the value (an "advanced return"), rather than moving the value to the result
-continuation and jumping to the return code. Then (according to policy), we
-can decide to advance all possible returns. If all uses of the result are
-Tail-P, then LTN can annotate the result continuation as :Unused, inhibiting
-emission of the default return code.
-
-[\#\#\# But not really. Now there is a single list of templates, and a given
-template has only one policy.]
-
-In LTN, we use the :Safe template as a last resort even when the policy is
-unsafe. Note that we don't try :Fast-Safe; if this is also a good unsafe
-template, then it should have the unsafe policies explicitly specified.
-
-With a :Fast-Safe template, the result type must be proven to satisfy the
-output type assertion. This means that a fast-safe template with a fixnum
-output type doesn't need to do fixnum overflow checking. [\#\#\# Not right to
-just check against the Node-Derived-Type, since type-check intersects with
-this.]
-
-It seems that it would be useful to have a kind of template where the args must
-be checked to be fixnum, but the template checks for overflow and signals an
-error. In the case where an output assertion is present, this would generate
-better code than conditionally branching off to make a bignum, and then doing a
-type check on the result.
-
- How do we deal with deciding whether to do a fixnum overflow check? This
- is perhaps a more general problem with the interpretation of result type
- restrictions in templates. It would be useful to be able to discriminate
- between the case where the result has been proven to be a fixnum and where
- it has simply been asserted to be so.
-
- The semantics of result type restriction is that the result must be proven
- to be of that type *except* for safe generators, which are assumed to
- verify the assertion. That way "is-fixnum" case can be a fast-safe
- generator and the "should-be-fixnum" case is a safe generator. We could
- choose not to have a safe "should-be-fixnum" generator, and let the
- unrestricted safe generator handle it. We would then have to do an
- explicit type check on the result.
-
- In other words, for all template except Safe, a type restriction on either
- an argument or result means "this must be true; if it is not the system may
- break." In contrast, in a Safe template, the restriction means "If this is
- not true, I will signal an error."
-
- Since the node-derived-type only takes into consideration stuff that can be
- proved from the arguments, we can use the node-derived-type to select
- fast-safe templates. With unsafe policies, we don't care, since the code
- is supposed to be unsafe.
-
-|\#
-
-Local TN assignment (LTN) assigns all the TNs needed to represent the values of
-continuations. This pass scans over the code for the component, examining each
-continuation and its destination. A number of somewhat unrelated things are
-also done at the same time so that multiple passes aren't necessary.
- -- Determine the Primitive-Type for each continuation value and assigns TNs
- to hold the values.
- -- Use policy information to determine the implementation strategy for each
- call to a known function.
- -- Clear the type-check flags in continuations whose destinations have safe
- implementations.
- -- Determine the value-passing strategy for each continuation: known or
- unknown.
- -- Note usage of unknown-values continuations so that stack analysis can tell
- when stack values must be discarded.
-
-If safety is more important that speed and space, then we consider generating
-type checks on the values of nodes whose CONT has the Type-Check flag set. If
-the destinatation for the continuation value is safe, then we don't need to do
-a check. We assume that all full calls are safe, and use the template
-information to determine whether inline operations are safe.
-
-This phase is where compiler policy switches have most of their effect. The
-speed/space/safety tradeoff can determine which of a number of coding
-strategies are used. It is important to make the policy choice in VMR
-conversion rather than in code generation because the cost and storage
-requirement information which drives TNBIND will depend strongly on what actual
-VOP is chosen. In the case of +/FIXNUM, there might be three or more
-implementations, some optimized for speed, some for space, etc. Some of these
-VOPS might be open-coded and some not.
-
-We represent the implementation strategy for a call by either marking it as a
-full call or annotating it with a "template" representing the open-coding
-strategy. Templates are selected using a two-way dispatch off of operand
-primitive-types and policy. The general case of LTN is handled by the
-LTN-Annotate function in the function-info, but most functions are handled by a
-table-driven mechanism. There are four different translation policies that a
-template may have:
-\begin{description}
-\item[Safe]
- The safest implementation; must do argument type checking.
-
-\item[Small]
- The (unsafe) smallest implementation.
-
-\item[Fast]
- The (unsafe) fastest implementation.
-
-\item[Fast-Safe]
- An implementation optimized for speed, but which does any necessary
- checks exclusive of argument type checking. Examples are array bounds
- checks and fixnum overflow checks.
-\end{description}
-
-Usually a function will have only one or two distinct templates. Either or
-both of the safe and fast-safe templates may be omitted; if both are specified,
-then they should be distinct. If there is no safe template and our policy is
-safe, then we do a full call.
-
-We use four different coding strategies, depending on the policy:
-\begin{description}
-\item[Safe:] safety $>$ space $>$ speed, or
-we want to use the fast-safe template, but there isn't one.
-
-\item[Small:] space $>$ (max speed safety)
-
-\item[Fast:] speed $>$ (max space safety)
-
-\item[Fast-Safe (and type check):] safety $>$ speed $>$ space, or we want to use
-the safe template, but there isn't one.
-\end{description}
-
-``Space'' above is actually the maximum of space and cspeed, under the theory
-that less code will take less time to generate and assemble. [\#\#\# This could
-lose if the smallest case is out-of-line, and must allocate many linkage
-registers.]
-
-\f
-\chapter{Control optimization}
-
-In this phase we annotate blocks with drop-throughs. This controls how code
-generation linearizes code so that drop-throughs are used most effectively. We
-totally linearize the code here, allowing code generation to scan the blocks
-in the emit order.
-
-There are basically two aspects to this optimization:
- 1] Dynamically reducing the number of branches taken v.s. branches not
- taken under the assumption that branches not taken are cheaper.
- 2] Statically minimizing the number of unconditional branches, saving space
- and presumably time.
-
-These two goals can conflict, but if they do it seems pretty clear that the
-dynamic optimization should get preference. The main dynamic optimization is
-changing the sense of a conditional test so that the more commonly taken branch
-is the fall-through case. The problem is determining which branch is more
-commonly taken.
-
-The most clear-cut case is where one branch leads out of a loop and the other
-is within. In this case, clearly the branch within the loop should be
-preferred. The only added complication is that at some point in the loop there
-has to be a backward branch, and it is preferable for this branch to be
-conditional, since an unconditional branch is just a waste of time.
-
-In the absence of such good information, we can attempt to guess which branch
-is more popular on the basis of difference in the cost between the two cases.
-Min-max strategy suggests that we should choose the cheaper alternative, since
-the percentagewise improvement is greater when the branch overhead is
-significant with respect to the cost of the code branched to. A tractable
-approximation of this is to compare only the costs of the two blocks
-immediately branched to, since this would avoid having to do any hairy graph
-walking to find all the code for the consequent and the alternative. It might
-be worthwhile discriminating against ultra-expensive functions such as ERROR.
-
-For this to work, we have to detect when one of the options is empty. In this
-case, the next for one branch is a successor of the other branch, making the
-comparison meaningless. We use dominator information to detect this situation.
-When a branch is empty, one of the predecessors of the first block in the empty
-branch will be dominated by the first block in the other branch. In such a
-case we favor the empty branch, since that's about as cheap as you can get.
-
-Statically minimizing branches is really a much more tractable problem, but
-what literature there is makes it look hard. Clearly the thing to do is to use
-a non-optimal heuristic algorithm.
-
-A good possibility is to use an algorithm based on the depth first ordering.
-We can modify the basic DFO algorithm so that it chooses an ordering which
-favors any drop-thrus that we may choose for dynamic reasons. When we are
-walking the graph, we walk the desired drop-thru arc last, which will place it
-immediately after us in the DFO unless the arc is a retreating arc.
-
-We scan through the DFO and whenever we find a block that hasn't been done yet,
-we build a straight-line segment by setting the drop-thru to the unreached
-successor block which has the lowest DFN greater than that for the block. We
-move to the drop-thru block and repeat the process until there is no such
-block. We then go back to our original scan through the DFO, looking for the
-head of another straight-line segment.
-
-This process will automagically implement all of the dynamic optimizations
-described above as long as we favor the appropriate IF branch when creating the
-DFO. Using the DFO will prevent us from making the back branch in a loop the
-drop-thru, but we need to be clever about favoring IF branches within loops
-while computing the DFO. The IF join will be favored without any special
-effort, since we follow through the most favored path until we reach the end.
-
-This needs some knowledge about the target machine, since on most machines
-non-tail-recursive calls will use some sort of call instruction. In this case,
-the call actually wants to drop through to the return point, rather than
-dropping through to the beginning of the called function.
-
-\f
-\chapter{VMR conversion}
-
-\#|
-Single-use let var continuation substitution not really correct, since it can
-cause a spurious type error. Maybe we do want stuff to prove that an NLX can't
-happen after all. Or go back to the idea of moving a combination arg to the
-ref location, and having that use the ref cont (with its output assertion.)
-This lossage doesn't seem very likely to actually happen, though.
-[\#\#\# must-reach stuff wouldn't work quite as well as combination substitute in
-psetq, etc., since it would fail when one of the new values is random code
-(might unwind.)]
-
-Is this really a general problem with eager type checking? It seems you could
-argue that there was no type error in this code:
- (+ :foo (throw 'up nil))
-But we would signal an error.
-
-
-Emit explicit you-lose operation when we do a move between two non-T ptypes,
-even when type checking isn't on. Can this really happen? Seems we should
-treat continuations like this as though type-check was true. Maybe LTN should
-leave type-check true in this case, even when the policy is unsafe. (Do a type
-check against NIL?)
-
-At continuation use time, we may in general have to do both a coerce-to-t and a
-type check, allocating two temporary TNs to hold the intermediate results.
-
-
-VMR Control representation:
-
-We represent all control transfer explicitly. In particular, :Conditional VOPs
-take a single Target continuation and a Not-P flag indicating whether the sense
-of the test is negated. Then an unconditional Branch VOP will be emitted
-afterward if the other path isn't a drop-through.
-
-So we linearize the code before VMR-conversion. This isn't a problem,
-since there isn't much change in control flow after VMR conversion (none until
-loop optimization requires introduction of header blocks.) It does make
-cost-based branch prediction a bit ucky, though, since we don't have any cost
-information in ICR. Actually, I guess we do have pretty good cost information
-after LTN even before VMR conversion, since the most important thing to know is
-which functions are open-coded.
-
-|\#
-
-VMR preserves the block structure of ICR, but replaces the nodes with a target
-dependent virtual machine (VM) representation. Different implementations may
-use different VMs without making major changes in the back end. The two main
-components of VMR are Temporary Names (TNs) and Virtual OPerations (VOPs). TNs
-represent the locations that hold values, and VOPs represent the operations
-performed on the values.
-
-A "primitive type" is a type meaningful at the VM level. Examples are Fixnum,
-String-Char, Short-Float. During VMR conversion we use the primitive type of
-an expression to determine both where we can store the result of the expression
-and which type-specific implementations of an operation can be applied to the
-value. [Ptype is a set of SCs == representation choices and representation
-specific operations]
-
-The VM specific definitions provide functions that do stuff like find the
-primitive type corresponding to a type and test for primitive type subtypep.
-Usually primitive types will be disjoint except for T, which represents all
-types.
-
-The primitive type T is special-cased. Not only does it overlap with all the
-other types, but it implies a descriptor ("boxed" or "pointer") representation.
-For efficiency reasons, we sometimes want to use
-alternate representations for some objects such as numbers. The majority of
-operations cannot exploit alternate representations, and would only be
-complicated if they had to be able to convert alternate representations into
-descriptors. A template can require an operand to be a descriptor by
-constraining the operand to be of type T.
-
-A TN can only represent a single value, so we bare the implementation of MVs at
-this point. When we know the number of multiple values being handled, we use
-multiple TNs to hold them. When the number of values is actually unknown, we
-use a convention that is compatible with full function call.
-
-Everything that is done is done by a VOP in VMR. Calls to simple primitive
-functions such as + and CAR are translated to VOP equivalents by a table-driven
-mechanism. This translation is specified by the particular VM definition; VMR
-conversion makes no assumptions about which operations are primitive or what
-operand types are worth special-casing. The default calling mechanisms and
-other miscellaneous builtin features are implemented using standard VOPs that
-must implemented by each VM.
-
-Type information can be forgotten after VMR conversion, since all type-specific
-operation selections have been made.
-
-Simple type checking is explicitly done using CHECK-xxx VOPs. They act like
-innocuous effectless/unaffected VOPs which return the checked thing as a
-result. This allows loop-invariant optimization and common subexpression
-elimination to remove redundant checks. All type checking is done at the time
-the continuation is used.
-
-Note that we need only check asserted types, since if type inference works, the
-derived types will also be satisfied. We can check whichever is more
-convenient, since both should be true.
-
-Constants are turned into special Constant TNs, which are wired down in a SC
-that is determined by their type. The VM definition provides a function that
-returns constant a TN to represent a Constant Leaf.
-
-Each component has a constant pool. There is a register dedicated to holding
-the constant pool for the current component. The back end allocates
-non-immediate constants in the constant pool when it discovers them during
-translation from ICR.
-
-[\#\#\# Check that we are describing what is actually implemented. But this
-really isn't very good in the presence of interesting unboxed
-representations...]
-Since LTN only deals with values from the viewpoint of the receiver, we must be
-prepared during the translation pass to do stuff to the continuation at the
-time it is used.
- -- If a VOP yields more values than are desired, then we must create TNs to
- hold the discarded results. An important special-case is continuations
- whose value is discarded. These continuations won't be annotated at all.
- In the case of a Ref, we can simply skip evaluation of the reference when
- the continuation hasn't been annotated. Although this will eliminate
- bogus references that for some reason weren't optimized away, the real
- purpose is to handle deferred references.
- -- If a VOP yields fewer values than desired, then we must default the extra
- values to NIL.
- -- If a continuation has its type-check flag set, then we must check the type
- of the value before moving it into the result location. In general, this
- requires computing the result in a temporary, and having the type-check
- operation deliver it in the actual result location.
- -- If the template's result type is T, then we must generate a boxed
- temporary to compute the result in when the continuation's type isn't T.
-
-
-We may also need to do stuff to the arguments when we generate code for a
-template. If an argument continuation isn't annotated, then it must be a
-deferred reference. We use the leaf's TN instead. We may have to do any of
-the above use-time actions also. Alternatively, we could avoid hair by not
-deferring references that must be type-checked or may need to be boxed.
-
-\f
-\section{Stack analysis}
-
-Think of this as a lifetime problem: a values generator is a write and a values
-receiver is a read. We want to annotate each VMR-Block with the unknown-values
-continuations that are live at that point. If we do a control transfer to a
-place where fewer continuations are live, then we must deallocate the newly
-dead continuations.
-
-We want to convince ourselves that values deallocation based on lifetime
-analysis actually works. In particular, we need to be sure that it doesn't
-violate the required stack discipline. It is clear that it is impossible to
-deallocate the values before they become dead, since later code may decide to
-use them. So the only thing we need to ensure is that the "right" time isn't
-later than the time that the continuation becomes dead.
-
-The only reason why we couldn't deallocate continuation A as soon as it becomes
-dead would be that there is another continuation B on top of it that isn't dead
-(since we can only deallocate the topmost continuation).
-
-The key to understanding why this can't happen is that each continuation has
-only one read (receiver). If B is on top of A, then it must be the case that A
-is live at the receiver for B. This means that it is impossible for B to be
-live without A being live.
-
-
-The reason that we don't solve this problem using a normal iterative flow
-analysis is that we also need to know the ordering of the continuations on the
-stack so that we can do deallocation. When it comes time to discard values, we
-want to know which discarded continuation is on the bottom so that we can reset
-SP to its start.
-
-[I suppose we could also decrement SP by the aggregate size of the discarded
-continuations.] Another advantage of knowing the order in which we expect
-continuations to be on the stack is that it allows us to do some consistency
-checking. Also doing a localized graph walk around the values-receiver is
-likely to be much more efficient than doing an iterative flow analysis problem
-over all the code in the component (not that big a consideration.)
-
-
-
-\#|
-Actually, what we do is do a backward graph walk from each unknown-values
-receiver. As we go, we mark each walked block with ther ordered list of
-continuations we believe are on the stack. Starting with an empty stack, we:
- -- When we encounter another unknown-values receiver, we push that
- continuation on our simulated stack.
- -- When we encounter a receiver (which had better be for the topmost
- continuation), we pop that continuation.
- -- When we pop all continuations, we terminate our walk.
-
-[\#\#\# not quite right... It seems we may run into "dead values" during the
-graph walk too. It seems that we have to check if the pushed continuation is
-on stack top, and if not, add it to the ending stack so that the post-pass will
-discard it.]
-
-
-
-[\#\#\# Also, we can't terminate our walk just because we hit a block previously
-walked. We have to compare the the End-Stack with the values received along
-the current path: if we have more values on our current walk than on the walk
-that last touched the block, then we need to re-walk the subgraph reachable
-from from that block, using our larger set of continuations. It seems that our
-actual termination condition is reaching a block whose End-Stack is already EQ
-to our current stack.]
-
-
-
-
-
-If at the start, the block containing the values receiver has already been
-walked, the we skip the walk for that continuation, since it has already been
-handled by an enclosing values receiver. Once a walk has started, we
-ignore any signs of a previous walk, clobbering the old result with our own,
-since we enclose that continuation, and the previous walk doesn't take into
-consideration the fact that our values block underlies its own.
-
-When we are done, we have annotated each block with the stack current both at
-the beginning and at the end of that block. Blocks that aren't walked don't
-have anything on the stack either place (although they may hack MVs
-internally).
-
-We then scan all the blocks in the component, looking for blocks that have
-predecessors with a different ending stack than that block's starting stack.
-(The starting stack had better be a tail of the predecessor's ending stack.)
-We insert a block intervening between all of these predecessors that sets SP to
-the end of the values for the continuation that should be on stack top. Of
-course, this pass needn't be done if there aren't any global unknown MVs.
-
-Also, if we find any block that wasn't reached during the walk, but that USEs
-an outside unknown-values continuation, then we know that the DEST can't be
-reached from this point, so the values are unused. We either insert code to
-pop the values, or somehow mark the code to prevent the values from ever being
-pushed. (We could cause the popping to be done by the normal pass if we
-iterated over the pushes beforehand, assigning a correct END-STACK.)
-
-[\#\#\# But I think that we have to be a bit clever within blocks, given the
-possibility of blocks being joined. We could collect some unknown MVs in a
-block, then do a control transfer out of the receiver, and this control
-transfer could be squeezed out by merging blocks. How about:
-
- (tagbody
- (return
- (multiple-value-prog1 (foo)
- (when bar
- (go UNWIND))))
-
- UNWIND
- (return
- (multiple-value-prog1 (baz)
- bletch)))
-
-But the problem doesn't happen here (can't happen in general?) since a node
-buried within a block can't use a continuation outside of the block. In fact,
-no block can have more then one PUSH continuation, and this must always be be
-last continuation. So it is trivially (structurally) true that all pops come
-before any push.
-
-[\#\#\# But not really: the DEST of an embedded continuation may be outside the
-block. There can be multiple pushes, and we must find them by iterating over
-the uses of MV receivers in LTN. But it would be hard to get the order right
-this way. We could easily get the order right if we added the generators as we
-saw the uses, except that we can't guarantee that the continuations will be
-annotated at that point. (Actually, I think we only need the order for
-consistency checks, but that is probably worthwhile). I guess the thing to do
-is when we process the receiver, add the generator blocks to the
-Values-Generators, then do a post-pass that re-scans the blocks adding the
-pushes.]
-
-I believe that above concern with a dead use getting mashed inside a block
-can't happen, since the use inside the block must be the only use, and if the
-use isn't reachable from the push, then the use is totally unreachable, and
-should have been deleted, which would prevent the prevent it from ever being
-annotated.
-]
-]
-|\#
-
-We find the partial ordering of the values globs for unknown values
-continuations in each environment. We don't have to scan the code looking for
-unknown values continuations since LTN annotates each block with the
-continuations that were popped and not pushed or pushed and not popped. This
-is all we need to do the inter-block analysis.
-
-After we have found out what stuff is on the stack at each block boundary, we
-look for blocks with predecessors that have junk on the stack. For each such
-block, we introduce a new block containing code to restore the stack pointer.
-Since unknown-values continuations are represented as <start, count>, we can
-easily pop a continuation using the Start TN.
-
-Note that there is only doubt about how much stuff is on the control stack,
-since only it is used for unknown values. Any special stacks such as number
-stacks will always have a fixed allocation.
-
-\f
-\section{Non-local exit}
-
-
-If the starting and ending continuations are not in the same environment, then
-the control transfer is a non-local exit. In this case just call Unwind with
-the appropriate stack pointer, and let the code at the re-entry point worry
-about fixing things up.
-
-It seems like maybe a good way to organize VMR conversion of NLX would be to
-have environment analysis insert funny functions in new interposed cleanup
-blocks. The thing is that we need some way for VMR conversion to:
- 1] Get its hands on the returned values.
- 2] Do weird control shit.
- 3] Deliver the values to the original continuation destination.
-I.e. we need some way to interpose arbitrary code in the path of value
-delivery.
-
-What we do is replace the NLX uses of the continuation with another
-continuation that is received by a MV-Call to %NLX-VALUES in a cleanup block
-that is interposed between the NLX uses and the old continuation's block. The
-MV-Call uses the original continuation to deliver it's values to.
-
-[Actually, it's not really important that this be an MV-Call, since it has to
-be special-cased by LTN anyway. Or maybe we would want it to be an MV call.
-If did normal LTN analysis of an MV call, it would force the returned values
-into the unknown values convention, which is probably pretty convenient for use
-in NLX.
-
-Then the entry code would have to use some special VOPs to receive the unknown
-values. But we probably need special VOPs for NLX entry anyway, and the code
-can share with the call VOPs. Also we probably need the technology anyway,
-since THROW will use truly unknown values.]
-
-
-On entry to a dynamic extent that has non-local-exists into it (always at an
-ENTRY node), we take a complete snapshot of the dynamic state:
- the top pointers for all stacks
- current Catch and Unwind-Protect
- current special binding (binding stack pointer in shallow binding)
-
-We insert code at the re-entry point which restores the saved dynamic state.
-All TNs live at a NLX EP are forced onto the stack, so we don't have to restore
-them, and we don't have to worry about getting them saved.
-
+++ /dev/null
-\chapter{Object Format}
-
-\f
-\section{Tagging}
-
-The following is a key of the three bit low-tagging scheme:
-\begin{description}
- \item[000] even fixnum
- \item[001] function pointer
- \item[010] even other-immediate (header-words, characters, symbol-value trap value, etc.)
- \item[011] list pointer
- \item[100] odd fixnum
- \item[101] structure pointer
- \item[110] odd other immediate
- \item[111] other-pointer to data-blocks (other than conses, structures,
- and functions)
-\end{description}
-
-This tagging scheme forces a dual-word alignment of data-blocks on the heap,
-but this can be pretty negligible:
-\begin{itemize}
-\item RATIOS and COMPLEX must have a header-word anyway since they are not a
- major type. This wastes one word for these infrequent data-blocks since
- they require two words for the data.
-
-\item BIGNUMS must have a header-word and probably contain only one other word
- anyway, so we probably don't waste any words here. Most bignums just
- barely overflow fixnums, that is by a bit or two.
-
-\item Single and double FLOATS?
- no waste, or
- one word wasted
-
-\item SYMBOLS have a pad slot (current called the setf function, but unused.)
-\end{itemize}
-Everything else is vector-like including code, so these probably take up
-so many words that one extra one doesn't matter.
-
-
-\f
-\section{GC Comments}
-
-Data-Blocks comprise only descriptors, or they contain immediate data and raw
-bits interpreted by the system. GC must skip the latter when scanning the
-heap, so it does not look at a word of raw bits and interpret it as a pointer
-descriptor. These data-blocks require headers for GC as well as for operations
-that need to know how to interpret the raw bits. When GC is scanning, and it
-sees a header-word, then it can determine how to skip that data-block if
-necessary. Header-Words are tagged as other-immediates. See the sections
-"Other-Immediates" and "Data-Blocks and Header-Words" for comments on
-distinguishing header-words from other-immediate data. This distinction is
-necessary since we scan through data-blocks containing only descriptors just as
-we scan through the heap looking for header-words introducing data-blocks.
-
-Data-Blocks containing only descriptors do not require header-words for GC
-since the entire data-block can be scanned by GC a word at a time, taking
-whatever action is necessary or appropriate for the data in that slot. For
-example, a cons is referenced by a descriptor with a specific tag, and the
-system always knows the size of this data-block. When GC encounters a pointer
-to a cons, it can transport it into the new space, and when scanning, it can
-simply scan the two words manifesting the cons interpreting each word as a
-descriptor. Actually there is no cons tag, but a list tag, so we make sure the
-cons is not nil when appropriate. A header may still be desired if the pointer
-to the data-block does not contain enough information to adequately maintain
-the data-block. An example of this is a simple-vector containing only
-descriptor slots, and we attach a header-word because the descriptor pointing
-to the vector lacks necessary information -- the type of the vector's elements,
-its length, etc.
-
-There is no need for a major tag for GC forwarding pointers. Since the tag
-bits are in the low end of the word, a range check on the start and end of old
-space tells you if you need to move the thing. This is all GC overhead.
-
-
-\f
-\section{Structures}
-
-A structure descriptor has the structure lowtag type code, making
-{\tt structurep} a fast operation. A structure
-data-block has the following format:
-\begin{verbatim}
- -------------------------------------------------------
- | length (24 bits) | Structure header type (8 bits) |
- -------------------------------------------------------
- | structure type name (a symbol) |
- -------------------------------------------------------
- | structure slot 0 |
- -------------------------------------------------------
- | ... structure slot length - 2 |
- -------------------------------------------------------
-\end{verbatim}
-
-The header word contains the structure length, which is the number of words
-(other than the header word.) The length is always at least one, since the
-first word of the structure data is the structure type name.
-
-\f
-\section{Fixnums}
-
-A fixnum has one of the following formats in 32 bits:
-\begin{verbatim}
- -------------------------------------------------------
- | 30 bit 2's complement even integer | 0 0 0 |
- -------------------------------------------------------
-\end{verbatim}
-or
-\begin{verbatim}
- -------------------------------------------------------
- | 30 bit 2's complement odd integer | 1 0 0 |
- -------------------------------------------------------
-\end{verbatim}
-
-Effectively, there is one tag for immediate integers, two zeros. This buys one
-more bit for fixnums, and now when these numbers index into simple-vectors or
-offset into memory, they point to word boundaries on 32-bit, byte-addressable
-machines. That is, no shifting need occur to use the number directly as an
-offset.
-
-This format has another advantage on byte-addressable machines when fixnums are
-offsets into vector-like data-blocks, including structures. Even though we
-previously mentioned data-blocks are dual-word aligned, most indexing and slot
-accessing is word aligned, and so are fixnums with effectively two tag bits.
-
-Two tags also allow better usage of special instructions on some machines that
-can deal with two low-tag bits but not three.
-
-Since the two bits are zeros, we avoid having to mask them off before using the
-words for arithmetic, but division and multiplication require special shifting.
-
-
-\f
-\section{Other-immediates}
-
-As for fixnums, there are two different three-bit lowtag codes for
-other-immediate, allowing 64 other-immediate types:
-\begin{verbatim}
-----------------------------------------------------------------
-| Data (24 bits) | Type (8 bits with low-tag) | 1 0 |
-----------------------------------------------------------------
-\end{verbatim}
-
-The type-code for an other-immediate type is considered to include the two
-lowtag bits. This supports the concept of a single "type code" namespace for
-all descriptors, since the normal lowtag codes are disjoint from the
-other-immediate codes.
-
-For other-pointer objects, the full eight bits of the header type code are used
-as the type code for that kind of object. This is why we use two lowtag codes
-for other-immediate types: each other-pointer object needs a distinct
-other-immediate type to mark its header.
-
-The system uses the other-immediate format for characters,
-the {\tt symbol-value} unbound trap value, and header-words for data-blocks on
-the heap. The type codes are laid out to facilitate range checks for common
-subtypes; for example, all numbers will have contiguous type codes which are
-distinct from the contiguous array type codes. See section
-\ref{data-blocks-and-o-i} for details.
-
-\f
-\section{Data-Blocks and Header-Word Format}
-
-Pointers to data-blocks have the following format:
-\begin{verbatim}
-----------------------------------------------------------------
-| Dual-word address of data-block (29 bits) | 1 1 1 |
-----------------------------------------------------------------
-\end{verbatim}
-
-The word pointed to by the above descriptor is a header-word, and it has the
-same format as an other-immediate:
-\begin{verbatim}
-----------------------------------------------------------------
-| Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
-----------------------------------------------------------------
-\end{verbatim}
-This is convenient for scanning the heap when GC'ing, but it does mean that
-whenever GC encounters an other-immediate word, it has to do a range check on
-the low byte to see if it is a header-word or just a character (for example).
-This is easily acceptable performance hit for scanning.
-
-The system interprets the data portion of the header-word for non-vector
-data-blocks as the word length excluding the header-word. For example, the
-data field of the header for ratio and complex numbers is two, one word each
-for the numerator and denominator or for the real and imaginary parts.
-
-For vectors and data-blocks representing Lisp objects stored like vectors, the
-system ignores the data portion of the header-word:
-\begin{verbatim}
-----------------------------------------------------------------
-| Unused Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
-----------------------------------------------------------------
-| Element Length of Vector (30 bits) | 0 0 |
-----------------------------------------------------------------
-\end{verbatim}
-
-Using a separate word allows for much larger vectors, and it allows {\tt
-length} to simply access a single word without masking or shifting. Similarly,
-the header for complex arrays and vectors has a second word, following the
-header-word, the system uses for the fill pointer, so computing the length of
-any array is the same code sequence.
-
-
-\f
-\section{Data-Blocks and Other-immediates Typing}
-
-\label{data-blocks-and-o-i}
-These are the other-immediate types. We specify them including all low eight
-bits, including the other-immediate tag, so we can think of the type bits as
-one type -- not an other-immediate major type and a subtype. Also, fetching a
-byte and comparing it against a constant is more efficient than wasting even a
-small amount of time shifting out the other-immediate tag to compare against a
-five bit constant.
-\begin{verbatim}
-Number (< 30)
- bignum 10
- ratio 14
- single-float 18
- double-float 22
- complex 26
-
-Array (>= 30 code 86)
- Simple-Array (>= 20 code 70)
- simple-array 30
- Vector (>= 34 code 82)
- simple-string 34
- simple-bit-vector 38
- simple-vector 42
- (simple-array (unsigned-byte 2) (*)) 46
- (simple-array (unsigned-byte 4) (*)) 50
- (simple-array (unsigned-byte 8) (*)) 54
- (simple-array (unsigned-byte 16) (*)) 58
- (simple-array (unsigned-byte 32) (*)) 62
- (simple-array single-float (*)) 66
- (simple-array double-float (*)) 70
- complex-string 74
- complex-bit-vector 78
- (array * (*)) -- general complex vector. 82
- complex-array 86
-
-code-header-type 90
-function-header-type 94
-closure-header-type 98
-funcallable-instance-header-type 102
-unused-function-header-1-type 106
-unused-function-header-2-type 110
-unused-function-header-3-type 114
-closure-function-header-type 118
-return-pc-header-type (a.k.a LRA) 122
-value-cell-header-type 126
-symbol-header-type 130
-base-character-type 134
-system-area-pointer-type (header type) 138
-unbound-marker 142
-weak-pointer-type 146
-structure-header-type 150
-\end{verbatim}
-\f
-\section{Strings}
-
-All strings in the system are C-null terminated. This saves copying the bytes
-when calling out to C. The only time this wastes memory is when the string
-contains a multiple of eight characters, and then the system allocates two more
-words (since Lisp objects are dual-word aligned) to hold the C-null byte.
-Since the system will make heavy use of C routines for systems calls and
-libraries that save reimplementation of higher level operating system
-functionality (such as pathname resolution or current directory computation),
-saving on copying strings for C should make C call out more efficient.
-
-The length word in a string header, see section "Data-Blocks and Header-Word
-Format", counts only the characters truly in the Common Lisp string.
-Allocation and GC will have to know to handle the extra C-null byte, and GC
-already has to deal with rounding up various objects to dual-word alignment.
-
-
-\f
-\section{Symbols and NIL}
-
-Symbol data-block has the following format:
-\begin{verbatim}
--------------------------------------------------------
-| 7 (data-block words) | Symbol Type (8 bits) |
--------------------------------------------------------
-| Value Descriptor |
--------------------------------------------------------
-| Function Pointer |
--------------------------------------------------------
-| Raw Function Address |
--------------------------------------------------------
-| Setf Function |
--------------------------------------------------------
-| Property List |
--------------------------------------------------------
-| Print Name |
--------------------------------------------------------
-| Package |
--------------------------------------------------------
-\end{verbatim}
-
-Most of these slots are self-explanatory given what symbols must do in Common
-Lisp, but a couple require comments. We added the Raw Function Address slot to
-speed up named call which is the most common calling convention. This is a
-non-descriptor slot, but since objects are dual word aligned, the value
-inherently has fixnum low-tag bits. The GC method for symbols must know to
-update this slot. The Setf Function slot is currently unused, but we had an
-extra slot due to adding Raw Function Address since objects must be dual-word
-aligned.
-
-The issues with nil are that we want it to act like a symbol, and we need list
-operations such as CAR and CDR to be fast on it. CMU Common Lisp solves this
-by putting nil as the first object in static space, where other global values
-reside, so it has a known address in the system:
-\begin{verbatim}
-------------------------------------------------------- <-- space
-| 0 | start
--------------------------------------------------------
-| 7 (data-block words) | Symbol Type (8 bits) |
-------------------------------------------------------- <-- nil
-| Value/CAR |
--------------------------------------------------------
-| Definition/CDR |
--------------------------------------------------------
-| Raw Function Address |
--------------------------------------------------------
-| Setf Function |
--------------------------------------------------------
-| Property List |
--------------------------------------------------------
-| Print Name |
--------------------------------------------------------
-| Package |
--------------------------------------------------------
-| ... |
--------------------------------------------------------
-\end{verbatim}
-In addition, we make the list typed pointer to nil actually point past the
-header word of the nil symbol data-block. This has usefulness explained below.
-The value and definition of nil are nil. Therefore, any reference to nil used
-as a list has quick list type checking, and CAR and CDR can go right through
-the first and second words as if nil were a cons object.
-
-When there is a reference to nil used as a symbol, the system adds offsets to
-the address the same as it does for any symbol. This works due to a
-combination of nil pointing past the symbol header-word and the chosen list and
-other-pointer type tags. The list type tag is four less than the other-pointer
-type tag, but nil points four additional bytes into its symbol data-block.
-
-
-\f
-;;;; Array Headers.
-
-The array-header data-block has the following format:
-\begin{verbatim}
-----------------------------------------------------------------
-| Header Len (24 bits) = Array Rank +5 | Array Type (8 bits) |
-----------------------------------------------------------------
-| Fill Pointer (30 bits) | 0 0 |
-----------------------------------------------------------------
-| Available Elements (30 bits) | 0 0 |
-----------------------------------------------------------------
-| Data Vector (29 bits) | 1 1 1 |
-----------------------------------------------------------------
-| Displacement (30 bits) | 0 0 |
-----------------------------------------------------------------
-| Displacedp (29 bits) -- t or nil | 1 1 1 |
-----------------------------------------------------------------
-| Range of First Index (30 bits) | 0 0 |
-----------------------------------------------------------------
- .
- .
- .
-
-\end{verbatim}
-The array type in the header-word is one of the eight-bit patterns from section
-"Data-Blocks and Other-immediates Typing", indicating that this is a complex
-string, complex vector, complex bit-vector, or a multi-dimensional array. The
-data portion of the other-immediate word is the length of the array header
-data-block. Due to its format, its length is always five greater than the
-array's number of dimensions. The following words have the following
-interpretations and types:
-\begin{description}
- \item[Fill Pointer:]
- This is a fixnum indicating the number of elements in the data vector
- actually in use. This is the logical length of the array, and it is
- typically the same value as the next slot. This is the second word, so
- LENGTH of any array, with or without an array header, is just four bytes
- off the pointer to it.
- \item[Available Elements:]
- This is a fixnum indicating the number of elements for which there is
- space in the data vector. This is greater than or equal to the logical
- length of the array when it is a vector having a fill pointer.
- \item[Data Vector:]
- This is a pointer descriptor referencing the actual data of the array.
- This a data-block whose first word is a header-word with an array type as
- described in sections "Data-Blocks and Header-Word Format" and
- "Data-Blocks and Other-immediates Typing"
- \item[Displacement:]
- This is a fixnum added to the computed row-major index for any array.
- This is typically zero.
- \item[Displacedp:]
- This is either t or nil. This is separate from the displacement slot, so
- most array accesses can simply add in the displacement slot. The rare
- need to know if an array is displaced costs one extra word in array
- headers which probably aren't very frequent anyway.
- \item[Range of First Index:]
- This is a fixnum indicating the number of elements in the first dimension
- of the array. Legal index values are zero to one less than this number
- inclusively. IF the array is zero-dimensional, this slot is
- non-existent.
- \item[... (remaining slots):]
- There is an additional slot in the header for each dimension of the
- array. These are the same as the Range of First Index slot.
-\end{description}
-
-\f
-\section{Bignums}
-
-Bignum data-blocks have the following format:
-\begin{verbatim}
--------------------------------------------------------
-| Length (24 bits) | Bignum Type (8 bits) |
--------------------------------------------------------
-| least significant bits |
--------------------------------------------------------
- .
- .
- .
-\end{verbatim}
-The elements contain the two's complement representation of the integer with
-the least significant bits in the first element or closer to the header. The
-sign information is in the high end of the last element.
-
-
-
-\f
-\section{Code Data-Blocks}
-
-A code data-block is the run-time representation of a "component". A component
-is a connected portion of a program's flow graph that is compiled as a single
-unit, and it contains code for many functions. Some of these functions are
-callable from outside of the component, and these are termed "entry points".
-
-Each entry point has an associated user-visible function data-block (of type
-{\tt function}). The full call convention provides for calling an entry point
-specified by a function object.
-
-Although all of the function data-blocks for a component's entry points appear
-to the user as distinct objects, the system keeps all of the code in a single
-code data-block. The user-visible function object is actually a pointer into
-the middle of a code data-block. This allows any control transfer within a
-component to be done using a relative branch.
-
-Besides a function object, there are other kinds of references into the middle
-of a code data-block. Control transfer into a function also occurs at the
-return-PC for a call. The system represents a return-PC somewhat similarly to
-a function, so GC can also recognize a return-PC as a reference to a code
-data-block. This representation is known as a Lisp Return Address (LRA).
-
-It is incorrect to think of a code data-block as a concatenation of "function
-data-blocks". Code for a function is not emitted in any particular order with
-respect to that function's function-header (if any). The code following a
-function-header may only be a branch to some other location where the
-function's "real" definition is.
-
-
-The following are the three kinds of pointers to code data-blocks:
-\begin{description}
- \item[Code pointer (labeled A below):]
- A code pointer is a descriptor, with other-pointer low-tag bits, pointing
- to the beginning of the code data-block. The code pointer for the
- currently running function is always kept in a register (CODE). In
- addition to allowing loading of non-immediate constants, this also serves
- to represent the currently running function to the debugger.
- \item[LRA (labeled B below):]
- The LRA is a descriptor, with other-pointer low-tag bits, pointing
- to a location for a function call. Note that this location contains no
- descriptors other than the one word of immediate data, so GC can treat
- LRA locations the same as instructions.
- \item[Function (labeled C below):]
- A function is a descriptor, with function low-tag bits, that is user
- callable. When a function header is referenced from a closure or from
- the function header's self-pointer, the pointer has other-pointer low-tag
- bits, instead of function low-tag bits. This ensures that the internal
- function data-block associated with a closure appears to be uncallable
- (although users should never see such an object anyway).
-
- Information about functions that is only useful for entry points is kept
- in some descriptors following the function's self-pointer descriptor.
- All of these together with the function's header-word are known as the
- "function header". GC must be able to locate the function header. We
- provide for this by chaining together the function headers in a NIL
- terminated list kept in a known slot in the code data-block.
-\end{description}
-
-A code data-block has the following format:
-\begin{verbatim}
-A -->
-****************************************************************
-| Header-Word count (24 bits) | Code-Type (8 bits) |
-----------------------------------------------------------------
-| Number of code words (fixnum tag) |
-----------------------------------------------------------------
-| Pointer to first function header (other-pointer tag) |
-----------------------------------------------------------------
-| Debug information (structure tag) |
-----------------------------------------------------------------
-| First constant (a descriptor) |
-----------------------------------------------------------------
-| ... |
-----------------------------------------------------------------
-| Last constant (and last word of code header) |
-----------------------------------------------------------------
-| Some instructions (non-descriptor) |
-----------------------------------------------------------------
-| (pad to dual-word boundary if necessary) |
-
-B -->
-****************************************************************
-| Word offset from code header (24) | Return-PC-Type (8) |
-----------------------------------------------------------------
-| First instruction after return |
-----------------------------------------------------------------
-| ... more code and LRA header-words |
-----------------------------------------------------------------
-| (pad to dual-word boundary if necessary) |
-
-C -->
-****************************************************************
-| Offset from code header (24) | Function-Header-Type (8) |
-----------------------------------------------------------------
-| Self-pointer back to previous word (with other-pointer tag) |
-----------------------------------------------------------------
-| Pointer to next function (other-pointer low-tag) or NIL |
-----------------------------------------------------------------
-| Function name (a string or a symbol) |
-----------------------------------------------------------------
-| Function debug arglist (a string) |
-----------------------------------------------------------------
-| Function type (a list-style function type specifier) |
-----------------------------------------------------------------
-| Start of instructions for function (non-descriptor) |
-----------------------------------------------------------------
-| More function headers and instructions and return PCs, |
-| until we reach the total size of header-words + code |
-| words. |
-----------------------------------------------------------------
-\end{verbatim}
-
-The following are detailed slot descriptions:
-\begin{description}
- \item[Code data-block header-word:]
- The immediate data in the code data-block's header-word is the number of
- leading descriptors in the code data-block, the fixed overhead words plus
- the number of constants. The first non-descriptor word, some code,
- appears at this word offset from the header.
- \item[Number of code words:]
- The total number of non-header-words in the code data-block. The total
- word size of the code data-block is the sum of this slot and the
- immediate header-word data of the previous slot.
- header-word.
- \item[Pointer to first function header:]
- A NIL-terminated list of the function headers for all entry points to
- this component.
- \item[Debug information:]
- The DEBUG-INFO structure describing this component. All information that
- the debugger wants to get from a running function is kept in this
- structure. Since there are many functions, the current PC is used to
- locate the appropriate debug information. The system keeps the debug
- information separate from the function data-block, since the currently
- running function may not be an entry point. There is no way to recover
- the function object for the currently running function, since this
- data-block may not exist.
- \item[First constant ... last constant:]
- These are the constants referenced by the component, if there are any.
-\vspace{1ex}
- \item[LRA header word:]
- The immediate header-word data is the word offset from the enclosing code
- data-block's header-word to this word. This allows GC and the debugger
- to easily recover the code data-block from a LRA. The code at the
- return point restores the current code pointer using a subtract immediate
- of the offset, which is known at compile time.
-\vspace{1ex}
- \item[Function entry point header-word:]
- The immediate header-word data is the word offset from the enclosing code
- data-block's header-word to this word. This is the same as for the
- retrun-PC header-word.
- \item[Self-pointer back to header-word:]
- In a non-closure function, this self-pointer to the previous header-word
- allows the call sequence to always indirect through the second word in a
- user callable function. See section "Closure Format". With a closure,
- indirecting through the second word gets you a function header-word. The
- system ignores this slot in the function header for a closure, since it
- has already indirected once, and this slot could be some random thing
- that causes an error if you jump to it. This pointer has an
- other-pointer tag instead of a function pointer tag, indicating it is not
- a user callable Lisp object.
- \item[Pointer to next function:]
- This is the next link in the thread of entry point functions found in
- this component. This value is NIL when the current header is the last
- entry point in the component.
- \item[Function name:]
- This function's name (for printing). If the user defined this function
- with DEFUN, then this is the defined symbol, otherwise it is a
- descriptive string.
- \item[Function debug arglist:]
- A printed string representing the function's argument list, for human
- readability. If it is a macroexpansion function, then this is the
- original DEFMACRO arglist, not the actual expander function arglist.
- \item[Function type:]
- A list-style function type specifier representing the argument signature
- and return types for this function. For example,
- \begin{verbatim}
-(function (fixnum fixnum fixnum) fixnum)
- \end{verbatim}
- or
- \begin{verbatim}
-(function (string &key (:start unsigned-byte)) string)
- \end{verbatim}
- This information is intended for machine readablilty, such as by the
- compiler.
-\end{description}
-
-\f
-\section{Closure Format}
-
-A closure data-block has the following format:
-\begin{verbatim}
-----------------------------------------------------------------
-| Word size (24 bits) | Closure-Type (8 bits) |
-----------------------------------------------------------------
-| Pointer to function header (other-pointer low-tag) |
-----------------------------------------------------------------
-| . |
-| Environment information |
-| . |
-----------------------------------------------------------------
-\end{verbatim}
-
-A closure descriptor has function low-tag bits. This means that a descriptor
-with function low-tag bits may point to either a function header or to a
-closure. The idea is that any callable Lisp object has function low-tag bits.
-Insofar as call is concerned, we make the format of closures and non-closure
-functions compatible. This is the reason for the self-pointer in a function
-header. Whenever you have a callable object, you just jump through the second
-word, offset some bytes, and go.
-
-
-\f
-\section{Function call}
-
-Due to alignment requirements and low-tag codes, it is not possible to use a
-hardware call instruction to compute the LRA. Instead the LRA
-for a call is computed by doing an add-immediate to the start of the code
-data-block.
-
-An advantage of using a single data-block to represent both the descriptor and
-non-descriptor parts of a function is that both can be represented by a
-single pointer. This reduces the number of memory accesses that have to be
-done in a full call. For example, since the constant pool is implicit in a
-LRA, a call need only save the LRA, rather than saving both the
-return PC and the constant pool.
-
-
-\f
-\section{Memory Layout}
-
-CMU Common Lisp has four spaces, read-only, static, dynamic-0, and dynamic-1.
-Read-only contains objects that the system never modifies, moves, or reclaims.
-Static space contains some global objects necessary for the system's runtime or
-performance (since they are located at a known offset at a know address), and
-the system never moves or reclaims these. However, GC does need to scan static
-space for references to moved objects. Dynamic-0 and dynamic-1 are the two
-heap areas for stop-and-copy GC algorithms.
-
-What global objects are at the head of static space???
-\begin{verbatim}
- NIL
- eval::*top-of-stack*
- lisp::*current-catch-block*
- lisp::*current-unwind-protect*
- FLAGS (RT only)
- BSP (RT only)
- HEAP (RT only)
-\end{verbatim}
-
-In addition to the above spaces, the system has a control stack, binding stack,
-and a number stack. The binding stack contains pairs of descriptors, a symbol
-and its previous value. The number stack is the same as the C stack, and the
-system uses it for non-Lisp objects such as raw system pointers, saving
-non-Lisp registers, parts of bignum computations, etc.
-
-
-\f
-\section{System Pointers}
-
-The system pointers reference raw allocated memory, data returned by foreign
-function calls, etc. The system uses these when you need a pointer to a
-non-Lisp block of memory, using an other-pointer. This provides the greatest
-flexibility by relieving contraints placed by having more direct references
-that require descriptor type tags.
-
-A system area pointer data-block has the following format:
-\begin{verbatim}
--------------------------------------------------------
-| 1 (data-block words) | SAP Type (8 bits) |
--------------------------------------------------------
-| system area pointer |
--------------------------------------------------------
-\end{verbatim}
-
-"SAP" means "system area pointer", and much of our code contains this naming
-scheme. We don't currently restrict system pointers to one area of memory, but
-if they do point onto the heap, it is up to the user to prevent being screwed
-by GC or whatever.
+++ /dev/null
-Todo:
-fasl.tex
-In good shape.
-
-object.tex
-Fairly good, but should probably be integrated with description of primitives
-in vm.tex.
-
-front.tex
-Needs updating cleanup scan. Not too bad.
-
-middle.tex
-Need VMR overview. New names for GTN/LTN? Needs general cleanup, but not too
-bad. NLX and stack are the worst.
-
-back.tex
-Pack and assembler need more info. General cleanup.
-
-
-compiler-overview.tex
-Adapt introductory material from /../fred/usr/ram/comp.mss, pap:talk.mss
-Division between ICR overview and ICR convert needs work.
-
-debugger.tex
-Needs much work. Merge much info from debug-info and debug-int. Duplicating a
-fair amount of stuff in the source may make sense where, since this is a part
-of the system that is generally interesting. And also, a part that people
-building on CMU CL might want to understand.
-
-glossary.tex
-Finish, integrate w/ main text?
-
-interpreter.tex
-Very sketchy and tentative. Needs to be fleshed out from the code.
-
-retargeting.tex
-Very rough. Needs to be merged with parts of vm.tex (call vops). Needs some
-additional text. Documentation of assembler, and all other exported
-interfaces. (Generate defined VOP descriptions from the core, keyed to files?)
-
-vm.tex
-This file should probably cease to exist, going into object, retargeting and
-introductory material. [Also other scrap in stuff/]
-
-
-[VMR and ICR overview also needed...]
-
-architecture.tex
-Missing sections on startup code, compiling, building.
-
-environment.tex
-Needs to be written: type system and info database interfaces.
-
-interface.tex
-Needs to be written: source paths and error message utilities.
-
-lowlev.tex
-Needs to be written. All manner of low-level stuff: memory layout and
-management, core file format, C interface, low-level debugging (and ldb.)
-
-\f
-Several different audiences:
- -- Curious compiler implementors (not a big priority. Downplay academic
- aspects, i.e. comparisons to other techniques, analysis of limitations,
- future work...) Compiler part can be more academic, and include some
- justifications of other design decisions.
- -- System maintainers.
- -- People retargeting the compiler.
- -- People bringing up the system in a new environment.
-
-Sys arch part:
- Package + file structure [system.txt]
- system building [compiling.txt]
- bootstrapping & cross compiling
-
-Compiler design:
- Overview (mirror structure of rest of the part)
- ICR data structure
- Front end [front.tex]
- Basic VMR data structures (no back-end stuff)
- Middle end [middle.tex]
- Back end + data structures [back.tex]
-
- Error system interface
- Source tracking
-
-Compiler retargeting:
- VM definition concepts [porting.txt, mail.txt, retargeting.tex]
- SCs, SBs, primitive-types
- Defining VOPS
- time specification
- defining
- and using the assembler
- Required VOPs [internal.txt, lowlev.txt, vm.mss]
- Standard primitives [vm.mss] (broken down by type, parallels object format
- section structure.)
- Customizing VMR conversion
- multiple hardware
- constant operands
- VM specific transforms
- special-case IR2 convert methods
-
-Run-time system:
- type system
- info database
- Data format [object.tex]
- Debugger:
- Info format [debug.txt]
- Stack parsing [debug.txt]
- Breakpoints
- Internal errors
- Signals
- Memory management: [William]
- heap Layout
- stacks
- GC
- misc implementation stuff: foreign call, assembly routines [lowlev.txt]
- LDB and low-level debugging
- core file format [William]
- fasl format [fasl.tex]
+++ /dev/null
-\part{Compiler Retargeting}
-
-[\#\#\#
-
-In general, it is a danger sign if a generator references a TN that isn't an
-operand or temporary, since lifetime analysis hasn't been done for that use.
-We are doing weird stuff for the old-cont and return-pc passing locations,
-hoping that the conflicts at the called function have the desired effect.
-Other stuff? When a function returns unknown values, we don't reference the
-values locations when a single-value return is done. But nothing is live at a
-return point anyway.
-
-
-
-Have a way for template conversion to special-case constant arguments?
-How about:
- If an arg restriction is (:satisfies [<predicate function>]), and the
- corresponding argument is constant, with the constant value satisfying the
- predicate, then (if any other restrictions are satisfied), the template
- will be emitted with the literal value passed as an info argument. If the
- predicate is omitted, then any constant will do.
-
- We could sugar this up a bit by allowing (:member <object>*) for
- (:satisfies (lambda (x) (member x '(<object>*))))
-
-We could allow this to be translated into a Lisp type by adding a new Constant
-type specifier. This could only appear as an argument to a function type.
-To satisfy (Constant <type>), the argument must be a compile-time constant of
-the specified type. Just Constant means any constant (i.e. (Constant *)).
-This would be useful for the type constraints on ICR transforms.
-
-
-Constant TNs: we count on being able to indirect to the leaf, and don't try to
-wedge the information into the offset. We set the FSC to an appropriate
-immediate SC.
-
- Allow "more operands" to VOPs in define-vop. You can't do much with the
- more operands: define-vop just fills in the cost information according to
- the loading costs for a SC you specify. You can't restrict more operands,
- and you can't make local preferences. In the generator, the named variable
- is bound to the TN-ref for the first extra operand. This should be good
- enough to handle all the variable arg VOPs (primarily function call and
- return). Usually more operands are used just to get TN lifetimes to work
- out; the generator actually ignores them.
-
- Variable-arg VOPs can't be used with the VOP macro. You must use VOP*.
- VOP* doesn't do anything with these extra operand except stick them on the
- ends of the operand lists passed into the template. VOP* is often useful
- within the convert functions for non-VOP templates, since it can emit a VOP
- using an already prepared TN-Ref list.
-
-
- It is pretty basic to the whole primitive-type idea that there is only one
- primitive-type for a given lisp type. This is really the same as saying
- primitive types are disjoint. A primitive type serves two somewhat
- unrelated purposes:
- -- It is an abstraction a Lisp type used to select type specific
- operations. Originally kind of an efficiency hack, but it lets a
- template's type signature be used both for selection and operand
- representation determination.
- -- It represents a set of possible representations for a value (SCs). The
- primitive type is used to determine the legal SCs for a TN, and is also
- used to determine which type-coercion/move VOP to use.
-
-]
-
-There are basically three levels of target dependence:
-
- -- Code in the "front end" (before VMR conversion) deals only with Lisp
- semantics, and is totally target independent.
-
- -- Code after VMR conversion and before code generation depends on the VM,
- but should work with little modification across a wide range of
- "conventional" architectures.
-
- -- Code generation depends on the machine's instruction set and other
- implementation details, so it will have to be redone for each
- implementation. Most of the work here is in defining the translation into
- assembly code of all the supported VOPs.
-
-
-\f
-\chapter{Storage bases and classes}
-New interface: instead of CURRENT-FRAME-SIZE, have CURRENT-SB-SIZE <name> which
-returns the current element size of the named SB.
-
-How can we have primitive types that overlap, i.e. (UNSIGNED-BYTE 32),
-(SIGNED-BYTE 32), FIXNUM?
-Primitive types are used for two things:
- Representation selection: which SCs can be used to represent this value?
- For this purpose, it isn't necessary that primitive types be disjoint,
- since any primitive type can choose an arbitrary set of
- representations. For moves between the overlapping representations,
- the move/load operations can just be noops when the locations are the
- same (vanilla MOVE), since any bad moves should be caught out by type
- checking.
- VOP selection:
- Is this operand legal for this VOP? When ptypes overlap in interesting
- ways, there is a problem with allowing just a simple ptype restriction,
- since we might want to allow multiple ptypes. This could be handled
- by allowing "union primitive types", or by allowing multiple primitive
- types to be specified (only in the operand restriction.) The latter
- would be long the lines of other more flexible VOP operand restriction
- mechanisms, (constant, etc.)
-
-
-
-Ensure that load/save-operand never need to do representation conversion.
-
-The PRIMITIVE-TYPE more/coerce info would be moved into the SC. This could
-perhaps go along with flushing the TN-COSTS. We would annotate the TN with
-best SC, which implies the representation (boxed or unboxed). We would still
-need represent the legal SCs for restricted TNs somehow, and also would have to
-come up with some other way for pack to keep track of which SCs we have already
-tried.
-
-A SC would have a list of "alternate" SCs and a boolean SAVE-P value that
-indicates it needs to be saved across calls in some non-SAVE-P SC. A TN is
-initially given its "best" SC. The SC is annotated with VOPs that are used for
-moving between the SC and its alternate SCs (load/save operand, save/restore
-register). It is also annotated with the "move" VOPs used for moving between
-this SC and all other SCs it is possible to move between. We flush the idea
-that there is only c-to-t and c-from-t.
-
-But how does this mesh with the idea of putting operand load/save back into the
-generator? Maybe we should instead specify a load/save function? The
-load/save functions would also differ from the move VOPs in that they would
-only be called when the TN is in fact in that particular alternate SC, whereas
-the move VOPs will be associated with the primary SC, and will be emitted
-before it is known whether the TN will be packed in the primary SC or an
-alternate.
-
-I guess a packed SC could also have immediate SCs as alternate SCs, and
-constant loading functions could be associated with SCs using this mechanism.
-
-So given a TN packed in SC X and a SC restriction for Y and Z, how do we know
-which load function to call? There would be ambiguity if X was an alternate
-for both Y and Z and they specified different load functions. This seems
-unlikely to arise in practice, though, so we could just detect the ambiguity
-and give an error at define-vop time. If they are doing something totally
-weird, they can always inhibit loading and roll their own.
-
-Note that loading costs can be specified at the same time (same syntax) as
-association of loading functions with SCs. It seems that maybe we will be
-rolling DEFINE-SAVE-SCS and DEFINE-MOVE-COSTS into DEFINE-STORAGE-CLASS.
-
-Fortunately, these changes will affect most VOP definitions very little.
-
-
-A Storage Base represents a physical storage resource such as a register set or
-stack frame. Storage bases for non-global resources such as the stack are
-relativized by the environment that the TN is allocated in. Packing conflict
-information is kept in the storage base, but non-packed storage resources such
-as closure environments also have storage bases.
-Some storage bases:
- General purpose registers
- Floating point registers
- Boxed (control) stack environment
- Unboxed (number) stack environment
- Closure environment
-
-A storage class is a potentially arbitrary set of the elements in a storage
-base. Although conceptually there may be a hierarchy of storage classes such
-as "all registers", "boxed registers", "boxed scratch registers", this doesn't
-exist at the implementation level. Such things can be done by specifying
-storage classes whose locations overlap. A TN shouldn't have lots of
-overlapping SC's as legal SC's, since time would be wasted repeatedly
-attempting to pack in the same locations.
-
-There will be some SC's whose locations overlap a great deal, since we get Pack
-to do our representation analysis by having lots of SC's. A SC is basically a
-way of looking at a storage resource. Although we could keep a fixnum and an
-unboxed representation of the same number in the same register, they correspond
-to different SC's since they are different representation choices.
-
-TNs are annotated with the primitive type of the object that they hold:
- T: random boxed object with only one representation.
- Fixnum, Integer, XXX-Float: Object is always of the specified numeric type.
- String-Char: Object is always a string-char.
-
-When a TN is packed, it is annotated with the SC it was packed into. The code
-generator for a VOP must be able to uniquely determine the representation of
-its operands from the SC. (debugger also...)
-
-Some SCs:
- Reg: any register (immediate objects)
- Save-Reg: a boxed register near r15 (registers easily saved in a call)
- Boxed-Reg: any boxed register (any boxed object)
- Unboxed-Reg: any unboxed register (any unboxed object)
- Float-Reg, Double-Float-Reg: float in FP register.
- Stack: boxed object on the stack (on cstack)
- Word: any 32bit unboxed object on nstack.
- Double: any 64bit unboxed object on nstack.
-
-We have a number of non-packed storage classes which serve to represent access
-costs associated with values that are not allocated using conflicts
-information. Non-packed TNs appear to already be packed in the appropriate
-storage base so that Pack doesn't get confused. Costs for relevant non-packed
-SC's appear in the TN-Ref cost information, but need not ever be summed into
-the TN cost vectors, since TNs cannot be packed into them.
-
-There are SCs for non-immediate constants and for each significant kind of
-immediate operand in the architecture. On the RT, 4, 8 and 20 bit integer SCs
-are probably worth having.
-
-Non-packed SCs:
- Constant
- Immediate constant SCs:
- Signed-Byte-<N>, Unsigned-Byte-<N>, for various architecture dependent
- values of <N>
- String-Char
- XXX-Float
- Magic values: T, NIL, 0.
-
-\f
-\chapter{Type system parameterization}
-
-The main aspect of the VM that is likely to vary for good reason is the type
-system:
-
- -- Different systems will have different ways of representing dynamic type
- information. The primary effect this has on the compiler is causing VMR
- conversion of type tests and checks to be implementation dependent.
- Rewriting this code for each implementation shouldn't be a big problem,
- since the portable semantics of types has already been dealt with.
-
- -- Different systems will have different specialized number and array types,
- and different VOPs specialized for these types. It is easy add this kind
- of knowledge without affecting the rest of the compiler. All you have to
- do is define the VOPs and translations.
-
- -- Different systems will offer different specialized storage resources
- such as floating-point registers, and will have additional kinds of
- primitive-types. The storage class mechanism handles a large part of this,
- but there may be some problem in getting VMR conversion to realize the
- possibly large hidden costs in implicit moves to and from these specialized
- storage resources. Probably the answer is to have some sort of general
- mechanism for determining the primitive-type for a TN given the Lisp type,
- and then to have some sort of mechanism for automatically using specialized
- Move VOPs when the source or destination has some particular primitive-type.
-
-\#|
-How to deal with list/null(symbol)/cons in primitive-type structure? Since
-cons and symbol aren't used for type-specific template selection, it isn't
-really all that critical. Probably Primitive-Type should return the List
-primitive type for all of Cons, List and Null (indicating when it is exact).
-This would allow type-dispatch for simple sequence functions (such as length)
-to be done using the standard template-selection mechanism. [Not a wired
-assumption]
-|\#
-
-
-\f
-\chapter{VOP Definition}
-
-Before the operand TN-refs are passed to the emit function, the following
-stuff is done:
- -- The refs in the operand and result lists are linked together in order using
- the Across slot. This list is properly NIL terminated.
- -- The TN slot in each ref is set, and the ref is linked into that TN's refs
- using the Next slot.
- -- The Write-P slot is set depending on whether the ref is an argument or
- result.
- -- The other slots have the default values.
-
-The template emit function fills in the Vop, Costs, Cost-Function,
-SC-Restriction and Preference slots, and links together the Next-Ref chain as
-appropriate.
-
-\f
-\section{Lifetime model}
-
-\#|
-Note in doc that the same TN may not be used as both a more operand and as any
-other operand to the same VOP, to simplify more operand LTN number coalescing.
-|\#
-
-It seems we need a fairly elaborate model for intra-VOP conflicts in order to
-allocate temporaries without introducing spurious conflicts. Consider the
-important case of a VOP such as a miscop that must have operands in certain
-registers. We allocate a wired temporary, create a local preference for the
-corresponding operand, and move to (or from) the temporary. If all temporaries
-conflict with all arguments, the result will be correct, but arguments could
-never be packed in the actual passing register. If temporaries didn't conflict
-with any arguments, then the temporary for an earlier argument might get packed
-in the same location as the operand for a later argument; loading would then
-destroy an argument before it was read.
-
-A temporary's intra-VOP lifetime is represented by the times at which its life
-starts and ends. There are various instants during the evaluation that start
-and end VOP lifetimes. Two TNs conflict if the live intervals overlap.
-Lifetimes are open intervals: if one TN's lifetime begins at a point where
-another's ends, then the TNs don't conflict.
-
-The times within a VOP are the following:
-
-:Load
- This is the beginning of the argument's lives, as far as intra-vop
- conflicts are concerned. If load-TNs are allocated, then this is the
- beginning of their lives.
-
-(:Argument <n>)
- The point at which the N'th argument is read for the last time (by this
- VOP). If the argument is dead after this VOP, then the argument becomes
- dead at this time, and may be reused as a temporary or result load-TN.
-
-(:Eval <n>)
- The N'th evaluation step. There may be any number of evaluation steps, but
- it is unlikely that more than two are needed.
-
-(:Result <n>)
- The point at which the N'th result is first written into. This is the
- point at which that result becomes live.
-
-:Save
- Similar to :Load, but marks the end of time. This is point at which result
- load-TNs are stored back to the actual location.
-
-In any of the list-style time specifications, the keyword by itself stands for
-the first such time, i.e.
- :argument <==> (:argument 0)
-
-
-Note that argument/result read/write times don't actually have to be in the
-order specified, but they must *appear* to happen in that order as far as
-conflict analysis is concerned. For example, the arguments can be read in any
-order as long no TN is written that has a life beginning at or after
-(:Argument <n>), where N is the number of an argument whose reading was
-postponed.
-
-[\#\#\# (???)
-
-We probably also want some syntactic sugar in Define-VOP for automatically
-moving operands to/from explicitly allocated temporaries so that this kind of
-thing is somewhat easy. There isn't really any reason to consider the
-temporary to be a load-TN, but we want to compute costs as though it was and
-want to use the same operand loading routines.
-
-We also might consider allowing the lifetime of an argument/result to be
-extended forward/backward. This would in many cases eliminate the need for
-temporaries when operands are read/written out of order.
-]
-
-\f
-\section{VOP Cost model}
-
-Note that in this model, if a operand has no restrictions, it has no cost.
-This makes make sense, since the purpose of the cost is to indicate the
-relative value of packing in different SCs. If the operand isn't required to
-be in a good SC (i.e. a register), then we might as well leave it in memory.
-The SC restriction mechanism can be used even when doing a move into the SC is
-too complex to be generated automatically (perhaps requiring temporary
-registers), since Define-VOP allows operand loading to be done explicitly.
-
-\f
-\section{Efficiency notes}
-
- In addition to
-being used to tell whether a particular unsafe template might get emitted, we
-can also use it to give better efficiency notes:
- -- We can say what is wrong with the call types, rather than just saying we
- failed to open-code.
- -- We can tell whether any of the "better" templates could possibly apply,
- i.e. is the inapplicability of a template because of inadequate type
- information or because the type is just plain wrong. We don't want to
- flame people when a template that couldn't possibly match doesn't match,
- e.g. complaining that we can't use fixnum+ when the arguments are known to
- be floats.
-
-
-This is how we give better efficiency notes:
-
-The Template-Note is a short noun-like string without capitalization or
-punctuation that describes what the template "does", i.e. we say
-"Unable to do ~A, doing ~A instead."
-
-The Cost is moved from the Vop-Info to the Template structure, and is used to
-determine the "goodness" of possibly applicable templates. [Could flush
-Template/Vop-Info distinction] The cost is used to choose the best applicable
-template to emit, and also to determine what better templates we might have
-been able to use.
-
-A template is possibly applicable if there is an intersection between all of
-the arg/result types and the corresponding arg/result restrictions, i.e. the
-template is not clearly impossible: more declarations might allow it to be
-emitted.
-
-\f
-\chapter{Assembler Retargeting}
-
-\f
-\chapter{Writing Assembly Code}
-
-VOP writers expect:
- MOVE
- You write when you port the assembler.)
- EMIT-LABEL
- Assembler interface like INST. Takes a label you made and says "stick it
- here."
- GEN-LABEL
- Returns a new label suitable for use with EMIT-LABEL exactly once and
- for referencing as often as necessary.
- INST
- Recognizes and dispatches to instructions you defined for assembler.
- ALIGN
- This takes the number of zero bits you want in the low end of the address
- of the next instruction.
- ASSEMBLE
- ASSEMBLE-ELSEWHERE
- Get ready for assembling stuff. Takes a VOP and arbitrary PROGN-style
- body. Wrap these around instruction emission code announcing the first
- pass of our assembler.
- CURRENT-NFP-TN
- This returns a TN for the NFP if the caller uses the number stack, or
- nil.
- SB-ALLOCATED-SIZE
- This returns the size of some storage based used by the currently
- compiling component.
- ...
-
-;;;
-;;; VOP idioms
-;;;
-
-STORE-STACK-TN
-LOAD-STACK-TN
- These move a value from a register to the control stack, or from the
- control stack to a register. They take care of checking the TN types,
- modifying offsets according to the address units per word, etc.
-
-\f
-\chapter{Required VOPS}
-
-
-Note: the move VOP cannot have any wired temps. (Move-Argument also?) This is
-so we can move stuff into wired TNs without stepping on our toes.
-
-
-We create set closure variables using the Value-Cell VOP, which takes a value
-and returns a value cell containing the value. We can basically use this
-instead of a Move VOP when initializing the variable. Value-Cell-Set and
-Value-Cell-Ref are used to access the value cell. We can have a special effect
-for value cells so that value cells references can be discovered to be common
-subexpressions or loop invariants.
-
-
-
-
-Represent unknown-values continuations as (start, count). Unknown values
-continuations are always outside of the current frame (on stack top). Within a
-function, we always set up and receive values in the standard passing
-locations. If we receive stack values, then we must BLT them down to the start
-of our frame, filling in any unsupplied values. If we generate unknown values
-(i.e. PUSH-VALUES), then we set the values up in the standard locations, then
-BLT them to stack top. When doing a tail-return of MVs, we just set them up in
-the standard locations and decrement SP: no BLT is necessary.
-
-Unknown argument call (MV-CALL) takes its arguments on stack top (is given a
-base pointer). If not a tail call, then we just set the arg pointer to the
-base pointer and call. If a tail call, we must BLT the arguments down to the
-beginning of the current frame.
-
-
-Implement more args by BLT'ing the more args *on top* of the current frame.
-This solves two problems:
- -- Any register more arguments can be made uniformly accessibly by copying
- them into memory. [We can't store the registers in place, since the
- beginning of the frame gets double use for storing the old-cont, return-pc
- and env.]
- -- It solves the deallocation problem: the arguments will be deallocated when
- the frame is returned from or a tail full call is done out of it. So
- keyword args will be properly tail-recursive without any special mechanism
- for squeezing out the more arg once the parsing is done. Note that a tail
- local call won't blast the more arg, since in local call the callee just
- takes the frame it is given (in this case containing the more arg).
-
-More args in local call??? Perhaps we should not attempt local call conversion
-in this case. We already special-case keyword args in local call. It seems
-that the main importance of more args is primarily related to full call: it is
-used for defining various kinds of frobs that need to take arbitrary arguments:
- -- Keyword arguments
- -- Interpreter stubs
- -- "Pass through" applications such as dispatch functions
-
-Given the marginal importance of more args in local call, it seems unworth
-going to any implementation difficulty. In fact, it seems that it would cause
-complications both at the VMR level and also in the VM definition. This being
-the case, we should flush it.
-
-
-\section{Function Call}
-
-
-\f
-\subsection{Registers and frame format}
-
-These registers are used in function call and return:
-
-A0..A{\it n}
- In full call, the first three arguments. In unknown values return, the
- first three return values.
-
-CFP
- The current frame pointer. In full call, this initially points to a
- partial frame large enough to hold the passed stack arguments (zero-length
- if none).
-
-CSP
- The current control stack top pointer.
-
-OCFP
- In full call, the passing location for the frame to return to.
-
- In unknown-values return of other than one value, the pointer to returned
- stack values. In such a return, OCFP is always initialized to point to
- the frame returned from, even when no stack values are returned. This
- allows OCFP to be used to restore CSP.
-
-LRA
- In full call, the passing location for the return PC.
-
-NARGS
- In full call, the number of arguments passed. In unknown-values return of
- other than one value, the number of values returned.
-
-\f
-\subsection{Full call}
-
-What is our usage of CFP, OCFP and CSP?
-
-It is an invariant that CSP always points after any useful information so that
-at any time an interrupt can come and allocate stuff in the stack.
-
-TR call is also a constraint: we can't deallocate the caller's frame before the
-call, since it holds the stack arguments for the call.
-
-What we do is have the caller set up CFP, and have the callee set CSP to CFP
-plus the frame size. The caller leaves CSP alone: the callee is the one who
-does any necessary stack deallocation.
-
-In a TR call, we don't do anything: CFP is left as CFP, and CSP points to the
-end of the frame, keeping the stack arguments from being trashed.
-
-In a normal call, CFP is set to CSP, causing the callee's frame to be allocated
-after the current frame.
-
-\f
-\subsection{Unknown values return}
-
-The unknown values return convention is always used in full call, and is used
-in local call when the compiler either can't prove that a fixed number of
-values are returned, or decides not to use the fixed values convention to allow
-tail-recursive XEP calls.
-
-The unknown-values return convention has variants: single value and variable
-values. We make this distinction to optimize the important case of a returner
-whose knows exactly one value is being returned. Note that it is possible to
-return a single value using the variable-values convention, but it is less
-efficient.
-
-We indicate single-value return by returning at the return-pc+4; variable value
-return is indicated by returning at the return PC.
-
-Single-value return makes only the following guarantees:
- A0 holds the value returned.
- CSP has been reset: there is no garbage on the stack.
-
-In variable value return, more information is passed back:
- A0..A2 hold the first three return values. If fewer than three values are
- returned, then the unused registers are initialized to NIL.
-
- OCFP points to the frame returned from. Note that because of our
- tail-recursive implementation of call, the frame receiving the values is
- always immediately under the frame returning the values. This means that
- we can use OCFP to index the values when we access them, and to restore
- CSP when we want to discard them.
-
- NARGS holds the number of values returned.
-
- CSP is always (+ OCFP (* NARGS 4)), i.e. there is room on the stack
- allocated for all returned values, even if they are all actually passed in
- registers.
-
-\f
-\subsection{External Entry Points}
-
-Things that need to be done on XEP entry:
- 1] Allocate frame
- 2] Move more arg above the frame, saving context
- 3] Set up env, saving closure pointer if closure
- 4] Move arguments from closure to local home
- Move old-cont and return-pc to the save locations
- 5] Argument count checking and dispatching
-
-XEP VOPs:
-
-Allocate-Frame
-Copy-More-Arg <nargs-tn> 'fixed {in a3} => <context>, <count>
-Setup-Environment
-Setup-Closure-Environment => <closure>
-Verify-Argument-Count <nargs-tn> 'count {for fixed-arg lambdas}
-Argument-Count-Error <nargs-tn> {Drop-thru on hairy arg dispatching}
-Use fast-if-=/fixnum and fast-if-</fixnum for dispatching.
-
-Closure vops:
-make-closure <fun entry> <slot count> => <closure>
-closure-init <closure> <values> 'slot
-
-
-Things that need to be done on all function entry:
- -- Move arguments to the variable home (consing value cells as necessary)
- -- Move environment values to the local home
- -- Move old-cont and return-pc to the save locations
-
-\f
-\section{Calls}
-
-Calling VOP's are a cross product of the following sets (with some members
-missing):
- Return values
- multiple (all values)
- fixed (calling with unknown values conventions, wanting a certain
- number.)
- known (only in local call where caller/callee agree on number of
- values.)
- tail (doesn't return but does tail call)
- What function
- local
- named (going through symbol, like full but stash fun name for error sys)
- full (have a function)
- Args
- fixed (number of args are known at compile-time)
- variable (MULTIPLE-VALUE-CALL and APPLY)
-
-Note on all jumps for calls and returns that we want to put some instruction
-in the jump's delay slot(s).
-
-Register usage at the time of the call:
-
-LEXENV
- This holds the lexical environment to use during the call if it's a closure,
- and it is undefined otherwise.
-
-CNAME
- This holds the symbol for a named call and garbage otherwise.
-
-OCFP
- This holds the frame pointer, which the system restores upon return. The
- callee saves this if necessary; this is passed as a pseudo-argument.
-
-A0 ... An
- These holds the first n+1 arguments.
-
-NARGS
- This holds the number of arguments, as a fixnum.
-
-LRA
- This holds the lisp-return-address object which indicates where to return.
- For a tail call, this retains its current value. The callee saves this
- if necessary; this is passed as a pseudo-argument.
-
-CODE
- This holds the function object being called.
-
-CSP
- The caller ignores this. The callee sets it as necessary based on CFP.
-
-CFP
- This holds the callee's frame pointer. Caller sets this to the new frame
- pointer, which it remembered when it started computing arguments; this is
- CSP if there were no stack arguments. For a tail call CFP retains its
- current value.
-
-NSP
- The system uses this within a single function. A function using NSP must
- allocate and deallocate before returning or making a tail call.
-
-Register usage at the time of the return for single value return, which
-goes with the unknown-values convention the caller used.
-
-A0
- The holds the value.
-
-CODE
- This holds the lisp-return-address at which the system continues executing.
-
-CSP
- This holds the CFP. That is, the stack is guaranteed to be clean, and there
- is no code at the return site to adjust the CSP.
-
-CFP
- This holds the OCFP.
-
-Additional register usage for multiple value return:
-
-NARGS
- This holds the number of values returned.
-
-A0 ... An
- These holds the first n+1 values, or NIL if there are less than n+1 values.
-
-CSP
- Returner stores CSP to hold its CFP + NARGS * <address units per word>
-
-OCFP
- Returner stores this as its CFP, so the returnee has a handle on either
- the start of the returned values on the stack.
-
-
-ALLOCATE FULL CALL FRAME.
-
-If the number of call arguments (passed to the VOP as an info argument)
-indicates that there are stack arguments, then it makes some callee frame for
-arguments:
- VOP-result <- CSP
- CSP <- CSP + value of VOP info arg times address units per word.
-
-In a call sequence, move some arguments to the right places.
-
-There's a variety of MOVE-ARGUMENT VOP's.
-
-FULL CALL VOP'S
-(variations determined by whether it's named, it's a tail call, there
-is a variable arg count, etc.)
-
- if variable number of arguments
- NARGS <- (CSP - value of VOP argument) shift right by address units per word.
- A0...An <- values off of VOP argument (just fill them all)
- else
- NARGS <- value of VOP info argument (always a constant)
-
- if tail call
- OCFP <- value from VOP argument
- LRA <- value from VOP argument
- CFP stays the same since we reuse the frame
- NSP <- NFP
- else
- OCFP <- CFP
- LRA <- compute LRA by adding an assemble-time determined constant to
- CODE.
- CFP <- new frame pointer (remembered when starting to compute args)
- This is CSP if no stack args.
- when (current-nfp-tn VOP-self-pointer)
- stack-temp <- NFP
-
- if named
- CNAME <- function symbol name
- the-fun <- function object out of symbol
-
- LEXENV <- the-fun (from previous line or VOP argument)
- CODE <- function-entry (the first word after the-fun)
- LIP <- calc first instruction addr (CODE + constant-offset)
- jump and run off temp
-
- <emit Lisp return address data-block>
- <default and move return values OR receive return values>
- when (current-nfp-tn VOP-self-pointer)
- NFP <- stack-temp
-
-Callee:
-
-XEP-ALLOCATE-FRAME
- emit function header (maybe initializes offset back to component start,
- but other pointers are set up at load-time. Pads
- to dual-word boundary.)
- CSP <- CFP + compile-time determined constant (frame size)
- if the function uses the number stack
- NFP <- NSP
- NSP <- NSP + compile-time determined constant (number stack frame size)
-
-SETUP-ENVIRONMENT
-(either use this or the next one)
-
-CODE <- CODE - assembler-time determined offset from function-entry back to
- the code data-block address.
-
-SETUP-CLOSURE-ENVIRONMENT
-(either use this or the previous one)
-After this the CLOSURE-REF VOP can reference closure variables.
-
-VOP-result <- LEXENV
-CODE <- CODE - assembler-time determined offset from function-entry back to
- the code data-block address.
-
-Return VOP's
-RETURN and RETURN-MULTIPLE are for the unknown-values return convention.
-For some previous caller this is either it wants n values (and it doesn't
-know how many are coming), or it wants all the values returned (and it
-doesn't know how many are coming).
-
-
-RETURN
-(known fixed number of values, used with the unknown-values convention
- in the caller.)
-When compiler invokes VOP, all values are already where they should be;
-just get back to caller.
-
-when (current-nfp-tn VOP-self-pointer)
- ;; The number stack grows down in memory.
- NSP <- NFP + number stack frame size for calls within the currently
- compiling component
- times address units per word
-CODE <- value of VOP argument with LRA
-if VOP info arg is 1 (number of values we know we're returning)
- CSP <- CFP
- LIP <- calc target addr
- (CODE + skip over LRA header word + skip over address units per branch)
- (The branch is in the caller to skip down to the MV code.)
-else
- NARGS <- value of VOP info arg
- nil out unused arg regs
- OCFP <- CFP (This indicates the start of return values on the stack,
- but you leave space for those in registers for convenience.)
- CSP <- CFP + NARGS * address-units-per-word
- LIP <- calc target addr (CODE + skip over LRA header word)
-CFP <- value of VOP argument with OCFP
-jump and run off LIP
-
-RETURN-MULTIPLE
-(unknown number of values, used with the unknown-values convention in
- the caller.)
-When compiler invokes VOP, it gets TN's representing a pointer to the
-values on the stack and how many values were computed.
-
-when (current-nfp-tn VOP-self-pointer)
- ;; The number stack grows down in memory.
- NSP <- NFP + number stack frame size for calls within the currently
- compiling component
- times address units per word
-NARGS <- value of VOP argument
-copy the args to the beginning of the current (returner's) frame.
- Actually some go into the argument registers. When putting the rest at
- the beginning of the frame, leave room for those in the argument registers.
-CSP <- CFP + NARGS * address-units-per-word
-nil out unused arg regs
-OCFP <- CFP (This indicates the start of return values on the stack,
- but you leave space for those in registers for convenience.)
-CFP <- value of VOP argument with OCFP
-CODE <- value of VOP argument with LRA
-LIP <- calc target addr (CODE + skip over LRA header word)
-jump and run off LIP
-
-
-Returnee
-The call VOP's call DEFAULT-UNKNOWN-VALUES or RECEIVE-UNKNOWN-VALUES after
-spitting out transfer control to get stuff from the returner.
-
-DEFAULT-UNKNOWN-VALUES
-(We know what we want and we got something.)
-If returnee wants one value, it never does anything to deal with a shortage
-of return values. However, if start at PC, then it has to adjust the stack
-pointer to dump extra values (move OCFP into CSP). If it starts at PC+N,
-then it just goes along with the "want one value, got it" case.
-If the returnee wants multiple values, and there's a shortage of return
-values, there are two cases to handle. One, if the returnee wants fewer
-values than there are return registers, and we start at PC+N, then it fills
-in return registers A1..A<desired values necessary>; if we start at PC,
-then the returnee is fine since the returning conventions have filled in
-the unused return registers with nil, but the returnee must adjust the
-stack pointer to dump possible stack return values (move OCFP to CSP).
-Two, if the returnee wants more values than the number of return registers,
-and it starts at PC+N (got one value), then it sets up returnee state as if
-an unknown number of values came back:
- A0 has the one value
- A1..An get nil
- NARGS gets 1
- OCFP gets CSP, so general code described below can move OCFP into CSP
-If we start at PC, then branch down to the general "got k values, wanted n"
-code which takes care of the following issues:
- If k < n, fill in stack return values of nil for shortage of return
- values and move OCFP into CSP
- If k >= n, move OCFP into CSP
-This also restores CODE from LRA by subtracting an assemble-time constant.
-
-RECEIVE-UKNOWN-VALUES
-(I want whatever I get.)
-We want these at the end of our frame. When the returnee starts starts at
-PC, it moves the return value registers to OCFP..OCFP[An] ignoring where
-the end of the stack is and whether all the return value registers had
-values. The returner left room on the stack before the stack return values
-for the register return values. When the returnee starts at PC+N, bump CSP
-by 1 and copy A0 there.
-This also restores CODE from LRA by subtracting an assemble-time constant.
-
-
-Local call
-
-There are three flavors:
- 1] KNOWN-CALL-LOCAL
- Uses known call convention where caller and callee agree where all
- the values are, and there's a fixed number of return values.
- 2] CALL-LOCAL
- Uses the unknown-values convention, but we expect a particular
- number of values in return.
- 3] MULTIPLE-CALL-LOCAL
- Uses the unknown-values convention, but we want all values returned.
-
-ALLOCATE-FRAME
-
-If the number of call arguments (passed to the VOP as an info argument)
-indicates that there are stack arguments, then it makes some callee frame for
-arguments:
- VOP-result1 <- CSP
- CSP <- CSP + control stack frame size for calls within the currently
- compiling component
- times address units per word.
- when (callee-nfp-tn <VOP info arg holding callee>)
- ;; The number stack grows down.
- ;; May have to round to dual-word boundary if machines C calling
- ;; conventions demand this.
- NSP <- NSP - number stack frame size for calls within the currently
- compiling component
- times address units per word
- VOP-result2 <- NSP
-
-KNOWN-CALL-LOCAL, CALL-LOCAL, MULTIPLE-CALL-LOCAL
-KNOWN-CALL-LOCAL has no need to affect CODE since CODE is the same for the
-caller/returnee and the returner. This uses KNOWN-RETURN.
-With CALL-LOCAL and MULTIPLE-CALL-LOCAL, the caller/returnee must fixup
-CODE since the callee may do a tail full call. This happens in the code
-emitted by DEFAULT-UNKNOWN-VALUES and RECEIVE-UNKNOWN-VALUES. We use these
-return conventions since we don't know what kind of values the returner
-will give us. This could happen due to a tail full call to an unknown
-function, or because the callee had different return points that returned
-various numbers of values.
-
-when (current-nfp-tn VOP-self-pointer) ;Get VOP self-pointer with
- ;DEFINE-VOP switch :vop-var.
- stack-temp <- NFP
-CFP <- value of VOP arg
-when (callee-nfp-tn <VOP info arg holding callee>)
- <where-callee-wants-NFP-tn> <- value of VOP arg
-<where-callee-wants-LRA-tn> <- compute LRA by adding an assemble-time
- determined constant to CODE.
-jump and run off VOP info arg holding start instruction for callee
-
-<emit Lisp return address data-block>
-<case call convention
- known: do nothing
- call: default and move return values
- multiple: receive return values
->
-when (current-nfp-tn VOP-self-pointer)
- NFP <- stack-temp
-
-KNOWN-RETURN
-
-CSP <- CFP
-when (current-nfp-tn VOP-self-pointer)
- ;; number stack grows down in memory.
- NSP <- NFP + number stack frame size for calls within the currently
- compiling component
- times address units per word
-LIP <- calc target addr (value of VOP arg + skip over LRA header word)
-CFP <- value of VOP arg
-jump and run off LIP
-
-
-
-\f
-\chapter{Standard Primitives}
-
-\f
-\chapter{Customizing VMR Conversion}
-
-Another way in which different implementations differ is in the relative cost
-of operations. On machines without an integer multiply instruction, it may be
-desirable to convert multiplication by a constant into shifts and adds, while
-this is surely a bad idea on machines with hardware support for multiplication.
-Part of the tuning process for an implementation will be adding implementation
-dependent transforms and disabling undesirable standard transforms.
-
-When practical, ICR transforms should be used instead of VMR generators, since
-transforms are more portable and less error-prone. Note that the Lisp code
-need not be implementation independent: it may contain all sorts of
-sub-primitives and similar stuff. Generally a function should be implemented
-using a transform instead of an VMR translator unless it cannot be implemented
-as a transform due to being totally evil or it is just as easy to implement as
-a translator because it is so simple.
-
-\f
-\section{Constant Operands}
-
-If the code emitted for a VOP when an argument is constant is very different
-than the non-constant case, then it may be desirable to special-case the
-operation in VMR conversion by emitting different VOPs. An example would be if
-SVREF is only open-coded when the index is a constant, and turns into a miscop
-call otherwise. We wouldn't want constant references to spuriously allocate
-all the miscop linkage registers on the off chance that the offset might not be
-constant. See the :constant feature of VOP primitive type restrictions.
-
-\f
-\section{Supporting Multiple Hardware Configurations}
-
-
-A winning way to change emitted code depending on the hardware configuration,
-i.e. what FPA is present is to do this using primitive types. Note that the
-Primitive-Type function is VM supplied, and can look at any appropriate
-hardware configuration switches. Short-Float can become 6881-Short-Float,
-AFPA-Short-Float, etc. There would be separate SBs and SCs for the registers
-of each kind of FP hardware, with the each hardware-specific primitive type
-using the appropriate float register SC. Then the hardware specific templates
-would provide AFPA-Short-Float as the argument type restriction.
-
-Primitive type changes:
-
-The primitive-type structure is given a new %Type slot, which is the CType
-structure that is equivalent to this type. There is also a Guard slot, with,
-if true is a function that control whether this primitive type is allowed (due
-to hardware configuration, etc.)
-
-We add new :Type and :Guard keywords to Def-Primitive-Type. Type is the type
-specifier that is equivalent (default to the primitive-type name), and Guard is
-an expression evaluated in the null environment that controls whether this type
-applies (default to none, i.e. constant T).
-
-The Primitive-Type-Type function returns the Lisp CType corresponding to a
-primitive type. This is the %Type unless there is a guard that returns false,
-in which case it is the empty type (i.e. NIL).
-
-[But this doesn't do what we want it to do, since we will compute the
-function type for a template at load-time, so they will correspond to whatever
-configuration was in effect then. Maybe we don't want to dick with guards here
-(if at all). I guess we can defer this issue until we actually support
-different FP configurations. But it would seem pretty losing to separately
-flame about all the different FP configurations that could be used to open-code
-+ whenever we are forced to closed-code +.
-
-If we separately report each better possibly applicable template that we
-couldn't use, then it would be reasonable to report any conditional template
-allowed by the configuration.
-
-But it would probably also be good to give some sort of hint that perhaps it
-would be a good time to make sure you understand how to tell the compiler to
-compile for a particular configuration. Perhaps if there is a template that
-applies *but for the guard*, then we could give a note. This way, if someone
-thinks they are being efficient by throwing in lots of declarations, we can let
-them know that they may have to do more.
-
-I guess the guard should be associated with the template rather than the
-primitive type. This would allow LTN and friends to easily tell whether a
-template applies in this configuration. It is also probably more natural for
-some sorts of things: with some hardware variants, it may be that the SBs and
-representations (SCs) are really the same, but there some different allowed
-operations. In this case, we could easily conditionalize VOPs without the
-increased complexity due to bogus SCs. If there are different storage
-resources, then we would conditionalize Primitive-Type as well.
-
-
-\f
-\section{Special-case VMR convert methods}
-
- (defun continuation-tn (cont \&optional (check-p t))
- ...)
-Return the TN which holds Continuation's first result value. In general
-this may emit code to load the value into a TN. If Check-P is true, then
-when policy indicates, code should be emitted to check that the value satisfies
-the continuation asserted type.
-
- (defun result-tn (cont)
- ...)
-Return the TN that Continuation's first value is delivered in. In general,
-may emit code to default any additional values to NIL.
-
- (defun result-tns (cont n)
- ...)
-Similar to Result-TN, except that it returns a list of N result TNs, one
-for each of the first N values.
-
-
-Nearly all open-coded functions should be handled using standard template
-selection. Some (all?) exceptions:
- -- List, List* and Vector take arbitrary numbers of arguments. Could
- implement Vector as a source transform. Could even do List in a transform
- if we explicitly represent the stack args using %More-Args or something.
- -- %Typep varies a lot depending on the type specifier. We don't want to
- transform it, since we want %Typep as a canonical form so that we can do
- type optimizations.
- -- Apply is weird.
- -- Funny functions emitted by the compiler: %Listify-Rest-Args, Arg,
- %More-Args, %Special-Bind, %Catch, %Unknown-Values (?), %Unwind-Protect,
- %Unwind, %%Primitive.
+++ /dev/null
-@make [Manual]
-@device [PostScript]
-@use (database "/usr/lisp/scribe/database/")
-@libraryfile [Mathematics10]
-@libraryfile [ArpaCredit]
-@libraryfile [table]
-@libraryfile [spice]
-@style(FontFamily=TimesRoman)
-@style(Date="March 1952")
-
-@commandstring(pusharrow = "@jsym<L>")
-@define(f, facecode f)
-
-@commandstring(InstrSection = "@tabclear@tabset[.5 in, 3.0 in]")
-@form(Instr = "@*@\@Parm[name]@\")
-@form(BInstr ="@*@\@Parm[name]@+[*]@\")
-@string(DinkyMachine = "IBM RT PC")
-@begin[TitlePage]
-@begin[TitleBox]
-@blankspace(0.25in)
-@heading[Internal Design of CMU Common Lisp
-on the IBM RT PC]
-@begin[Center]
-@b{David B. McDonald
-Scott E. Fahlman
-Skef Wholey
-
-@value[Date]
-
-CMU-CS-87-157
-}
-@end[Center]
-@end[TitleBox]
-@center[@b<Abstract>]
-@begin[Text]
-CMU Common Lisp is an implementation of Common Lisp that currently runs on
-the IBM RT PC under Mach, a Berkeley Unix 4.3 binary compatible operating
-system. This document describes low level
-details of the implementation. In particular, it describes the data
-formats used for all Lisp objects, the assembler language routines
-(miscops) used to support compiled code, the function call and return
-mechanism, and other design information necessary to understand the
-underlying structure of the CMU Common Lisp implementation on the IBM RT PC
-under the Mach operating system.
-@end[Text]
-
-@begin[ResearchCredit]
-@ArpaCredit[Contract=Strategic87-90]
-@end[ResearchCredit]
-@end[TitlePage]
-
-@heading [Acknowledgments]
-
-This document is based heavily on the document @i[Revised Internal Design
-of Spice Lisp] by Skef Wholey, Scott Fahlman, and Joseph Ginder.
-
-The FASL file format was designed by Guy L. Steele Jr. and Walter van
-Roggen, and the appendix on this subject is their document with very few
-modifications.
-
-@chapter [Introduction]
-
-@section [Scope and Purpose]
-
-This document describes a new implementation of CMU Common Lisp (nee Spice
-Lisp) as it is implemented on the @value(DinkyMachine) running Mach, a
-Berkeley Unix 4.3 binary compatible operating system. This design is
-undergoing rapid change, and for the present is not guaranteed to
-accurately describe any past, present, or future implementation of CMU
-Common Lisp. All questions and comments on this material should be
-directed to David B. McDonald (David.McDonald@@CS.CMU.EDU).
-
-This document specifies the hand-coded assembler routines (miscops) and
-virtual memory architecture of the @value(DinkyMachine) CMU Common Lisp system.
-This is a working document, and it will change frequently as the system is
-developed and maintained. If some detail of the system does not agree with
-what is specified here, it is to be considered a bug.
-
-@section [Notational Conventions]
-@index [Bit numbering]
-@index [Byte numbering]
-CMU Common Lisp objects are 32 bits long. The high-order bit of each word is
-numbered 0; the low-order bit is numbered 31. If a word is broken into smaller
-units, these are packed into the word from left to right. For example, if we
-break a word into bytes, byte 0 would occupy bits 0-7, byte 1 would occupy
-8-15, byte 2 would occupy 16-23, and byte 3 would occupy 24-31.
-
-All CMU Common Lisp documentation uses decimal as the default radix; other
-radices will be indicated by a subscript (as in 77@-[8]) or by a clear
-statement of what radix is in use.
-
-@chapter [Data Types and Object Formats]
-
-@section [Lisp Objects]
-@index [Lisp objects]
-
-Lisp objects are 32 bits long. They come in 32 basic types, divided into three
-classes: immediate data types, pointer types, and forwarding pointer types.
-The storage formats are as follows:
-
-@index [Immediate object format]
-@index [Pointer object format]
-@begin [verbatim, group]
-
-@b[Immediate Data Types:]
- 0 4 5 31
-------------------------------------------------------------------------
-| Type Code (5) | Immediate Data (27) |
-------------------------------------------------------------------------
-
-@b[Pointer and Forwarding Types:]
- 0 4 5 6 7 29 31
-------------------------------------------------------------------------
-| Type Code (5) | Space Code (2) | Pointer (23) | Unused (2) |
-------------------------------------------------------------------------
-@end [verbatim]
-
-@section [Table of Type Codes]
-@index [Type codes]
-
-@begin [verbatim, group]
-
-Code Type Class Explanation
----- ---- ----- -----------
-0 + Fixnum Immediate Positive fixnum, miscop code, etc.
-1 GC-Forward Pointer GC forward pointer, used during GC.
-4 Bignum Pointer Bignum.
-5 Ratio Pointer Two words: numerator, denominator.
-6 + Short Float Immediate Positive short flonum.
-7 - Short Float Immediate Negative short flonum.
-8 Single Float Pointer Single precision float.
-9 Double Float Pointer Double precision float (?).
-9 Long Float Pointer Long float.
-10 Complex Pointer Two words: real, imaginary parts.
-11 String Pointer Character string.
-12 Bit-Vector Pointer Vector of bits
-13 Integer-Vector Pointer Vector of integers
-14 General-Vector Pointer Vector of Lisp objects.
-15 Array Pointer Array header.
-16 Function Pointer Compiled function header.
-17 Symbol Pointer Symbol.
-18 List Pointer Cons cell.
-20 C. S. Pointer Pointer Pointer into control stack.
-21 B. S. Pointer Pointer Pointer into binding stack.
-26 Interruptible Immediate Marks a miscop as interruptible.
-27 Character Immediate Character object.
-28 Values-Marker Immediate Multiple values marker.
-29 Catch-All Immediate Catch-All object.
-30 Trap Immediate Illegal object trap.
-31 - Fixnum Immediate Negative fixnum.
-@end [verbatim]
-
-@section [Table of Space Codes]
-@index [Space codes]
-
-@begin [verbatim, group]
-
-Code Space Explanation
----- ----- -----------
-0 Dynamic-0 Storage normally garbage collected, space 0.
-1 Dynamic-1 Storage normally garbage collected, space 1.
-2 Static Permanent objects, never moved or reclaimed.
-3 Read-Only Objects never moved, reclaimed, or altered.
-@end [verbatim]
-
-@section [Immediate Data Type Descriptions]
-
-@begin [description]
-
-@index [Fixnum format]
-Fixnum@\A 28-bit two's complement integer. The sign bit is stored redundantly
-in the top 5 bits of the word.
-
-@index [Short float format]
-Short-Float@\The sign bit is stored as part of the type code,
-allowing a 28 bit signed short float format. The format of short floating
-point numbers is:
-@begin [verbatim]
- 0 3 4 5 12 13 31
----------------------------------------------------------------
-| Type code (4) | Sign (1) | Exponent (8) | Mantissa (19) |
----------------------------------------------------------------
-@end [verbatim]
-The floating point number is the same format as the @value(DinkyMachine)
-supports for single precision numbers, except it has been shifted right
-by four bits for the type code. The result of any operation is therefore
-truncated. Long floating point numbers are also available if you need
-more accuracy and better error propagation properties.
-
-@index [Character object]
-Character@\A character object holding a character code, control bits, and font
-in the following format:
-@begin [verbatim, group]
- 0 4 6 7 8 15 16 23 24 31
----------------------------------------------------------------
-| Type code (5) | Unused (3) | Font (8) | Bits (8) | Code (8) |
----------------------------------------------------------------
-@end [verbatim]
-
-@index [Values-Marker]
-Values-Marker@\Used to mark the presence of multiple values on the stack. The
-low 16 bits indicate how many values are being returned. Note that only 65535
-values can be returned from a multiple-values producing form. These are pushed
-onto the stack in order, and the Values-Marker is returned in register A0.
-
-@index [Catch-All object]
-Catch-All@\Object used as the catch tag for unwind-protects. Special things
-happen when a catch frame with this as its tag is encountered during a throw.
-See section @ref[Catch] for details.
-
-@index[Trap]
-@index[Illegal object trap]
-Trap@\Illegal object trap. This value is used in symbols to signify an
-undefined value or definition.
-
-@index[Interruptible Marker]
-Interruptible-Marker@\Object used to mark a miscop as interruptible. This
-object is put in one of the registers and signals to the interrupt handler
-that the miscop can be interrupted safely. Only miscops that can take a long
-time (e.g., length when passed a circular list, system call miscops that
-may wait indefinitely) are marked this way.
-@end [description]
-
-@section [Pointer-Type Objects and Spaces]
-@index [Pointer object format]
-@index [Virtual memory]
-
-Each of the pointer-type lisp objects points into a different space in virtual
-memory. There are separate spaces for Bit-Vectors, Symbols, Lists, and so on.
-The 5-bit type-code provides the high-order virtual address bits for the
-object, followed by the 2-bit space code, followed by the 25-bit pointer
-address. This gives a 30-bit virtual address to a 32-bit word; since the
-@value(DinkyMachine) is a byte-addressed machine, the two low-order
-bits are 0. In effect we have carved a 30-bit space into a fixed set
-of 23-bit subspaces, not all of which are used.
-
-@index [Space codes]
-The space code divides each of the type spaces into four sub-spaces,
-as shown in the table above. At any given time, one of the dynamic
-spaces is considered newspace, while the other is oldspace.
-During a stop and copy garbage collection, a ``flip'' can be done, turning the
-old newspace into the new oldspace. All type-spaces are flipped at once.
-Allocation of new dynamic objects always occurs in newspace.
-
-@index [Static space]
-@index [Read-only space]
-Optionally, the user (or system functions) may allocate objects in
-static or read-only space. Such objects are never reclaimed once they
-are allocated -- they occupy the space in which they were initially
-allocated for the lifetime of the Lisp process. The advantage of
-static allocation is that the GC never has to move these objects,
-thereby saving a significant amount of work, especially if the objects
-are large. Objects in read-only space are static, in that they are
-never moved or reclaimed; in addition, they cannot be altered once
-they are set up. Pointers in read-only space may only point to
-read-only or static space, never to dynamic space. This saves even
-more work, since read-only space does not need to be scavenged, and
-pages of read-only material do not need to be written back onto the
-disk during paging.
-
-Objects in a particular type-space will contain either pointers to
-garbage-collectible objects or words of raw non-garbage-collectible bits, but
-not both. Similarly, a space will contain either fixed-length objects or
-variable-length objects, but not both. A variable-length object always
-contains a 24-bit length field right-justified in the first word, with
-the positive fixnum type-code in the high-order five bits. The remaining three
-bits can be used for sub-type information. The length field gives the
-size of the object in 32-bit words, including the header word. The
-garbage collector needs this information when the object is moved, and
-it is also useful for bounds checking.
-
-The format of objects in each space are as follows:
-
-@begin [description]
-@index [Symbol]
-@index [Value cell]
-@index [Definition cell]
-@index [Property list cell]
-@index [Plist cell]
-@index [Print name cell]
-@index [Pname cell]
-@index [Package cell]
-Symbol@\Each symbol is represented as a
-fixed-length block of boxed Lisp cells. The number of cells
-per symbol is 5, in the following order:
-@begin [verbatim, group]
-0 Value cell for shallow binding.
-1 Definition cell: a function or list.
-2 Property list: a list of attribute-value pairs.
-3 Print name: a string.
-4 Package: the obarray holding this symbol.
-@end [verbatim]
-
-@index [List cell]
-List@\A fixed-length block of two boxed Lisp cells, the CAR and the CDR.
-
-@index [General-Vector format]
-@index [G-Vector format]
-@index [Vector format]
-General-Vector@\Vector of lisp objects, any length. The first word is a fixnum
-giving the number of words allocated for the vector (up to 24 bits). The
-highest legal index is this number minus 2. The second word is vector entry 0,
-and additional entries are allocated contiguously in virtual memory. General
-vectors are sometimes called G-Vectors. (See section @ref[Vectors] for further
-details.)
-
-@index [Integer-Vector format]
-@index [I-Vector format]
-@index [Vector format]
-Integer-Vector@\Vector of integers, any length. The 24 low bits of the first
-word give the allocated length in 32-bit words. The low-order 28 bits of the
-second word gives the length of the vector in entries, whatever the length of
-the individual entries may be. The high-order 4 bits of the second word
-contain access-type information that yields, among other things, the number of
-bits per entry. Entry 0 is left-justified in the third word of the vector.
-Bits per entry will normally be powers of 2, so they will fit neatly into
-32-bit words, but if necessary some empty space may be left at the low-order
-end of each word. Integer vectors are sometimes called I-Vectors. (See
-section @ref[Vectors] for details.)
-
-@index [Bit-Vector format]
-@index [Vector format]
-Bit-Vector@\Vector of bits, any length. Bit-Vectors are represented in a form
-identical to I-Vectors, but live in a different space for efficiency reasons.
-
-@index [Bignum format]
-@label [Bignums]
-Bignum@\Bignums are infinite-precision integers, represented in a format
-identical to G-Vectors. Each bignum is stored as a series of 32-bit words,
-with the low-order word stored first. The representation is two's complement,
-but the sign of the number is redundantly encoded in the type field of the
-fixnum in the header word. If this fixnum is non-negative, then so is the
-bignum, if it is negative, so is the bignum.
-
-@index [Flonum format]
-@index [Flonum formats]
-@index [Floating point formats]
-Floats@\Floats are stored as two or more consecutive words of bits, in the
-following format:
-@begin [verbatim, group]
----------------------------------------------------------------
-| Header word, used only for GC forward pointers. |
----------------------------------------------------------------
-| Appropriate number of 32-bit words in machine format |
----------------------------------------------------------------
-@end [verbatim]
-The number of words used to represent a floating point number is one plus the
-size of the floating point number being stored. The floating point numbers
-will be represented in whatever format the @value(DinkyMachine) expects. The
-extra header word is needed so that a valid floating point number is not
-mistaken for a gc-forward pointer during a garbage collection.
-
-@index [Ratio format]
-Ratio@\Ratios are stored as two consecutive words of Lisp objects, which should
-both be integers.
-
-@index [Complex number format]
-Complex@\Complex numbers are stored as two consecutive words of Lisp objects,
-which should both be numbers.
-
-@index [Array format]
-Array@\This is actually a header which holds the accessing and
-other information about the array. The actual array contents are held in a
-vector (either an I-Vector or G-Vector) pointed to by an entry in
-the header. The header is identical in format to a G-Vector. For
-details on what the array header contains, see section @ref[Arrays].
-
-@index [String format]
-String@\A vector of bytes. Identical in form to I-Vectors with the access type
-always 8-Bit. However, instead of accepting and returning fixnums, string
-accesses accept and return character objects. Only the 8-bit code field is
-actually stored, and the returned character object always has bit and font
-values of 0.
-
-@index [Function object format]
-Function @\A compiled CMU Common Lisp function consists of both lisp
-objects and raw bits for the code. The Lisp objects are stored in
-the Function space in a format identical to that used for general
-vectors, with a 24-bit length field in the first word. This object
-contains assorted parameters needed by the calling machinery, a
-pointer to an 8-bit I-Vector containing the compiled code, a number
-of pointers to symbols used as special variables within the function,
-and a number of lisp objects used as constants by the function.
-@end [description]
-
-@section [Forwarding Pointers]
-@index [Forwarding pointers]
-
-@begin [description]
-@index [GC-Forward pointer]
-GC-Forward@\When a data structure is transported into newspace, a GC-Forward
-pointer is left behind in the first word of the oldspace object. This points
-to the same type-space in which it is found. For example, a GC-Forward in
-G-Vector space points to a structure in the G-Vector newspace. GC-Forward
-pointers are only found in oldspace.
-@end [description]
-
-@section [System and Stack Spaces]
-@index [System table space]
-@index [Stack spaces]
-@index [Control stack space]
-@index [Binding stack space]
-@index [Special binding stack space]
-
-The virtual addresses below 08000000@-[16] are not occupied by Lisp objects,
-since Lisp objects with type code 0 are positive fixnums. Some of this space
-is used for other purposes by Lisp. A couple of pages (4096 byte pages)
-at address 00100000@-[16] contain tables that Lisp needs to access
-frequently. These include the allocation table, the active-catch-frame,
-information to link to C routines, etc. Memory at location 00200000@-[16]
-contains code for various miscops. Also, any C code loaded into a running
-Lisp process is loaded after the miscops. The format of the allocation
-table is described in chapter @ref[Alloc-Chapter].
-
-The control stack grows upward (toward higher addresses) in memory,
-and is a framed stack. It contains only general Lisp objects (with
-some random things encoded as fixnums). Every object
-pointed to by an entry on this stack is kept alive. The frame for a
-function call contains an area for the function's arguments, an area
-for local variables, a pointer to the caller's frame, and a pointer
-into the binding stack. The frame for a Catch form contains similar
-information. The precise stack format can be found in chapter
-@ref[Runtime].
-
-The special binding stack grows downward. This stack is used to hold
-previous values of special variables that have been bound. It grows and
-shrinks with the depth of the binding environment, as reflected in the
-control stack. This stack contains symbol-value pairs, with only boxed
-Lisp objects present.
-
-All Lisp objects are allocated on word boundaries, since the
-@value(DinkyMachine) can only access words on word boundaries.
-
-@section [Vectors and Arrays]
-@label [Vectors]
-@index [Vectors]
-
-Common Lisp arrays can be represented in a few different ways in CMU Common
-Lisp -- different representations have different performance advantages.
-Simple general vectors, simple vectors of integers, and simple strings are
-basic CMU Common Lisp data types, and access to these structures is quicker
-than access to non-simple (or ``complex'') arrays. However, all
-multi-dimensional arrays in CMU Common Lisp are complex arrays, so
-references to these are always through a header structure.
-
-@subsection [General Vectors]
-@index [General-Vector format]
-
-G-Vectors contain Lisp objects. The format is as follows:
-
-@begin [verbatim, group]
-------------------------------------------------------------------
-| Fixnum code (5) | Subtype (3) | Allocated length (24) |
-------------------------------------------------------------------
-| Vector entry 0 (Additional entries in subsequent words) |
-------------------------------------------------------------------
-@end [verbatim]
-
-The first word of the vector is
-a header indicating its length; the remaining words hold the boxed entries of
-the vector, one entry per 32-bit word. The header word is of type fixnum. It
-contains a 3-bit subtype field, which is used to indicate several special types
-of general vectors. At present, the following subtype codes are defined:
-
-@index [DEFSTRUCT]
-@index [Hash tables]
-@begin [itemize, spread 0, spacing 1]
-0 Normal. Used for assorted things.
-
-1 Named structure created by DEFSTRUCT, with type name in entry 0.
-
-2 EQ Hash Table, last rehashed in dynamic-0 space.
-
-3 EQ Hash Table, last rehashed in dynamic-1 space.
-
-4 EQ Hash Table, must be rehashed.
-@end [itemize]
-
-Following the subtype is a 24-bit field indicating how many 32-bit words are
-allocated for this vector, including the header word. Legal indices into the
-vector range from zero to the number in the allocated length field minus 2,
-inclusive. Normally, the index is checked on every access to the vector.
-Entry 0 is stored in the second word of the vector, and subsequent entries
-follow contiguously in virtual memory.
-
-Once a vector has been allocated, it is possible to reduce its length by using
-the Shrink-Vector miscop, but never to increase its length, even back to
-the original size, since the space freed by the reduction may have been
-reclaimed. This reduction simply stores a new smaller value in the length
-field of the header word.
-
-It is not an error to create a vector of length 0, though it will always be an
-out-of-bounds error to access such an object. The maximum possible length for
-a general vector is 2@+[24]-2 entries, and that can't fit in the available
-space. The maximum length is 2@+[23]-2 entries, and that is only possible if
-no other general vectors are present in the space.
-
-@index [Bignum Format]
-Bignums are identical in format to G-Vectors although each entry is a 32-bit
-integer, and thus only assembler routines should ever access an entry.
-
-@index [Function object format]
-@index [Array format]
-Objects of type Function and Array are identical in format to
-general vectors, though they have their own spaces.
-
-@subsection [Integer Vectors]
-@index [Integer-Vector format]
-
-I-Vectors contain unboxed items of data, and their format is more complex. The
-data items come in a variety of lengths, but are of constant length within a
-given vector. Data going to and from an I-Vector are passed as Fixnums, right
-justified. Internally these integers are stored in packed form, filling 32-bit
-words without any type-codes or other overhead. The format is as follows:
-
-@begin [verbatim, group]
-----------------------------------------------------------------
-| Fixnum code (5) | Subtype (3) | Allocated length (24) |
-----------------------------------------------------------------
-| Access type (4) | Number of entries (28) |
-----------------------------------------------------------------
-| Entry 0 left justified |
-----------------------------------------------------------------
-@end [verbatim]
-
-The first word of an I-Vector
-contains the Fixnum type-code in the top 5 bits, a 3-bit subtype code in the
-next three bits, and the total allocated length of the vector (in 32-bit words)
-in the low-order 24 bits. At present, the following subtype codes are defined:
-@begin [itemize, spread 0, spacing 1]
-0 Normal. Used for assorted things.
-
-1 Code. This is the code-vector for a function object.
-@end [itemize]
-
-The second word of the vector is the one that is looked at every
-time the vector is accessed. The low-order 28 bits of this word
-contain the number of valid entries in the vector, regardless of how
-long each entry is. The lowest legal index into the vector is always
-0; the highest legal index is one less than this number-of-entries
-field from the second word. These bounds are checked on every access.
-Once a vector is allocated, it can be reduced in size but not increased.
-The Shrink-Vector miscop changes both the allocated length field
-and the number-of-entries field of an integer vector.
-
-@index [Access-type codes]
-The high-order 4 bits of the second word contain an access-type code
-which indicates how many bits are occupied by each item (and therefore
-how many items are packed into a 32-bit word). The encoding is as follows:
-@begin [verbatim, group]
-0 1-Bit 8 Unused
-1 2-Bit 9 Unused
-2 4-Bit 10 Unused
-3 8-Bit 11 Unused
-4 16-Bit 12 Unused
-5 32-Bit 13 Unused
-6 Unused 14 Unused
-7 Unused 15 Unused
-@end [verbatim]
-
-In I-Vectors, the data items are packed into the third and subsequent
-words of the vector. Item 0 is left justified in the third word,
-item 1 is to its right, and so on until the allocated number of items
-has been accommodated. All of the currently-defined access types
-happen to pack neatly into 32-bit words, but if this should not be
-the case, some unused bits would remain at the right side of each
-word. No attempt will be made to split items between words to use up
-these odd bits. When allocated, an I-Vector is initialized to all
-0's.
-
-As with G-Vectors, it is not an error to create an I-Vector of length
-0, but it will always be an error to access such a vector. The
-maximum possible length of an I-Vector is 2@+[28]-1 entries or
-2@+[23]-3 words, whichever is smaller.
-
-@index [String format]
-Objects of type String are identical in format to I-Vectors, though they have
-their own space. Strings always have subtype 0 and access-type 3 (8-Bit).
-Strings differ from normal I-Vectors in that the accessing miscops accept
-and return objects of type Character rather than Fixnum.
-
-@subsection [Arrays]
-@label [Arrays]
-@index [Arrays]
-
-An array header is identical in form to a G-Vector. Like any G-Vector, its
-first word contains a fixnum type-code, a 3-bit subtype code, and a 24-bit
-total length field (this is the length of the array header, not of the vector
-that holds the data). At present, the subtype code is always 0. The entries
-in the header-vector are interpreted as follows:
-
-@index [Array header format]
-@begin [description]
-0 Data Vector @\This is a pointer to the I-Vector, G-Vector, or string that
-contains the actual data of the array. In a multi-dimensional array, the
-supplied indices are converted into a single 1-D index which is used to access
-the data vector in the usual way.
-
-1 Number of Elements @\This is a fixnum indicating the number of elements for
-which there is space in the data vector.
-
-2 Fill Pointer @\This is a fixnum indicating how many elements of the data
-vector are actually considered to be in use. Normally this is initialized to
-the same value as the Number of Elements field, but in some array applications
-it will be given a smaller value. Any access beyond the fill pointer is
-illegal.
-
-3 Displacement @\This fixnum value is added to the final code-vector index
-after the index arithmetic is done but before the access occurs. Used for
-mapping a portion of one array into another. For most arrays, this is 0.
-
-4 Range of First Index @\This is the number of index values along the first
-dimension, or one greater than the largest legal value of this index (since the
-arrays are always zero-based). A fixnum in the range 0 to 2@+[24]-1. If any
-of the indices has a range of 0, the array is legal but will contain no data
-and accesses to it will always be out of range. In a 0-dimension array, this
-entry will not be present.
-
-5 - N Ranges of Subsequent Dimensions
-@end [description]
-
-The number of dimensions of an array can be determined by looking at the length
-of the array header. The rank will be this number minus 6. The maximum array
-rank is 65535 - 6, or 65529.
-
-The ranges of all indices are checked on every access, during the conversion to
-a single data-vector index. In this conversion, each index is added to the
-accumulating total, then the total is multiplied by the range of the following
-dimension, the next index is added in, and so on. In other words, if the data
-vector is scanned linearly, the last array index is the one that varies most
-rapidly, then the index before it, and so on.
-
-@section [Symbols Known to the Assembler Routines]
-@label [Known-Objects]
-
-A large number of symbols will be pre-defined when a CMU Common Lisp system
-is fired up. A few of these are so fundamental to the operation of the
-system that their addresses have to be known to the assembler routines.
-These symbols are listed here. All of these symbols are in static space,
-so they will not move around.
-
-@begin [description]
-@index [NIL]
-NIL @\94000000@-[16] The value of NIL is always NIL; it is an error
-to alter it. The plist of NIL is always NIL; it is an error to alter
-it. NIL is unique among symbols in that it is stored in Cons cell
-space and thus you can take its CAR and CDR, yielding NIL in either
-case. NIL has been placed in Cons cell space so that the more common
-operations on lists will yield the desired results. This slows down
-some symbol operations but this should be insignificant compared to
-the savings in list operations. A test for NIL for the
-@value(DinkyMachine) is:
-@begin(Example)
- xiu R0,P,X'9400'
- bz IsNIL or bnz IsNotNIL
-@end(Example)
-
-@index [T]
-T @\8C000000@-[16] The value of T is always T; it is an error
-to alter it. A similar sequence of code as for NIL above can test for T,
-if necessary.
-
-@index [%SP-Internal-Apply]
-%SP-Internal-Apply @\8C000014@-[16] The function stored in the definition cell
-of this symbol is called by an assembler routine whenever compiled code calls
-an interpreted function.
-
-@index [%SP-Internal-Error]
-%SP-Internal-Error @\8C000028@-[16] The function stored in the definition cell
-of this symbol is called whenever an error is detected during the execution of
-an assembler routine. See section @ref[Errors] for details.
-
-@index [%SP-Software-Interrupt-Handler]
-%SP-Software-Interrupt-Handler @\8C00003C@-[16] The function stored in the
-definition cell of this symbol is called whenever a software interrupt occurs.
-See section @ref[Interrupts] for details.
-
-@index [%SP-Internal-Throw-Tag]
-%SP-Internal-Throw-Tag @\8C000050@-[16] This symbol is bound to the tag being
-thrown when a Catch-All frame is encountered on the stack. See section
-@ref[Catch] for details.
-
-@index [%Initial-function]
-%Initial-function@\8c000064@-[16] This symbol's function cell should contain
-a function that is called when the initial core image is started. This
-function should initialize all the data structures that Lisp needs to run.
-
-@index [%Link-table-header]
-%Link-table-header@\8c000078@-[16] This symbol's value cell contains a pointer
-to the link table information.
-
-@index [Current-allocation-space]
-Current-allocation-space@\8c00008c@-[16] This symbol's value cell contains
-an encoded form of the current space that new lisp objects are to be allocated
-in.
-
-@index [%SP-bignum/fixnum]
-%SP-bignum/fixnum@\8c0000a0@-[16] This function is invoked by the miscops
-when a division of a bignum by a fixnum results in a ratio.
-
-@index [%SP-fixnum/bignum]
-%SP-bignum/bignum@\8c0000b4@-[16] This
-function is invoked by the miscops when a division of a fixnum by a
-bignum results in a ratio.
-
-@index [%SP-bignum/bignum]
-%SP-bignum/bignum@\8c0000c8@-[16] This function is invoked by the miscops
-when a division of a bignum by a bignum results in a ratio.
-
-@index [%SP-abs-ratio]
-%SP-abs-ratio@\8c0000dc@-[16] This function is invoked by the miscops
-when the absolute value of a ratio is taken.
-
-@index [%SP-abs-complex]
-%SP-abs-complex@\8c0000f0@-[16] This function is invoked by the miscops
-when the absolute value of a complex is taken.
-
-@index [%SP-negate-ratio]
-%SP-negate-ratio@\8c000104@-[16] This function is invoked by the miscops
-when a ratio is to be negated.
-
-@index [%SP-negate-complex]
-%SP-negate-ratio@\8c000118@-[16] This function is invoked by the miscops
-when a complex is to be negated.
-
-@index[%SP-integer+ratio]
-%SP-integer+ratio@\8c00012c@-[16] This function is invoked by the miscops
-when a fixnum or bignum is added to a ratio.
-
-@index[%SP-ratio+ratio]
-%SP-ratio+ratio@\8c000140@-[16] This function is invoked by the miscops
-when a ratio is added to a ratio.
-
-@index[%SP-complex+number]
-%SP-complex+number@\8c000154@-[16] This function is invoked by the miscops
-when a complex is added to a number.
-
-@index[%SP-number+complex]
-%SP-number+complex@\8c000168@-[16] This function is invoked by the miscops
-when a number is added to a complex.
-
-@index[%SP-complex+complex]
-%SP-complex+complex@\8c00017c@-[16] This function is invoked by the miscops
-when a number is added to a complex.
-
-@index[%SP-1+ratio]
-%SP-1+ratio@\8c000190@-[16] This function is invoked by the miscops when
-1 is added to a ratio.
-
-@index[%SP-1+complex]
-%SP-1+complex@\8c000190@-[16] This function is invoked by the miscops when
-1 is added to a complex.
-
-@index[%SP-ratio-integer]
-%SP-ratio-integer@\8c0001b8@-[16] This function is invoked by the miscops
-when an integer is subtracted from a ratio.
-
-@index[%SP-ratio-ratio]
-%SP-ratio-ratio@\8c0001cc@-[16] This function is invoked by the miscops
-when an ratio is subtracted from a ratio.
-
-@index[%SP-complex-number]
-%SP-complex-number@\8c0001e0@-[16] This function is invoked by the miscops
-when a complex is subtracted from a number.
-
-@index[%SP-number-complex]
-%SP-number-complex@\8c0001f4@-[16] This function is invoked by the miscops
-when a number is subtracted from a complex.
-
-@index[%SP-complex-complex]
-%SP-complex-complex@\8c000208@-[16] This function is invoked by the miscops
-when a complex is subtracted from a complex.
-
-@index[%SP-1-complex]
-%SP-1-complex@\8c000230@-[16] This function is invoked by the miscops when
-1 is subtracted from a complex.
-
-@index[%SP-ratio*ratio]
-%SP-ratio*ratio@\8c000244@-[16] This function is invoked by the miscops to
-multiply two ratios.
-
-@index[%SP-number*complex]
-%SP-number*complex@\8c000258@-[16] This function is invoked by the miscops to
-multiply a number by a complex.
-
-@index[%SP-complex*number]
-%SP-complex*number@\8c00026c@-[16] This function is invoked by the miscops to
-multiply a complex by a number.
-
-@index[%SP-complex*complex]
-%SP-complex*complex@\8c000280@-[16] This function is invoked by the miscops
-to multiply a complex by a complex.
-
-@index[%SP-integer/ratio]
-%SP-integer/ratio@\8c000294@-[16] This function is invoked by the miscops to
-divide an integer by a ratio.
-
-@index[%SP-ratio/integer]
-%SP-ratio/integer@\8c0002a8@-[16] This function is invoked by the miscops to
-divide a ratio by an integer.
-
-@index[%SP-ratio/ratio]
-%SP-ratio/ratio@\8c0002bc@-[16] This function is invoked by the miscops to
-divide a ratio by a ratio.
-
-@index[%SP-number/complex]
-%SP-number/complex@\8c0002d0@-[16] This function is invoked by the miscops to
-divide a number by a complex.
-
-@index[%SP-complex/number]
-%SP-complex/number@\8c0002e4@-[16] This function is invoked by the miscops to
-divide a complex by a number.
-
-@index[%SP-complex/complex]
-%SP-complex/complex@\8c0002f8@-[16] This function is invoked by the miscops
-to divide a complex by a complex.
-
-@index[%SP-integer-truncate-ratio]
-%SP-integer-truncate-ratio@\8c00030c@-[16] This function is invoked by the
-miscops to truncate an integer by a ratio.
-
-@index[%SP-ratio-truncate-integer]
-%SP-ratio-truncate-integer@\8c000320@-[16] This function is invoked by the
-miscops to truncate a ratio by an integer.
-
-@index[%SP-ratio-truncate-ratio]
-%SP-ratio-truncate-ratio@\8c000334@-[16] This function is invoked by the
-miscops to truncate a ratio by a ratio.
-
-@index[%SP-number-truncate-complex]
-%SP-number-truncate-complex@\8c000348@-[16] This function is invoked by the
-miscops to truncate a number by a complex.
-
-@index[%SP-complex-truncate-number]
-%SP-complex-truncate-number@\8c00035c@-[16] This function is invoked by the
-miscops to truncate a complex by a number.
-
-@index[%SP-complex-truncate-complex]
-%SP-complex-truncate-complex@\8c000370@-[16] This function is invoked by
-the miscops to truncate a complex by a complex.
-
-@index[maybe-gc]
-Maybe-GC@\8c000384@-[16] This function may be invoked by any miscop that
-does allocation. This function determines whether it is time to garbage
-collect or not. If it is it performs a garbage collection. Whether it
-invokes a garbage collection or not, it returns the single argument passed
-to it.
-
-@index[Lisp-environment-list]
-Lisp-environment-list@\8c000398@-[16] The value of this symbol is
-set to the a list of the Unix environment strings passed into the Lisp
-process. This list by Lisp to obtain various environment information, such
-as the user's home directory, etc.
-
-@index[Call-lisp-from-c]
-Call-lisp-from-C@\8c0003ac@-[16] This function is called whenever a
-C function called by Lisp tries to call a Lisp function.
-
-@index[Lisp-command-line-list]
-Lisp-command-line-list@\8c0003c0@-[16] The value of this symbol is
-set to the list of strings passed into the Lisp process as the command
-line.
-
-@index[*Nameserverport*]
-*Nameserverport*@\8c0003d4@-[16] The value of this symbol is set to
-the C global variable name_server_port. This allows Lisp to access the
-name server.
-
-@index[*Ignore-Floating-Point-Underflow*]
-*Ignore-Floating-Point-Underflow*@\8c0003e8@-[16] If the the value of this
-symbol is NIL then an error is signalled when floating point underflow
-occurs, otherwise the operation quietly returns zero.
-@End[description]
-
-@chapter [Runtime Environment]
-@index [Runtime Environment]
-@label [Runtime]
-
-@section [Register Allocation]
-@index [Register allocation]
-To describe the assembler support routines in chapter @ref[Instr-Chapter] and
-the complicated
-control conventions in chapter @ref[Control-Conventions] requires that we talk
-about the allocation of the 16 32-bit general purpose registers provided
-by the @value(DinkyMachine).
-@begin [description]
-@index [Program-Counter register]
-Program-Counter (PC) [R15]@\This register contains an index into the current
-code vector when a Lisp function is about to be called. When a miscop is
-called, it contains the return address. It may be used as a super temporary
-between miscop and function calls.
-
-@index [Active-Function-Pointer register]
-Active-Function-Pointer (AF) [R14]@\This register contains a pointer to the
-active function object. It is used to access the symbol and constant area for
-the currently running function.
-
-@index [Active-Frame-Pointer register]
-Active-Frame-Pointer (FP) [R13]@\This register contains a pointer to the
-current active frame on the control stack. It is used to access the arguments
-and local variables stored on the control stack.
-
-@index [Binding-Stack-Pointer register]
-Binding-Stack-Pointer (BS) [R12]@\This register contains the current binding
-stack pointer. The binding stack is a downward growing stack and follows
-a decrement-write/increment-read discipline.
-
-@index [Local registers]
-Local registers (L0-L4) [R7-R11]@\These registers contain locals and saved
-arguments for the currently executing function. Functions may use these
-registers, so that stack accesses can be reduced, since a stack access is
-relatively expensive compared to a register access.
-
-@index [Argument registers]
-Argument register (A0, A1, A2) [R1, R3, R5]@\These registers contain arguments
-to a function or miscop that has just been called. On entry to a function
-or miscop, they contain the first three arguments. The first thing a function
-does is to move the contents of these registers into the local registers.
-
-@index [Miscop argument register]
-Miscop argument register (A3) [R4]@\This register is used to pass a fourth
-argument to miscops requiring four or more arguments. It is also used as a
-super temporary by the compiler.
-
-@index [Control-Stack-Pointer register]
-Control-Stack-Pointer (CS) [R6]@\The stack pointer for the control stack, an
-object of type Control-Stack-Pointer. Points to the last used word in
-Control-Stack space; this upward growing stack uses a
-increment-write/read-decrement discipline.
-
-@index [Non-Lisp temporary registers]
-Non-Lisp temporary registers (NL0, NL1) [R0, R2]@\These registers are used to
-contain non-Lisp values. They will normally be used during miscop calls, but
-may also be used in in-line code to contain temporary data. These are the only
-two registers never examined by the garbage collector, so no pointers to Lisp
-objects should be stored here (since they won't get updated during a garbage
-collection).
-@end [description]
-
-@section [Function Object Format]
-@label [Fn-Format]
-
-Each compiled function is represented in the machine as a Function
-Object. This is identical in form to a G-Vector of lisp objects, and
-is treated as such by the garbage collector, but it exists in a
-special function space. (There is no particular reason for this
-distinction. We may decide later to store these things in G-Vector
-space, if we become short on spaces or have some reason to believe
-that this would improve paging behavior.) Usually, the function
-objects and code vectors will be kept in read-only space, but nothing
-should depend on this; some applications may create, compile, and
-destroy functions often enough to make dynamic allocation of function
-objects worthwhile.
-
-@index [Code vector]
-@index [Constants in code] The function object contains a vector of
-header information needed by the function-calling mechanism: a
-pointer to the I-Vector that holds the actual code. Following this
-is the so-called ``symbols and constants'' area. The first few
-entries in this area are fixnums that give the offsets into the code
-vector for various numbers of supplied arguments. Following this
-begin the true symbols and constants used by the function. Any
-symbol used by the code as a special variable.
-Fixnum constants can be generated faster
-with in-line code than they can be accessed from the function-object,
-so they are not stored in the constants area.
-
-The subtype of the G-Vector header indicates the type of the function:
-@begin(Itemize, spacing 1, spread 0)
-0 - A normal function (expr).
-
-1 - A special form (fexpr).
-
-2 - A defmacro macroexpansion function.
-
-3 - An anonymous expr. The name is the name of the parent function.
-
-4 - A compiled top-level form.
-@end(Itemize)
-Only the fexpr information has any real meaning to the system. The rest
-is there for the printer and anyone else who cares.
-
-
-After the one-word G-Vector header, the entries of the function object
-are as follows:
-
-@begin [verbatim, group]
-0 Name of the innermost enclosing named function.
-1 Pointer to the unboxed Code vector holding the instructions.
-2 A fixnum with bit fields as follows:
- 24 - 31: The minimum legal number of args (0 to 255).
- 16 - 23: The maximum number of args, not counting &rest (0 to 255).
- The fixnum has a negative type code, if the function accepts a &rest
- arg and a positive one otherwise.
-3 A string describing the source file from which the function was defined.
- See below for a description of the format.
-4 A string containing a printed representation of the argument list, for
- documentation purposes. If the function is a defmacro macroexpansion
- function, the argument list will be the one originally given to defmacro
- rather than the actual arglist to the expansion function.
-5 The symbols and constants area starts here.
- This word is entry 0 of the symbol/constant area.
- The first few entries in this area are fixnums representing the
- code-vector entry points for various numbers of optional arguments.
-@end [verbatim]
-
-@section [Defined-From String Format]
-@label [Defined-From-String-Format]
-@index [Defined-From String Format]
-
-The defined-from string may have any of three different formats, depending
-on which of the three compiling functions compiled it:
-@begin(Description)
-compile-file "@i[filename user-time universal-time]"@\ The @i[filename] is
-the namestring of the truename of the file the function was defined from.
-The time is the file-write-date of the file.
-
-compile "Lisp on @i[user-time], machine @i[machine universal-time]"@\
-The time is the time that the function was compiled. @i[Machine] is the
-machine-instance of the machine on which the compilation was done.
-
-compile-from-stream "@i[stream] on @i[user-time], machine @i[machine-instance
-universal-time]"@\@i[Stream] is the printed representation of the stream
-compiled from. The time is the time the compilation started.
-@end(Description)
-
-An example of the format of @i[user-time] is 6-May-86 1:04:44. The
-@i[universal-time] is the same time represented as a decimal integer.
-It should be noted that in each case, the universal time is the last
-thing in the string.
-
-@section [Control-Stack Format]
-@label [Control-Stack-Format]
-@index [Control-stack format]
-
-The CMU Common Lisp control stack is a framed stack. Call frames, which hold
-information for function calls, are intermixed with catch frames, which hold
-information used for non-local exits. In addition, the control stack is used
-as a scratchpad for random computations.
-
-@subsection [Call Frames]
-@index [Open frame]
-@index [Active frame]
-
-At any given time, the machine contains pointers to the current top
-of the control stack and the start of the current active frame (in
-which the current function is executing). In addition, there is a
-pointer to the current top of the special binding stack. CMU Common Lisp
-on the Perq also has a pointer to an open frame. An open frame is
-one which has been partially built, but which is still having
-arguments for it computed. When all the arguments have been computed
-and saved on the frame, the function is then started. This means
-that the call frame is completed, becomes the current active frame,
-and the function is executed. At this time, special variables may be
-bound and the old values are saved on the binding stack. Upon
-return, the active frame is popped away and the result is either sent
-as an argument to some previously opened frame or goes to some other
-destination. The binding stack is popped and old values are
-restored.
-
-On the @value(DinkyMachine), open frames still exist, however, no register is
-allocated to point at the most recent one. Instead, a count of the arguments
-to the function is kept. In most cases, a known fixed number of arguments are
-passed to a function, and this is all that is needed to calculate the correct
-place to set the active frame pointer.
-In some cases, it is not as simple, and runtime calculations are necessary to
-set up the frame pointer. These calculations are simple except in some very
-strange cases.
-
-The active frame contains pointers to the previously-active frame and
-to the point to which the binding stack will be popped
-on exit, among other things. Following this is a vector of storage locations
-for the function's arguments and local variables. Space is allocated for the
-maximum number of arguments that the function can take, regardless of how many
-are actually supplied.
-
-In an open frame, stack space is allocated up to the point where the arguments
-are stored. Nothing is stored in the frame
-at this time. Thus, as arguments are computed, they can simply be pushed on
-the stack. Since the first three arguments are passed in registers, it is
-sometimes necessary to save these values when succeeding arguments are
-complicated. When the function is finally started, the remainder of the frame
-is built (including storing all the
-registers that must be saved). A call frame looks like this:
-@begin [verbatim, group]
-0 Saved local 0 register.
-1 Saved local 1 register.
-2 Saved local 2 register.
-3 Saved local 3 register.
-4 Saved local 4 register.
-5 Pointer to previous binding stack.
-6 Pointer to previous active frame.
-7 Pointer to previous active function.
-8 Saved PC of caller. A fixnum.
-9 Args-and-locals area starts here. This is entry 0.
-@end [verbatim]
-The first slot is pointed to by the Active-Frame register if this frame is
-currently active.
-
-@subsection [Catch Frames]
-@index [Catch]
-@index [Catch frames]
-
-Catch frames contain much of the same information that call frames
-do, and have a very similar format. A catch frame holds the function
-object for the current function, a stack pointer to the current
-active frame, a pointer to the current top of the binding stack, and
-a pointer to the previous catch frame. When a Throw occurs, an
-operation similar to returning from this catch frame (as if it
-were a call frame) is performed, and the stacks are unwound to the
-proper place for continued execution in the current function. A
-catch frame looks like this:
-@begin [verbatim, group]
-0 Pointer to current binding stack.
-1 Pointer to current active frame.
-2 Pointer to current function object.
-3 Destination PC for a Throw.
-4 Tag caught by this catch frame.
-5 Pointer to previous catch frame.
-@end [verbatim]
-The conventions used to manipulate call and catch frames are described in
-chapter @ref[Control-Conventions].
-
-@section [Binding-Stack Format]
-@index [Binding stack format]
-
-Each entry of the binding-stack consists of two boxed (32-bit) words. Pushed
-first is a pointer to the symbol being bound. Pushed second is the symbol's
-old value (any boxed item) that is to be restored when the binding stack is
-popped.
-
-@chapter [Storage Management]
-@index [Storage management]
-@index [Garbage Collection]
-@label [Alloc-Chapter]
-
-@index [Free-Storage pointer]
-@index [Clean-Space pointer]
-New objects are allocated from the lowest unused addresses within the specified
-space. Each allocation call specifies how many words are wanted, and a
-Free-Storage pointer is incremented by that amount. There is one of these
-Free-Storage pointers for each space, and it points to the lowest free address
-in the space. There is also a Clean-Space pointer associated with each space
-that is used during garbage collection. These pointers are stored in a table
-which is indexed by the type and space code. The
-address of the Free-Storage pointer for a given space is
-@begin[verbatim]
- (+ alloc-table-base (lsh type 5) (lsh space 3)).
-@end[verbatim]
-The address of the Clean-Space pointer is
-@begin[verbatim]
- (+ alloc-table-base (lsh type 5) (lsh space 3) 4).
-@end[verbatim]
-
-Common Lisp on the @value(DinkyMachine) uses a stop-and-copy garbage collector
-to reclaim storage. The Collect-Garbage miscop performs a full GC. The
-algorithm used is a degenerate form of Baker's incremental garbage collection
-scheme. When the Collect-Garbage miscop is executed, the following
-happens:
-@begin[enumerate]
-The current newspace becomes oldspace, and the current oldspace becomes
-newspace.
-
-The newspace Free-Storage and Clean-Space pointers are initialized to point to
-the beginning of their spaces.
-
-The objects pointed at by contents of all the registers containing Lisp objects
-are transported if necessary.
-
-The control stack and binding stack are scavenged.
-
-Each static pointer space is scavenged.
-
-Each new dynamic space is scavenged. The scavenging of the dynamic spaces
-continues until an entire pass through all of them does not result in anything
-being transported. At this point, every live object is in newspace.
-@end[enumerate]
-A Lisp-level GC function returns the oldspace pages to Mach.
-
-@index [Transporter]
-@section [The Transporter]
-The transporter moves objects from oldspace to newspace. It is given an
-address @i[A], which contains the object to be transported, @i[B]. If @i[B] is
-an immediate object, a pointer into static space, a pointer into read-only
-space, or a pointer into newspace, the transporter does nothing.
-
-If @i[B] is a pointer into oldspace, the object it points to must be
-moved. It may, however, already have been moved. Fetch the first
-word of @i[B], and call it @i[C]. If @i[C] is a GC-forwarding
-pointer, we form a new pointer with the type code of @i[B] and the
-low 27 bits of @i[C]. Write this into @i[A].
-
-If @i[C] is not a GC-forwarding pointer, we must copy the object that
-@i[B] points to. Allocate a new object of the same size in newspace,
-and copy the contents. Replace @i[C] with a GC-forwarding pointer to
-the new structure, and write the address of the new structure back
-into @i[A].
-
-Hash tables maintained with an EQ relation need special treatment by the
-transporter. Whenever a G-Vector with subtype 2 or 3 is transported to
-newspace, its subtype code is changed to 4. The Lisp-level hash-table
-functions will see that the subtype code has changed, and re-hash the entries
-before any access is made.
-
-@index [Scavenger]
-@section [The Scavenger] The scavenger looks through an area of
-pointers for pointers into oldspace, transporting the objects they
-point to into newspace. The stacks and static spaces need to be
-scavenged once, but the new dynamic spaces need to be scavenged
-repeatedly, since new objects will be allocated while garbage
-collection is in progress. To keep track of how much a dynamic space
-has been scavenged, a Clean-Space pointer is maintained. The
-Clean-Space pointer points to the next word to be scavenged. Each
-call to the scavenger scavenges the area between the Clean-Space
-pointer and the Free-Storage pointer. The Clean-Space pointer is
-then set to the Free-Storage pointer. When all Clean-Space pointers
-are equal to their Free-Storage pointers, GC is complete.
-
-To maintain (and create) locality of list structures, list space is
-treated specially. When a list cell is transported, if the cdr points
-to oldspace, it is immediately transported to newspace. This continues until
-the end of the list is encountered or a non-oldspace pointer occurs in the cdr
-position. This linearizes lists in the cdr direction which should
-improve paging performance.
-
-@section [Purification]
-@index [Purification]
-@label [PURIFY]
-
-Garbage is created when the files that make up a CMU Common Lisp system are
-loaded. Many functions are needed only for initialization and
-bootstrapping (e.g. the ``one-shot'' functions produced by the compiler for
-random forms between function definitions), and these can be thrown away
-once a full system is built. Most of the functions in the system, however,
-will be used after initialization. Rather than bend over backwards to make
-the compiler dump some functions in read-only space and others in dynamic
-space (which involves dumping their constants in the proper spaces, also),
-@i[everything] is dumped into dynamic space. A purify miscop is provided
-that does a garbage collection and moves accessible information in dynamic
-space into read-only or static space.
-
-@chapter [Assembler Support Routines]
-@label [Instr-Chapter]
-@index [Assembler Support Routines]
-
-To support compiled Common Lisp code many hand coded assembler
-language routines (miscops) are required. These routines accept
-arguments in the three argument registers, the special miscop
-argument register, and in a very few cases on the stack. The current
-register assignments are:
-@begin(Itemize, spread 0, spacing 1)
-A0 contains the first argument.
-
-A1 contains the second argument.
-
-A2 contains the third argument.
-
-A3 contains the fourth argument.
-@end(itemize)
-The rest of the arguments are passed on the stack with the last
-argument at the end of the stack. All arguments on the stack must be
-popped off the stack by the miscop. All miscops return their
-values in register A0. A few miscops return two or three values,
-these are all placed in the argument registers. The main return
-value is stored in register A0, the others in A1 and A2. The
-compiler must generate code to use the multiple values correctly,
-i.e., place the return values on the stack and put a values marker in
-register A0 if multiple-values are wanted. Otherwise the compiler
-can use the value(s) it needs and ignore the rest. NB: Most of the
-miscops follow this scheme, however, a few do not. Any
-discrepancies are explained in the description of particular
-miscops.
-
-Several of the instructions described in the Perq Internal Design Document do
-not have associated miscops, rather they have been code directly in-line.
-Examples of these instructions include push, pop, bind, bind-null, many of the
-predicates, and a few other instructions. Most of these instructions can be
-performed in 4 or fewer @value(DinkyMachine) instructions and the overhead of
-calling a miscop seemed overly expensive. Some instructions are encoded
-in-line or as a miscop call depending on settings of compiler optimization
-switches. If space is more important than speed, then some Perq instructions
-are compiled as calls to out of line miscops rather than generating in-line
-code.
-
-@section [Miscop Descriptions]
-@label[macro-codes]
-
-There are 10 classes of miscops: allocation, stack manipulation,
-list manipulation, symbol manipulation, array manipulation, type predicate,
-arithmetic and logical, function call and return,
-miscellaneous, and system hacking.
-
-@subsection [Allocation]
-@instrsection
-All non-immediate objects are allocated in the ``current allocation space,''
-which is dynamic space, static space, or read-only space. The current
-allocation space is initially dynamic space, but can be changed by using the
-Set-Allocation-Space miscop below. The current allocation space can be
-determined by using the Get-Allocation-Space miscop. One usually wants to
-change the allocation space around some section of code; an unwind protect
-should be used to insure that the allocation space is restored to some safe
-value.
-
-@begin(Description)
-@index [Get-Allocation-Space]
-Get-Allocation-Space (@i[])@\returns 0, 2, or 3 if the current allocation
-space is dynamic, static, or read-only, respectively.
-
-@index [Set-Allocation-Space]
-Set-Allocation-Space (@i[X])@\sets the current allocation space to dynamic,
-static, or read-only if @i[X] is 0, 2, or 3 respectively. Returns @i[X].
-
-@index [Alloc-Bit-Vector]
-Alloc-Bit-Vector (Length)@\returns a new bit-vector @i[Length] bits long,
-which is allocated in the current allocation space. @i[Length] must be a
-positive fixnum.
-
-@index [Alloc-I-Vector]
-Alloc-I-Vector (@i[Length A])@\returns a new I-Vector @i[Length]
-bytes long, with the access code specified by @i[A]. @i[Length] and
-@i[A] must be positive fixnums.
-
-@index [Alloc-String]
-Alloc-String (@i[Length])@\ returns a new string @i[Length] characters long.
-@i[Length] must be a fixnum.
-
-@index [Alloc-Bignum]
-Alloc-Bignum (@i[Length])@\returns a new bignum @i[Length] 32-bit words long.
-@i[Length] must be a fixnum.
-
-@index [Make-Complex]
-Make-Complex (@i[Realpart Imagpart])@\returns a new complex number with the
-specified @i[Realpart] and @i[Imagpart]. @i[Realpart] and @i[Imagpart] should
-be the same type of non-complex number.
-
-@index [Make-Ratio]
-Make-Ratio (@i[Numerator Denominator])@\returns a new ratio with the
-specified @i[Numerator] and @i[Denominator]. @i[Numerator] and
-@i[Denominator] should be integers.
-
-@index [Alloc-G-Vector]
-Alloc-G-Vector (@i[Length Initial-Element])@\returns a new G-Vector
-with @i[Length] elements initialized to @i[Initial-Element].
-@i[Length] should be a fixnum.
-
-@index [Static-Alloc-G-Vector]
-Static-G-Vector (@i[Length Initial-Element])@\returns a new G-Vector in
-static allocation space with @i[Length] elements initialized to
-@i[Initial-Element].
-
-@index [Vector]
-Vector (@i[Elt@-[0] Elt@-[1] ... Elt@-[Length - 1] Length])@\returns a new
-G-Vector containing the specified @i[Length] elements. @i[Length] should be a
-fixnum and is passed in register A0. The rest of the arguments are passed on
-the stack.
-
-@index [Alloc-Function]
-Alloc-Function (@i[Length])@\returns a new function with @i[Length] elements.
-@i[Length] should be a fixnum.
-
-@index [Alloc-Array]
-Alloc-Array (@i[Length])@\returns a new array with @i[Length] elements.
-@i[Length] should be a fixnum.
-
-@index [Alloc-Symbol]
-Alloc-Symbol (@i[Print-Name])@\returns a new symbol with the print-name as
-@i[Print-Name]. The value is initially Trap, the definition is Trap,
-the property list and the package are initially NIL. The symbol is
-not interned by this operation -- that is done in Lisp code.
-@i[Print-Name] should be a simple-string.
-
-@index [Cons]
-Cons (@i[Car Cdr])@\returns a new cons with the specified @i[Car] and @i[Cdr].
-
-@index [List]
-List (@i[Elt@-[0] Elt@-[1] ... Elt@-[CE - 1] Length])@\returns a new list
-containing the @i[Length] elements. @i[Length] should be fixnum and is
-passed in register NL0. The first three arguments are passed in A0, A1, and
-A2. The rest of the arguments are passed on the stack.
-
-@index [List*]
-List* (@i[Elt@-[0] Elt@-[1] ... Elt@-[CE - 1] Length])@\returns a list* formed
-by the @i[Length-1] elements. The last element is placed in the cdr of the
-last element of the new list formed. @i[Length] should be a fixnum and is
-passed in register NL0. The first three arguments are passed in A0, A1, and
-A2. The rest of the arguments are passed on the stack.
-
-@index[mv-list]
-MV-List (@i[Elt@-<0> Elt@-<1> ... Elt@-<CE - 1> Length])@\returns a list
-formed from the elements, all of which are on the stack. @i[Length] is
-passed in register A0. This miscop is invoked when multiple values from
-a function call are formed into a list.
-@end(Description)
-
-@subsection [Stack Manipulation]
-@instrsection
-
-@begin(Description)
-@index [Push]
-Push (@i[E])@\pushes E on to the control stack.
-
-@index [Pop]
-Pop (@i[E])@\pops the top item on the control stack into @i[E].
-
-@index [NPop]
-NPop (@i[N])@\If @i[N] is positive, @i[N] items are popped off of the stack.
-If @i[N] is negative, NIL is pushed onto the stack -@i[N] times. @i[N] must be
-a fixnum.
-
-@index [Bind-Null]
-Bind-Null (@i[E])@\pushes @i[E] (which must be a symbol) and its current value
-onto the binding stack, and sets the value of @i[E] to NIL. Returns NIL.
-
-@index [Bind]
-Bind (Value Symbol)@\pushes @i[Symbol] (which must be a symbol) and its current
-value onto the binding stack, and sets the value cell of @i[Symbol] to
-@i[Value]. Returns @i[Symbol].
-
-@index [Unbind]
-Unbind (@i[N])@\undoes the top @i[N] bindings on the binding stack.
-@end(Description)
-
-@subsection [List Manipulation]
-@instrsection
-
-@begin(Description)
-@index [Car]
-@index [Cdr]
-@index [Caar]
-@index [Cadr]
-@index [Cdar]
-@index [Cddr]
-Car, Cdr, Caar, Cadr, Cdar, Cddr (@i[E])@\returns the car, cdr, caar, cadr,
-cdar, or cddr of @i[E] respectively.
-
-@index [Set-Cdr]
-@index [Set-Cddr]
-Set-Cdr, Set-Cddr (@i[E])@\The cdr or cddr of the contents of @i[E] is stored
-in @i[E]. The contents of @i[E] should be either a list or NIL.
-
-@index [Set-Lpop]
-Set-Lpop (@i[E])@\The car of the contents of @i[E] is returned;
-the cdr of the contents of @i[E] is stored in @i[E]. The contents of @i[E]
-should be a list or NIL.
-
-@index [Spread]
-Spread (@i[E])@\pushes the elements of the list @i[E] onto the stack in
-left-to-right order.
-
-@index [Replace-Car]
-@index [Replace-Cdr]
-Replace-Car, Replace-Cdr (@i[List Value])@\sets the car or cdr of the @i[List]
-to @i[Value] and returns @i[Value].
-
-@index [Endp]
-Endp (X)@\sets the condition code eq bit to 1 if @i[X] is NIL, or 0 if @i[X] is
-a cons cell. Otherwise an error is signalled.
-
-@index [Assoc]
-@index [Assq]
-Assoc, Assq (@i[List Item])@\returns the first cons in the association-list
-@i[List] whose car is EQL to @i[Item]. If the = part of the EQL comparison
-bugs out (and it can if the numbers are too complicated), a Lisp-level Assoc
-function is called with the current cdr of the @i[List]. Assq returns the
-first cons in the association-list @i[List] whose car is EQ to @i[Item].
-
-@index [Member]
-@index [Memq] Member, Memq (@i[List Item])@\returns the first cons in
-the list @i[List] whose car is EQL to @i[Item]. If the = part of the
-EQL comparison bugs out, a Lisp-level Member function is called with
-the current cdr of the @i[List]. Memq returns the first cons in
-@i[List] whose car is EQ to the @i[Item].
-
-@index [GetF]
-
-GetF (@i[List Indicator Default])@\searches for the @i[Indicator] in
-the list @i[List], cddring down as the Common Lisp form GetF would.
-If @i[Indicator] is found, its associated value is returned,
-otherwise @i[Default] is returned.
-@end(Description)
-
-@subsection [Symbol Manipulation]
-@instrsection
-
-Most of the symbol manipulation miscops are compiled in-line rather than
-actual calls.
-
-@begin(Description)
-@index [Get-Value]
-Get-Value (@i[Symbol])@\returns the value of @i[Symbol] (which must be a
-symbol). An error is signalled if @i[Symbol] is unbound.
-
-@index [Set-Value]
-Set-Value (@i[Symbol Value])@\sets the value cell of the symbol @i[Symbol] to
-@i[Value]. @i[Value] is returned.
-
-@index [Get-Definition]
-Get-Definition (@i[Symbol])@\returns the definition of the symbol
-@i[Symbol]. If @i[Symbol] is undefined, an error is signalled.
-
-@index [Set-Definition]
-Set-Definition (@i[Symbol Definition])@\sets the definition of the symbol
-@i[Symbol] to @i[Definition]. @i[Definition] is returned.
-
-@index [Get-Plist]
-Get-Plist (@i[Symbol])@\returns the property list of the symbol @i[Symbol].
-
-@index [Set-Plist]
-Set-Plist (@i[Symbol Plist])@\sets the property
-list of the symbol @i[Symbol] to
-@i[Plist]. @i[Plist] is returned.
-
-@index [Get-Pname]
-Get-Pname (@i[Symbol])@\returns the print name of the symbol @i[Symbol].
-
-@index [Get-Package]
-Get-Package (@i[Symbol])@\returns the package cell of the symbol @i[Symbol].
-
-@index [Set-Package]
-Set-Package (@i[Symbol Package])@\sets the package cell of the symbol
-@i[Symbol] to @i[Package]. @i[Package] is returned.
-
-@index [Boundp]
-Boundp (@i[Symbol])@\sets the eq condition code bit to 1 if the symbol
-@i[Symbol] is bound; sets it to 0 otherwise.
-
-@index [FBoundp]
-FBoundp (@i[Symbol])@\sets the eq condition code bit to 1 if the symbol
-@i[Symbol] is defined; sets it to 0 otherwise.
-
-@index [Get]
-Get (@i[Symbol] @i[Indicator] @i[Default])@\searches the property list of
-@i[Symbol] for @i[Indicator] and returns the associated value. If
-@i[Indicator] is not found, @i[Default] is returned.
-
-@index [Put]
-Put (@i[Symbol] @i[Indicator] @i[Value])@\searches the property list of
-@i[Symbol] for @i[Indicator] and replaces the associated value with @i[Value].
-If @i[Indicator] is not found, the @i[Indicator] @i[Value] pair are consed onto
-the front of the property list.
-@end(Description)
-
-@subsection [Array Manipulation]
-@instrsection
-
-Common Lisp arrays have many manifestations in CMU Common Lisp. The CMU
-Common Lisp data types Bit-Vector, Integer-Vector, String, General-Vector,
-and Array are used to implement the collection of data types the Common
-Lisp manual calls ``arrays.''
-
-In the following miscop descriptions, ``simple-array'' means an array
-implemented in CMU Common Lisp as a Bit-Vector, I-Vector, String, or
-G-Vector. ``Complex-array'' means an array implemented as a CMU Common Lisp
-Array object. ``Complex-bit-vector'' means a bit-vector implemented as a
-CMU Common Lisp array; similar remarks apply for ``complex-string'' and so
-forth.
-
-@begin(Description)
-@index [Vector-Length] @index [G-Vector-Length] @index
-[Simple-String-Length] @index [Simple-Bit-Vector-Length] Vector-Length
-(@i[Vector])@\returns the length of the one-dimensional Common Lisp array
-@i[Vector]. G-Vector-Length, Simple-String-Length, and
-Simple-Bit-Vector-Length return the lengths of G-Vectors, CMU Common Lisp
-strings, and CMU Common Lisp Bit-Vectors respectively. @i[Vector] should
-be a vector of the appropriate type.
-
-@index [Get-Vector-Subtype]
-Get-Vector-Subtype (@i[Vector])@\returns the subtype field of the vector
-@i[Vector] as an integer. @i[Vector] should be a vector of some sort.
-
-@index [Set-Vector-Subtype]
-Set-Vector-Subtype (@i[Vector A])@\sets the subtype field of the vector
-@i[Vector] to @i[A], which must be a fixnum.
-
-@index [Get-Vector-Access-Code]
-Get-Vector-Access-Code (@i[Vector])@\returns the access code of the I-Vector
-(or Bit-Vector) @i[Vector] as a fixnum.
-
-@index [Shrink-Vector]
-Shrink-Vector (@i[Vector Length])@\sets the length field and the
-number-of-entries field of the vector @i[Vector] to @i[Length]. If the vector
-contains Lisp objects, entries beyond the new end are set to Trap.
-Returns the shortened vector. @i[Length] should be a fixnum. One cannot
-shrink array headers or function headers.
-
-@index [Typed-Vref]
-Typed-Vref (@i[A Vector I])@\returns the @i[I]'th element of the I-Vector
-@i[Vector] by indexing into it as if its access-code were @i[A]. @i[A] and
-@i[I] should be fixnums.
-
-@index [Typed-Vset]
-Typed-Vset (@i[A Vector I Value])@\sets the @i[I]'th element of the I-Vector
-@i[Vector] to @i[Value] indexing into @i[Vector] as if its access-code were
-@i[A]. @i[A], @i[I], and @i[Value] should be fixnums. @i[Value] is returned.
-
-@index [Header-Length]
-Header-Length (@i[Object])@\returns the number of Lisp objects in the header of
-the function or array @i[Object]. This is used to find the number of
-dimensions of an array or the number of constants in a function.
-
-@index [Header-Ref]
-Header-Ref (@i[Object I])@\returns the @i[I]'th element of the function or
-array header @i[Object]. @i[I] must be a fixnum.
-
-@index [Header-Set]
-Header-Set (@i[Object I Value])@\sets the @i[I]'th element of the function of
-array header @i[Object] to @i[Value], and pushes @i[Value]. @i[I] must be a
-fixnum.
-@end(Description)
-
-The names of the miscops used to reference and set elements of arrays are
-based somewhat on the Common Lisp function names. The SVref, SBit, and SChar
-miscops perform the same operation as their Common Lisp namesakes --
-referencing elements of simple-vectors, simple-bit-vectors, and simple-strings
-respectively. Aref1 references any kind of one dimensional array.
-The names of setting functions are derived by replacing ``ref'' with ``set'',
-``char'' with ``charset'', and ``bit'' with ``bitset.''
-
-@begin(Description)
-@index [Aref1]
-@index [SVref]
-@index [SChar]
-@index [SBit]
-Aref1, SVref, SChar, SBit (@i[Array I])@\returns the @i[I]'th element of the
-one-dimensional
-array @i[Array]. SVref pushes an element of a G-Vector; SChar an element of a
-string; Sbit an element of a Bit-Vector. @i[I] should be a fixnum.
-
-@index [Aset1]
-@index [SVset]
-@index [SCharset]
-@index [SBitset]
-Aset1, SVset, SCharset, SBitset (@i[Array I Value])@\sets the @i[I]'th element
-of the one-dimensional
-array @i[Array] to @i[Value]. SVset sets an element of a G-Vector; SCharset an
-element of a string; SBitset an element of a Bit-Vector. @i[I] should be a
-fixnum and @i[Value] is returned.
-
-@index [CAref2]
-@index [CAref3]
-CAref2, CAref3 (@i[Array I1 I2])@\returns the element (@i[I1], @i[I2]) of the
-two-dimensional array @i[Array]. @i[I1] and @i[I2] should be
-fixnums. CAref3 pushes the element (@i[I1], @i[I2], @i[I3]).
-
-@index [CAset2]
-@index [CAset3]
-CAset2, CAset3 (@i[Array I1 I2 Value]) @\sets the element (@i[I1], @i[I2]) of
-the two-dimensional array @i[Array] to @i[Value] and returns @i[Value].
-@i[I1] and @i[I2] should be fixnums. CAset3 sets the element (@i[I1], @i[I2],
-@i[I3]).
-
-@index [Bit-Bash]
-Bit-Bash (@i[V1 V2 V3 Op])@\@i[V1], @i[V2], and @i[V3] should be bit-vectors
-and @i[Op] should be a fixnum. The elements of the bit vector @i[V3] are
-filled with the result of @i[Op]'ing the corresponding elements of @i[V1] and
-@i[V2]. @i[Op] should be a Boole-style number (see the Boole miscop in
-section @ref[Boole-Section]).
-@end(Description)
-
-The rest of the miscops in this section implement special cases of sequence or
-string operations. Where an operand is referred to as a string, it may
-actually be an 8-bit I-Vector or system area pointer.
-
-@begin(Description)
-@index [Byte-BLT]
-Byte-BLT (@i[Src-String Src-Start Dst-String Dst-Start Dst-End])@\
-moves bytes from @i[Src-String] into @i[Dst-String] between @i[Dst-Start]
-(inclusive) and @i[Dst-End] (exclusive). @i[Dst-Start] - @i[Dst-End] bytes are
-moved. If the substrings specified overlap, ``the right thing happens,'' i.e.
-all the characters are moved to the right place. This miscop corresponds
-to the Common Lisp function REPLACE when the sequences are simple-strings.
-
-@index [Find-Character]
-Find-Character (@i[String Start End Character])@\
-searches @i[String] for the @i[Character] from @i[Start] to @i[End]. If the
-character is found, the corresponding index into @i[String] is returned,
-otherwise NIL is returned. This miscop corresponds to the Common Lisp
-function FIND when the sequence is a simple-string.
-
-@index [Find-Character-With-Attribute]
-Find-Character-With-Attribute (@i[String Start End Table Mask])@\
-The codes of the characters of @i[String] from @i[Start] to @i[End] are used as
-indices into the @i[Table], which is an I-Vector of 8-bit bytes. When the
-number picked up from the table bitwise ANDed with @i[Mask] is non-zero, the
-current index into the @i[String] is returned.
-
-@index [SXHash-Simple-String]
-SXHash-Simple-String (@i[String Length])@\Computes the hash code of the first
-@i[Length] characters of @i[String] and pushes it on the stack. This
-corresponds to the Common Lisp function SXHASH when the object is a
-simple-string. The @i[Length] operand can be Nil, in which case the length of
-the string is calculated in assembler.
-@end(Description)
-
-@subsection [Type Predicates]
-@instrsection
-
-Many of the miscops described in this sub-section can be coded in-line rather
-than as miscops. In particular, all the predicates on basic types are coded
-in-line with default optimization settings in the compiler. Currently, all of
-these predicates set the eq condition code bit to return an indication of
-whether the predicate is true or false. This is so that the
-@value(DinkyMachine) branch instructions can be used directly without having to
-test for NIL. However, this only works if the value of the predicate is needed
-for a branching decision. In the cases where the value is actually needed, T
-or NIL is generated in-line according to whether the predicate is true or
-false. At some point it might be worthwhile having two versions of these
-predicates, one which sets the eq condition code bit, and one which returns T
-or NIL. This is especially true if space becomes an issue.
-
-@begin(Description)
-@index [Bit-Vector-P]
-Bit-Vector-P (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is
-a Common Lisp bit-vector or 0 if it is not.
-
-@index [Simple-Bit-Vector-P]
-Simple-Bit-Vector-P (@i[Object])@\sets the eq condition code bit to 1 if
-@i[Object] is a CMU Common Lisp bit-vector or 0 if it is not.
-
-@index [Simple-Integer-Vector-P]
-Simple-Integer-Vector-P (@i[Object])@\sets the eq condition code bit to 1
-if @i[Object] is a CMU Common Lisp I-Vector or 0 if it is not.
-
-@index [StringP]
-StringP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-Common Lisp string or 0 if it is not.
-
-@index [Simple-String-P]
-Simple-String-P (@i[Object])@\sets the eq condition code bit to 1 if
-@i[Object] is a CMU Common Lisp string or 0 if it is not.
-
-@index [BignumP]
-BignumP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-bignum or 0 if it is not.
-
-@index [Long-Float-P]
-Long-Float-P (@i[Object])@\sets the eq condition code bit to 1 if
-@i[Object] is a long-float or 0 if it is not.
-
-@index [ComplexP]
-ComplexP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-complex number or 0 if it is not.
-
-@index [RatioP]
-RatioP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-ratio or 0 if it is not.
-
-@index [IntegerP]
-IntegerP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-fixnum or bignum or 0 if it is not.
-
-@index [RationalP]
-RationalP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-fixnum, bignum, or ratio or 0 if it is not.
-
-@index [FloatP]
-FloatP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-short-float or long-float or 0 if it is not.
-
-@index [NumberP]
-NumberP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-number or 0 if it is not.
-
-@index [General-Vector-P]
-General-Vector-P (@i[Object])@\sets the eq condition code bit to 1 if
-@i[Object] is a Common Lisp general vector or 0 if it is not.
-
-@index [Simple-Vector-P]
-Simple-Vector-P (@i[Object])@\sets the eq condition code bit to 1 if @i[Object]
-is a CMU Common Lisp G-Vector or 0 if it is not.
-
-@index [Compiled-Function-P]
-Compiled-Function-P (@i[Object])@\sets the eq condition code bit to 1 if
-@i[Object] is a compiled function or 0 if it is not.
-
-@index [ArrayP]
-ArrayP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-Common Lisp array or 0 if it is not.
-
-@index [VectorP]
-VectorP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-Common Lisp vector of 0 if it is not.
-
-@index [Complex-Array-P]
-Complex-Array-P (@i[Object])@\sets the eq condition code bit to 1 if @i[Object]
-is a CMU Common Lisp array or 0 if it is not.
-
-@index [SymbolP]
-SymbolP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-symbol or 0 if it is not.
-
-@index [ListP]
-ListP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a cons
-or NIL or 0 if it is not.
-
-@index [ConsP]
-ConsP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a cons
-or 0 if it is not.
-
-@index [FixnumP]
-FixnumP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is a
-fixnum or 0 if it is not.
-
-@index [Single-Float-P]
-Single-Float-P (@i[Object])@\sets the eq condition code bit to 1 if @i[Object]
-is a single-float or 0 if it is not.
-
-@index [CharacterP]
-CharacterP (@i[Object])@\sets the eq condition code bit to 1 if @i[Object] is
-a character or 0 if it is not.
-@end(Description)
-
-@subsection [Arithmetic]
-@instrsection
-
-@begin(Description)
-@index [Integer-Length]
-Integer-Length (@i[Object])@\returns the integer-length (as defined in the
-Common Lisp manual) of the integer @i[Object].
-
-@index [Logcount]
-Logcount (@i[Object])@\returns the number of 1's if @i[object] is a
-positive integer, the number of 0's if @i[object] is a negative integer,
-and signals an error otherwise.
-
-@index [Float-Short]
-Float-Short (@i[Object])@\returns a short-float corresponding to the number
-@i[Object].
-
-@index [Float-Long]
-Float-Long (@i[Number])@\returns a long float formed by coercing @i[Number] to
-a long float. This corresponds to the Common Lisp function Float when given a
-long float as its second argument.
-
-@index [Realpart]
-Realpart (@i[Number])@\returns the realpart of the @i[Number].
-
-@index [Imagpart]
-Imagpart (@i[Number])@\returns the imagpart of the @i[Number].
-
-@index [Numerator]
-Numerator (@i[Number])@\returns the numerator of the rational @i[Number].
-
-@index [Denominator]
-Denominator (@i[Number])@\returns the denominator of the rational @i[Number].
-
-@index [Decode-Float]
-Decode-Float (@i[Number])@\performs the Common Lisp Decode-Float function,
-returning 3 values.
-
-@index [Scale-Float]
-Scale-Float (@i[Number X])@\performs the Common Lisp Scale-Float function,
-returning the result.
-
-@index[=]
-= (@i[X Y])@\sets the condition codes according to whether @i[X] is equal
-to @i[Y]. Both @i[X] and @i[Y] must be numbers, otherwise an error is
-signalled. If a rational is compared with a flonum, the rational is
-converted to a flonum of the same type first. If a short flonum is compared
-with a long flonum, the short flonum is converted to a long flonum.
-Flonums must be exactly equal (after conversion) for the condition codes to
-be set to equality. This miscop also deals with complex numbers.
-
-@index [Compare]
-Compare (@i[X Y])@\sets the condition codes according to
-whether @i[X] is less than, equal to, or greater than @i[Y]. @i[X]
-and @i[Y] must be numbers. Conversions as described in = above are done
-as necessary. This miscop replaces the < and > instructions on the Perq,
-so that the branch on condition instructions can be used more
-effectively. The value of < and > as defined for the Perq are
-only generated if necessary, i.e., the result is saved. If @i[X] or @i[Y]
-is a complex number, an error is signalled.
-
-@index [Truncate]
-Truncate (@i[N X])@\performs the Common Lisp TRUNCATE operation. There are 3
-cases depending on @i[X]:
-@Begin[Itemize]
-If @i[X] is fixnum 1, return two items: a fixnum or bignum
-representing the integer part of @i[N] (rounded toward 0), then either 0 if
-@i[N] was already an integer or the fractional part of @i[N] represented as a
-flonum or ratio with the same type as @i[N].
-
-If @i[X] and @i[N] are both fixnums or bignums and @i[X] is not 1, divide
-@i[N] by @i[X]. Return two items: the integer quotient (a fixnum or
-bignum) and the integer remainder.
-
-If either @i[X] or @i[N] is a flonum or ratio, return a fixnum or bignum
-quotient (the true quotient rounded toward 0), then a flonum or ratio
-remainder. The type of the remainder is determined by the same type-coercion
-rules as for +. The value of the remainder is equal to @i[N] - @i[X] *
-@i[Quotient].
-@End[Itemize]
-On the @value(DinkyMachine), the integer part is returned in register A0, and
-the remainder in A1.
-
-@index [+]
-@index [-]
-@index [*]
-@index [/]
-+, -, *, / (@i[N X])@\returns @i[N] + @i[X]. -, *, and / are similar.
-
-@index [Fixnum*Fixnum]
-@index [Fixnum/Fixnum]
-Fixnum*Fixnum, Fixnum/Fixnum (@i[N X])@\returns @i[N] * @i[X], where
-both @i[N] and @i[X] are fixnums. Fixnum/ is similar.
-
-@index [1+]
-1+ (@i[E])@\returns @i[E] + 1.
-
-@index [1-]
-1- (@i[E])@\returns @i[E] - 1.
-
-@index [Negate]
-Negate (@i[N])@\returns -@i[N].
-
-@index [Abs]
-Abs (@i[N])@\returns |@i[N]|.
-
-@index [GCD]
-GCD (@i[N X])@\returns the greatest common divisor of the integers @i[N] and @i[X].
-
-@index [Logand]
-@index [Logior]
-@index [Logxor]
-Logand (@i[N X])@\returns the bitwise and of the integers @i[N] and @i[X].
-Logior and Logxor are analogous.
-
-@index [Lognot]
-Lognot (@i[N])@\returns the bitwise complement of @i[N].
-
-@index [Boole]
-@label [Boole-Section]
-Boole (@i[Op X Y])@\performs the Common Lisp Boole operation @i[Op] on @i[X],
-and @i[Y]. The Boole constants for CMU Common Lisp are these:
-@begin [verbatim, group]
- boole-clr 0
- boole-set 1
- boole-1 2
- boole-2 3
- boole-c1 4
- boole-c2 5
- boole-and 6
- boole-ior 7
- boole-xor 8
- boole-eqv 9
- boole-nand 10
- boole-nor 11
- boole-andc1 12
- boole-andc2 13
- boole-orc1 14
- boole-orc2 15
-@end [verbatim]
-
-@index [Ash]
-Ash (@i[N X])@\performs the Common Lisp ASH operation on @i[N] and @i[X].
-
-@index [Ldb]
-Ldb (@i[S P N])@\All args are integers; @i[S] and @i[P] are non-negative.
-Performs the Common Lisp LDB operation with @i[S] and @i[P] being the size and
-position of the byte specifier.
-
-@index [Mask-Field]
-Mask-Field (@i[S P N])@\performs the Common Lisp Mask-Field operation with
-@i[S] and @i[P] being the size and position of the byte specifier.
-
-@index [Dpb]
-Dpb (@i[V S P N])@\performs the Common Lisp DPB operation with @i[S] and @i[P]
-being the size and position of the byte specifier.
-
-@index [Deposit-Field]
-Deposit-Field (@i[V S P N])@\performs the Common Lisp Deposit-Field operation
-with @i[S] and @i[P] as the size and position of the byte specifier.
-
-@index [Lsh]
-Lsh (@i[N X])@\returns a fixnum that is @i[N] shifted left by @i[X] bits, with
-0's shifted in on the right. If @i[X] is negative, @i[N] is shifted to the
-right with 0's coming in on the left. Both @i[N] and @i[X] should be fixnums.
-
-@index [Logldb]
-Logldb (@i[S P N])@\All args are fixnums. @i[S] and @i[P] specify a ``byte''
-or bit-field of any length within @i[N]. This is extracted and is returned
-right-justified as a fixnum. @i[S] is the length of the field in bits; @i[P]
-is the number of bits from the right of @i[N] to the beginning of the
-specified field. @i[P] = 0 means that the field starts at bit 0 of @i[N], and
-so on. It is an error if the specified field is not entirely within the 26
-bits of @i[N]
-
-@index [Logdpb]
-Logdpb (@i[V S P N])@\All args are fixnums. Returns a number equal to @i[N],
-but with the field specified by @i[P] and @i[S] replaced by the @i[S] low-order
-bits of @i[V]. It is an error if the field does not fit into the 26 bits of
-@i[N].
-
-@index[Sin]@index[Cos]@index[Tan]@index[Atan]
-Sin(@i[X]), Cos(@i[X]), Tan(@i[X]), and Atan(@i[X])@\accept a single number
-@i[X] as argument and return the sine, cosine, tangent, and arctangent of
-the number respectively. These miscops take advantage of the hardware
-support provided on the IBM RT PC if it is available, otherwise they escape
-to Lisp code to calculate the appropriate result.
-
-@index[Log]
-Log(@i[X])@\returns the natural log of the number @i[X]. This miscop uses
-the hardware operation if it is available, otherwise it escapes to Lisp
-code to calculate the result.
-
-@index[Exp]
-Exp(@i[X])@\returns e raised to the power @i[X]. This miscop uses the
-hardware operation if it is available, otherwise it escapes to Lisp code to
-calculate the result.
-
-@index[Sqrt]
-Sqrt(@i[X])@\returns the square root of @i[X]. This miscop uses the
-hardware operation if it is available, otherwise it escapes to Lisp code to
-calculate the result.
-@end(Description)
-
-@subsection [Branching]
-All branching is done with @value(DinkyMachine) branch instructions.
-Instructions are generated to set the condition code bits appropriately, and
-a branch which tests the appropriate condition code bit is generated.
-
-@subsection [Function Call and Return]
-@instrsection
-
-@begin(Description)
-@index [Call]
-Call()@\A call frame for a function is opened. This is explained in
-more detail in the next chapter.
-
-@index [Call-0]
-Call-0 (@i[F])@\@i[F] must be an executable function, but is a
-function of 0 arguments. Thus, there is no need to collect arguments. The
-call frame is opened and activated in a single miscop.
-
-@index [Call-Multiple]
-Call-Multiple ()@\Just like a Call miscop, but it marks the frame
-to indicate that multiple values will be accepted. See
-section @ref[Multi].
-
-@index[Set-Up-Apply-Args]
-Set-Up-Apply-Args ()@\is called to handle the last argument of a
-function called by apply. All the other arguments will have been
-properly set up by this time. Set-up-apply-args places the values of
-the list passed as the last argument to apply in their proper
-locations, whether they belong in argument registers or on the stack.
-It updates the NArgs register with the actual count of the arguments
-being passed to the function. When Set-up-apply-args returns, all the
-arguments to the function being applied are in their correct
-locations, and the function can be invoked normally.
-
-@index[Start-Call-Interpreter]
-Start-Call-Interpreter (@i[NArgs])@\is called from the interpreter to
-start a function call. It accepts the number of arguments that are
-pushed on the stack in register A0. Just below the arguments is the
-function to call; just below the function is the area to store the
-preserved registers. This miscop sets up the argument registers
-correctly, moves any other arguments down on the stack to their
-proper place, and invokes the function.
-
-@index[Invoke1]
-Invoke1 (@i[Function] @i[Argument])@\is similar to Start-Call-Interpreter,
-but is simpler, since the @i[Function] is being called with only a
-single @i[Argument].
-
-@index[Invoke1*]
-Invoke1* (@i[Function] @i[Argument])@\is similar to Invoke1, but the
-@i[Function] being called is called for one value, rather than multiple ones.
-
-@index [Start-call-mc]
-Start-call-mc ()@\is called when the compiler generates code for the
-form multiple-value-call. Register A0 contains the function to be
-called, A1 contains a 0 if the call if for a single value, and 1
-otherwise, NArgs contains the number of arguments that are stored on
-the stack. The argument registers are set up correctly, and the
-excess values moved down on the stack if necessary. Finally, the
-function is actually invoked.
-
-@index [Push-Last]
-Push-Last ()@\closes the currently open call frame, and initiates a function
-call.
-
-@index [Return]
-Return (@i[X])@\Return from the current function call. After the current
-frame is popped off the stack, @i[X] is returned in register A0 as the result
-Being returned. See section @ref[Return] for more details.
-
-@index [Return-From]
-Return-From (@i[X] @i[F])@\is similar to Return, except it accepts the frame
-to return from as an additional argument.
-
-@index [Return-1-Value-Any-Bind]
-Return-1-Value-Any-Bind (@i[X])@\is similar to return, except only
-one value is returned. Any number of bindings are undone during the
-return operation.
-
-@index [Return-Mult-Value-0-Bind]
-Return-Mult-Value-0-Bind (@i[X])@\is similar to return, except multiple values
-may be returned, but the binding stack does not have to be popped.
-
-@index [Link-Address-Fixup]
-Link-Address-Fixup (@i[Symbol NArgs Code-Vector Offset])@\finds the
-correct link table entry for @i[Symbol] with @i[NArgs] (@i[NArgs]
-specifies the fixed number of arguments and a flag if more may be
-passed). It smashes the @i[Code-Vector] at @i[Offset] to generate
-code to point at the absolute address of the link table entry.
-
-@index [Miscop-Fixup]
-Miscop-Fixup (@i[Code-Vector Offset Index])@\smashes @i[Code-Vector] at
-@i[Offset] with the correct value for the miscop specified by @i[Index] in a
-transfer vector of all the miscops.
-
-@index [Make-Compiled-Closure]
-Make-Compiled-Closure (@i[env fcn offset])@\returns a new function object
-that is a copy of the function object @i[fcn] which has the @i[env]
-information stored at @i[offset]. Compiled lexical closures are now
-represented as real function objects rather than as lists. This miscop is
-necessary to support this change.
-
-@index [Reset-link-table]
-Reset-link-table (@i[function])@\resets all the link table entries for
-@i[function] to the default action. This is necessary because Portable
-Commonloops updates generic function objects by copying new information
-into the function object. The link table must be updated to reflect this
-or the wrong function will be called.
-
-@index[Interrupt-Handler]
-@begin[Multiple]
-Interrupt-Handler (@i[Signal Code Signal-Context])@\gets the first
-indication that a Unix signal has occurred. This miscop does not follow
-the normal Lisp calling conventions at all. Instead it follows the
-standard IBM RT PC calling conventions for C or other algorithmic
-languages. On entry the registers are as follows:
-@begin(Description)
-R0@\Pointer to C data area for Interrupt-Handler. Currently this data area
-only holds a pointer to the entry point for Interrupt-Handler and nothing
-else.
-
-R1@\Pointer to a C stack that contains information about the signal.
-
-R2@\Contains the @i[Signal] number that caused the interrupt to happen.
-
-R3@\Contains the @i[Code] that further specifies what caused the interrupt
-(if necessary).
-
-R4@\Contains a pointer to the @i[signal-context] which contains
-information about where the interrupt occurred, the saved registers, etc.
-
-R5-R14@\Contain unknown values.
-
-R15@\is the return PC which will return from the interrupt handler and
-restart the computation.
-@end(Description)
-Interrupt-Handler determines whether it is safe to take the interrupt now,
-i.e., it is executing in Lisp code, C code, or an interruptible miscop. An
-interruptible miscop is one that has been specially written to make sure
-that it is safe to interrupt it at any point and is possible that it will
-never return of its own accord (e.g., length which could be passed a
-circular list, some of the system call miscops, etc.). If it is safe to
-take the interrupt, the signal-context is modified so that control will
-transfer to the miscop interrupt-routine when the interrupt-handler returns
-normally (i.e., after the kernel has done the necessary bookkeeping). If
-it is unsafe to take the interrupt (i.e., it is executing in an
-non-interruptible miscop), then the return PC of the miscop is modified to
-be interrupt-routine and interrupt-handler returns to the kernel. In
-either case interrupts are disabled and information is stored in a global
-Lisp data area, so that the interrupt-routine miscop can retrieve the
-important information about the interrupt.
-@end[Multiple]
-
-Interrupt-Routine ()@\gets control when it is safe to take an interrupt.
-It saves the current state of the computation on the appropriate stack (on
-the C stack if it was executing in C or on the Lisp stack if in Lisp)
-including all the registers, some control information specifying whether
-the computation was in C code, Lisp code, whether it should form a PC in
-register R15. When a PC has to be formed in R15, R14 will contain a pointer
-to the active function and R15 will contain an index into the code vector
-associated with the active function. Reforming the PC is necessary so
-it is possible to restart a computation even after a garbage collection
-may have moved the function. Once this information is stored,
-interrupt-routine invokes the Lisp function %sp-software-interrupt-routine
-which moves the processing of the interrupt to Lisp code.
-
-@index [Break-Return]
-Break-Return (@i[])@\returns from a function called by the
-interrupt-routine miscop. The only function that should ever do this is
-%sp-software-interrupt-routine. This miscop expects the stack to be in a
-format that is generated during an interrupt and should not be used for
-anything else.
-
-@index [Catch]
-Catch (@i[Tag PC])@\builds a catch frame. @i[Tag] is the tag caught by this
-catch frame, @i[PC] is a saved-format PC (i.e., an index into the current code
-vector). See section @ref[Catch] for details.
-
-@index [Catch-Multiple]
-Catch-Multiple (@i[Tag PC])@\builds a multiple-value catch frame. @i[Tag] is
-the tag caught by this catch frame, and @i[PC] is a saved-format PC. See
-section @ref[Catch] for details.
-
-@index [Catch-All]
-Catch-All (@i[PC])@\builds a catch frame whose tag is the special Catch-All
-object. @i[PC] is the saved-format PC, which is the address to branch to if
-this frame is thrown through. See section @ref[Catch] for details.
-
-@index [Throw]
-Throw (@i[X Tag])@\@i[Tag] is the throw-tag, normally a symbol. @i[X] is the
-value to be returned. See section @ref[Catch] for a description of how this
-miscop works.
-
-@index[Rest-Entry-0]@index[Rest-Entry-1]@index[Rest-Entry-2]@index[Rest-Entry]
-Rest-Entry-0, Rest-Entry-1, Rest-Entry-2, Rest-Entry@\are miscops
-that do the processing for a function at its &rest entry point.
-Rest-Entry-@i[i] are miscops that are invoked by functions that have
-0, 1, or 2 arguments before the &rest argument. Rest-entry is
-invoked for all other cases, and is passed an additional argument in
-A3 which is the number of non-&rest arguments. These miscops form
-the &rest arg list and set up all the registers to have the
-appropriate values. In particular, the non-&rest arguments are copied
-into preserved registers, and the &rest arg list is built and stored
-in the appropriate preserved register or on the stack as appropriate.
-
-@index[Call-Foreign]
-Call-Foreign (@i[C-Function Arguments NArgs])@\establishes the C
-environment so that C code can be called correctly. @i[C-Function] is a
-pointer to the data area for a C function, the first word of which is a
-pointer to the entry point of the C function. @i[Arguments] is a block of
-storage that contains the @i[NArgs] arguments to be passed to the C
-function. The first four of these arguments are passed in registers R2
-through R5 respectively, the rest are moved onto the C stack in the proper
-location. When the C function returns, Call-Foreign restores the Lisp
-environment and returns as its value the integer in R2.
-
-@index[Call-Lisp]
-Call-Lisp (@i[Arg@-<1> ... Arg@-<2>])@\is a Lisp miscop that gets control
-when a C function needs to call a Lisp function. Lisp provides a mechanism
-for setting up an object that looks like a C procedure pointer. The code
-pointer in this object always points at Call-Lisp. Additional data in this
-procedure pointer is the Lisp function to call and the number of arguments
-that it should be called with. Call-Lisp restores the Lisp environment,
-saves the state of the C computation, moves the C arguments into the
-correct places for a call to a Lisp function and then invokes the special
-Lisp function call-lisp-from-c. This Lisp function actually invokes the
-correct Lisp function. Call-Lisp never regains control.
-
-@index[Return-To-C]
-Return-To-C (@i[C-Stack-Pointer Value])@\is used in the
-function call-lisp-from-c to return control to C from a Lisp function
-called by C. @i[C-Stack-Pointer] is the C stack pointer when the call-lisp
-miscop got control. The C stack pointer argument is used to restore the C
-environment to what it was at the time the call to Lisp was made.
-@i[Value] is the value returned from Lisp and is passed back to C in
-register R2. Currently, it is not possible to return other than a single
-32 bit quantity.
-
-@index[Reset-C-Stack]
-Reset-C-Stack ()@\is invoked when a Lisp function called by C throws out
-past where it should return to C. Reset-C-Stack restores the C stack to
-what it was before the original call to C happened. This is so that in the
-future, the C stack will not contain any garbage that should not be there.
-
-@index[Set-C-Procedure-Pointer]
-Set-C-Procedure-Pointer (@i[Sap] @i[I] @I[Proc])@\sets the @i[I/2]'th
-element of @i[sap] to be the data part of the statically allocated g-vector
-@i[Proc]. This is used to set up a C procedure argument in the argument
-block that is passed to call-foreign.
-
-@end(Description)
-
-@subsection [Miscellaneous]
-@instrsection
-
-@begin(Description)
-@index [Eq]
-Eq (@i[X Y])@\sets the eq condition code bit to 1 if @i[X] and @i[Y] are the
-same object, 0 otherwise.
-
-@index [Eql]
-Eql (@i[X Y])@\sets the eq condition code bit to 1 if @i[X] and @i[Y] are the
-same object or if
-@i[X] and @i[Y] are numbers of the same type with the same value, 0 otherwise.
-
-@index [Make-Predicate]
-Make-Predicate (@i[X])@\returns NIL if @i[X] is NIL or T if it is not.
-
-@index [Not-Predicate]
-Not-Predicate (@i[X])@\returns T if @i[X] is NIL or NIL if it is not.
-
-@index [Values-To-N]
-Values-To-N (@i[V])@\@i[V] must be a Values-Marker. Returns the number
-of values indicated in the low 24 bits of @i[V] as a fixnum.
-
-@index [N-To-Values]
-N-To-Values (@i[N])@\@i[N] is a fixnum. Returns a Values-Marker with the
-same low-order 24 bits as @i[N].
-
-@index [Force-Values]
-Force-Values (@i[VM])@\If the @i[VM] is a Values-Marker, do
-nothing; if not, push @i[VM] and return a Values-Marker 1.
-
-@index [Flush-Values]
-Flush-Values (@i[])@\is a no-op for the @value(DinkyMachine), since the only
-time that a Flush-Values miscop is generated is in some well-defined cases
-where all the values are wanted on the stack.
-@end(Description)
-
-@subsection [System Hacking]
-@label [System-Hacking-Instructions]
-@instrsection
-
-@begin(Description)
-@index [Get-Type]
-Get-Type (@i[Object])@\returns the five type bits of the @i[Object] as a
-fixnum.
-
-@index [Get-Space]
-Get-Space (@i[Object])@\returns the two space bits of @i[Object] as a
-fixnum.
-
-@index [Make-Immediate-Type]
-Make-Immediate-Type (@i[X A])@\returns an object whose type bits are the
-integer @i[A] and whose other bits come from the immediate object or pointer
-@i[X]. @i[A] should be an immediate type code.
-
-@index [8bit-System-Ref]
-8bit-System-Ref (@i[X I])@\@i[X] must be a system area pointer, returns
-the @i[I]'th byte of @i[X], indexing into @i[X] directly. @i[I]
-must be a fixnum.
-
-@index [8bit-System-Set]
-8bit-System-Set (@i[X I V])@\@i[X] must be a system area pointer, sets the
-@i[I]'th element of @i[X] to @i[V], indexing into @i[X] directly.
-
-@index [16bit-System-Ref]
-16bit-System-Ref (@i[X I])@\@i[X] must be a system area pointer, returns the
-@i[I]'th 16-bit word of @i[X], indexing into @i[X] directly.
-
-@index [Signed-16bit-System-Ref]
-Signed-16bit-System-Ref (@i[X I])@\@i[X] must be a system area pointer,
-returns the @i[I]'th 16-bit word of @i[X] extending the high order bit as
-the sign bit.
-
-@Index [16bit-System-Set]
-16bit-System-Set (@i[X I V])@\@i[X] must be a system area pointer, sets the
-@i[I]'th element of @i[X] to @i[V], indexing into @i[X] directly.
-
-@Index [Signed-32bit-System-Ref]
-Signed-32bit-System-Ref (@i[X I])@\@i[X] must be a system area pointer and
-@i[I] an even fixnum, returns the @i[I]/2'th 32 bit word as a signed
-quantity.
-
-@Index [Unsigned-32bit-System-Ref]
-Unsigned-32bit-System-Ref (@i[X I])@\@i[X] must be a system area pointer and
-@i[I] an even fixnum, returns the @i[I]/2'th 32 bit word as an unsigned
-quantity.
-
-@Index [Signed-32bit-System-Set]
-Signed-32bit-System-Set (@i[X I V])@\@i[X] must be a system area pointer,
-@i[I] an even fixnum, and @i[V] an integer, sets the @i[I]/2'th element of
-@i[X] to @i[V].
-
-@index[Sap-System-Ref]
-Sap-System-Ref (@i[X I])@\@i[X] must be a system area pointer and @i[I] and
-even fixnum, returns the @i[I]/2'th element of @i[X] as a system area
-pointer.
-
-@index[Sap-System-Set]
-Sap-System-Set (@i[X I V])@\@i[X] and @i[V] must be a system area pointers
-and @i[I] an even fixnum, sets the @i[I]/2'th element of @i[X] to @i[V].
-
-@index[Pointer-System-Set]
-Pointer-System-Set (@i[X I])@\@i[X] must be a system area pointer, @i[I] an
-even fixnum, and @i[V] a pointer (either system area pointer or Lisp
-pointer), sets the @i[I]/2'th element of @i[X] to the pointer @i[V]. If
-the pointer is a Lisp pointer, the pointer stored is to the first word of
-data (i.e., the header word(s) are bypassed).
-
-@index[Sap-Int]
-Sap-Int (@i[X])@\@i[X] should be a system area pointer, returns a Lisp
-integer containing the system area pointer. This miscop is useful when it
-is necessary to do arithmetic on system area pointers.
-
-@index[Int-Sap]
-Int-Sap (@i[X])@\@i[X] should be an integer (fixnum or bignum), returns a
-system area pointer. This miscop performs the inverse operation of sap-int.
-
-@index[Check-<=]
-Check-<= (@i[X] @i[Y])@\checks to make sure that @i[X] is less than or
-equal to @i[Y]. If not, then check-<= signals an error, otherwise it just
-returns.
-
-@index [Collect-Garbage]
-Collect-Garbage (@i[])@\causes a stop-and-copy GC to be performed.
-
-@index [Purify]
-Purify (@i[])@\is similar to collect-garbage, except it copies Lisp objects
-into static or read-only space. This miscop needs Lisp level code to get
-the process started by putting some root structures into the correct space.
-
-@index [Newspace-Bit]
-Newspace-Bit (@i[])@\returns 0 if newspace is currently space 0 or 1 if it is
-1.
-
-@index [Save]
-Save (@i[*current-alien-free-pointer*] @i[Checksum] @I[memory])@\Save takes
-a snap short of the current state of the Lisp computation. The value of
-the symbol *Current-alien-free-pointer* must be passed to save, so that it
-can save the static alien data structures. The parameter @i[checksum]
-specifies whether a checksum should be generated for the saved image.
-Currently, this parameter is ignored and no checksum is generated. The
-parameter @i[memory] should be be a pointer to a block of memory where the
-saved core image will be stored. Save returns the size of the core image
-generated.
-
-@index [Syscall0]
-@index [Syscall1]
-@index [Syscall2]
-@index [Syscall3]
-@index [Syscall4]
-@index [Syscall]
-Syscall0 Syscall1 Syscall2 Syscall3 Syscall4 Syscall (@i[number]
-@i[arg@-<1> ... arg@-<n>])@\is for making syscalls to the Mach kernel. The
-argument @i[number] should be the number of the syscall. Syscall0 accepts
-no arguments to the syscall; syscall1 accepts one argument to the syscall,
-etc. Syscall accepts five or more arguments to the syscall.
-
-@index[Unix-write]
-Unix-Write (@i[fd buffer offset length])@\performs a Unix write syscall to
-the file descriptor @i[fd]. @i[Buffer] should contain the data to be
-written; @i[Offset] should be an offset into buffer from which to start
-writing; and @i[length] is the number of bytes of data to write.
-
-@index[Unix-fork]
-Unix-Fork ()@\performs a Unix fork operation returning one or two values.
-If an error occurred, the value -1 and the error code is returned. If no
-error occurred, 0 is returned in the new process and the process id of the
-child process is returned in the parent process.
-
-@index [Arg-In-Frame] Arg-In-Frame (@i[N F])@\@i[N] is a fixnum, @i[F] is a
-control stack pointer as returned by the Active-Call-Frame miscop. It
-returns the item in slot @i[N] of the args-and-locals area of call frame
-@i[F].
-
-@index [Active-Call-Frame]
-Active-Call-Frame (@i[])@\returns a control-stack pointer to the start of the
-currently active call frame. This will be of type Control-Stack-Pointer.
-
-@index [Active-Catch-Frame]
-Active-Catch-Frame (@i[])@\returns the control-stack pointer to the start of
-the currently active catch frame. This is Nil if there is no active catch.
-
-@index [Set-Call-Frame]
-Set-Call-Frame (@i[P])@\@i[P] must be a control stack pointer. This becomes
-the current active call frame pointer.
-
-@index [Current-Stack-Pointer]
-Current-Stack-Pointer (@i[])@\returns the Control-Stack-Pointer that points
-to the current top of the stack (before the result of this operation is
-pushed). Note: by definition, this points to the
-to the last thing pushed.
-
-@index [Current-Binding-Pointer]
-Current-Binding-Pointer (@i[])@\returns a Binding-Stack-Pointer that points
-to the first word above the current top of the binding stack.
-
-@index [Read-Control-Stack]
-Read-Control-Stack (@i[F])@\@i[F] must be a control stack pointer. Returns
-the Lisp object that resides at this location. If the addressed object is
-totally outside the current stack, this is an error.
-
-@index [Write-Control-Stack]
-Write-Control-Stack (@i[F V])@\@i[F] is a stack pointer, @i[V] is any Lisp
-object. Writes @i[V] into the location addressed. If the addressed cell is
-totally outside the current stack, this is an error. Obviously, this should
-only be used by carefully written and debugged system code, since you can
-destroy the world by using this miscop.
-
-@index [Read-Binding-Stack]
-Read-Binding-Stack (@i[B])@\@i[B] must be a binding stack pointer. Reads and
-returns the Lisp object at this location. An error if the location specified
-is outside the current binding stack.
-
-@index [Write-Binding-Stack]
-Write-Binding-Stack (@i[B V])@\@i[B] must be a binding stack pointer. Writes
-@i[V] into the specified location. An error if the location specified is
-outside the current binding stack.
-@end(Description)
-
-@chapter [Control Conventions]
-@label [Control-Conventions]
-@index [Hairy stuff]
-
-@section [Function Calls]
-@index [Call]
-@index [Call-0]
-@index [Call-Multiple]
-
-On the Perq function calling is done by micro-coded instructions. The
-instructions perform a large number of operations, including determining
-whether the function being called is compiled or interpreted, determining that
-a legal number of arguments are passed, and branching to the correct entry
-point in the function. To do all this on the @value(DinkyMachine) would
-involve a large amount of computation. In the general case, it is necessary to
-do all this, but in some common cases, it is possible to short circuit most of
-this work.
-
-To perform a function call in the general case, the following steps occur:
-@begin(Enumerate)
-
-Allocate space on the control stack for the fix-sized part of a call
-frame. This space will be used to store all the registers that must
-be preserved across a function call.
-
-Arguments to the function are now evaluated. The first three
-arguments are stored in the argument registers A0, A1, and A2. The
-rest of the arguments are stored on the stack as they are evaluated.
-Note that during the evaluation of arguments, the argument registers
-may be used and may have to be stored in local variables and restored
-just before the called function is invoked.
-
-Load R0 with the argument count.
-
-Load the PC register with the offset into the current code vector of
-the place to return to when the function call is complete.
-
-If this call is for multiple values, mark the frame as accepting
-multiple values, by making the fixnum offset above negative by oring
-in the negative fixnum type code.
-
-Store all the registers that must be preserved over the function call in the
-current frame.
-@end(Enumerate)
-
-At this point, all the arguments are set up and all the registers have been
-saved. All the code to this point is done inline. If the object being called
-as a function is a symbol, we get the definition from the definition cell of
-the symbol. If this definition is the trap object, an undefined symbol error
-is generated. The function calling mechanism diverges at this point depending
-on the type of function being called, i.e., whether it is a compiled function
-object or a list.
-
-If we have a compiled function object, the following steps are performed (this
-code is out of line):
-@begin(Enumerate)
-Load the active function register with a pointer to the compiled function
-object.
-
-The active frame register is set to the start of the current frame.
-
-Note the number of arguments evaluated. Let this be K. The correct
-entry point in the called function's code vector must be computed as
-a function of K and the number of arguments the called function
-wants:
-@begin(Enumerate, spread 0, spacing 1)
-If K < minimum number of arguments, signal an error.
-
-If K > maximum number of arguments and there is no &rest argument,
-signal an error.
-
-If K > maximum number of arguments and there is a &rest argument,
-start at offset 0 in the code vector. This entry point must collect
-the excess arguments into a list and leave the &rest argument in the
-appropriate argument register or on the stack as appropriate.
-
-If K is between the minimum and maximum arguments (inclusive), get
-the starting offset from the appropriate slot of the called
-function's function object. This is stored as a fixnum in slot K -
-MIN + 6 of the function object.
-@end(Enumerate)
-
-Load one of the Non-Lisp temporary registers with the address of the
-code vector and add in the offset calculated above. Then do a branch
-register instruction with this register as the operand. The called
-function is now executing at the appropriate place.
-@end(enumerate)
-
-If the function being called is a list, %SP-Internal-Apply must be called to
-interpret the function with the given arguments. Proceed as follows:
-@begin(Enumerate)
-Note the number of arguments evaluated for the current open frame (call this N)
-and the frame pointer for the frame (call it F). Also remember the lambda
-expression in this frame (call it L).
-
-Load the active function register with the list L.
-
-Load the PC register with 0.
-
-Allocate a frame on the control stack for the call to %SP-Internal-Apply.
-
-Move the contents of the argument registers into the local registers L0, L1,
-and L2 respectively.
-
-Store all the preserved registers in the frame.
-
-Place N, F, and L into argument registers A0, A1, and A2 respectively.
-
-Do the equivalent of a start call on %SP-Internal-Apply.
-@end(Enumerate) %SP-Internal-Apply, a function of three arguments,
-now evaluates the call to the lambda-expression or interpreted
-lexical closure L, obtaining the arguments from the frame pointed to
-by F. The first three arguments must be obtained from the frame that
-%SP-Internal-Apply runs in, since they are stored in its stack frame
-and not on the stack as the rest of the arguments are. Prior to
-returning %SP-Internal-Apply sets the Active-Frame register to F, so
-that it returns from frame F.
-
-The above is the default calling mechanism. However, much of the
-overhead can be reduced. Most of the overhead is incurred by having
-to check the legality of the function call everytime the function is
-called. In many situations where the function being called is a
-symbol, this checking can be done only once per call site by
-introducing a data structure called a link table. The one exception
-to this rule is when the function apply is used with a symbol. In
-this situation, the argument count checks are still necessary, but
-checking for whether the function is a list or compiled function
-object can be bypassed.
-
-The link table is a hash table whose key is based on the name of the
-function, the number of arguments supplied to the call and a flag
-specifying whether the call is done through apply or not. Each entry
-of the link table consists of two words:
-@begin(Enumerate)
-The address of the function object associated with the symbol being
-called. This is here, so that double indirection is not needed to
-access the function object which must be loaded into the active
-function register. Initially, the symbol is stored in this slot.
-
-The address of the instruction in the function being called to start
-executing when this table entry is used. Initially, this points to
-an out of line routine that checks the legality of the call and
-calculates the correct place to jump to in the called function. This
-out of line routine replaces the contents of this word with the
-correct address it calculated. In the case when the call is caused
-by apply, this will often be an out of line routine that checks the
-argument count and calculates where to jump. In the case where the
-called function accepts &rest arguments and the minimum number of
-arguments passed is guaranteed to be greater than the maximum number
-of arguments, then a direct branch to the &rest arg entry point is
-made.
-@end(Enumerate)
-
-When a compiled file is loaded into the lisp environment, all the
-entries for the newly loaded functions will be set to an out of line
-routine mentioned above. Also, during a garbage collection the
-entries in this table must be updated when a function object for a
-symbol is moved.
-
-The @value(DinkyMachine) code to perform a function call using the link table
-becomes:
-@begin(Example)
- cal CS,CS,%Frame-Size ; Alloc. space on control st.
-
- <Code to evaluate arguments to the function>
-
- cau NL1,0,high-half-word(lte(function nargs flag))
- oil NL1,0,low-half-word(lte(function nargs flag))
- cal PC,0,return-tag ; Offset into code vector.
- <oiu PC,PC,#xF800 ; Mark if call-multiple frame>
- stm L0,CS,-(%Frame-Size-4) ; Save preserved regs.
- lm AF,NL1,0 ; Link table entry contents.
- bnbrx pz,R15 ; Branch to called routine.
- cal FP,CS,-(%Frame-Size-4) ; Get pointer to frame.
-return-tag:
-@end(Example)
-The first two instructions after the arguments are evaled get the
-address of the link table entry into a register. The two 16-bit half
-word entries are filled in at load time. The rest of the
-instructions should be fairly straight forward.
-
-@section(Returning from a Function Call)
-@label(Return)
-@index(Return)
-
-Returning from a function call on the Perq is done by a micro-coded
-instruction. On the @value(DinkyMachine), return has to do the following:
-@begin(enumerate)
-Pop the binding stack back to the binding stack pointer stored in the frame
-we're returning from. For each symbol/value pair popped of the binding stack,
-restore that value for the symbol.
-
-Save the current value of the frame pointer in a temporary registers. This
-will be used to restore the control stack pointer at the end.
-
-Restore all the registers that are preserved across a function call.
-
-Get a pointer to the code vector for the function we're returning to. This is
-retrieved from the code slot of what is now the active function.
-
-Make sure the relative PC (which is now in a register) is positive and add it
-to the code vector pointer above, giving the address of the instruction to
-return to.
-
-If the function is returning multiple values do a block transfer of all the
-return values down over the stack frame just released, i.e., the first return
-value should be stored where the temporarily saved frame pointer points to.
-In effect the return values can be pushed onto the stack using the saved frame
-pointer above as a stack pointer that is incremented everytime a value is
-pushed. Register A0 can be examined to determine the number of values that
-must be transferred.
-
-Set the control stack register to the saved frame pointer above. NB: it may
-have been updated if multiple values are being returned.
-
-Resume execution of the calling function.
-@end(enumerate)
-
-Again, it is not always necessary to use the general return code. At compile
-time it is often possible to determine that no special symbols have to be
-unbound and/or only one value is being returned. For example the code to
-perform a return when only one value is returned and it is unnecessary to
-unbind any special symbols is:
-@begin(Example)
- cas NL1,FP,0 ; Save frame register.
- lm L0,FP,0 ; Restore all preserved regs.
- ls A3,AF,%function-code ; Get pointer to code vector.
- niuo PC,PC,#x07FF ; Make relative PC positive.
- cas PC,A3,PC ; Get addr. of instruction
- bnbrx pz,PC ; to return to and do so while
- cas CS,NL1,0 ; updating control stack reg.
-@end(Example)
-
-
-@subsection [Returning Multiple-Values]
-@label [Multi]
-@index [Multiple values]
-
-If the current frame can accept multiple values and a values marker is in
-register A0 indicating N values on top of the stack, it is necessary to copy
-the N return values down to the top of the control stack after the current
-frame is popped off. Thus returning multiple values is similar to the
-above, but a block transfer is necessary to move the returned values down to
-the correct location on the control stack.
-
-In tail recursive situations, such as in the last form of a PROGN, one
-function, FOO, may want to call another function, BAR, and return ``whatever
-BAR returns.'' Call-Multiple is used in this case. If BAR returns multiple
-values, they will all be passed to FOO. If FOO's caller wants multiple values,
-the values will be returned. If not, FOO's Return instruction will see that
-there are multiple values on the stack, but that multiple values will not be
-accepted by FOO's caller. So Return will return only the first value.
-
-@section [Non-Local Exits]
-@label [Catch]
-@index [Catch]
-@index [Throw]
-@index [Catch-All object]
-@index [Unwind-Protect]
-@index [Non-Local Exits]
-
-The Catch and Unwind-Protect special forms are implemented using
-catch frames. Unwind-Protect builds a catch frame whose tag is the
-Catch-All object. The Catch miscop creates a catch frame for a
-given tag and PC to branch to in the current instruction. The Throw
-miscop looks up the stack by following the chain of catch frames
-until it finds a frame with a matching tag or a frame with the
-Catch-All object as its tag. If it finds a frame with a matching
-tag, that frame is ``returned from,'' and that function is resumed.
-If it finds a frame with the Catch-All object as its tag, that frame
-is ``returned from,'' and in addition, %SP-Internal-Throw-Tag is set
-to the tag being searched for. So that interrupted cleanup forms
-behave correctly, %SP-Internal-Throw-Tag should be bound to the
-Catch-All object before the Catch-All frame is built. The protected
-forms are then executed, and if %SP-Internal-Throw-Tag is not the
-Catch-All object, its value is thrown to. Exactly what we do is
-this:
-@begin [enumerate]
-Put the contents of the Active-Catch register into a register, A.
-Put NIL into another register, B.
-
-If A is NIL, the tag we seek isn't on the stack. Signal an
-Unseen-Throw-Tag error.
-
-Look at the tag for the catch frame in register A. If it's the tag
-we're looking for, go to step 4. If it's the Catch-All object and B
-is NIL, copy A to B. Set A to the previous catch frame and go back
-to step 2.
-
-If B is non-NIL, we need to execute some cleanup forms. Return into
-B's frame and bind %SP-Internal-Throw-Tag to the tag we're searching
-for. When the cleanup forms are finished executing, they'll throw to
-this tag again.
-
-If B is NIL, return into this frame, pushing the return value (or
-BLTing the multiple values if this frame accepts multiple values and
-there are multiple values).
-@end [enumerate]
-
-If no form inside of a Catch results in a Throw, the catch frame
-needs to be removed from the stack before execution of the function
-containing the throw is resumed. For now, the value produced by the
-forms inside the Catch form are thrown to the tag. Some sort of
-specialized miscop could be used for this, but right now we'll
-just go with the throw. The branch PC specified by a Catch
-miscop is part of the constants area of the function object,
-much like the function's entry points.
-
-@section [Escaping to Lisp code]
-@label [Escape]
-@index [Escape to Lisp code convention]
-
-Escaping to Lisp code is fairly straight forward. If a miscop discovers that
-it needs to call a Lisp function, it creates a call frame on the control
-stack and sets it up so that the called function returns to the function that
-called the miscop. This means it is impossible to return control to a miscop
-from a Lisp function.
-
-@section [Errors]
-@label [Errors]
-@index [Errors]
-
-When an error occurs during the execution of a miscop, a call
-to %SP-Internal-Error is performed. This call is a break-type call,
-so if the error is proceeded (with a Break-Return instruction), no
-value will be returned.
-
-
-%SP-Internal-Error is passed a fixnum error code as its first
-argument. The second argument is a fixnum offset into the current
-code vector that points to the location immediately following the
-instruction that encountered the trouble. From this offset, the
-Lisp-level error handler can reconstruct the PC of the losing
-instruction, which is not readily available in the micro-machine.
-Following the offset, there may be 0 - 2 additional arguments that
-provide information of possible use to the error handler. For
-example, an unbound-symbol error will pass the symbol in question as
-the third arg.
-
-The following error codes are currently defined. Unless otherwise
-specified, only the error code and the code-vector offset are passed
-as arguments.
-
-@begin
-[description]
-1 Object Not List@\The object is passed as the third argument.
-
-2 Object Not Symbol@\The object is passed as the third argument.
-
-3 Object Not Number@\The object is passed as the third argument.
-
-4 Object Not Integer@\The object is passed as the third argument.
-
-5 Object Not Ratio@\The object is passed as the third argument.
-
-6 Object Not Complex@\The object is passed as the third argument.
-
-7 Object Not Vector@\The object is passed as the third argument.
-
-8 Object Not Simple Vector@\The object is passed as the third argument.
-
-9 Illegal Function Object@\The object is passed as the third argument.
-
-10 Object Not Header@\The object (which is not an array or function header)
-is passed as the third argument.
-
-11 Object Not I-Vector@\The object is passed as the third argument.
-
-12 Object Not Simple Bit Vector@\The object is passed as the third argument.
-
-13 Object Not Simple String@\The object is passed as the third argument.
-
-14 Object Not Character@\The object is passed as the third argument.
-
-15 Object Not Control Stack Pointer@\The object is passed as the third
-argument.
-
-16 Object Not Binding Stack Pointer@\The object is passed as the third
-argument.
-
-17 Object Not Array@\The object is passed as the third argument.
-
-18 Object Not Non-negative Fixnum@\The object is passed as the third
-argument.
-
-19 Object Not System Area Pointer@\The object is passed as the third
-argument.
-
-20 Object Not System Pointer@\The object is passed as the third argument.
-
-21 Object Not Float@\The object is passed as the third argument.
-
-22 Object Not Rational@\The object is passed as the third argument.
-
-23 Object Not Non-Complex Number@\A complex number has been passed to
-the comparison routine for < or >. The complex number is passed as the
-third argument.
-
-25 Unbound Symbol @\Attempted access to the special value of an unbound
-symbol. Passes the symbol as the third argument to %Sp-Internal-Error.
-
-26 Undefined Symbol @\Attempted access to the definition cell of an undefined
-symbol. Passes the symbol as the third argument to %Sp-Internal-Error.
-
-27 Altering NIL @\Attempt to bind or setq the special value of NIL.
-
-28 Altering T @\Attempt to bind or setq the special value of T.
-
-30 Illegal Vector Access Type @\The specified access type is returned as the
-third argument.
-
-31 Illegal Vector Size @\Attempt to allocate a vector with negative size or
-size too large for vectors of this type. Passes the requested size as the
-third argument.
-
-32 Vector Index Out of Range @\The specified index is out of bounds for
-this vector. The bad index is passed as the third argument.
-
-33 Illegal Vector Index@\The specified index is not a positive fixnum. The
-bad index is passed as the third argument.
-
-34 Illegal Shrink Vector Value@\The specified value to shrink a vector to is
-not a positive fixnum. The bad value is passed as the third argument.
-
-35 Not A Shrink@\The specified value is greater than the current size of the
-vector being shrunk. The bad value is passed as the third argument.
-
-36 Illegal Data Vector@\The data vector of an array is illegal. The bad
-vector is passed as the third value.
-
-37 Array has Too Few Indices@\An attempt has been made to access
-an array as a two or three dimensional array when it has fewer than two
-or three dimensions, respectively.
-
-38 Array has Too Many Indices@\An attempt has been made to access an array
-as a two or three dimensional array when it has more than two or three
-dimensions, respectively.
-
-40 Illegal Byte Specifier@\A bad byte specifier has been passed to one
-of the byte manipulation miscops. The offending byte specifier is passed
-as the third argument.
-
-41 Illegal Position in Byte Specifier@\A bad position has been given in a
-byte specifier that has been passed to one of the byte manipulation
-miscops. The offending byte specifier is passed as the third
-argument.
-
-42 Illegal Size in Byte Specifier@\A bad size has been given in a
-byte specifier that has been passed to one of the byte manipulation
-miscops. The offending byte specifier is passed as the third
-argument.
-
-43 Illegal Shift Count@\A shift miscop has encountered non fixnum shift
-count. The offending shift count is passed as the third argument.
-
-44 Illegal Boole Operation@\The operation code passed to the boole miscop
-is either not a fixnum or is out of range. The operation code is passed as
-the third argument.
-
-50 Too Few Arguments@\Too few arguments have been passed to a function. The
-number of arguments actually passed is passed as the third argument, and the
-function is passed as the fourth.
-
-51 Too Many Arguments@\Too many arguments have been passed to a function.
-The number of arguments actually passed is passed as the third argument, and
-the function is passed as the fourth.
-
-52 Last Apply Arg Not a List@\The last argument to a function being
-invoked by apply is not a list. The last argument is passed as the third
-argument.
-
-53 Deleted Link Table Entry@\An attempt has been made to call a function
-through a link table entry which no longer exists. This is a serious
-internal error and should never happen.
-
-55 Error Not <=@\The check-<= miscop will invoke this error if the condition
-is false. The two arguments are passed as the third and fourth arguments
-to %SP-internal-error.
-
-60 Divide by 0@\An division operation has done a division by zero. The
-two operands are passed as the third and fourth arguments.
-
-61 Unseen Throw Tag@\An attempt has been made to throw to a tag that is
-not in the current catch hierarchy. The offending tag is passed as the
-third argument.
-
-62 Short Float Underflow@\A short float operation has resulted in
-underflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-63 Short Float Overflow@\A short float operation has resulted in
-overflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-64 Single Float Underflow@\A single float operation has resulted in
-underflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-65 Single Float Overflow@\A single float operation has resulted in
-overflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-66 Long Float Underflow@\A long float operation has resulted in
-underflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-67 Long Float Overflow@\A long float operation has resulted in
-overflow. The two arguments to the operation are passed as the third
-and fourth arguments.
-
-68 Monadic Short Float Underflow@\A short float operation has resulted in
-underflow. The argument to the operation is passed as the third argument.
-
-69 Monadic Short Float Overflow@\A short float operation has resulted in
-overflow. The argument to the operation is passed as the third argument.
-
-70 Monadic Long Float Underflow@\A long float operation has resulted in
-underflow. The argument to the operation is passed as the third argument.
-
-71 Monadic Long Float Overflow@\A long float operation has resulted in
-overflow. The argument to the operation is passed as the third argument.
-@end [description]
-
-@section [Trapping to the Mach Kernel]
-@label [Trap]
-@index [Trapping to the kernel]
-@index [Kernel traps]
-
-Trapping to the Mach kernel is done through one of the syscall0, syscall1,
-syscall2, syscall3, syscall4, or syscall miscops. The first argument to
-these miscops is the number of the Unix syscall that is to be invoked. Any
-other arguments the syscall requires are passed in order after the first
-one. Syscall0 accepts only the syscall number and no other arguments;
-syscall1 accepts the syscall number and a single argument to the syscall;
-etc. Syscall accepts the syscall number and five or more arguments to the
-Unix syscall. These syscalls generally return two values: the result twice
-if the syscall succeeded and a -1 and the Unix error code if the syscall
-failed.
-
-@section [Interrupts]
-@label [Interrupts]
-@index [Interrupts]
-
-An interface has been built to the general signal mechanism defined by the
-Unix operating system. As mentioned in the section on function call and
-return miscops, several miscops are defined that support the lowest level
-interface to the Unix signal mechanism. The manual @I[CMU Common Lisp
-User's Manual, Mach/IBM RT PC Edition] contains descriptions of functions
-that allow a user to set up interrupt handlers for any of the Unix signals
-from within Lisp.
-
-@appendix [Fasload File Format]
-@section [General]
-
-The purpose of Fasload files is to allow concise storage and rapid
-loading of Lisp data, particularly function definitions. The intent
-is that loading a Fasload file has the same effect as loading the
-ASCII file from which the Fasload file was compiled, but accomplishes
-the tasks more efficiently. One noticeable difference, of course, is
-that function definitions may be in compiled form rather than
-S-expression form. Another is that Fasload files may specify in what
-parts of memory the Lisp data should be allocated. For example,
-constant lists used by compiled code may be regarded as read-only.
-
-In some Lisp implementations, Fasload file formats are designed to
-allow sharing of code parts of the file, possibly by direct mapping
-of pages of the file into the address space of a process. This
-technique produces great performance improvements in a paged
-time-sharing system. Since the Mach project is to produce a
-distributed personal-computer network system rather than a
-time-sharing system, efficiencies of this type are explicitly @i[not]
-a goal for the CMU Common Lisp Fasload file format.
-
-On the other hand, CMU Common Lisp is intended to be portable, as it will
-eventually run on a variety of machines. Therefore an explicit goal
-is that Fasload files shall be transportable among various
-implementations, to permit efficient distribution of programs in
-compiled form. The representations of data objects in Fasload files
-shall be relatively independent of such considerations as word
-length, number of type bits, and so on. If two implementations
-interpret the same macrocode (compiled code format), then Fasload
-files should be completely compatible. If they do not, then files
-not containing compiled code (so-called "Fasdump" data files) should
-still be compatible. While this may lead to a format which is not
-maximally efficient for a particular implementation, the sacrifice of
-a small amount of performance is deemed a worthwhile price to pay to
-achieve portability.
-
-The primary assumption about data format compatibility is that all
-implementations can support I/O on finite streams of eight-bit bytes.
-By "finite" we mean that a definite end-of-file point can be detected
-irrespective of the content of the data stream. A Fasload file will
-be regarded as such a byte stream.
-
-@section [Strategy]
-
-A Fasload file may be regarded as a human-readable prefix followed by
-code in a funny little language. When interpreted, this code will
-cause the construction of the encoded data structures. The virtual
-machine which interprets this code has a @i[stack] and a @i[table],
-both initially empty. The table may be thought of as an expandable
-register file; it is used to remember quantities which are needed
-more than once. The elements of both the stack and the table are
-Lisp data objects. Operators of the funny language may take as
-operands following bytes of the data stream, or items popped from the
-stack. Results may be pushed back onto the stack or pushed onto the
-table. The table is an indexable stack that is never popped; it is
-indexed relative to the base, not the top, so that an item once
-pushed always has the same index.
-
-More precisely, a Fasload file has the following macroscopic
-organization. It is a sequence of zero or more groups concatenated
-together. End-of-file must occur at the end of the last group. Each
-group begins with a series of seven-bit ASCII characters terminated
-by one or more bytes of all ones (FF@-(16)); this is called the
-@i[header]. Following the bytes which terminate the header is the
-@i[body], a stream of bytes in the funny binary language. The body
-of necessity begins with a byte other than FF@-(16). The body is
-terminated by the operation @f[FOP-END-GROUP].
-
-The first nine characters of the header must be "@f[FASL FILE]" in
-upper-case letters. The rest may be any ASCII text, but by
-convention it is formatted in a certain way. The header is divided
-into lines, which are grouped into paragraphs. A paragraph begins
-with a line which does @i[not] begin with a space or tab character,
-and contains all lines up to, but not including, the next such line.
-The first word of a paragraph, defined to be all characters up to but
-not including the first space, tab, or end-of-line character, is the
-@i[name] of the paragraph. A Fasload file header might look something like
-this:
-@begin(verbatim)
-FASL FILE >SteelesPerq>User>Guy>IoHacks>Pretty-Print.Slisp
-Package Pretty-Print
-Compiled 31-Mar-1988 09:01:32 by some random luser
-Compiler Version 1.6, Lisp Version 3.0.
-Functions: INITIALIZE DRIVER HACK HACK1 MUNGE MUNGE1 GAZORCH
- MINGLE MUDDLE PERTURB OVERDRIVE GOBBLE-KEYBOARD
- FRY-USER DROP-DEAD HELP CLEAR-MICROCODE
- %AOS-TRIANGLE %HARASS-READTABLE-MAYBE
-Macros: PUSH POP FROB TWIDDLE
-@r[<one or more bytes of FF@-(16)>]
-@end(verbatim)
-The particular paragraph names and contents shown here are only intended as
-suggestions.
-
-@section [Fasload Language]
-
-Each operation in the binary Fasload language is an eight-bit
-(one-byte) opcode. Each has a name beginning with "@f[FOP-]". In
-the following descriptions, the name is followed by operand
-descriptors. Each descriptor denotes operands that follow the opcode
-in the input stream. A quantity in parentheses indicates the number
-of bytes of data from the stream making up the operand. Operands
-which implicitly come from the stack are noted in the text. The
-notation "@PushArrow stack" means that the result is pushed onto the
-stack; "@PushArrow table" similarly means that the result is added to the
-table. A construction like "@i[n](1) @i[value](@i[n])" means that
-first a single byte @i[n] is read from the input stream, and this
-byte specifies how many bytes to read as the operand named @i[value].
-All numeric values are unsigned binary integers unless otherwise
-specified. Values described as "signed" are in two's-complement form
-unless otherwise specified. When an integer read from the stream
-occupies more than one byte, the first byte read is the least
-significant byte, and the last byte read is the most significant (and
-contains the sign bit as its high-order bit if the entire integer is
-signed).
-
-Some of the operations are not necessary, but are rather special
-cases of or combinations of others. These are included to reduce the
-size of the file or to speed up important cases. As an example,
-nearly all strings are less than 256 bytes long, and so a special
-form of string operation might take a one-byte length rather than a
-four-byte length. As another example, some implementations may
-choose to store bits in an array in a left-to-right format within
-each word, rather than right-to-left. The Fasload file format may
-support both formats, with one being significantly more efficient
-than the other for a given implementation. The compiler for any
-implementation may generate the more efficient form for that
-implementation, and yet compatibility can be maintained by requiring
-all implementations to support both formats in Fasload files.
-
-Measurements are to be made to determine which operation codes are
-worthwhile; little-used operations may be discarded and new ones
-added. After a point the definition will be "frozen", meaning that
-existing operations may not be deleted (though new ones may be added;
-some operations codes will be reserved for that purpose).
-
-@begin(description)
-0 @f[ ] @f[FOP-NOP] @\
-No operation. (This is included because it is recognized
-that some implementations may benefit from alignment of operands to some
-operations, for example to 32-bit boundaries. This operation can be used
-to pad the instruction stream to a desired boundary.)
-
-1 @f[ ] @f[FOP-POP] @f[ ] @PushArrow @f[ ] table @\
-One item is popped from the stack and added to the table.
-
-2 @f[ ] @f[FOP-PUSH] @f[ ] @i[index](4) @f[ ] @PushArrow @f[ ] stack @\
-Item number @i[index] of the table is pushed onto the stack.
-The first element of the table is item number zero.
-
-3 @f[ ] @f[FOP-BYTE-PUSH] @f[ ] @i[index](1) @f[ ] @PushArrow @f[ ] stack @\
-Item number @i[index] of the table is pushed onto the stack.
-The first element of the table is item number zero.
-
-4 @f[ ] @f[FOP-EMPTY-LIST] @f[ ] @PushArrow @f[ ] stack @\
-The empty list (@f[()]) is pushed onto the stack.
-
-5 @f[ ] @f[FOP-TRUTH] @f[ ] @PushArrow @f[ ] stack @\
-The standard truth value (@f[T]) is pushed onto the stack.
-
-6 @f[ ] @f[FOP-SYMBOL-SAVE] @f[ ] @i[n](4) @f[ ] @i[name](@i[n])
-@f[ ] @PushArrow @f[ ] stack & table@\
-The four-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the default package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-7 @f[ ] @f[FOP-SMALL-SYMBOL-SAVE] @f[ ] @i[n](1) @f[ ] @i[name](@i[n]) @f[ ] @PushArrow @f[ ] stack & table@\
-The one-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the default package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-8 @f[ ] @f[FOP-SYMBOL-IN-PACKAGE-SAVE] @f[ ] @i[index](4)
-@f[ ] @i[n](4) @f[ ] @i[name](@i[n])
-@f[ ] @PushArrow @f[ ] stack & table@\
-The four-byte @i[index] specifies a package stored in the table.
-The four-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-9 @f[ ] @f[FOP-SMALL-SYMBOL-IN-PACKAGE-SAVE] @f[ ] @i[index](4)
-@f[ ] @i[n](1) @f[ ] @i[name](@i[n]) @f[ ]
-@PushArrow @f[ ] stack & table@\
-The four-byte @i[index] specifies a package stored in the table.
-The one-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-10 @f[ ] @f[FOP-SYMBOL-IN-BYTE-PACKAGE-SAVE] @f[ ] @i[index](1)
-@f[ ] @i[n](4) @f[ ] @i[name](@i[n])
-@f[ ] @PushArrow @f[ ] stack & table@\
-The one-byte @i[index] specifies a package stored in the table.
-The four-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-11@f[ ] @f[FOP-SMALL-SYMBOL-IN-BYTE-PACKAGE-SAVE] @f[ ] @i[index](1)
-@f[ ] @i[n](1) @f[ ] @i[name](@i[n]) @f[ ]
-@PushArrow @f[ ] stack & table@\
-The one-byte @i[index] specifies a package stored in the table.
-The one-byte operand @i[n] specifies the length of the print name
-of a symbol. The name follows, one character per byte,
-with the first byte of the print name being the first read.
-The name is interned in the specified package,
-and the resulting symbol is both pushed onto the stack and added to the table.
-
-12 Unused.
-
-13 @f[ ] @f[FOP-DEFAULT-PACKAGE] @f[ ] @i[index](4) @\
-A package stored in the table entry specified by @i[index] is made
-the default package for future @f[FOP-SYMBOL] and @f[FOP-SMALL-SYMBOL]
-interning operations. (These package FOPs may change or disappear
-as the package system is determined.)
-
-14 @f[ ] @f[FOP-PACKAGE] @f[ ] @PushArrow @f[ ] table @\
-An item is popped from the stack; it must be a symbol. The package of
-that name is located and pushed onto the table.
-
-15 @f[ ] @f[FOP-LIST] @f[ ] @i[length](1) @f[ ] @PushArrow @f[ ] stack @\
-The unsigned operand @i[length] specifies a number of
-operands to be popped from the stack. These are made into a list
-of that length, and the list is pushed onto the stack.
-The first item popped from the stack becomes the last element of
-the list, and so on. Hence an iterative loop can start with
-the empty list and perform "pop an item and cons it onto the list"
-@i[length] times.
-(Lists of length greater than 255 can be made by using @f[FOP-LIST*]
-repeatedly.)
-
-16 @f[ ] @f[FOP-LIST*] @f[ ] @i[length](1) @f[ ] @PushArrow @f[ ] stack @\
-This is like @f[FOP-LIST] except that the constructed list is terminated
-not by @f[()] (the empty list), but by an item popped from the stack
-before any others are. Therefore @i[length]+1 items are popped in all.
-Hence an iterative loop can start with
-a popped item and perform "pop an item and cons it onto the list"
-@i[length]+1 times.
-
-17-24 @f[ ] @f[FOP-LIST-1], @f[FOP-LIST-2], ..., @f[FOP-LIST-8] @\
-@f[FOP-LIST-@i{k}] is like @f[FOP-LIST] with a byte containing @i[k]
-following it. These exist purely to reduce the size of Fasload files.
-Measurements need to be made to determine the useful values of @i[k].
-
-25-32 @f[ ] @f[FOP-LIST*-1], @f[FOP-LIST*-2], ..., @f[FOP-LIST*-8] @\
-@f[FOP-LIST*-@i{k}] is like @f[FOP-LIST*] with a byte containing @i[k]
-following it. These exist purely to reduce the size of Fasload files.
-Measurements need to be made to determine the useful values of @i[k].
-
-33 @f[ ] @f[FOP-INTEGER] @f[ ] @i[n](4) @f[ ] @i[value](@i[n]) @f[ ]
-@PushArrow @f[ ] stack @\
-A four-byte unsigned operand specifies the number of following
-bytes. These bytes define the value of a signed integer in two's-complement
-form. The first byte of the value is the least significant byte.
-
-34 @f[ ] @f[FOP-SMALL-INTEGER] @f[ ] @i[n](1) @f[ ] @i[value](@i[n])
-@f[ ] @PushArrow @f[ ] stack @\
-A one-byte unsigned operand specifies the number of following
-bytes. These bytes define the value of a signed integer in two's-complement
-form. The first byte of the value is the least significant byte.
-
-35 @f[ ] @f[FOP-WORD-INTEGER] @f[ ] @i[value](4) @f[ ] @PushArrow @f[ ] stack @\
-A four-byte signed integer (in the range -2@+[31] to 2@+[31]-1) follows the
-operation code. A LISP integer (fixnum or bignum) with that value
-is constructed and pushed onto the stack.
-
-36 @f[ ] @f[FOP-BYTE-INTEGER] @f[ ] @i[value](1) @f[ ] @PushArrow @f[ ] stack @\
-A one-byte signed integer (in the range -128 to 127) follows the
-operation code. A LISP integer (fixnum or bignum) with that value
-is constructed and pushed onto the stack.
-
-37 @f[ ] @f[FOP-STRING] @f[ ] @i[n](4) @f[ ] @i[name](@i[n])
-@f[ ] @PushArrow @f[ ] stack @\
-The four-byte operand @i[n] specifies the length of a string to
-construct. The characters of the string follow, one per byte.
-The constructed string is pushed onto the stack.
-
-38 @f[ ] @f[FOP-SMALL-STRING] @f[ ] @i[n](1) @f[ ] @i[name](@i[n]) @f[ ] @PushArrow @f[ ] stack @\
-The one-byte operand @i[n] specifies the length of a string to
-construct. The characters of the string follow, one per byte.
-The constructed string is pushed onto the stack.
-
-39 @f[ ] @f[FOP-VECTOR] @f[ ] @i[n](4) @f[ ] @PushArrow @f[ ] stack @\
-The four-byte operand @i[n] specifies the length of a vector of LISP objects
-to construct. The elements of the vector are popped off the stack;
-the first one popped becomes the last element of the vector.
-The constructed vector is pushed onto the stack.
-
-40 @f[ ] @f[FOP-SMALL-VECTOR] @f[ ] @i[n](1) @f[ ] @PushArrow @f[ ] stack @\
-The one-byte operand @i[n] specifies the length of a vector of LISP objects
-to construct. The elements of the vector are popped off the stack;
-the first one popped becomes the last element of the vector.
-The constructed vector is pushed onto the stack.
-
-41 @f[ ] @f[FOP-UNIFORM-VECTOR] @f[ ] @i[n](4) @f[ ] @PushArrow @f[ ] stack @\
-The four-byte operand @i[n] specifies the length of a vector of LISP objects
-to construct. A single item is popped from the stack and used to initialize
-all elements of the vector. The constructed vector is pushed onto the stack.
-
-42 @f[ ] @f[FOP-SMALL-UNIFORM-VECTOR] @f[ ] @i[n](1) @f[ ] @PushArrow @f[ ] stack @\
-The one-byte operand @i[n] specifies the length of a vector of LISP objects
-to construct. A single item is popped from the stack and used to initialize
-all elements of the vector. The constructed vector is pushed onto the stack.
-
-43 @f[ ] @f[FOP-INT-VECTOR] @f[ ] @i[n](4) @f[ ] @i[size](1) @f[ ] @i[count](1) @f[ ]
-@i[data](@ceiling<@i[n]/@i[count]>@ceiling<@i[size]*@i[count]/8>) @f[ ]
-@PushArrow @f[ ] stack @\
-The four-byte operand @i[n] specifies the length of a vector of
-unsigned integers to be constructed. Each integer is @i[size]
-bits big, and are packed in the data stream in sections of
-@i[count] apiece. Each section occupies an integral number of bytes.
-If the bytes of a section are lined up in a row, with the first
-byte read at the right, and successive bytes placed to the left,
-with the bits within a byte being arranged so that the low-order bit
-is to the right, then the integers of the section are successive
-groups of @i[size] bits, starting from the right and running across
-byte boundaries. (In other words, this is a consistent
-right-to-left convention.) Any bits wasted at the left end of
-a section are ignored, and any wasted groups in the last section
-are ignored.
-It is permitted for the loading implementation to use a vector
-format providing more precision than is required by @i[size].
-For example, if @i[size] were 3, it would be permitted to use a vector
-of 4-bit integers, or even vector of general LISP objects filled
-with integer LISP objects. However, an implementation is expected
-to use the most restrictive format that will suffice, and is expected
-to reconstruct objects identical to those output if the Fasload file
-was produced by the same implementation.
-(For the PERQ U-vector formats, one would have
-@i[size] an element of {1, 2, 4, 8, 16}, and @i[count]=32/@i[size];
-words could be read directly into the U-vector.
-This operation provides a very general format whereby almost
-any conceivable implementation can output in its preferred packed format,
-and another can read it meaningfully; by checking at the beginning
-for good cases, loading can still proceed quickly.)
-The constructed vector is pushed onto the stack.
-
-44 @f[ ] @f[FOP-UNIFORM-INT-VECTOR] @f[ ] @i[n](4) @f[ ] @i[size](1) @f[ ]
-@i[value](@ceiling<@i[size]/8>) @f[ ] @PushArrow @f[ ] stack @\
-The four-byte operand @i[n] specifies the length of a vector of unsigned
-integers to construct.
-Each integer is @i[size] bits big, and is initialized to the value
-of the operand @i[value].
-The constructed vector is pushed onto the stack.
-
-45 @f[ ] @f[FOP-FLOAT] @f[ ] @i[n](1) @f[ ] @i[exponent](@ceiling<@i[n]/8>) @f[ ]
-@i[m](1) @f[ ] @i[mantissa](@ceiling<@i[m]/8>) @f[ ] @PushArrow @f[ ] stack @\
-The first operand @i[n] is one unsigned byte, and describes the number of
-@i[bits] in the second operand @i[exponent], which is a signed
-integer in two's-complement format. The high-order bits of
-the last (most significant) byte of @i[exponent] shall equal the sign bit.
-Similar remarks apply to @i[m] and @i[mantissa]. The value denoted by these
-four operands is @i[mantissa]@f[x]2@+{@i[exponent]-length(@i[mantissa])}.
-A floating-point number shall be constructed which has this value,
-and then pushed onto the stack. That floating-point format should be used
-which is the smallest (most compact) provided by the implementation which
-nevertheless provides enough accuracy to represent both the exponent
-and the mantissa correctly.
-
-46-51 Unused
-
-52 @f[ ] @f[FOP-ALTER] @f[ ] @i[index](1) @\
-Two items are popped from the stack; call the first @i[newval] and
-the second @i[object]. The component of @i[object] specified by
-@i[index] is altered to contain @i[newval]. The precise operation
-depends on the type of @i[object]:
-@begin(description)
-List @\ A zero @i[index] means alter the car (perform @f[RPLACA]),
-and @i[index]=1 means alter the cdr (@f[RPLACD]).
-
-Symbol @\ By definition these indices have the following meaning,
-and have nothing to do with the actual representation of symbols
-in a given implementation:
-@begin(description)
-0 @\ Alter value cell.
-
-1 @\ Alter function cell.
-
-2 @\ Alter property list (!).
-@end(description)
-
-Vector (of any kind) @\ Alter component number @i[index] of the vector.
-
-String @\ Alter character number @i[index] of the string.
-@end(description)
-
-53 @f[ ] @f[FOP-EVAL] @f[ ] @PushArrow @f[ ] stack @\
-Pop an item from the stack and evaluate it (give it to @f[EVAL]).
-Push the result back onto the stack.
-
-54 @f[ ] @f[FOP-EVAL-FOR-EFFECT] @\
-Pop an item from the stack and evaluate it (give it to @f[EVAL]).
-The result is ignored.
-
-55 @f[ ] @f[FOP-FUNCALL] @f[ ] @i[nargs](1) @f[ ] @PushArrow @f[ ] stack @\
-Pop @i[nargs]+1 items from the stack and apply the last one popped
-as a function to
-all the rest as arguments (the first one popped being the last argument).
-Push the result back onto the stack.
-
-56 @f[ ] @f[FOP-FUNCALL-FOR-EFFECT] @f[ ] @i[nargs](1) @\
-Pop @i[nargs]+1 items from the stack and apply the last one popped
-as a function to
-all the rest as arguments (the first one popped being the last argument).
-The result is ignored.
-
-57 @f[ ] @f[FOP-CODE-FORMAT] @f[ ] @i[id](1) @\
-The operand @i[id] is a unique identifier specifying the format
-for following code objects. The operations @f[FOP-CODE]
-and its relatives may not
-occur in a group until after @f[FOP-CODE-FORMAT] has appeared;
-there is no default format. This is provided so that several
-compiled code formats may co-exist in a file, and so that a loader
-can determine whether or not code was compiled by the correct
-compiler for the implementation being loaded into.
-So far the following code format identifiers are defined:
-@begin(description)
-0 @\ PERQ
-
-1 @\ VAX
-
-3 @\ @value(DinkyMachine)
-@end(description)
-
-58 @f[ ] @f[FOP-CODE] @f[ ] @i[nitems](4) @f[ ] @i[size](4) @f[ ]
-@i[code](@i[size]) @f[ ] @PushArrow @f[ ] stack @\
-A compiled function is constructed and pushed onto the stack.
-This object is in the format specified by the most recent
-occurrence of @f[FOP-CODE-FORMAT].
-The operand @i[nitems] specifies a number of items to pop off
-the stack to use in the "boxed storage" section. The operand @i[code]
-is a string of bytes constituting the compiled executable code.
-
-59 @f[ ] @f[FOP-SMALL-CODE] @f[ ] @i[nitems](1) @f[ ] @i[size](2) @f[ ]
-@i[code](@i[size]) @f[ ] @PushArrow @f[ ] stack @\
-A compiled function is constructed and pushed onto the stack.
-This object is in the format specified by the most recent
-occurrence of @f[FOP-CODE-FORMAT].
-The operand @i[nitems] specifies a number of items to pop off
-the stack to use in the "boxed storage" section. The operand @i[code]
-is a string of bytes constituting the compiled executable code.
-
-60 @f[ ] @f[FOP-STATIC-HEAP] @\
-Until further notice operations which allocate data structures
-may allocate them in the static area rather than the dynamic area.
-(The default area for allocation is the dynamic area; this
-default is reset whenever a new group is begun.
-This command is of an advisory nature; implementations with no
-static heap can ignore it.)
-
-61 @f[ ] @f[FOP-DYNAMIC-HEAP] @\
-Following storage allocation should be in the dynamic area.
-
-62 @f[ ] @f[FOP-VERIFY-TABLE-SIZE] @f[ ] @i[size](4) @\
-If the current size of the table is not equal to @i[size],
-then an inconsistency has been detected. This operation
-is inserted into a Fasload file purely for error-checking purposes.
-It is good practice for a compiler to output this at least at the
-end of every group, if not more often.
-
-63 @f[ ] @f[FOP-VERIFY-EMPTY-STACK] @\
-If the stack is not currently empty,
-then an inconsistency has been detected. This operation
-is inserted into a Fasload file purely for error-checking purposes.
-It is good practice for a compiler to output this at least at the
-end of every group, if not more often.
-
-64 @f[ ] @f[FOP-END-GROUP] @\
-This is the last operation of a group. If this is not the
-last byte of the file, then a new group follows; the next
-nine bytes must be "@f[FASL FILE]".
-
-65 @f[ ] @f[FOP-POP-FOR-EFFECT] @f[ ] stack @f[ ] @PushArrow @f[ ] @\
-One item is popped from the stack.
-
-66 @f[ ] @f[FOP-MISC-TRAP] @f[ ] @PushArrow @f[ ] stack @\
-A trap object is pushed onto the stack.
-
-67 @f[ ] @f[FOP-READ-ONLY-HEAP] @\
-Following storage allocation may be in a read-only heap.
-(For symbols, the symbol itself may not be in a read-only area,
-but its print name (a string) may be.
-This command is of an advisory nature; implementations with no
-read-only heap can ignore it, or use a static heap.)
-
-68 @f[ ] @f[FOP-CHARACTER] @f[ ] @i[character](3) @f[ ] @PushArrow @f[ ] stack @\
-The three bytes specify the 24 bits of a CMU Common Lisp character object.
-The bytes, lowest first, represent the code, control, and font bits.
-A character is constructed and pushed onto the stack.
-
-69 @f[ ] @f[FOP-SHORT-CHARACTER] @f[ ] @i[character](1) @f[ ]
-@PushArrow @f[ ] stack @\
-The one byte specifies the lower eight bits of a CMU Common Lisp character
-object (the code). A character is constructed with zero control
-and zero font attributes and pushed onto the stack.
-
-70 @f[ ] @f[FOP-RATIO] @f[ ] @PushArrow @f[ ] stack @\
-Creates a ratio from two integers popped from the stack.
-The denominator is popped first, the numerator second.
-
-71 @f[ ] @f[FOP-COMPLEX] @f[ ] @PushArrow @f[ ] stack @\
-Creates a complex number from two numbers popped from the stack.
-The imaginary part is popped first, the real part second.
-
-72 @f[ ] @f[FOP-LINK-ADDRESS-FIXUP] @f[ ] @i[nargs](1) @f[ ] @i[restp](1)
-@f[ ] @i[offset](4) @f[ ] @PushArrow @f[ ] stack @\
-Valid only for when FOP-CODE-FORMAT corresponds to the Vax or the
-@value(DinkyMachine).
-This operation pops a symbol and a code object from the stack and pushes
-a modified code object back onto the stack according to the needs of the
-runtime code linker on the Vax or @value(DinkyMachine).
-
-73 @f[ ] @f[FOP-LINK-FUNCTION-FIXUP] @f[ ] @i[offset](4) @f[ ]
-@PushArrow @f[ ] stack @\
-Valid only for when FOP-CODE-FORMAT corresponds to the Vax or the
-@value(DinkyMachine).
-This operation pops a symbol and a code object from the stack and pushes
-a modified code object back onto the stack according to the needs of the
-runtime code linker on the Vax or the @value(DinkyMachine).
-
-74 @f[ ] @f[FOP-FSET] @f[ ] @\
-Pops the top two things off of the stack and uses them as arguments to FSET
-(i.e. SETF of SYMBOL-FUNCTION).
-
-128 @f[ ] @f[FOP-LINK-ADDRESS-FIXUP] @f[ ] @i[nargs] @f[ ] @i[flag] @f[ ]
-@i[offset] @f[ ]@\Valid only when FOP-CODE-FORMAT corresponds to the
-@value(DinkyMachine). This operation pops a symbol and a function object
-off the stack. The code vector in the function object is modified
-according to the needs of the runtime code linker of the @value(DinkyMachine)
-and pushed back on the stack. This FOP links in calls to other functions.
-
-129 @f[ ] @f[FOP-MISCOP-FIXUP] @f[ ] @i[index](2) @f[ ] @i[offset](4) @f[ ]@\
-Valid only when FOP-CODE-FORMAT corresponds to the @value(DinkyMachine).
-This operation pops a code object from the stack and pushes a
-modified code object back onto the stack according to the needs of
-the runtime code linker on the @value(DinkyMachine). This FOP links in
-calls to the assembler language support routines.
-
-130 @f[ ] @f[FOP-ASSEMBLER-ROUTINE] @f[ ] @i[code-length] @f[ ] @\
-Valid only when FOP-CODE-FORMAT corresponds to the @value(DinkyMachine).
-This operation loads assembler code into the assembler code space of the
-currently running Lisp.
-
-131 @f[ ] @f[FOP-FIXUP-MISCOP-ROUTINE] @f[ ]@\Valid only when FOP-CODE-FORMAT
-corresponds to the @value(DinkyMachine). This operation pops a list of
-external references, a list of external labels defined, the name, and the
-code address off the stack. This information is saved, so that after
-everything is loaded, all the external references can be resolved.
-
-132 @f[ ] @f[FOP-FIXUP-ASSEMBLER-ROUTINE] @f[ ]@\is similar to
-FOP-FIXUP-MISCOP-ROUTINE, except it is for internal assembler routines
-rather than ones visible to Lisp.
-
-133 @f[ ] @f[FOP-FIXUP-USER-MISCOP-ROUTINE] @f[ ]@\is similar to
-FOP-FIXUP-MISCOP-ROUTINE, except it is for routines written by users who
-have an extremely good understanding of the system internals.
-
-134 @f[ ] @f[FOP-USER-MISCOP-FIXUP] @f[ ] @i[offset](4) @f[ ]@\is similar
-to FOP-MISCOP-FIXUP, but is used to link in user defined miscops.
-
-255 @f[ ] @f[FOP-END-HEADER] @\ Indicates the end of a group header, as described above.
-@end(description)
-
-@Appendix[Building CMU Common Lisp]
-
-@section(Introduction)
-This document explains how to build a working Common Lisp from source code on
-the IBM RT PC under the Mach operating system. You should already have a
-working Common Lisp running on an IBM RT PC before trying to build a new Common
-Lisp.
-
-Throughout this document the following terms are used:
-@begin(Description)
-Core file@\A core file is a file containing an image of a Lisp system. The
-core file contains header information describing where the data in the rest of
-the file should be placed in memory. There is a simple C program which reads a
-core file into memory at the correct locations and then jumps to a location
-determined by the contents of the core file. The C code includes the X
-window system version 10 release 4 which may be called from Lisp.
-
-
-Cold core file @\A cold core file contains enough of the Lisp system to make it
-possible to load in the rest of the code necessary to generate a full Common
-Lisp. A cold core file is generated by the program Genesis.
-
-Miscops@\Miscops are assembler language routines that are used to support
-compiled Lisp code. A Lisp macro assembler provides a
-convenient mechanism for writing these assembler language routines.
-
-Matchmaker@\Matchmaker is a program developed to automatically generate
-remote procedure call interfaces between programs. Matchmaker accepts
-a description of a remote procedure call interface and generates code
-that implements it.
-@end(Description)
-
-There are many steps required to go from sources to a working Common Lisp
-system. Each step will be explained in detail in the following sections.
-It is possible to perform more than one step with one invocation of Lisp.
-However, I recommend that each step be started with a fresh Lisp. There
-is some small chance that something done in one step will adversely affect
-a following step if the same Lisp is used. The scripts for each
-step assume that you are in the user package which is the default when
-Lisp first starts up. If you change to some other package, some of these
-steps may not work correctly.
-
-In many of the following steps, there are lines setting up search lists so that
-command files know where to find the sources. What I have done is create a
-init.lisp file that sets up these search lists for me. This file is
-automatically loaded from the user's home directory (as determined by the
-@b[HOME] environment variable) when you start up Lisp. Note that my init.lisp
-file is included with the sources. You may have to modify it, if you change
-where the lisp sources are.
-
-@section(Installing Source Code)
-With this document, you should also have received a tape cartridge in tar
-format containing the complete Common Lisp source code. You should create
-some directory where you want to put the source code. For the following
-discussion, I will assume that the source code lives in the directory
-/usr/lisp. To install the source code on your machine, issue the following
-commands:
-@begin(Example)
-cd /usr/lisp
-tar xvf <tape device>
-@end(Example)
-The first command puts you into the directory where you want the source code,
-and the second extracts all the files and sub-directories from the tape into
-the current directory. <Tape device> should be the name of the tape device on
-your machine, usually /dev/st0.
-
-The following sub-directories will be created by tar:
-@begin(Description)
-bin@\contains a single executable file, lisp, which is a C program
-used to start up Common Lisp.
-
-clc@\contains the Lisp source code for the Common Lisp compiler and assembler.
-
-code@\contains the Lisp source code that corresponds to all the functions,
-variables, macros, and special forms described in @i[Common Lisp: The Language]
-by Guy L. Steele Jr., as well as some Mach specific files.
-
-hemlock@\contains the Lisp source code for Hemlock, an emacs-like editor
-written completely in Common Lisp.
-
-icode@\contains Matchmaker generated code for interfaces to Inter Process
-Communication (IPC) routines. This code is used to communicate with other
-processes using a remote procedure call mechanism. Under Mach, all the
-facilities provided by Mach beyond the normal Berkeley Unix 4.3 system
-calls are accessed from Lisp using this IPC mechanism. Currently, the
-code for the Mach, name server, Lisp typescript, and Lisp eval server
-interfaces reside in this directory.
-
-idefs@\contains the Matchmaker definition files used to generate the Lisp
-code in the icode directory.
-
-lib@\contains files needed to run Lisp. The file lisp.core is known as a
-Lisp core file and is loaded into memory by the lisp program mentioned
-above in the entry for the bin directory. This file has a format which
-allows it to be mapped into memory at the correct locations. The files
-spell-dictionary.text and spell-dictionary.bin are the text and binary
-form of a dictionary, respectively, used by Hemlock's spelling checker and
-corrector. The two files hemlock.cursor and hemlock.mask are used by
-Hemlock when running under the X window system.
-
-miscops@\contains the Lisp assembler source code for all the miscops
-that support low level Lisp functions, such as storage allocation,
-complex operations that can not performed in-line, garbage collection, and
-other operations. These routines are written in assembler, so that they
-are as efficient as possible. These routines use a very short calling
-sequence, so calling them is very cheap compared to a normal Lisp
-function call.
-
-mm@\contains the Lisp source code for the Matchmaker program. This program
-is used to generate the Lisp source code files in icode from the corresponding
-matchmaker definitions in idefs.
-
-pcl@\contains the Lisp source code for a version of the Common Lisp Object
-System (originally Portable Common Loops),
-an object oriented programming language built on top of Common Lisp.
-
-X@\contains the C object files for X version 10 release 4 C library
-routines. These are linked with the lisp startup code, so that X is
-available from Lisp.
-
-scribe@\contains Scribe source and postscript output for the manuals
-describing various aspects of the CMU Common Lisp implementation.
-
-demos@\contains the Lisp source code for various demonstration programs.
-This directory contains the Gabriel benchmark set (bmarks.lisp) and
-a sub-directory containing the Soar program which is also used for
-benchmarking purposes. There may be other programs and/or sub-directories
-here that you may look at.
-@end(Description)
-These directories contain source files as well as Lisp object files.
-This means it is not necessary to go through all the steps to
-build a new a Common Lisp, only those steps that are affected by
-a modification to the sources. For example, modifying the compiler
-will require recompiling everything. Modifying a miscop file should
-require only reassembling that particular file and rebuilding the
-cold core file and full core file.
-
-As well as the directories mentioned above, there are also several files
-contained in the top-level directory. These are:
-@begin(Description)
-init.lisp@\is a Lisp init file I use. This sets up some standard search
-lists, as well as defines a Hemlock mode for editing miscop
-source files.
-
-lisp.c@\contains the C code used to start up the lisp core image under Mach.
-
-lispstart.s@\contains some assembler language code that is invoked by lisp.c
-to finish the process of starting up the lisp process.
-
-makefile@\contains make definitions for compiling lisp.c and lispstart.s
-into the lisp program.
-
-rg@\contains some adb commands that can be read into adb while debugging a lisp
-process. It prints out all the registers, the name of the currently
-executing Lisp function, and sets an adb variable to the current stack frame
-which is used by the following file.
-
-st@\contains some adb commands that can be read into adb while debugging
-a lisp process. It prints out a Lisp stack frame and the name of the
-function associated with the stack frame. It also updates the adb variable
-mentioned above to point to the next stack frame. Repeatedly reading this
-file into adb will produce a backtrace of all the active call frames
-on the Lisp stack.
-
-ac@\contains some adb commands that print out the current values of the
-active catch pointer. This points to the head of a list of catch frames
-that exist on the control stack.
-
-cs@\contains some adb commands that print out the contents of a catch
-frame. Reading cs into adb several times in a row (after reading ac once)
-will print out the catch frames in order.
-@end(Description)
-
-@section(Compiling the Lisp Startup Program)
-To compile the lisp start up program, you should be in the top level directory
-of the sources (/usr/lisp) and type:
-@begin(Example)
-make lisp
-@end(Example)
-This will compile the file lisp.c, assemble the file lispstart.s and produce
-an executable file lisp. Currently the default location for the lisp core
-file is /usr/misc/.lisp/lib/lisp.core. If you want to change this default
-location, edit the file lisp.c and change the line
-@begin(Example)
-#define COREFILE "/usr/misc/.lisp/lib/lisp.core"
-@end(Example)
-to refer to the file where you intend to put the core file.
-
-This step takes a few seconds.
-
-@section(Assembling Assembler routines)
-The standard core image includes a Lisp macro assembler. To assemble all
-the miscops, the following steps should be performed:
-@begin(Example)
-(compile-file "/usr/lisp/clc/miscasm.lisp")
-(load "/usr/lisp/clc/miscasm.fasl")
-(setf (search-list "msc:") '("/usr/lisp/miscops/"))
-(clc::asm-files)
-@end(Example)
-The first line compiles a file that contains a couple of functions used to
-assemble miscop source files. The second line loads the resulting compiled
-file into the currently executing core image. The third line defines the
-msc search list which is used by the function clc::asm-files to locate
-the miscop sources. The terminal will display information as each file
-is assembled. For each file a .fasl, a .list, and an .err file will be
-generated in /usr/lisp/miscops.
-
-This step takes about half an hour.
-
-@section(Compiling the Compiler)
-
-To compile the compiler is simple:
-@begin(Example)
-(setf (search-list "clc:") '("/usr/lisp/clc/"))
-(load "clc:compclc.lisp")
-@end(Example)
-The first line just sets up a search list variable clc, so that the file
-compclc.lisp can find the compiler sources. The terminal will display
-information as each file is assembled. For each file a .fasl and an .err file
-will be generated. A log of the compiler output is also displayed on the
-terminal.
-
-This step takes about forty-five minutes.
-
-@section(Compiling the Lisp Sources)
-
-Compiling the Lisp source code is also easy:
-@begin(Example)
-(setf (search-list "code:") '("/usr/lisp/code/"))
-(load "code:worldcom.lisp")
-@end(Example)
-Again, the first line defines a search list variable, so that the file
-worldcom.lisp can find the Lisp sources. As each file is compiled, the
-name of the file is printed on the terminal. For each file a .fasl will be
-generated. Also, a single error log will be generated in the file
-code:compile-lisp.log.
-
-This step takes about an hour and a half.
-
-@section(Compiling Hemlock)
-
-Compiling the Hemlock source code is done as follows:
-@begin(Example)
-(setf (search-list "hem:") '("/usr/lisp/hemlock/"))
-(load "hem:ctw.lisp")
-@end(Example)
-Again, the first line defines a search list variable, so that ctw.lisp can
-find the Hemlock sources. As each file is compiled, the name of the file is
-printed on the terminal. For each file a .fasl will be generated. Also, a
-single error log will be generated in the file hem:lossage.log.
-
-This step takes about forty-five minutes.
-
-@section(Compiling Matchmaker)
-Compiling the matchmaker sources is done as follows:
-@begin(Example)
-(setf (search-list "mm:") '("/usr/lisp/mm"))
-(compile-file "mm:mm.lisp")
-(load "mm:mm.fasl")
-(compile-mm)
-@end(Example)
-The first line sets up a search list, so that the matchmaker sources can be
-found. The second line compiles the file containing a function for compiling
-the matchmaker sources. The third line loads the file just
-compiled, and the final line invokes the function compile-mm which compiles the
-matchmaker sources. For each file, a .fasl and .err file is generated. Also,
-a log of the compiler output is printed to the terminal.
-
-This step takes about 15 minutes
-
-@section(Generating Lisp Source Files from Matchmaker Definition Files)
-The following sequence of commands is necessary to generate the Lisp
-files for the Mach interface:
-@begin(Example)
-(setf (search-list "mm:") '("/usr/lisp/mm/"))
-(setf (search-list "idefs:") '("/usr/lisp/idefs/"))
-(setf (search-list "icode:") '("/usr/lisp/icode/"))
-(setf (search-list "code:") '("/usr/lisp/code/"))
-(setf (default-directory) "/usr/lisp/icode/")
-(load "code:mm-interfaces.lisp")
-@end(Example)
-The first four lines set up search lists for mm (matchmaker sources), idefs
-(matchmaker interface definition files), icode (Lisp matchmaker interface
-sources), and code (Lisp code sources). The fifth line changes the current
-working directory to be /usr/lisp/icode. This is where the output from
-matchmaker will be placed. And finally, the last line invokes matchmaker on
-the matchmaker definition files for all the interfaces.
-
-Matchmaker generates three files for each interface XXX:
-@begin(Description)
-XXXdefs.lisp@\contains constants and record definitions for the interface.
-
-XXXmsgdefs.lisp@\contains definitions of offsets to important fields in the
-messages that are sent to and received from the interface.
-
-XXXuser.lisp@\contains code for each remote procedure, that sends a message
-to the server and receives the reply from the server (if appropriate).
-Each of these functions returns one or more values. The first value
-returned is a general return which specifies whether the remote procedure
-call succeeded or gives an indication of why it failed. Other values may
-be returned depending on the particular remote procedure. These values are
-returned using the multiple value mechanism of Common Lisp.
-@end(Description)
-
-This step takes about five minutes.
-
-@section(Compiling Matchmaker Generated Lisp Files)
-To compile the matchmaker generated Lisp files the following steps should
-be performed:
-@begin(Example)
-(setf (search-list "code:") '("/usr/lisp/code/"))
-(setf (search-list "icode:") '("/usr/lisp/icode/"))
-(load "code:comutil.lisp")
-@end(Example)
-The first two lines set up search lists for the code and icode directory.
-The final line loads a command file that compiles the Mach interface
-definition in the correct order. Note that once the files are compiled,
-the XXXmsgdefs files are no longer needed. The file
-/usr/lisp/icode/lossage.log contains a listing of all the error messages
-generated by the compiler.
-
-This step takes about fifteen minutes.
-
-@section(Compiling the Common Lisp Object System)
-
-To compile the Common Lisp Object System (CLOS) do the following:
-@begin(Example)
-(setf (search-list "pcl:") '("/usr/lisp/pcl/"))
-(rename-package (find-package "CLOS") "OLD-CLOS")
-(compile-file "pcl:defsys.lisp")
-(load "pcl:defsys.fasl")
-(clos::compile-pcl)
-@end(Example)
-The first line sets up a search list as usual. The second line renames the
-CLOS package to be the OLD-CLOS package. This is so that the current version
-of CLOS doesn't interfere with the compilation process. The third line
-compiles a file containing some functions for building CLOS. The fourth
-line loads in the result of the previous compilation. The final line
-compiles all the CLOS files necessary for a working CLOS system.
-
-The file /usr/lisp/pcl/test.lisp is a file that contains some test functions.
-To run it through CLOS build a new Lisp and start up a fresh Lisp
-resulting from the build and do the following:
-@begin(Example)
-(in-package 'clos)
-(compile-file "/usr/lisp/pcl/test.lisp")
-(load "/usr/lisp/pcl/test.fasl")
-@end(Example)
-This sequence of function calls puts you in the CLOS package, compiles the
-test file and then loads it. As the test file is loaded, it executes several
-tests. It will print out a message specifying whether each test passed or
-failed.
-
-Currently, CLOS is built into the standard core.
-
-This step takes about 30 minutes.
-
-@section(Compiling Genesis)
-To compile genesis do the following:
-@begin(Example)
-(compile-file "/usr/lisp/clc/genesis.lisp")
-@end(Example)
-Genesis is used to build a cold core file. Compiling Genesis takes about five
-minutes.
-
-@section(Building a Cold Core File)
-Once all the files have been assembled or compiled as described above, it is
-necessary to build a cold core file as follows:
-@begin(Example)
-(setf (search-list "code:") '("/usr/lisp/code/"))
-(setf (search-list "icode:") '("/usr/lisp/icode/"))
-(setf (search-list "msc:") '("/usr/lisp/miscops/"))
-(load "/usr/lisp/clc/genesis.fasl")
-(load "code:worldbuild.lisp")
-@end(Example)
-The first three lines set up search lists for the code, icode, and miscops
-subdirectories. The fourth line loads in the program Genesis which builds
-the cold core file. The last line calls Genesis on a list of the files that
-are necessary to build the cold core file. As each file is being processed,
-its name is printed to the terminal. Genesis generates two files:
-/usr/lisp/ilisp.core and /usr/lisp/lisp.map. Ilisp.core is the cold core
-file and lisp.map is a file containing the location of all the functions
-and miscops in the cold core file. Lisp.map is useful for debugging the
-cold core file.
-
-This step takes from about fifteen minutes.
-
-@section(Building a Full Common Lisp)
-The cold core file built above does not contain some of the more useful
-programs such as the compiler and hemlock. To build these into a core, it is
-necessary to do the following:
-@begin(Example)
-lisp -c /usr/lisp/ilisp.core
-(in-package "USER")
-(load (open "/usr/lisp/code/worldload.lisp"))
-@end(Example)
-The first line invokes the lisp startup program specifying the cold core
-file just built as the core file to load. This cold core file is set up
-to do a significant amount of initialization and it is quite possible that
-some bug will occur during this initialization process. After about a
-minute, you should get a prompt of the form:
-@begin(Example)
-CMU Common Lisp kernel core image 2.7(?).
-[You are in the Lisp Package.]
-*
-@end(Example)
-The following two lines should then be entered. The first of these puts
-you into the User package which is the package you should be in when the
-full core is first started up. It is necessary to add this line, because
-the current package is rebound while a file is loaded. The last line loads
-in a file that loads in the compiler, hemlock, and some other files not yet
-loaded. The open call is @b[essential] otherwise when the full core is
-started up, load will try to close the file and probably invalidate memory
-that is needed. When load is passed a stream, it does not automatically
-close the stream. With a file name it now does after a recent bug fix.
-This file prompts for the versions of the Lisp system, the compiler, and
-hemlock. You should enter versions that make sense for your installation.
-It then purifies the core image. Up to this point most of the Lisp system
-has been loaded into dynamic space. Only a few symbols and some other data
-structures are in static space. The process of purification moves Lisp
-objects into static and read-only space, leaving very little in dynamic
-space. Having the Lisp system in static and read-only space reduces the
-amount of work the garbage collector has to do. Only those objects needed
-in the final core file are retained. Finally, a new core file is generated
-and is written to the file /usr/lisp/nlisp.core. Also, the currently
-running Lisp should go through the default initialization process, finally
-prompting for input with an asterisk. At this point you have successfully
-built a new core file containing a complete Common Lisp implementation.
-
-This step takes about thirty minutes.
-
-@section(Debugging)
-Debugging Lisp code is much easier with a fully functional Lisp. However, it
-is quite possible that a change made in the system can cause a bug to happen
-while running the cold core file. If this happens, it is best to use adb to
-track down the problem. Unfortunately, the core file (i.e., the
-remains of a process normally created by Unix when a process dies) generated by
-such a bug will be of no use. To get some useful information, follow these
-steps:
-@begin(Enumerate)
-Look at the file /usr/lisp/lisp.map and find the entry points for the
-miscop routines error0, error1, and error2. These entry points are
-used to invoke the Lisp error system from the miscops. Write down
-the numbers beside these names. They are the addresses (in hex) of where
-the miscops are located when the cold core file is loaded into memory.
-
-Run adb on the lisp file, i.e.:
-@begin(example)
-adb lisp
-@end(Example)
-
-Set a breakpoint at the lispstart entry point:
-@begin(Example)
-lispstart:b
-@end(Example)
-
-Start the lisp program running, telling it to use ilisp.core (I'm
-assuming you're in /usr/lisp):
-@begin(Example)
-:r -c ilisp.core
-@end(Example)
-
-After a while, you will hit the lispstart breakpoint. The core file has been
-mapped into memory, but control is still in the C startup code. At this point,
-you should enter breakpoints for all the error entry points described above.
-
-Continue running the program by typing :c. Shortly after this, the C lisp
-program will give up control to Lisp proper. Lisp will start doing its
-initialization and will probably hit one of the error break points.
-At that point you can look around at the state and try and discover
-what has gone wrong. Note that the two files rg and st are useful at this
-point. Also, you should look at the document @i[Internal Design of Common
-Lisp on the IBM RT PC] by David B. McDonald, Scott E. Fahlman, and Skef
-Wholey so that you know the internal data structures.
-@end(Enumerate)
-
-@section(Running the Soar Benchmark)
-To compile the soar benchmark, you should do the following:
-@begin(Example)
-(compile-file "/usr/lisp/demos/soar/soar.lisp")
-@end(Example)
-
-To run the benchmark, you should start up a fresh Lisp and do the following:
-@begin(Example)
-(load "/usr/lisp/demos/soar/soar.fasl")
-(load "/usr/lisp/demos/soar/default.soar")
-(load "/usr/lisp/demos/soar/eight.soar")
-(user-select 'first)
-(init-soar)
-(time (run))
-@end(Example)
-The first two lines load in the standard Soar system. The third line loads in
-information about the eight puzzle which is a standard Soar puzzle that has
-been run on several different machines. The fourth line sets up the puzzle
-conditions so that it will select a goal to work on automatically. The fifth
-line initializes Soar's working memory, etc. The final line is the one that
-actually runs the benchmark. Soar prints out a fair amount of information as
-it solves the puzzle. The final state should be numbered 143 when it finishes.
-The time macro prints out information about information various resources after
-the eight puzzle has run.
-
-@section(Summary)
-I have tried to present sufficient information here to allow anyone to be
-able to build a Common Lisp system under Mach from the sources. I am sure
-there are many tricks that I have learned to use to reduce the amount of grief
-necessary to build a system. My best recommendation is to go slowly. Start
-by building a system from the sources provided on the tape. Make sure you
-are comfortable doing that before you try modifying anything.
-
-Some hints on building the system which you may find useful:
-@begin(Itemize)
-If you change the compiler, you will have to recompile all the sources before
-the change is reflected in a system. Changing the compiler is probably the
-most dangerous change you can make, since an error here means that
-nothing will work. In particular, this is the time you are going to need
-to get familiar with adb and the internal structure of the Lisp, since a
-serious error in the compiler will show up during the initialization of the
-cold core file.
-
-Changing the miscops should be done with care. They follow a fairly rigid
-convention and you should understand all the information provided in
-@i[Internal Design of Common Lisp on the IBM RT PC] before making any changes
-to miscops. You will probably need to get familiar with adb to debug some of
-the changes. Note that this requires building a new cold core file and a final
-core file before the change is reflected in the system.
-
-Changing sources in the code directory should be fairly straight forward. The
-only time this will cause trouble is if you change something that a lot of
-files depend on in which case you will have to recompile everything and build
-a new cold core file and a core file.
-
-Changing hemlock should have no adverse effect on system integrity.
-
-If you make a fairly major change, it is a good idea to go through the complete
-process of building a core file at least two or three times. If things are
-still working at the end of this, your change is probably correct and shouldn't
-cause any serious trouble.
-
-Finally, always keep at least one backup copy of a good core image around.
-If you build a bad core file over an existing one and can't back up, it is
-possible that you may not be able to recover from a serious error.
-@end(Itemize)
+++ /dev/null
-\part{Run-Time system}
-\include{environment}
-\include{interpreter}
-\include{debugger}
-\include{object}
-\include{lowlev}
-\include{fasl}
+++ /dev/null
-\chapter{Introduction} % -*- Dictionary: design -*-
-
-(defun gvp (f)
- (with-open-file (s f :direction :output :if-exists :supersede)
- (maphash \#'(lambda (k v)
- (declare (ignore v))
- (format s "~A~%" k))
- (c::backend-template-names c::*backend*))))
-
-\f
-\section{Scope and Purpose}
-
-This document describes the Virtual Machine that serves as the basis for the
-portable implementation of \ccl. The Virtual Machine (hereafter referred to as
-the VM) provides a layer of abstraction that hides low-level details of
-hardware and implementation strategy, while still revealing enough of the
-implementation so that most of the system can be written at the VM level or
-above.
-
-\begin{comment}
-
-{\#\#\# Shouldn't specify VOPs. Instead, should specify which \clisp functions
-are primitive and which subprimitives exist. Isn't really anyone's business
-which VOPs actually exist. Each primitive function or subprimitive is
-implemented either as a VOP or as expansion into Lisp code, at the particular
-implementation's discretion.
-
-From this point of view, the document is expressing the contract that the Lisp
-level code outside of the compiler must satisfy. All functions must ultimately
-be defined in terms of primitive functions and sub-primitives.
-
-The responsibility of the compiler is to implement these primitive operations,
-and also to implement special forms, variables and function calling.
-
-VOPs emitted by the hard-wired translators for non-function nodes are a
-somewhat different story. Each implementation will presumably implement all
-these VOPs in order to avoid having to rewrite IR2 translation. We also need
-to spend quite a bit of time discussing the semantics of these operations,
-since they don't just correspond to some \clisp function with type constraints.
-
-Hard-wired stuff:
-
-function call
-variable access:
- global
- function
- constant
- closure
- local
-closure creation
-non-local exit
-special binding/unbinding
-TN hacking:
- move VOPs
- TN address (???)
-Conditionals:
- Basic conditionals: EQ, ...
- Interface to generation of other conditional VOPs.
-
-Some VOPs don't need to be implemented at all:
- VOPs to delimit the lifetimes of big stack TNs such as catch blocks
- Others? Move VOPs might be defined in terms of an implementation supplied
- move routine, since we probably also need this info outside of VOP generators
- so that implicit moves can be generated.
-
-
-Type testing/checking (somehow)
-
-}
-
-What this document talks about:
-
-Interface between compiler front-end and back end. (VOPs)
- Primitive \clisp operations directly supported by the VM.
- Support for complex language features such as function call.
-
-Sub-primitives that allow system code to do things not possible in \clisp.
-
-Descriptions of how the current \ccl system uses VM facilities, especially
-non-standard ones accessed through sub-primitives.
-
-Notes about known portability problems.
-
-Guidelines for writing portable \ccl system code. To some degree these
-guidelines are implied by statements that certain things are true of \ccl
-system code.
-
-Descriptions of data structures that are not directly used by the VM, such as
-debug information and Core files.
-
-Descriptions of data structures that are directly used by the VM, such as
-symbols and arrays.
-
-
-Who should read it:
-
-People who want to port \ccl.
-People who want to understand the compiler.
-People who want to understand how \ccl works.
-People who need to write portable \ccl system code.
-People such as debugger writers who need to access \ccl\t()'s internal data
-structures.
-
-What it won't do:
-
-Tell you things that are obviously implementation dependent, such as type
-systems or memory management disciplines. See the the various implementation
-VM documents.
-
-Tell you only what you need to know. Programmers shouldn't exploit properties
-of the VM documented here unless there is no way to do the same thing in
-portable \clisp.
-
-Tell you how the compiler works. In order to understand some of the subtleties
-of VOP descriptions, you will have to understand the IR2 representation and how
-it fits into the rest of the compiler.
-
-Tell you anything about \clisp semantics. When some part of the VM has a
-direct relationship to \clisp semantics, the relationship will be directly
-stated using \clisp terminology, since a restatement of the semantics is likely
-to be inaccurate or misleading. Exceptions will be made only when some
-implication of the \clisp semantics is non-obvious.
-
-Tell you everything about how \ccl works. This document only offers
-information that is likely to be needed by programmers doing a port or writing
-system code; portable, self-contained parts of the system are totally ignored.
-This document deliberately avoids replicating information that is easily
-available in the system sources, since such replicated information is always
-incorrect somewhere. In some cases, a forwarding pointer to the appropriate
-source will be given.
-
-
-Things the VM won't do:
-
-The VM specification does not totally solve the problem of porting \ccl, since
-it is inevitable that it will not map cleanly to all possible combinations of
-hardware and operating systems. The VM should not be regarded as being cast in
-concrete, since changes in many characteristics would only affect a tiny
-fraction of the system sources.
-
-One current major problem with porting is that large pieces of functionality
-are entirely within the VM, and would need to be reimplemented for each port.
-A major goal for future work on the system is moving code out of the VM, both
-by supporting a "fast-call" convention that allows reasonable use of Lisp in
-the out of line implementation of VOPs, and by having a "bugout" mechanism that
-allows the VM to call Lisp functions to implement the hard cases in some VOPs.
-
-The VM is designed to support conventional, untagged, general register
-architectures. Suitably lobotomized, it could be mapped to less flexible
-hardware such as "Lisp machines", but the compiler would have serious
-difficulties supporting stack architectures.
-
-The VM does not support concurrent lightweight processes. Locking primitives
-and deep-binding of specials would be needed.
-
-The VM does not deal with operating systems interface issues at all. A minimal
-port would require implementing at least file and terminal I/O streams. \ccl
-implements system interfaces using Aliens and other facilities built on top of
-them.
-
-\end{comment}
-
-
-
-Major components:
-\begin{itemize}
-Specific virtual operations implemented by the VM (VOPs). VOPs are primarily
-the concern of the compiler, since it translates Lisp code into VOPs and then
-translates VOPs into the implementation.
-
-Sub-primitives that are used by Lisp code needing to perform operations
-below the Lisp level. The compiler implements some sub-primitives directly
-using VOPs, while others are translated into Lisp code. Sub-primitives provide
-a layer of insulation between the Lisp system code and the VM, since the Lisp
-code may assume the existence of operations that are not implemented directly
-by the VM. Only sub-primitives with fairly portable semantics are documented
-here. Others are in implementation-specific VM documentation.
-\end{itemize}
-
-\comment<Not all sub-primitives are VOPs, and most VOPs are not sub-primitives.>
-
-
-\f
-\subsection{VOP base name rules}
-
-The names of VOPs that implement functions are based on the function name.
-Other VOPs may use any base that doesn't conflict with a function name. There
-are some rules used to obtain the base name for related operations.
-
-To get the name of a setting operation, replace the string "{\tt ref}" in the name
-with "{\tt set}". If "{\tt ref}" doesn't appear in the name, add the prefix "{\tt set-}" to the
-base name. For example, {\tt svref} becomes {\tt svset}, and {\tt symbol-value}
-becomes {\tt set-symbol-value}.
-
-To get the name of a conditional VOP from the name of a predicate, add the
-prefix "{\tt if-}" to the predicate name. For example, {\tt eq} becomes {\tt if-eq}.
-{\tt eq} by itself would be a VOP that returned true or false value.
-
-Some operations check for some error condition, magically signalling the error
-through an implicit control transfer. These operations are prefixed with
-"{\tt check-}", as in {\tt check-fixnum} and {\tt check-bound}.
-
-
-\f
-\subsection{VOP name prefixes and suffixes}
-
-Prefixes and suffixes are added to the base to get the names of variant
-versions of the VOP. The fully general VOP name looks like this:
-\begin{format}
- {"{\tt small-}" | "{\tt fast-}"} {\it name}{"{\tt -c}" {\it info}}{"{\tt /}" {\it type}{"{\tt =>}" {\it result-type}}
-\end{format}
-The "{\tt small-}" and "{\tt fast-}" prefixes indicates that the VOP does minimal
-safety checking and is optimized for space or speed, respectively. The absence
-of a prefix indicates the safest (or only) version. Usually if the "{\tt small-}"
-VOP exists, it will be a synonym for either the fast version or the safe
-version, depending on which is smaller.
-
-The "{\tt -c}" suffix indicates that the some info that is passed as a normal
-argument to the base version of the VOP is passed as Codegen-Info in this
-version. A typical use would be for VOPs where it is important to use a
-different version when one of the arguments is a compile time constant.
-{\it info} is some (possibly null) string that indicates which "{\tt -c}" variant
-is involved.
-
-The "{\tt /}{\it type}" suffix asserts that all operands that could be of {\it type} are.
-For example, {\tt +/fixnum} adds two fixnums returning a fixnum, while
-{\tt length/simple-vector} finds the length of a simple vector, but the result isn't
-a simple vector.
-
-The "{\tt =>}{\it result-type}" suffix supplies a result type assertion on the
-operation.
-
-A not totally silly example of all these modifiers simultaneously is
- {\tt fast-+-c/fixnum=>integer}. This operation would this operation adds two
-fixnums, one of which is a constant passed as codegen info, resulting in an
-integer. The implementation is optimized for speed at the expense of space and
-safety.
-
-
-\f
-\chapter{Data Types and Storage Resources}
-
-\f
-\section{Lisp Objects}
-\index{Lisp objects}
-
-A Lisp object is fixed-size data structure that is organized in a way mandated
-by the VM implementation. The fixed format allows the VM to determine the type
-of the object. \comment<Virtual type? VM type? Implementation type?
-...provides the VM enough information about the type of the object for the VM
-to implement the VM-level semantics... ...supports the "dynamic types"...>
-
-Lisp objects are stored in locations known as cells.
-
-
-Has major types: immediate and non-immediate.
-Non-immediate objects may have a subtype.
-Non-immediate types:
- symbol (nil may be weird)
- cons
- ratio
- complex
- some float types
- g-vector
- i-vector
- string
- bit-vector
- environment (always has subtype)
- array header
- bignum
- structure
- pc (code vector)
- stack closure (control stack pointer)
-
-Non-immediate objects are allocated in "type spaces". The type space of an
-object is characterized by a small integer known as the type code. Any two
-objects of one of the above boxed types will always have the same type code.
-{But not really... Some types might be allocated in different type spaces at
-different times. (?)}
-
-The type code doesn't totally describe the object. In general, subtype
-information may be involved.
-
-
-Immediate types:
- character
- fixnum
- unbound trap
- short float
-
-
-\f
-\section{Type VOPs}
-
-We consider control transfer to be the fundamental result of comparison, rather
-than anything such as a condition code. Although most compilers with whizzy
-register allocation seem to explicitly allocate and manipulate the condition
-codes, it seems that any benefit is small in our case. This is partly because
-our VOPs are at a somewhat higher level, making it difficult to tell which VOPs
-do and don't trash the the CC. Explicitly incorporating condition codes in our
-VM also introduces another architecture dependency.
-
-At the IR2 level, we have a class of IF-XXX VOPs which transfer control to one
-of two places on the basis of some test on the operands. When generating code
-for a predicate, we peek at the destination IF node to find where to transfer
-control to.
-
-The exact representation of type tests in IR2 will be fairly implementation
-dependent, since it will depend on the specific type system for the given
-implementation. For example, if an implementation can test some types with a
-simple tag check, but other types require reading a field from the object in
-addition, then the two different kinds of checks should be distinct at the VOP
-level, since this will allow the VOP cost and storage information to be more
-accurate. Generation of type tests should be factored out of code which would
-otherwise be more portable. Probably the IR2 translator for TYPEP and the type
-check generation code are the only places that should know about how type tests
-are represented in IR2.
-
-if-type (object)
-if-type-range
- If-Type Tests whether Object has the type code that is passed in the
- codegen info. If-Type-Range tests for a range of type codes.
-
-{small, fast} if-vector-type (object)
- Test that Object is either of the specified type code, or is a 1d array
- header with data having the specified type code.
-
-if-vector-subtype (object)
- Test the subtype field of a vector-like object. It is assumed that the
- object has already been determined to be vector-like.
-
-if-fixnump (object)
-if-short-float-p
-if-characterp
- The rationale behind having these as separate VOPs is that they are likely
- to be immediate types, and thus may have bizzare type schemes.
-
-if-consp (object)
-if-listp
- We have distinct operations for these predicates since one or the other
- isn't a simple tag test, but we don't know which one.
-
-if-rationalp (object)
-if-floatp
-if-integerp
-if-numberp
-if-vectorp
-if-functionp
- The rationale behind having these operations is that they may take a lot of
- code, so it is reasonable to put them out of line.
-
-
-\f
-\section{Type Sub-primitives}
-
-change-type (object) => result
- Change the type of an object according to codegen info. The meaning of
- this is highly type-system dependent, but it doesn't matter, since the
- compiler will never emit this VOP directly. The only way that it can show
- up is through %Primitive.
-get-type
-
-
-Storage resources:
-
-Boxed and unboxed locations:
-Non-immediate objects may not be stored in unboxed locations.
-Things not lisp objects may not be stored in boxed locations.
-
-Control stack is boxed.
-Optional number stack is unboxed.
-Heap environment is boxed.
-Fixed number of registers, some boxed and some unboxed.
-
-PCs may be stored on the control stack or in boxed registers, subject to the
-constraint that a corresponding environment is also stored. Locations
-containing PCs don't need to be zeroed when they are no longer used; nothing
-bad will happen if an old PC is unaccompanied by an environment.
-
-
-\item[Trap]Illegal object trap. This value is used in symbols to signify an
-undefined value or definition.
-
-\f
-\chapter{Characters}
-
-
-Character is an immediate type. Characters are manipulated primarily by
-converting into an integer and accessing these fields:
-\begin{description}
-\item[{\tt %character-code-byte}]The character code. This is effectively required to
-start at bit 0, since \cl equates {\tt char-int} to {\tt char-code} when there is
-no bits or font. All current \ccl systems use ASCII for the character codes,
-and define {\tt \#\newline} to be a linefeed, but system code should not count on
-this.
-
-\item[{\tt %character-control-byte}]The character bits. Character bits are used by
-Hemlock to describe modifiers in keyboard events, but there is no assumption of
-any general portable significance of character bits.
-
-{\tt %character-font-byte}\\The character font. This is not used by \ccl, and is
-not particularly useful.
-\end{description}
-
-Characters should be converted to and from integers by using the \clisp
-{\tt char-int} and {\tt int-char} functions, which the compiler translates into
-these VOPs:
-\begin{example}
-char-int (char) => int
-int-char (int) => char
-\end{example}
-In the common case where Char is known to be a {\tt string-char}, these
-operations are equivalent to {\tt char-code} and {\tt code-char}. In addition to
-providing a portable interface to character conversion, the VOP representation
-of this type conversion allows the compiler to avoid unnecessary boxing and
-unboxing of character objects.
-
-Existing code explicitly converts fixnums to characters by using the
-Make-Immediate-Type sub-primitive with %Character-Type. Currently conversion
-of characters to fixnums is rather confused. Originally, characters were a
-subtype of the Misc type code, and the result of the Make-Fixnum sub-primitive
-had to be masked with {\tt %character-int-mask}; some code still does this, while
-other code may not.
-
-Character comparisons could be implemented by doing numeric comparisons on the
-result of {\tt char-int}, or by using {\tt eq} in the case of {\tt char=}, but this
-can result in unnecessary type conversions. Instead, the compiler uses these
-conditional VOPs:
-\begin{example}
-if-char= (x y)
-if-char< (x y)
-if-char> (x y)
-\end{example}
-
-\f
-\chapter{Symbols}
-
-
-Symbols are currently fairly boring in \ccl, containing only the obvious slots:
-\begin{description}
-{\tt %symbol-value-slot}\\The current dynamic value of this symbol. If the
-symbol is currently unbound, then the value of this slot is the unbound marker.
-
-{\tt %symbol-function-slot}\\The global function function definition of this
-symbol. If the symbol is not fbound, then this slot holds the unbound marker.
-
-\multiple{
-{\tt %symbol-plist-slot} \*
-{\tt %symbol-name-slot} \*
-{\tt %symbol-package-slot}
-}\\The property list, print name and package for this symbol.
-\end{description}
-
-
-\f
-\section{Sub-primitives}
-
-The {\tt alloc-symbol} sub-primitive allocates a new symbol object. {\it name} is
-the simple-string that is to be the name of the symbol.
-\begin{example}
-alloc-symbol (name) => symbol
-\end{example}
-
-The {\tt set-symbol-package} sub-primitive is used by system code that must set
-the symbol package.
-\begin{example}
-set-symbol-package (symbol new-value)
-\end{example}
-
-
-\f
-\section{Accessor VOPs}
-
-These VOPs read the global symbol value and definition cells. {\tt constant-ref}
-may only be used on symbols that have been defined to be constants. Since a
-constant cannot change in value and cannot be dynamically bound, the compiler
-may be able to compile uses of {\tt constant-ref} more efficiently. Unsafe
-versions of these VOPs may not check for the slot being unbound, which the
-corresponding \clisp functions are required to do.
-\begin{example}
-{small, fast} symbol-value (symbol) => value
-{small, fast} constant-ref (symbol) => value
-{small, fast} symbol-function (symbol) => value
-\end{example}
-
-These VOPs set the global symbol value and definition cells. {\tt makunbound}
-and {\tt fmakunbound} are implemented by setting the value to the unbound marker.
-\begin{example}
-{small, fast} set-symbol-value (symbol new-value)
-{small, fast} set-symbol-function (symbol new-value)
-\end{example}
-
-The \clisp accessors for other symbol slots are translated into uses of the
-{\tt slot-ref} and {\tt slot-set} VOPs.
-
-
-\f
-\section{Special Binding}
-
-These VOPs implement dynamic binding of special variables using shallow
-binding. {\tt bind} binds {\it symbol} to the specified {\it value}, while
-{\tt unbind} undoes the most recent {\it count} special bindings on the binding
-stack.
-\begin{example}
-bind (symbol value)
-unbind (count)
-\end{example}
-
-\f
-\section{Property Lists}
-
-The {\tt get} VOP implements the corresponding \clisp function, while {\tt put}
-implements its setf-inverse.
-\begin{example}
-get (symbol indicator default) => value
-put (symbol indicator value)
-\end{example}
-
-\f
-\chapter{Lists}
-
-
-cons
-
-list<n> (elt0 ... elt<n-1>) => list
-list (elt0 ... elt<n-1> more-elts) => list
- For some small N, we have fixed-arg versions of List. For larger lists, we
- pass in additional elements in a stack TN (possibly required to be on stack
- top). List* is similar.
-
-
-These VOPs implement the corresponding \clisp functions:
-\begin{example}
-{small, fast} car (list) => value
-{small, fast} cdr (list) => value
-\end{example}
-
-These VOPs set the car or cdr of a cons:
-\begin{example}
-{small, fast} set-car (cons new-value)
-{small, fast} set-cdr (cons new-value)
-\end{example}
-
-These VOPs implement the \clisp {\tt assoc} and {\tt member} functions with test
-functions of {\tt eql} and {\tt eq}:
-\begin{example}
-assoc (item alist) => cons-or-nil
-assq (item alist) => cons-or-nil
-member (item list) => cons-or-nil
-memq (item list) => cons-or-nil
-\end{example}
-
-
-{\tt getf} implements the corresponding \clisp function, while {\tt putf} is used
-to implement its setf-inverse. {\tt putf} returns the new value for the list so
-that it may stored back into the place.
-\begin{example}
-getf (list indicator default) => value
-putf (list indicator new-value) => list
-\end{example}
-
-\f
-\chapter{Numbers}
-
-\index{Fixnum format}
-Fixnum\\An N-bit two's complement integer.
-
-\index{Short float format}
-Short-Float\\An immediate float format.
-
-\index{Bignum format}
-\label{Bignums}
-Bignum\\Bignums are infinite-precision integers, represented somehow.
-
-\index{Flonum format}
-\index{Floating point formats}
-Floats\\Floats are stored as consecutive words of bits.
-
-\index{Ratio format}
-Ratio\\Ratios are stored as two consecutive words of Lisp objects, which should
-both be integers.
-
-\index{Complex number format}
-Complex\\Complex numbers are stored as two consecutive words of Lisp objects,
-which should both be numbers.
-
-
-\f
-\section{Number VOPs}
-
-integer-length
-{small, fast} integer-length/fixnum
-
-float=>xxx-float
-
-realpart
-lmagpart
-numerator
-denominator
-decode-float
-{small, fast} decode-float/xxx-float
-scale-float
-{small, fast} scale-float/xxx-float
-
-if-= (x y)
-{small, fast} if-=/fixnum
-{small, fast} if-=/xxx-float
- Do numeric comparison of X and Y. The codegen-info contains the
- continuations to transfer to in the true and false cases. Same for <, >.
-
-+ (x y) => z
-{small, fast} +/fixnum
-{small, fast} +/fixnum=>integer
-{small, fast} +/xxx-float
- Same for -, *. Fixnum multiplication by a constant power of 2 (or near
- power of 2) can be done by a transform.
-
-/ (x y) => z
-{small, fast} //xxx-float
-
-negate
-{small, fast} negate/fixnum
-{small, fast} negate/fixnum=>integer
-{small, fast} negate/xxx-float
- Ditto for Abs.
-
-truncate (x y) => q r
-{small, fast} truncate/fixnum
-
-logand (x y) => z
-{small, fast} logand/fixnum
- Ditto for logior, logxor.
-
-lognot (n) => z
-{small, fast} lognot/fixnum
-
-ash (n x) => z
-{small, fast} ash/fixnum
-{small, fast} ash-c/fixnum
-
-ldb
-dpb
-mask-field
-deposit-field
- These will only be used as a last resort. There should be transforms that
- turn fixnum operations with constant byte-specifiers into standard logical
- operations.
-
-\f
-\section{Number Sub-primitives}
-
-
-alloc-bignum
-make-complex
-make-ratio
-lsh
-logldb
-logdpb
-
-
-\f
-\chapter{Arrays}
-
-\cl arrays can be represented in a few different ways in \rtccl --
-different representations have different performance advantages. Simple
-general vectors, simple vectors of integers, and simple strings are basic \rtccl
- data types, and access to these structures is quicker than access to
-non-simple (or ``complex'') arrays. However, all multi-dimensional arrays in
-\rtccl are complex arrays, so references to these are always through a
-header structure.
-
-
-Once a vector has been allocated, it is possible to reduce its length by using
-the Shrink-Vector sub-primitive, but never to increase its length, even back to
-the original size, since the space freed by the reduction may have been
-reclaimed.
-
-
-\f
-\subsection{Arrays}
-\label{Arrays}
-\index{Arrays}
-
-An array header is identical in form to a G-Vector. At present, the following
-subtype codes are defined:
-\begin{itemize, spread 0, spacing 1}
-0 Normal.
-1 Array is displaced to another array (which may be simple).
-\end{itemize}
-The entries in the header-vector are interpreted as follows:
-
-\index{Array header format}
-\begin{description}
-0 Data Vector \\This is a pointer to the I-Vector, G-Vector, or string that
-contains the actual data of the array. In a multi-dimensional array, the
-supplied indices are converted into a single 1-D index which is used to access
-the data vector in the usual way. If the array is displaced, then this is
-the array displaced to, which may be an array header. In general, array
-access must loop until it finds an actual data vector.
-
-1 Number of Elements \\This is a fixnum indicating the number of elements for
-which there is space in the data vector.
-
-2 Fill Pointer \\This is a fixnum indicating how many elements of the data
-vector are actually considered to be in use. Normally this is initialized to
-the same value as the Number of Elements field, but in some array applications
-it will be given a smaller value. Any access beyond the fill pointer is
-illegal.
-
-3 Displacement \\This fixnum value is added to the final code-vector index
-after the index arithmetic is done but before the access occurs. Used for
-mapping a portion of one array into another. For most arrays, this is 0.
-
-4 Range of First Index \\This is the number of index values along the first
-dimension, or one greater than the largest legal value of this index (since the
-arrays are always zero-based). A fixnum in the range 0 to 2\+{24}-1. If any
-of the indices has a range of 0, the array is legal but will contain no data
-and accesses to it will always be out of range. In a 0-dimension array, this
-entry will not be present.
-
-5 - N Ranges of Subsequent Dimensions
-\end{description}
-
-The number of dimensions of an array can be determined by looking at the length
-of the array header. The rank will be this number minus 6. The maximum array
-rank is 65535 - 6, or 65529.
-
-The ranges of all indices are checked on every access, during the conversion to
-a single data-vector index. In this conversion, each index is added to the
-accumulating total, then the total is multiplied by the range of the following
-dimension, the next index is added in, and so on. In other words, if the data
-vector is scanned linearly, the last array index is the one that varies most
-rapidly, then the index before it, and so on.
-
-
-\f
-\section{Array VOPs}
-
-alloc-bit-vector
-alloc-i-vector
-alloc-string
-alloc-g-vector
- Initialized and uninitialized versions?
-
-
-length (sequence) => size
-{small, fast} length/vector
-{small, fast} length/simple-vector
-{small, fast} length/simple-string
-{small, fast} length/simple-bit-vector
-
-aref1 (vector index) => value
-{small, fast} aref1/simple-vector
-{small, fast} aref1/simple-string
-{small, fast} aref1/simple-bit-vector
-{small, fast} aref1/simple-array-XXX-float
-
-aset1 (vector index new-value)
-{small, fast} aset1/simple-vector
-{small, fast} aset1/simple-string
-{small, fast} aset1/simple-bit-vector
-{small, fast} aset1/simple-array-XXX-float
-
-{small, fast} aref1/simple-array-unsigned-byte (vector index) => value
-{small, fast} aset1/simple-array-unsigned-byte (vector index new-value)
- Byte size is codegen info.
-
-aref<N> (array index0 ... index<n-1>) => value
-aset<N> (array index0 ... index<n-1> new-value)
- For some small value of N. Of course, higher dimensional arrays can also
- be specialized in seven different ways.... Multi-dimensional simple array
- reference with known dimensions can be open-coded using a transform (useful
- for benchmarks.)
-
-
-\f
-\section{Array Sub-primitives}
-
-alloc-array
-vector-subtype
-set-vector-subtype
-vector-access-code
-set-vector-access-code
-shrink-vector
-
-typed-vref
-typed-vset
-
-header-length (header) => size
-header-ref (header index) => value
-header-set (header index new-value)
-
-bit-bash
-byte-blt
-{reverse-}find-character
-{reverse-}find-character-with-attribute
-{reverse-}string-compare
-sxhash-simple-string
-sxhash-simple-substring
-
-\f
-\chapter{Structures}
-
-{small, fast} structure-ref (s) => value
-{small, fast} structure-set (s new-value)
- Read and write structure slots. Defstruct slot description is in codegen
- info.
-
-alloc-structure
-
-\f
-\chapter{Runtime Environment}
-\index{Runtime Environment}
-\label{Runtime}
-
-\f
-\section{Register Allocation}
-\index{Register allocation}
-
-The main idea is to globally allocate only those registers with global
-significance.
-
-We permanently dedicate the CONT register to point to the current control stack
-environment. This is the "frame pointer" in standard terminology. It isn't
-possible to get pack to allocate this register on an as-needed basis due to the
-classic phase-ordering problem. We need to know if TNs are allocated on the
-stack before we can determine tell how badly we need a frame pointer register.
-This is of little significance with the control stack environment, since we
-almost always need one, and if there are any stack TNs, we must allocate the
-frame pointer in a register, since there is nowhere else to put it. The
-problem is more severe with a number stack environment pointer. We can't
-dedicate a register to it, since we usually don't have any TNs on the number
-stack. The only easy solution is to always allocate the number stack
-environment pointer on the control stack. This really isn't too bad, when you
-compare the cost of doing an extra memory reference to get at the number stack
-to the cost of number-consing.
-
-We also dedicate the ENV register to the current constant pool. It would be
-possible to explicitly allocate the constant pointer as needed if we explicitly
-represented non-immediate constant access by a VOP, but this would be extra
-work, and there are major advantages to representing all constants using TNs.
-Another potential efficiency advantage is since the same constant pool is
-shared by all the code in a component, we need only initialize ENV on entry to
-the component. When we make local calls, we don't have to do anything to make
-the constants available to the callee.
-
-Since the constant pool will also contain the code vector and the debug info,
-having it in a known place may make life easier for GC and the debugger. We
-may not be able to count on it too much, though, since ENV holds other things
-will calls are in progress, and might be pretty random if we jumped into
-hyperspace.
-
-\f
-Runtime environment:
-
-CONT: the current control stack context.
-PC is assumed to be accessible to the debugger when an error happens.
-Current-Catch: pointer to the current catch frame. Format of frame is assumed.
-Current-Unwind-Protect: current unwind protect frame. Similar to catch.
-
-If shallow-bind, binding stack and binding stack pointer.
-If deep-bind, current special binding. Format of binding frame assumed.
-
-Everything depends on the current environment, which is CONT.
-
-
-PC
-OLD-CONT
-ENV
-A<n>
-CONT
-CS
-
-\f
-\section{Other Dynamic State}
-
-There are some dynamic state variables that are stored in known memory
-locations, rather than having a dedicated register:
-\begin{description}
-binding stack pointer\\The current pointer to the top of the binding stack.
-
-current catch\\The pointer to the current catch block.
-
-current unwind-protect\\The pointer to the current unwind-protect block.
-\end{description}
-
-
-
-\f
-\section{Control-Stack Format}
-\label{Control-Stack-Format}
-\index{Control-stack format}
-
-
-The control stack contains only Lisp objects. Every object pointed to by an
-entry on this stack is kept alive.
-
-The \rtccl control stack does not have a rigid frame structure. The compiler
-is allowed a large amount of freedom in the use of the stack so that it choose
-the best calling sequences. Mostly the compiler is the only system that cares
-how the stack is laid out, so this isn't a big problem. See chapter
-\ref{debug-info} for a description of the structures which allow the debugger
-to parse the stack.
-
-
-
-\section{Values Passing Conventions}
-
-
-The first {\it nregs} arguments are passed in registers, where nregs is an
-implementation dependent constant. Any additional arguments are the block of
-storage between CONT and CS on the control stack. The first nregs locations in
-this block of storage are unused so that register more-args can be stored on
-the stack without having to BLT the stack values up.
-
-Returning unknown values are passed in a similar way, but the stack values
-block is between OLD-CONT and CS. There isn't any underneath the values: on
-return OLD-CONT is always what CS was when the function was called. The
-function returned to must copy the values into the desired location in its
-frame and deallocate excess stuff on the top of the stack.
-
-More args are represented by a pointer to the block of values and a count. The
-function that originally created the more arg must allocate and deallocate this
-stuff somehow. In the case of a local call to a more arg entry, we can just
-allocate it as a TN. The external entry point for a more arg entry is more
-magical.
-
-
-
-The caller allocates the environment for the called function, stores the
-arguments into it, and jumps to the function. The caller makes the called
-environment current, passing in the return OLD-CONT and PC as explicit arguments.
-
-When returning values, the returner directly stores the return values into the
-frame being returned to. This works even though the caller doesn't know what
-function it is returning to, since the same return locations are allocated in
-all frames.
-
-In a tail-recursive call, we can destructively modify the current frame and
-jump right to the callee, rather than allocating a new frame. We can do this
-because TNBind globally allocates frame locations; all frames are the same size
-and have the same TNs in the same place.
-
-\f
-\section{Binding-Stack Format}
-\index{Binding stack format}
-\comment<In a symbol chapter?>
-
-
-The special binding stack is used to hold previous values of special variables
-that have been bound. It grows and shrinks with the depth of the binding
-environment, as reflected in the control stack. This stack contains
-symbol-value pairs, with only boxed Lisp objects present.
-
-Each entry of the binding-stack consists of two boxed (32-bit) words. Pushed
-first is a pointer to the symbol being bound. Pushed second is the symbol's
-old value (any boxed item) that is to be restored when the binding stack is
-popped.
-
-\f
-\chapter{Functions}
-
-Function calling is a way of life.
-
-every function is a closure. pointer to current closure is passed in ENV
-unless it isn't (in local call may be elsewhere).
-
-The description of the representation of functions and the function calling
-conventions is a large part of the VM description, since:
- Function calling is one of the most complicated facilities provided by the
- VM.
-
- Everything that happens, happens in a function, so all parts of the system
- tend to get dragged in.
-
-
-Aspects of function call:
- Control
- Environment CONT, ENV
- Argument/value passing
- Argument/value count dispatching
-
-
-
-\f
-\section{Function Object Format}
-\label{Fn-Format}
-
-The old notion of a "function object" is now broken down into four different
-parts:
-\begin{description}
-Function entry\\A function entry is a structure that holds the information
-that we need to call a function. This is the user visible function object.
-
-Environment\\The environment is stuff that a function needs when it runs.
-This includes constants computed at load time and variables closed over at run
-time. Environment information may be allocated in the function entry structure
-after the required linkage information.
-
-Entry information\\This is information about a specific function entry that is
-occasionally referenced at run time, but need not be immediately accessible.
-Entry information will be either allocated in the function entry
-or in the environment that it points to.
-
-Debug information\\This is information about a function that isn't normally
-needed at run time. Debug information can be found by poking around in
-environment objects.
-\end{description}
-See chapter \ref{control-conventions} for a description of how function objects
-are used.
-
-\f
-\section{Environment Object Sub-primitives}
-
-alloc-code ?
-alloc-closure?
-
-\f
-\subsection{Debug Information Location}
-
-If present, debug information is stored immediately following any fixed
-information in the environment object. It may be necessary to chain up
-multiple levels of environments to find the debug information. The debug
-information can be recognized because it is represented by a defstruct
-structure. See chapter \ref{debug-info} for a description of the debug
-information.
-
-\f
-\section{Function Calls}
-\index{function call}
-
-\ccl supports three major calling conventions. The convention used
-depends on the amount of information available at compile time:
-\begin{description}
-Local\\Local call is used when the call and the called function are
-compiled at the same time. Using the term "convention" to describe this
-call mechanism is somewhat of a misnomer, since the compiler can do
-whatever it wants.
-
-Named\\Named call is used when the call is to a global function whose name
-is known at compile time.
-
-Anonymous\\Anonymous call is used when the function called is unknown until
-run time.
-\end{description}
-
-\#|
-IR2 function call:
-
-Environment manipulation code is always emitted at the location of the Bind or
-Return node for a Lambda.
-
-Implicit args to functions in IR2:
- old-cont: cont to restore on return
- return-pc: pc to return to
- env: pointer to current closure (if heap)
- closure<n>: closed values for current closure (if stack)
-
-Other info needed for IR2 conversion of functions:
- base pointers for all heap closures consed by this function
- also have passing locs for each explicit arg
- return strategy (known or unknown) and return locs
-
-All arguments including implicit ones must have both a passing TN and a
-permanent TN. Passing locs for let calls can be the actual TN that holds the
-variable in the case of local variables. Set closure variables must still have
-a separate passing TN.
-
-If we know the values counts for the argument continuations, then we compile
-local mv-calls by moving the TNs for the values continuations into the argument
-passing locations. Other mv-calls must be compiled using various hairy
-stack-hacking VOPs and unknown argument count call VOPs.
-
-For now, we will create the callee's frame just before the call, instead of
-creating it before the evaluation of the first argument. If we created the
-environment early, then we would be able to move the argument values directly
-into the frame, instead of having to store them somewhere else for a while.
-The problem with early creation is that lifetime analysis gets confused because
-there is more than one instance of the same TN present simultaneously in the
-case where there are nested calls to the same function.
-
-It turns out that there isn't a problem with a simple self-call, because the TN
-in the called frame is really the "same" TN as the one in the current frame,
-due to the restricted way in which we use the passing TNs.
-
-We emit code for external entry points during IR2 conversion. The external
-entry point is the place where we start running in a full call from a
-function-entry. It does arg count checking and dispatching, moves the
-arguments into the passing locations for the for the lambda being called, and
-calls the lambda, moving the results into the standard locations if there
-aren't there already.
-|\#
-
-
-In IR2, the environment manipulation semantics of function call are decoupled
-from the control semantics. When allocating closure variables for a Let, it is
-possible to do environment manipulation with only the normal sequential control
-flow. In the case of a Let call with the same environment, we neither
-manipulate the environment nor transfer control; we merely initialize the
-variables with Move VOPs.
-
-If a local function returns a known number of values which is less than the
-number expected by the caller, then additional code must be inserted at the
-return site which sets the unused values to NIL.
-
-The full function call mechanism must effectively be a subset of the local call
-mechanism, since the two mechanisms must mesh at entry points and full function
-calls. A full call turns into some kind of full call VOP. There are different
-VOPs for calling named functions and closures. We also have tail-recursive
-full call VOPs. Arguments are set up using Move VOPs, just as for local call.
-The only difference is that the passing locations and conventions are
-restricted to the standard ones.
-
-The gory details of arg count checking and dispatching are buried in the
-Function-Entry VOP, which takes a functional and a list of continuations, one
-pointing to each external entry.
-
-\f
-\subsection{Local Call}
-\index{local call}
-
-Named and anonymous call are called full calls, to distinguish them from
-local call. When making full calls, the compiler must make many worst-case
-assumptions that aren't necessary in a local call. The advantage of local
-call is that the compiler can choose to use only those parts of the full
-call sequence that are actually necessary.
-
-In local call, we always know the function being called, so we never have
-to do argument count checking, and can always use an immediate branch for
-the control transfer. If the function doesn't return to more than one
-place, then can just use a simple branch, or even drop through.
-
-The argument passing TNs may be allocated anywhere. The caller allocates the
-stack frame for the called function, moving any non-register arguments into the
-passing locations in the callee's frame.
-
-If we are calling a local function that doesn't always return the same
-number of values, then we must use the same values returning mechanism that
-is used in full call, but we don't have to use the standard registers.
-
-A tail-recursive local call doesn't require any call VOP. We just use Move
-VOPs to put the arguments into the passing locations and then jump to the the
-start of the code for the function. We don't have to do any stack hackery
-since we use the same stack frame format for all the functions compiled at the
-same time. In many cases tail-recursive local calls can be entirely optimized
-away, since they involve only some moves and a branch. We preference the
-argument values to the passing locations of the called function, making it
-likely that no move will be necessary. Often the control transfer can be done
-by simply dropping through.
-
-We have to do some funny stuff with local calls in order to get the lifetimes
-for the passing locations right, since lifetime analysis skips directly from
-the return point to the call point, ignoring the uses of the passing locations
-in the called function. Similarly, we pretend that a block ending in a return
-has no successors.
-
-call-local (arg*) "fun" => value
-multiple-call-local (arg*) "fun" => start end val0 ... val<n>
- Call-Local is used for calls to local functions that are forced to use the
- unknown-values passing convention. Value is the first return value
- register; we don't really do anything to it, but we specify it as a result
- to represent the assignment done by the calling function.
-
- Multiple-Call-Local is similar, but specifies all the values used by the
- unknown-values convention. Default-Values may be used to receive a
- specific number of values.
-
-known-call-local (arg*) "fun" => value*
- This VOP is used for local calls to functions where we can determine at
- compile time that the number of values returned is always the same. In
- this case, we don't need to indicate the number of values, and can pass
- them in separate TNs. The Values are the actual return locations. We
- don't really do anything to the return values; we just specify them as
- results to represent the assignment done by the called function.
-
-known-return (return-pc value*) "fun"
- This VOP is used for returning from local calls using the known return
- values convention. The specified return Values are moved into the passing
- locations in the caller's frame.
-
-
-If we know that the function we are calling is non-recursive, then we can
-compile it much like a tail-recursive call. We must have a call VOP to compute
-the return PC, but we don't need to allocate a frame or save registers. We
-just set up the arguments in the frame and do the call.
-
-We require simple functions to use the known-values convention. It would be
-possible to support unknown values, but it would potentially require BLT'ing
-return values out of the frame and on to the top of the stack. Supporting
-unknown values would also require a bunch more VOPs, since we need different
-call and return VOPs for simple call.
-
-Known values return causes no problems, since the callee knows how many values
-are wanted. We store the values directly into the current frame, since it is
-also the caller's frame.
-
-known-call-simple () "fun" => return-pc
-known-return-simple (return-pc) "fun"
- Similar to the non-simple VOPs, but don't allocate or deallocate frames,
- and assume that argument and value passing is done with explicit Move VOPs.
-
-\f
-\subsection{Full Call}
-\index{full call}
-
-Both named and anonymous call are optimized for calls where the number of
-arguments is known at compile time. Unknown argument calls are a
-pathological case of anonymous call; this case will be ignored in the main
-discussion. The difference between named and anonymous calls is in the
-argument count dispatching mechanism.
-
-Named call allows an arbitrary number of entry points, with start PCs at
-arbitrary locations in the code vector. The link-table mechanism described
-below allows named calls to jump directly to the actual entry point without any
-run-time argument count or type checking checking.
-
-Anonymous call has a fixed number of entry points, with start PCs at fixed
-locations in the code vector. This allows calls to be made without knowing
-what function is being called, but has more run-time overhead. The object
-called must be checked to be a valid function-entry object. The entry PC must
-be computed from the function entry, and argument count checking must be done
-if there are more than three required or optional arguments.
-
-Argument passing in full call is conceptually similar to local call, but the
-caller can't allocate the entire frame for the callee, since it doesn't know
-how much stack is needed. Instead we allocate the frame in two parts. The
-caller only allocates the beginning of the frame, which contains the stack
-arguments in fixed locations. We leave the first <n> locations unused so that
-the called function can move register more args onto the stack without having
-to BLT down any stack arguments.
-
-The place in the code where a full call jumps in is called an external entry
-point. The external entry point allocates the rest of the stack frame and then
-does a local call to the actual entry-point function, fetching the arguments
-from the standard passing locations. Usually we can do a tail-recursive local
-call.
-
-There are two main cases where the call from the external entry point cannot be
-tail-recursive:
- -- It is desirable to use the known-values convention for calling the
- entry-point function if the entry-point is used in other local calls
- (perhaps because of recursion). In this case, the called function stores
- the return values back into the frame allocated by the external entry point
- and then returns back to it. The external entry point must then return
- these values using the standard unknown-values convention.
- -- In a more-arg entry point we don't know how many stack arguments there are
- at the beginning of the frame, so we can't really use the frame allocated
- by the external entry point at all. Instead we do a local call to the
- more-arg entry point, passing in a pointer to the first extra value. When
- the function returns, we deallocate the crap on the stack and then return
- the values. It is still o.k. to use the known-values return convention
- from the more-arg entry since the extra arg values are no longer needed by
- the time the returning function stores the return values back into the
- external entry point frame.
-
-
-In full call we must always use the unknown-values convention for return. The
-first <n> values are passed in the standard argument registers. The Old-Cont
-register holds the Start of the values block and SP points to the End.
-
-
-{small, fast} call (function arg0 ... arg<n>) "nargs" => value
-{small, fast} call-named (arg0 ... arg<n>) "nargs" "name" => value
- Call-Closure calls Function with the specified register arguments,
- returning the first value as the result. "nargs" is the total number of
- arguments passed. Only the register arguments actually passed should be
- specified as operands.
-
- Call-Named is similar, but calls a global function specified at compile
- time by "name".
-
-{small, fast} tail-call (function pc arg0 ... arg<n>) "nargs"
-{small, fast} tail-call-named (pc arg0 ... arg<n>) "nargs" "name"
- Similar to the standard call VOPs, but passes PC as the return PC, rather
- than returning to the call site. These VOPs have no results since they
- don't return.
-
-{small, fast} multiple-call (function arg0 ... arg<n>) "nargs"
- => start end val0 ... val<n>
-{small, fast} multiple-call-named (arg0 ... arg<n>) "nargs" "name"
- => start end val0 ... val<n>
- These VOPs are similar to the standard call VOPs, but allow any number of
- values to be received by returning all the value passing registers as
- results. A specific number of values may be received by using
- Default-Values.
-
-call-unknown (function count arg0 ... arg<n>) => start end val0 ... val<n>
-tail-call-unknown (function pc count arg0 ... arg<n>)
- Call a function with an unknown number of arguments. Used for apply and
- hairy multiple-value-call.
-
-Function-Entry () "function" => env return-pc old-cont arg*
- This marks the place where we jump into a component for an external
- entry point. It represents whatever magic is necessary to do argument
- count checking and dispatching. The external entry points for each
- argument count will be successors of the entry-vector block (might be in
- the same block if only one?)
-
- Function-Entry also represents argument passing by specifying the actual
- external passing locations as results, thus marking the beginning of their
- lifetimes. All passing locations actually used by any entry point are
- specified as Args, including stack arguments.
- {\#\#\# Do we really need this? If we do, then we probably also need similar
- entry markers for local functions. The lifetimes don't really need to be
- explicitly bounded, since an entry point is effectively "the end of the
- world."}
-
-\f
-\section(Returning from a Function Call)
-\label(Return)
-\index(Return)
-
-
-return (return-pc value)
-multiple-return (return-pc start end val0 ... val<n>)
- Return Value from the current function, jumping back to the location
- specified by Return-PC. {Perhaps allow to return any fixed, known number
- of values.}
-
- Multiple-Return is similar, but allows an arbitrary number of values to be
- returned. End - Start is the total number of values returned. Start
- points to the beginning of the block of return values, but the first <n>
- values val0 ... val<n> are actually returned in registers.
-
-default-values (start end val0 ... val<n>) => val0 ... val<j>
- This VOP is used when we want to receive exactly J values. If fewer than J
- values were supplied, then missing values are defaulted to NIL. As a
- side-effect, this VOP pops off any returned stack values.
-
-\f
-\section{Saving and Restoring Registers}
-
-We use a caller-saves convention. The caller explicitly emits saving and
-restoring code. Tail-recursive calls don't need
-any register saving since we never come back.
-
-
-\f
-\chapter{Non-local exits}
-
-\f
-\subsection{Unwind Blocks}
-\index{Catch}
-\index{Catch frames}
-
-There is one aspect of the control stack format that is fixed, and which
-concerns us at this level. This is the format of the "frames" which mark the
-destination of non-local exits, such as for BLOCK and CATCH. These frames are
-collectively known as unwind blocks. The basic unwind block is used for
-lexical exists such as BLOCK, and for UNWIND-PROTECT. Its format is the
-following:
-\begin{verbatim}
-0 Pointer to current unwind-protect.
-1 Control stack context to restore on entry.
-2 PC to enter at.
-\end{verbatim}
-
-The unwind block for CATCH is identical except for additional cells
-containing the catch tag and previous catch.
-\begin{verbatim}
-0 Pointer to current unwind-protect.
-1 Control stack context to restore on entry.
-2 PC to enter at.
-3 Catch tag.
-4 Previous catch.
-\end{verbatim}
-
-The conventions used to manipulate unwind blocks are described in chapter
-\ref{Control-Conventions}.
-
-
-\f
-\section{Non-Local Exits}
-\label{Catch}
-\index{Catch}
-\index{Throw}
-\index{Unwinding}
-\index{Unwind-Protect}
-\index{Non-Local Exits}
-
-In the normal flow of control, each function that is called executes until it
-reaches a return point; under these conditions no special effort is needed to
-restore the environment as long as each function undoes any change that it
-makes to the dynamic state before it returns. When we make a non-local
-transfer, we skip a potentially arbitrary collection of these cleanup actions.
-Since we cannot in general know what changes have been made to the dynamic
-environment below us on the stack, we must restore a snapshot of the dynamic
-environment at the re-entry point.
-
-We represent the closed continuation by the pointer to the unwind-block for the
-reentry point. At the exit point, we just pass this stack pointer to the
-Unwind VOP, which deals with processing any unwind-protects. When Unwind is
-done, it grabs the re-entry PC out of the location at the stack pointer and
-jumps in.
-
-Catch and Unwind-Protect work in pretty much the same way. We make a stack TN
-to hold the catch frame or whatever, allocate TNs in them to represent the
-slots, and then initialize them. The frame can be explicitly linked in by TN
-manipulations, since the active catch and whatnot are represented by TNs.
-Since allocation of the frame is decoupled from linking and unlinking, some of
-this stuff could be moved out of loops. We will need a VOP for loading the PC
-for an arbitrary continuation so that we can set up the reentry PC. This can
-be done using the Call VOP. Using a call instruction is probably a good way to
-get a PC on most architectures anyway.
-
-These TNs are allocated by Pack like any others; we use special alloc and
-dealloc VOPs to delimit the aggregate lifetimes.
-
-In the non-local case, the the Block, Catch and Unwind-Protect special forms
-are implemented using unwind blocks. The unwind blocks are built by move
-operations emitted inline by the compiler. The compiler adds and removes
-catches and unwind protects by explicit moves to the locations that hold the
-current catch and unwind protect blocks. The entry PC is loaded using the Call
-VOP.
-
-The Unwind miscop is the basis non-local exits. It takes the address of an
-unwind block and processes unwind-protects until the current unwind-protect is
-the one recorded in the unwind block, then jumps in at the entry in the unwind
-block. The entry for the unwind block is responsible for restoring any state
-other than the current unwind-protect.
-
-Unwind is used directly to implement non-local Return-From. The address of the
-unwind block is stored in a closure variable.
-
-Catch just does a scan up the chain of Catch blocks, starting at the current
-catch. When it finds the right one, it calls unwind on it.
-
-Unwind-protects are represented by unwind blocks linked into the current
-unwind-protect chain. The cleanup code is entered just like any other any
-other unwind entry. As before, the entry is responsible for establishing the
-correct dynamic environment for the cleanup code. The target unwind block is
-passed in some non-argument register. When the cleanup code is done, it
-just calls Unwind with the block passed in. The cleanup code must be careful
-not to trash the argument registers or CS, since there may be multiple values
-lurking out there.
-
-With Catch/Throw, we always use the variable values return value passing convention,
-since we don't know how many values the catch wants. With Block/Return-From,
-we can do whatever we want, since the returner and receiver know each other.
-
-If a Block or Catch receives stack values, it must call a VOP that BLT's the
-values down the stack, squeezing out any intermediate crud.
-
-
-unwind (context)
-throw (tag)
- Unwind does a non-local exit, unwinding to the place indicated by Context.
- Context is a pointer to a block of storage allocated on the control stack,
- containing the entry PC, current environment and current unwind-protect.
- We scan up the stack, processing unwind-protects until we reach the entry
- point. The values being returned are passed in the standard locations.
- Throw is similar, but does a dynamic lookup for the Tag to determine what
- context to unwind to.
-
;;; four numeric fields, is used for versions which aren't released
;;; but correspond only to CVS tags or snapshots.
-"0.pre7.6"
+"0.pre7.7"
projects / sbcl.git / commitdiff