0.9.12.10:

[sbcl.git] / TODO

diff --git a/TODO b/TODO
index 8aceeeb..0dde9c9 100644 (file)
--- a/TODO
+++ b/TODO
@@ -1,182 +1,193 @@
-    Accumulation of half-understood design decisions eventually
-    chokes a program as a water weed chokes a canal. By refactoring
-    you can ensure that your full understanding of how the program
-    should be designed is always reflected in the program. As a
-    water weed quickly spreads its tendrils, partially understood
-    design decisions quickly spread their effects throughout your
-    program. No one or two or even ten individual actions will be
-    enough to eradicate the problem.
-       -- Martin Fowler, _Refactoring: Improving the Design
-          of Existing Code_, p. 360 
-===============================================================================
-some things that I'd like to do in 0.6.x, in no particular order:
--------------------------------------------------------------------------------
-PROBLEM:
-           The batch-related command line options for SBCL don't work
-       properly.
-           A small part of making them work properly is making sure that
-       verbose GC messages end up piped to error output.
-           Make sure that when the system dies due to an unhandled error
-       in batch mode, the error is printed successfully, whether
-       FINISH-OUTPUT or an extra newline or whatever is required.
-           Make sure that make.sh dies gracefully when one of the SBCLs
-       it's running dies with an error.
-MUSING:
-           Actually, the ANSI *DEBUGGER-HOOK* variable might be a better
-       place to put the die-on-unhandled-error functionality.
-FIX:
-       ??
--------------------------------------------------------------------------------
-PROBLEM:
-           As long as I'm working on the batch-related command-line options,
-       it would be reasonable to add one more option to "do what I'd want",
-       testing standard input for TTY-ness and running in no-programmer
-       mode if so.
-FIX:
-       ?? Do it.
--------------------------------------------------------------------------------
-PROBLEM:
-           In order to make a well-behaved backtrace when a batch program
-       terminates abnormally, it should be limited in length.
-FIX:
-       ?? Add a *DEBUG-BACKTRACE-COUNT* variable, initially set to 64,
-         to provide a default for the COUNT argument to BACKTRACE.
--------------------------------------------------------------------------------
-PROBLEM:
-           I used CMU CL for years, and dozens of times I cursed the
-       inadequate breakpoint-based TRACE facility which doesn't work on
-       some functions, and I never realized that there's a wrapper-based
-       facility too until I was wading through the source code for SBCL.
-           Yes, I know I should have RTFM, but there is a lot of M..
-FIX:
-       ?? possibility 1: Add error-handling code in ntrace.lisp to
-         catch failure to set breakpoints and retry using 
-         wrapper-based tracing.
-       ?? possibility 2: Add error-handling code in ntrace.lisp to
-         catch failure to catch failure to set breakpoints and output
-         a message suggesting retrying with wrapper-based breakpoints
-       ?? possibility 3: Fix the breakpoint-based TRACE facility so that
-         it always works.
--------------------------------------------------------------------------------
-PROBLEM:
-           When cross-compiling host-byte-comp.lisp, I get bogus
-       warnings
-               caught STYLE-WARNING:
-                 undefined function: %%DEFCONSTANT
-               caught STYLE-WARNING:
-                 This function is undefined:
-                 %%DEFCONSTANT
-MUSING:
-           The best way to clean this up would be as a side-effect of
-       a larger cleanup, making all the %%DEFFOO stuff use EVAL-WHEN
-       instead of IR1 magic.
-           There's probably some way to do it with a quick local hack too.
-FIX:
-       ??
--------------------------------------------------------------------------------
-PROBLEM:
-           My system of parallel build directories doesn't seem to add value.
-FIX:
-       ?? Replace it with a system where fasl output files live in the 
-         same directories as the sources and have names a la
-         "foo.fasl-from-host and "foo.fasl-from-xc".
--------------------------------------------------------------------------------
-PROBLEM:
-           It might be good to use the syntax (DEBUGGER-SPECIAL *PRINT-LEVEL*)
-       etc. to control the in-the-debug-context special variables. Then we 
-       wouldn't have to pick and choose which variables we shadow in the
-       debugger.
-           The shadowing values could also be made persistent between
-       debugger invocations, so that entering the debugger, doing
-       (SETF *PRINT-LEVEL* 2), and exiting the debugger would leave
-       (DEBUGGER-SPECIAL *PRINT-LEVEL*) set to 2, and upon reentry to the
-       debugger, *PRINT-LEVEL* would be set back to 2.
-FIX:
-       ??
--------------------------------------------------------------------------------
-PROBLEM:
-           The :SB-TEST target feature should do something.
-FIX:
-       ??
--------------------------------------------------------------------------------
-PROBLEM:
-           I still haven't cleaned up the cut-and-paste programming in 
-               * DEF-BOOLEAN-ATTRIBUTE, DELETEF-IN, and PUSH-IN
-               * SB!SYS:DEF!MACRO ASSEMBLE and SB!XC:DEFMACRO ASSEMBLE
-FIX:
-       ??
--------------------------------------------------------------------------------
-PROBLEM:
-           We be able to get rid of the IR1 interpreter, which would
-       not only get rid of all the code in *eval*.lisp, but also allow us to
-       reduce the number of special cases elsewhere in the system. (Try
-       grepping for 'interpret' sometime.:-) Making this usable might
-       require cleaning up %DEFSTRUCT, %DEFUN, etc. to use EVAL-WHEN
-       instead of IR1 transform magic, which would be a good
-       thing in itself, but might be a fair amount of work.)
-FIX:
-       ?? Delete, delete, delete.
--------------------------------------------------------------------------------
-PROBLEM:
-           The hashing code is new and should be tested.
-FIX:
-       ?? Enable the existing test code.
--------------------------------------------------------------------------------
-PROBLEM:
-           My ad hoc system of revision control is looking pretty clunky,
-       and I've pretty much stopped doing stuff to confuse CVS (like moving
-       directories around).
-FIX:
-       ?? Check into CVS. 
-       ?? Make sure that the tags in FILE-COMMENTs expand correctly.
-       ?? See about automatically propagating version information 
-          from CVS into the runtime.c banner message and the 
-          LISP-IMPLEMENTATION-VERSION string.
-===============================================================================
-other known issues with no particular target date:
-
-user manual including, at a minimum, updated versions of the
-CMU CL user manual information on the compiler and the alien
-interface
+planned incompatible changes in 0.8.x:
+  * (not done yet, but planned:) When the profiling interface settles
+    down, it might impact TRACE. They both encapsulate functions, and
+    it's not clear yet how e.g. UNPROFILE will interact with TRACE
+    and UNTRACE. (This shouldn't matter, though, unless you are using
+    profiling. If you never profile anything, TRACE should continue to
+    behave as before.)
+  * (not done yet, but planned:) Inlining can now be controlled the
+    ANSI way, without MAYBE-INLINE, since the idiom
+       (DECLAIM (INLINE FOO))
+       (DEFUN FOO (..) ..)
+       (DECLAIM (NOTINLINE FOO))
+       (DEFUN BAR (..) (FOO ..))
+       (DEFUN BLETCH (..) (DECLARE (INLINE FOO)) (FOO ..))
+    now does what ANSI says it should. The CMU-CL-style 
+    SB-EXT:MAYBE-INLINE declaration is now deprecated and ignored.

 
 

-bugs listed on the man page
+for early 0.8.x:

 
 

-more regression tests
+* test file reworking
+       ** ports with less than 256Mb of heap (sparc, ppc and mips)
+               now don't fail bit-vector.impure-cload.lisp
+* faster bootstrapping (both make.sh and slam.sh)
+       ** added mechanisms for automatically finding dead code, and
+               used them to remove dead code
+       ** moved stuff from warm init into cold init where possible
+               (so that slam.sh will run faster and also just because
+               ideally everything would be in cold init)
+       ** profiled and tweaked
+* fixed (TRACE :REPORT PROFILE ...) interface to profiling
+* more EVAL/EVAL-WHEN/%COMPILE/DEFUN/DEFSTRUCT cleanup:
+       ** made %COMPILE understand magicality of DEFUN FOO
+               w.r.t. e.g. preexisting inlineness of FOO
+       ** used %COMPILE where COMPILE-TOP-LEVEL used to be used
+       ** removed now-redundant COMPILE-TOP-LEVEL and 
+               FUNCTIONAL-KIND=:TOP-LEVEL stuff from the compiler
+       ** (ideally, but perhaps too hard, given what I've discovered
+               about the godawful internals of function debug names):
+               made FUNCTION-NAME logic work on closures, so that
+               various public functions like CL:PACKAGEP which
+               are now implemented as closures (because
+               they're structure slot accessors) won't be so
+               nasty in the debugger
+* outstanding embarrassments
+       ** :IGNORE-ERRORS-P cruft in stems-and-flags.lisp-expr. (It's
+               reasonable to support this as a crutch when initially
+               bootstrapping from balky xc hosts with their own
+               idiosyncratic ideas of what merits FAILURE-P, but it's
+               embarrassing to have to use it when bootstrapping 
+               under SBCL!),
+* fixups now feasible because of pre7 changes
+       ** ANSIfied DECLAIM INLINE stuff (deprecating MAYBE-INLINE,
+               including e.g. on the man page)
+       ** (maybe) allow INLINE of a recursive function, so that the
+               body is inlined once
+* miscellaneous simple refactoring
+       * belated renaming:
+               ** renamed %PRIMITIVE to %VOP
+               ** A few hundred things named FN and FCN should be
+                       named FUN (but maybe not while drichards is
+                       working on a Windows port).
+       * These days ANSI C has inline functions, so..
+               ** redid many cpp macros as inline functions: 
+                       HeaderValue, Pointerp, CEILING, ALIGNED_SIZE,
+                       GET_FREE_POINTER, SET_FREE_POINTER,
+                       GET_GC_TRIGGER, SET_GC_TRIGGER, GetBSP, SetBSP,
+                       os_trunc_foo(), os_round_up_foo()
+               ** removed various avoid-evaluating-C-macro-arg-twice
+                       cruft
+* Either get rid of or at least rework the fdefinition/encapsulation
+       system so that (SYMBOL-FUNCTION 'FOO) is identically equal to
+       (FDEFINITION 'FOO).
+* Make the system sources understandable to the system, so that
+       searching for sources doesn't error out quite so often
+       (e.g. in error handlers)
+       ** provided a suitable readtable for reading in the source
+               files when necessary, and a mechanism for activating
+               this readtable rather than the standard one.
+* Some work on conditions emitted by the system
+       ** eliminated COMPILER-WARN and COMPILER-STYLE-WARN, which
+               were simply limited versions of WARN and STYLE-WARN.
+       ** made STYLE-WARN parallel WARN more closely (by accepting
+               a condition type, which should be a subtype of
+               STYLE-WARNING, and initargs, as well as a format
+               string and format arguments for SIMPLE-STYLE-WARNING.
+               (WARN can also be used to signal STYLE-WARNINGs, but
+               STYLE-WARN helps to document the code)
+       ** eliminated use of INHIBIT-WARNINGS by code emitted by the
+               system from user code.
+       ** caused use of INHIBIT-WARNINGS to signal a STYLE-WARNING.
+       ** eliminated use of INHIBIT-WARNINGS within the system
+       ** deprecated INHIBIT-WARNINGS, causing its use to signal a
+               full WARNING.
+       ** began work on developing a class hierarchy of conditions
+               along semantic lines.
+       ** annotated conditions emitted by the system to have
+               references to documentation where applicable, so that
+               users can easily find an explanation for the
+               conditions they're seeing.

 
 

-various bugs fixed in CMUCL since this code was forked off of it
-ca. 19980801, since most of these haven't been fixed yet in SBCL
+=======================================================================
+for 0.9:

 
 

-byte compilation of appropriate parts of the system, so that the
-system core isn't so big
+[ note: much of the below refers to preparation for merging SB-PCL:FOO
+  and CL:FOO.  However, it turned out to be surprisingly
+  straightforward to do this notional end goal without doing many of
+  the preparatory operations.  That doesn't mean that plenty of the
+  goals below aren't worthwhile, but the motivation is somewhat
+  different. ]

 
 

-uninterning needed-only-at-init-time stuff after init is complete,
-so that the system core isn't so big
+* refactored in preparation for moving CLOS into cold init and merging
+       SB-PCL:FOO with CL:FOO (for FOO=CLASS, FOO=CLASS-OF, etc.)
+       ** systematized support for MOP (more regression tests, maybe) 
+               to try to make sure things don't get mislaid in the 
+               upcoming CLOS restructuring
+       ** extracted type system (and maybe CLASSOIDs) from SB-KERNEL 
+               into new SB-TYPE package
+       ** reimplemented GENERIC-FUNCTION as a primitive object (or
+               maybe made SB-MOP:FUNCALLABLE-STANDARD-OBJECT the
+               primitive object, and then let GENERIC-FUNCTIONs
+               inherit from that) instead of structures with
+               :ALTERNATE-METACLASS and funcallableness. Now
+               FUNCALLABLE-INSTANCE can go away. (And now the new
+               funcallable primitive objects need to go into
+               collections like *FUN-HEADER-WIDETAGS* where
+               FUNCALLABLE-INSTANCE objects used to be.)
+       ** reimplemented CONDITIONs as primitive objects instead of 
+               structures with :ALTERNATE-METACLASS. Now (between
+               this and the change to GENERIC-FUNCTIONs)
+               DEFSTRUCT :ALTERNATE-METACLASS can go away.
+       ** (maybe) Now INSTANCE_POINTER_LOWTAG can become just
+               STRUCTURE_POINTER_LOWTAG, and the concept of
+               SB-KERNEL:INSTANCE (including INSTANCEP, 
+               (SPECIFIER-TYPE 'INSTANCE), etc.) can go away.
+* moved CLOS into cold init, in order to allow CLOS to be used in the
+       implementation of the core system (e.g. the type system and the
+       compiler) and in order to support merger of CL:CLASS with 
+       SB-PCL:CLASS
+* (maybe) eliminated warm init altogether in favor of cold init
+* (maybe, especially if warm init can be eliminated) rationalized
+       the build process, fixing miscellaneous pre-0.5.0 stuff that's
+       transparently not the right thing
+       ** removed separate build directories, now just building in 
+               place with .sbclcoldfasl extensions
+* (maybe) more refactoring in preparation for merging SB-PCL:FOO
+       into CL:FOO: reimplemented type system OO dispatch
+       (!DEFINE-TYPE-METHOD, etc.) in terms of CLOS OO dispatch
+* added some automatic tests for basic binary compatibility, in hopes
+       that it might be practical to maintain binary compatibility
+       between minor maintenance releases on the stable branch (but no
+       promises, sorry, since I've never tried to do this before, and 
+       have no idea how much of a pain this'll be)
+========================================================================
+for 1.0 (fixes of lower priority which I'd nonetheless be embarrassed
+to leave unfixed in 1.0):
+* all too many BUGS entries and FIXMEs
+=======================================================================
+other priorities, no particular time:

 
 

-Search for unused external symbols (ones which are not bound, fbound,
-types, or whatever, and also have no other uses as e.g. flags) and
-delete them. This should make the system core a little smaller, but
-is mostly useful just to make the source code smaller and simpler.
-
-The eventual plan is for SBCL to bootstrap itself in two phases. In
-the first phase, the cross-compilation host is any old ANSI Common
-Lisp (not necessarily SBCL) and the cross-compiler won't handle some
-optimizations because the code it uses to implement them is not
-portable. In the second phase, the cross-compilation host will be
-required to be a compatible version of SBCL, and the cross-compiler
-will take advantage of that to implement all optimizations. The
-current version of SBCL only knows how to do the first of those two
-phases, with a fully-portable cross-compiler, so some optimizations
-are not done. Probably the most important consequence of this is that
-because the fully-portable cross-compiler isn't very smart about
-dealing with immediate values which are of specialized array type
-(e.g. (SIMPLE-ARRAY (UNSIGNED-BYTE 4) 1)) the system sometimes has to
-use unnecessarily-general array types internally.
+* bug fixes, especially really annoying bugs (ANSI or not) and any
+       ANSI bugs (i.e. not just bugs in extras like the debugger or
+       "declarations are assertions", but violations of the standard)
+* better communication with the outside world (scratching WHN's
+       personal itch): I don't want socket-level stuff so much as I
+       want RPC-level or higher (CORBA?) interfaces and (possibly
+       through RPC or CORBA) GUI support
+* Especially when ldb is not compiled in, the default "assertion failed"
+       behaviour in many parts of the runtime is unfriendly.  It may
+       be appropriate to look at some of these and see if they can be 
+       handled in some less abrupt way than aborting
+=======================================================================
+important but out of scope (for WHN, anyway: Patches from other people
+are still welcome!) until after 1.0:
+       * DYNAMIC-EXTENT
+       * sadly deteriorated support for ANSI-style block compilation
+               (static linking of DEFUNs within a single file or 
+               WITH-COMPILATION-UNIT)
+       * various GC issues (exuberant cut-and-paste coding,
+               possibly dangerously over-conservative handling
+               of neighbors of function objects, general GC efficiency)
+       * package issues other than SB!TYPE, SB!MOP, and dead exported
+               symbols
+       * Any systematic effort to fix compiler consistency checks is
+               out of scope. (However, it still might be possible to
+               determine that some or all of them are hopelessly stale
+               and delete them.)
+=======================================================================
+other known issues with no particular target date:

 
 

-adding new FOPs to provide something like CMU CL's FOP-SYMBOL-SAVE and
-FOP-SMALL-SYMBOL-SAVE functionality, so that fasl files will be more
-compact. (FOP-SYMBOL-SAVE used *PACKAGE*, which was concise but allowed
-obscure bugs. Something like FOP-LAST-PACKAGE-SYMBOL-SAVE could have
-much of the same conciseness advantage without the bugs.)
+bugs listed on the man page

 
 hundreds of FIXME notes in the sources from WHN
 
 
 hundreds of FIXME notes in the sources from WHN
 

@@ -187,3 +198,24 @@ or probably also other codes that I haven't noticed or have forgotten.
 (Things marked as KLUDGE are in general things which are ugly or
 confusing, but that, for whatever reason, may stay that way
 indefinitely.)
 (Things marked as KLUDGE are in general things which are ugly or
 confusing, but that, for whatever reason, may stay that way
 indefinitely.)

+=======================================================================
+"There's nothing an agnostic can't do as long as he doesn't know
+whether he believes in anything or not."
+  -- Monty Python.
+
+"God grant me serenity to accept the code I cannot change, courage to
+change the code I can, and wisdom to know the difference."
+  -- Erik Naggum
+
+"Accumulation of half-understood design decisions eventually chokes a
+program as a water weed chokes a canal. By refactoring you can ensure
+that your full understanding of how the program should be designed is
+always reflected in the program. As a water weed quickly spreads its
+tendrils, partially understood design decisions quickly spread their
+effects throughout your program. No one or two or even ten individual
+actions will be enough to eradicate the problem."
+  -- Martin Fowler, in _Refactoring: Improving the Design of Existing
+     Code_, p. 360 
+
+"I wish I didn't know now what I didn't know then."
+  -- Bob Seger