2 @comment node-name, next, previous, up
5 SBCL supports a fairly low-level threading interface that maps onto
6 the host operating system's concept of threads or lightweight
7 processes. This means that threads may take advantage of hardware
8 multiprocessing on machines that have more than one CPU, but it does
9 not allow Lisp control of the scheduler. This is found in the
12 Threads are part of the default build on x86[-64] Linux only.
14 They are also experimentally supported on: x86[-64] Darwin (Mac OS X),
15 x86[-64] FreeBSD, x86 SunOS (Solaris), and PPC Linux. On these platforms
16 threads must be explicitly enabled at build-time, see @file{INSTALL} for
25 * Waitqueue/condition variables::
27 * Sessions/Debugging::
29 * Implementation (Linux x86/x86-64)::
32 @node Threading basics
33 @comment node-name, next, previous, up
34 @section Threading basics
37 (make-thread (lambda () (write-line "Hello, world")))
40 @subsection Thread Objects
42 @include struct-sb-thread-thread.texinfo
43 @include var-sb-thread-star-current-thread-star.texinfo
44 @include fun-sb-thread-list-all-threads.texinfo
45 @include fun-sb-thread-thread-alive-p.texinfo
46 @include fun-sb-thread-thread-name.texinfo
47 @include fun-sb-thread-main-thread-p.texinfo
48 @include fun-sb-thread-main-thread.texinfo
50 @subsection Making, Returning From, Joining, and Yielding Threads
52 @include fun-sb-thread-make-thread.texinfo
53 @include macro-sb-thread-return-from-thread.texinfo
54 @include fun-sb-thread-abort-thread.texinfo
55 @include fun-sb-thread-join-thread.texinfo
56 @include fun-sb-thread-thread-yield.texinfo
58 @subsection Asynchronous Operations
60 @include fun-sb-thread-interrupt-thread.texinfo
61 @include fun-sb-thread-terminate-thread.texinfo
63 @subsection Miscellaneous Operations
65 @include fun-sb-thread-symbol-value-in-thread.texinfo
67 @subsection Error Conditions
69 @include condition-sb-thread-thread-error.texinfo
70 @include fun-sb-thread-thread-error-thread.texinfo
72 @c @include condition-sb-thread-symbol-value-in-thread-error.texinfo
73 @include condition-sb-thread-interrupt-thread-error.texinfo
74 @include condition-sb-thread-join-thread-error.texinfo
76 @node Special Variables
77 @comment node-name, next, previous, up
78 @section Special Variables
80 The interaction of special variables with multiple threads is mostly
81 as one would expect, with behaviour very similar to other
86 global special values are visible across all threads;
88 bindings (e.g. using LET) are local to the thread;
90 threads do not inherit dynamic bindings from the parent thread
93 The last point means that
98 (sb-thread:make-thread (lambda () (print *x*))))
101 prints @code{0} and not @code{1} as of 0.9.6.
103 @node Atomic Operations
104 @comment node-name, next, previous, up
105 @section Atomic Operations
107 Following atomic operations are particularly useful for implementing
110 @include macro-sb-ext-atomic-decf.texinfo
111 @include macro-sb-ext-atomic-incf.texinfo
112 @include macro-sb-ext-atomic-pop.texinfo
113 @include macro-sb-ext-atomic-push.texinfo
114 @include macro-sb-ext-atomic-update.texinfo
115 @include macro-sb-ext-compare-and-swap.texinfo
117 @unnumberedsubsec CAS Protocol
119 Our @code{compare-and-swap} is user-extensible using a protocol
120 similar to @code{setf}, allowing users to add CAS support to new
121 places via eg. @code{defcas}.
123 At the same time, new atomic operations can be built on top of CAS
124 using @code{get-cas-expansion}. See @code{atomic-update},
125 @code{atomic-push}, and €@code{atomic-pop} for example of how to do
128 @include macro-sb-ext-cas.texinfo
129 @include macro-sb-ext-define-cas-expander.texinfo
130 @include macro-sb-ext-defcas.texinfo
131 @include fun-sb-ext-get-cas-expansion.texinfo
134 @comment node-name, next, previous, up
135 @section Mutex Support
137 Mutexes are used for controlling access to a shared resource. One
138 thread is allowed to hold the mutex, others which attempt to take it
139 will be made to wait until it's free. Threads are woken in the order
140 that they go to sleep.
142 There isn't a timeout on mutex acquisition, but the usual WITH-TIMEOUT
143 macro (which throws a TIMEOUT condition after n seconds) can be used
144 if you want a bounded wait.
147 (defpackage :demo (:use "CL" "SB-THREAD" "SB-EXT"))
151 (defvar *a-mutex* (make-mutex :name "my lock"))
154 (format t "Thread ~A running ~%" *current-thread*)
155 (with-mutex (*a-mutex*)
156 (format t "Thread ~A got the lock~%" *current-thread*)
158 (format t "Thread ~A dropped lock, dying now~%" *current-thread*))
160 (make-thread #'thread-fn)
161 (make-thread #'thread-fn)
164 @include struct-sb-thread-mutex.texinfo
165 @include fun-sb-thread-make-mutex.texinfo
166 @include fun-sb-thread-mutex-name.texinfo
167 @include fun-sb-thread-mutex-value.texinfo
168 @include fun-sb-thread-grab-mutex.texinfo
169 @include fun-sb-thread-release-mutex.texinfo
170 @include macro-sb-thread-with-mutex.texinfo
171 @include macro-sb-thread-with-recursive-lock.texinfo
172 @include fun-sb-thread-get-mutex.texinfo
175 @comment node-name, next, previous, up
178 Semaphores are among other things useful for keeping track of a
179 countable resource, eg. messages in a queue, and sleep when the
180 resource is exhausted.
182 @include struct-sb-thread-semaphore.texinfo
183 @include fun-sb-thread-make-semaphore.texinfo
184 @include fun-sb-thread-signal-semaphore.texinfo
185 @include fun-sb-thread-wait-on-semaphore.texinfo
186 @include fun-sb-thread-try-semaphore.texinfo
187 @include fun-sb-thread-semaphore-count.texinfo
188 @include fun-sb-thread-semaphore-name.texinfo
190 @include struct-sb-thread-semaphore-notification.texinfo
191 @include fun-sb-thread-make-semaphore-notification.texinfo
192 @include fun-sb-thread-semaphore-notification-status.texinfo
193 @include fun-sb-thread-clear-semaphore-notification.texinfo
195 @node Waitqueue/condition variables
196 @comment node-name, next, previous, up
197 @section Waitqueue/condition variables
199 These are based on the POSIX condition variable design, hence the
200 annoyingly CL-conflicting name. For use when you want to check a
201 condition and sleep until it's true. For example: you have a shared
202 queue, a writer process checking ``queue is empty'' and one or more
203 readers that need to know when ``queue is not empty''. It sounds
204 simple, but is astonishingly easy to deadlock if another process runs
205 when you weren't expecting it to.
207 There are three components:
211 the condition itself (not represented in code)
214 the condition variable (a.k.a waitqueue) which proxies for it
217 a lock to hold while testing the condition
220 Important stuff to be aware of:
224 when calling condition-wait, you must hold the mutex. condition-wait
225 will drop the mutex while it waits, and obtain it again before
226 returning for whatever reason;
229 likewise, you must be holding the mutex around calls to
233 a process may return from condition-wait in several circumstances: it
234 is not guaranteed that the underlying condition has become true. You
235 must check that the resource is ready for whatever you want to do to
241 (defvar *buffer-queue* (make-waitqueue))
242 (defvar *buffer-lock* (make-mutex :name "buffer lock"))
244 (defvar *buffer* (list nil))
247 (with-mutex (*buffer-lock*)
249 (condition-wait *buffer-queue* *buffer-lock*)
251 (unless *buffer* (return))
252 (let ((head (car *buffer*)))
253 (setf *buffer* (cdr *buffer*))
254 (format t "reader ~A woke, read ~A~%"
255 *current-thread* head))))))
260 (with-mutex (*buffer-lock*)
263 (+ (char-code #\A) (random 26)))))))
264 (setf *buffer* (cons el *buffer*)))
265 (condition-notify *buffer-queue*))))
267 (make-thread #'writer)
268 (make-thread #'reader)
269 (make-thread #'reader)
272 @include struct-sb-thread-waitqueue.texinfo
273 @include fun-sb-thread-make-waitqueue.texinfo
274 @include fun-sb-thread-waitqueue-name.texinfo
275 @include fun-sb-thread-condition-wait.texinfo
276 @include fun-sb-thread-condition-notify.texinfo
277 @include fun-sb-thread-condition-broadcast.texinfo
280 @comment node-name, next, previous, up
283 These are based on the Linux kernel barrier design, which is in turn
284 based on the Alpha CPU memory model. They are presently implemented for
285 x86, x86-64, and PPC systems, and behave as compiler barriers on all
288 In addition to explicit use of the @code{sb-thread:barrier} macro, the
289 following functions and macros also serve as @code{:memory} barriers:
293 @code{sb-ext:atomic-decf}, @code{sb-ext:atomic-incf}, @code{sb-ext:atomic-push},
294 and @code{sb-ext:atomic-pop}.
296 @code{sb-ext:compare-and-swap}.
298 @code{sb-thread:get-mutex}, @code{sb-thread:release-mutex},
299 @code{sb-thread:with-mutex} and @code{sb-thread:with-recursive-lock}.
301 @code{sb-thread:signal-semaphore}, @code{sb-thread:try-semaphore} and
302 @code{sb-thread:wait-on-semaphore}.
304 @code{sb-thread:condition-wait}, @code{sb-thread:condition-notify} and
305 @code{sb-thread:condition-broadcast}.
308 @include macro-sb-thread-barrier.texinfo
310 @node Sessions/Debugging
311 @comment node-name, next, previous, up
312 @section Sessions/Debugging
314 If the user has multiple views onto the same Lisp image (for example,
315 using multiple terminals, or a windowing system, or network access)
316 they are typically set up as multiple @dfn{sessions} such that each
317 view has its own collection of foreground/background/stopped threads.
318 A thread which wishes to create a new session can use
319 @code{sb-thread:with-new-session} to remove itself from the current
320 session (which it shares with its parent and siblings) and create a
322 # See also @code{sb-thread:make-listener-thread}.
324 Within a single session, threads arbitrate between themselves for the
325 user's attention. A thread may be in one of three notional states:
326 foreground, background, or stopped. When a background process
327 attempts to print a repl prompt or to enter the debugger, it will stop
328 and print a message saying that it has stopped. The user at his
329 leisure may switch to that thread to find out what it needs. If a
330 background thread enters the debugger, selecting any restart will put
331 it back into the background before it resumes. Arbitration for the
332 input stream is managed by calls to @code{sb-thread:get-foreground}
333 (which may block) and @code{sb-thread:release-foreground}.
335 @node Foreign threads
336 @comment node-name, next, previous, up
337 @section Foreign threads
339 Direct calls to @code{pthread_create} (instead of @code{MAKE-THREAD})
340 create threads that SBCL is not aware of, these are called foreign
341 threads. Currently, it is not possible to run Lisp code in such
342 threads. This means that the Lisp side signal handlers cannot work.
343 The best solution is to start foreign threads with signals blocked,
344 but since third party libraries may create threads, it is not always
345 feasible to do so. As a workaround, upon receiving a signal in a
346 foreign thread, SBCL changes the thread's sigmask to block all signals
347 that it wants to handle and resends the signal to the current process
348 which should land in a thread that does not block it, that is, a Lisp
351 The resignalling trick cannot work for synchronously triggered signals
352 (SIGSEGV and co), take care not to trigger any. Resignalling for
353 synchronously triggered signals in foreign threads is subject to
354 @code{--lose-on-corruption}, see @ref{Runtime Options}.
356 @node Implementation (Linux x86/x86-64)
357 @comment node-name, next, previous, up
358 @section Implementation (Linux x86/x86-64)
360 Threading is implemented using pthreads and some Linux specific bits
363 On x86 the per-thread local bindings for special variables is achieved
364 using the %fs segment register to point to a per-thread storage area.
365 This may cause interesting results if you link to foreign code that
366 expects threading or creates new threads, and the thread library in
367 question uses %fs in an incompatible way. On x86-64 the r12 register
370 Queues require the @code{sys_futex()} system call to be available:
371 this is the reason for the NPTL requirement. We test at runtime that
372 this system call exists.
374 Garbage collection is done with the existing Conservative Generational
375 GC. Allocation is done in small (typically 8k) regions: each thread
376 has its own region so this involves no stopping. However, when a
377 region fills, a lock must be obtained while another is allocated, and
378 when a collection is required, all processes are stopped. This is
379 achieved by sending them signals, which may make for interesting
380 behaviour if they are interrupted in system calls. The streams
381 interface is believed to handle the required system call restarting
382 correctly, but this may be a consideration when making other blocking
383 calls e.g. from foreign library code.
385 Large amounts of the SBCL library have not been inspected for
386 thread-safety. Some of the obviously unsafe areas have large locks
387 around them, so compilation and fasl loading, for example, cannot be
388 parallelized. Work is ongoing in this area.
390 A new thread by default is created in the same POSIX process group and
391 session as the thread it was created by. This has an impact on
392 keyboard interrupt handling: pressing your terminal's intr key
393 (typically @kbd{Control-C}) will interrupt all processes in the
394 foreground process group, including Lisp threads that SBCL considers
395 to be notionally `background'. This is undesirable, so background
396 threads are set to ignore the SIGINT signal.
398 @code{sb-thread:make-listener-thread} in addition to creating a new
399 Lisp session makes a new POSIX session, so that pressing
400 @kbd{Control-C} in one window will not interrupt another listener -
401 this has been found to be embarrassing.