From: Olof-Joachim Frahm Date: Thu, 27 Nov 2014 21:49:47 +0000 (+0100) Subject: Add unix lisp post again. X-Git-Url: http://repo.macrolet.net/gitweb/?p=blog.git;a=commitdiff_plain;h=437bacef342364f5f4a8bb620a2e887e2789a7a7 Add unix lisp post again. --- diff --git a/unix-lisp.post b/unix-lisp.post new file mode 100644 index 0000000..232df27 --- /dev/null +++ b/unix-lisp.post @@ -0,0 +1,81 @@ +;;;;; +title: Lisp layered on Unix +tags: lisp,unix,emacs +date: 2014-11-27 21:21:10 +format: md +;;;;; + +[This blog post][heart] on LispCast is a pretty good start to get thinking +about the intricacies of the interaction between Lisp (Machine) ideas and the +current Unix environment. Of course that includes plan9 on the one side and +Emacs on the other. + +[heart]: http://www.lispcast.com/the-heart-of-unix (The Heart of Unix) + +# Lisp shell + +There is [scsh](???), but it's not really what I'm looking for. Using emacs as +login shell (with the `eshell` package) comes closest to it regarding both +with existing commands and integration of Lisp-based ones. However, while +pipes work as expected with `eshell`, data is still passed around as +(formatted) text. There doesn't seem to be an easy way to pass around +in-memory objects, at least while staying in Emacs itself. That would of +course mean to reimplement some (larger?) parts of that system. + +This all ties in to the idea that unstructured text isn't the best idea to +represent data between processes. Even though Unix pipes are extremely useful, +the ecosystem of shell and C conventions means that the obvious way isn't +completely correct, meaning that there are edge cases to consider. The best is +something as innocent as `ls | wc -l`, which will break, depending on the shell +settings, with some (unlikely) characters in filenames, i.e. newlines. + +# Common formats + +One of the problems is obviously that in order to pass around structured data, +i.e. objects, all participants have to understand their format. Passing +references won't work without OS support though. + +Instead of having unstructured streams, use streams of (data) objects. The +distinction here is Plain Old Objects (PODs) instead of objects with an +associated behaviour. + +Let's take a look at standard Unix command line tools (I'm using GNU Coreutils +here) in order to reproduce the behaviour and/or intent behind them: + +## Output of entire files + +The first command here is `cat`. Although GNU `cat` includes additional +transformations, this command concatenates files. Similar to the description, +we can image a `CAT` to perform a similar operation on streams of objects. + +It doesn't make much sense to concatenate a HTML document and an MP3 file +(hence you won't do it in most cases anyway). However, since files are +unstructured, `cat` can work on them. + +# Registering functionality + +Although you can call commands individually on files, some of them form an +ad-hoc service interface already: The C compiler, along with the toolchain +forms one such interface, where you're required to use the same interface if +you want to seamlessly replace one part of the toolchain. + +Same goes for the Coreutils: As long as you honour the interface, programs can +be replaced with different implementations. + +# Interactive commands + +Emacs has a special form `interactive` to indicate whether a command can be +directly called via the command prompt. There is also special handling there +to accomodate the use of interactive arguments. This is something that can be +generalised to the OS. An example of where this already happens is the +`.mailcap` file and the `.desktop` files from desktop suites. + +# Threading and job control + +Unfortunately getting proper job control to work will be a bit of a problem in +any Lisp implementation, since the best way to implement the concurrent jobs is +using threads, which are not particularly suited for handling the multitude of +signals and other problems associated with them. Even without job control +pipelines implemented in Lisp require shared in-memory communication channels, +so something like (object-based) streams, mailboxes, or queues are necessary to +move IO between different threads. diff --git a/unix-lisp.post.unpublished b/unix-lisp.post.unpublished deleted file mode 100644 index 8bb702d..0000000 --- a/unix-lisp.post.unpublished +++ /dev/null @@ -1,81 +0,0 @@ -;;;;; -title: Lisp layered on Unix -tags: lisp,unix,emacs -date: 2014-11-27 21:31:10 -format: md -;;;;; - -[This blog post][heart] on LispCast is a pretty good start to get thinking -about the intricacies of the interaction between Lisp (Machine) ideas and the -current Unix environment. Of course that includes plan9 on the one side and -Emacs on the other. - -[heart]: http://www.lispcast.com/the-heart-of-unix (The Heart of Unix) - -# Lisp shell - -There is [scsh](???), but it's not really what I'm looking for. Using emacs as -login shell (with the `eshell` package) comes closest to it regarding both -with existing commands and integration of Lisp-based ones. However, while -pipes work as expected with `eshell`, data is still passed around as -(formatted) text. There doesn't seem to be an easy way to pass around -in-memory objects, at least while staying in Emacs itself. That would of -course mean to reimplement some (larger?) parts of that system. - -This all ties in to the idea that unstructured text isn't the best idea to -represent data between processes. Even though Unix pipes are extremely useful, -the ecosystem of shell and C conventions means that the obvious way isn't -completely correct, meaning that there are edge cases to consider. The best is -something as innocent as `ls | wc -l`, which will break, depending on the shell -settings, with some (unlikely) characters in filenames, i.e. newlines. - -# - -One of the problems is obviously that in order to pass around structured data, -i.e. objects, all participants have to understand their format. Passing -references won't work without OS support though. - -Instead of having unstructured streams, use streams of (data) objects. The -distinction here is Plain Old Objects (PODs) instead of objects with an -associated behaviour. - -Let's take a look at standard Unix command line tools (I'm using GNU Coreutils -here) in order to reproduce the behaviour and/or intent behind them: - -## Output of entire files - -The first command here is `cat`. Although GNU `cat` includes additional -transformations, this command concatenates files. Similar to the description, -we can image a `CAT` to perform a similar operation on streams of objects. - -It doesn't make much sense to concatenate a HTML document and an MP3 file -(hence you won't do it in most cases anyway). However, since files are -unstructured, `cat` can work on them. - -# Registering functionality - -Although you can call commands individually on files, some of them form an -ad-hoc service interface already: The C compiler, along with the toolchain -forms one such interface, where you're required to use the same interface if -you want to seamlessly replace one part of the toolchain. - -Same goes for the Coreutils: As long as you honour the interface, programs can -be replaced with different implementations. - -# Interactive commands - -Emacs has a special form `interactive` to indicate whether a command can be -directly called via the command prompt. There is also special handling there -to accomodate the use of interactive arguments. This is something that can be -generalised to the OS. An example of where this already happens is the -`.mailcap` file and the `.desktop` files from desktop suites. - -# Threading and job control - -Unfortunately getting proper job control to work will be a bit of a problem in -any Lisp implementation, since the best way to implement the concurrent jobs is -using threads, which are not particularly suited for handling the multitude of -signals and other problems associated with them. Even without job control -pipelines implemented in Lisp require shared in-memory communication channels, -so something like (object-based) streams, mailboxes, or queues are necessary to -move IO between different threads.