A lot has changed since my last post nearly a year ago. I want to write about my experience coming from a ML-family language to a Lisp. It’s not going to be very informative as a tutorial; it’s mostly scattershot notes and things I’ve found interesting along the way.

Background

My very first exposure to functional programming was through a book called The Little Schemer: a fairly thin, flourescent, mind-melting paperback that I’m not quite sure how I came upon. I’d been programming for ten years but it was like my inner programmer was reincarnated.

I was working mostly with .NET at the time so F# was a natural choice for functional exploration, even though it wasn’t a Lisp. After a few years of F# I found myself back in Lispland.

My new language was Clojure and it was totally foreign. So many parentheses instead of semantic whitespace. No rigid type system to hold me close. How am I going to deal with these parentheses? (Thank you, Parinfer.)

While this was another functional language, it encouraged a very different style of programming—one I wasn’t used to and thought I wouldn’t like. The chaotic ambiguity of dynamic typing. How am I supposed to feel my program is even possibly correct if the compiler isn’t telling me so? ML is so expressive in its ability to model a problem domain with algebraic data types. How could I ever do without it?

First Impressions

First I looked for analogues of ML/F#. I quickly found all the familiar functional tidbits pretty much unchanged.

I wondered why there’s a reduce in clojure.core but no fold? (It’s just a different arity of reduce that’s missing the initial state.)

There were record types but they didn’t seem often used. Polymorphism via protocols and multimethods seemed like powerful tools, but I only see them used sparingly.

Where’s the pattern matching? I found clojure.core.match but I’ve never felt motivated enough to try it, nor have I seen it used. I have no idea about the state of pattern matching in other Lisps, but I assume it’s a similar situation. cond gets you most of the way there I suppose?

No tuple type? Vectors are tuples. “What if one of the vectors is of a different length?” I gasped as I clutched my pearl monads.

I was sure I’d miss discriminated union types. There seemed like no stand-in for those, although I think clojure.spec’s or might facilitate something similar.

The reality of the Clojure language design started to sink in. The language itself was extremely basic. There was so little syntax to learn that I could be productive almost immediately. Conversely, even when writing F# daily I sometimes forgot less common syntax. (Do I need a semicolon or comma in this record definition?!)

I was able to easily dive into clojure.core source interactively while I coded, which usually required third-party tools for FSharp.Core and .NET in general. I do find myself reading more third-party Clojure code out of necessity, a little more than I needed to with F#/.NET, and I chalk it up to needing to read code to see what it does rather than making assumptions based on the types it works on.

Keywords

These are everywhere in Clojure. I think of these as more-or-less interned strings that are granted some special abilities.

:foo ;; a keyword

You can think of that as "foo" for most intents and purposes, and there’s a pair of functions for converting between keywords and strings.

(keyword "foo")
=> :foo
(name :foo)
=> "foo"

Keywords are the de facto type for keys in map literals e.g.

{:name "Bill", :age 23}

Keywords can also be used as functions of maps i.e. to get a value from the map by keyword: (:foo m). They can be namespaced to avoid naming collisions e.g. :my-namespace/foo. Note the name of the namespaced keyword is just the part after the /.

Keywords don’t have to be declared anywhere before use; they’re just literals. I’ve read/heard the Clojure designers encourage the use of namespaced keywords, but I rarely encounter them outside clojure.spec definitions.

It’s also common to keywordize ingested data structures like JSON objects, XML DOMs, etc.

Arity

The really neat-o thing about Clojure function arities is the ability to define functions of arbitrary arity. In these cases the numerous arguments are bound to a collection: either a vector or a map, and a map can be thought of as a collection of key/value tuples. A basic example of this is the + function:

(+ x y & more)

Here, more can be any number of additional arguments. This ties into another concept that was new to me: applying a sequence of arguments to a function.

(apply + [1 2 3 4 5 6 7 8 9])
=> 45

Clojure has a nice compact way of defining multiple function arities: a single function symbol form with each arglist + implementation in parentheses:

(defn foo
 ([a]
  (println a))
 ([a b]
  (println a b)))

And you do get some compile-time safety checks on function arities.

I wondered what the equivalent of F#’s Seq.zip was in Clojure but a quick search turned up nothing. I didn’t realize you could simply pass more than one collection to map, and the step function would then take an argument per collection. This seemed really like a really elegant use of variable-arity functions and there are several examples of this technique in clojure.core.

Multi-arity functions are also one way Clojure deals with named optional arguments. Both these function calls use the same function definition:

(foo 123)
(foo 123 :append true :path "~/.sup")

The additional “arguments” are actually treated as a map and I’ve seen this style referred to as kwargs. It’s a form of destructuring in the function signature like this:

[num & {:keys [append path]}]

This syntax obviously requires an even number of key/value pair arguments.

Destructuring

I used destructuring in F# most often in pattern matching clauses. In Clojure I’m finding most destructuring I do involves getting values from maps, followed by grabbing items from sequences by position.

The map destructuring syntax is powerful and a bit confusing at first although it seems so simple now. It even allows you to specify default values for keys that aren’t present in the map, all in one line:

{:keys [foo] :or {foo 1} :as m}

foo will have the map’s value for key :foo, or 1 if the map doesn’t have a :foo key, and m is bound to the entire map. This makes getting values from maps very easy and terse. You can also alias destructurings inline:

{another-name :foo}

Another neat feature is positional destructuring, commonly used with vectors. In addition to the traditional head and tail destructuring of sequences, you have more flexibility with the & syntax.

[x & xs] [1 2 3]

Here, x is 1 and xs is [2 3]. But you can also do this:

[x & [next]] [1 2 3]

Here, x is 1 and next is 2. The nested destructuring indicates we only care about the first element of the tail.

You can also destructure more than one binding before the tail like so:

[x y & zs] [1 2 3]

Here, x is 1, y is 2, and zs is [3].

In the “anything goes” spirit, positional destructuring doesn’t throw exceptions on “failure”:

[x y z] [1 2]

Here, z will simply be nil as there is no third element. The same goes for map destructuring i.e. you just get nil back for keys that aren’t in the map.

All this destructuring is also available in function argument signatures! This allows for flexible function signatures and can get a lot of argument destructuring “out of the way” of the function’s body—a nice separation of concerns. In some sense, you can imagine all Clojure function argument bindings as a destructured vector.

Destructurings can be nested but it becomes unreadable two–three levels deep.

Data

Clojure and F# have similar data structures but working with them in Clojure feels lower-ceremony. The literal syntax is nice and terse and all the basic data structures can be conveniently composed and transformed.

Maps would seem to be Clojure’s commonplace record type, even though it has defrecord which seems somewhat deprecated. Something that bugged me about this initially was there were no guarantees a record (map) would contain what you expected, or that it wouldn’t contain a bunch of stuff you didn’t expect! I came to realize this doesn’t matter in practice as often as I thought.

clojure.spec

I can’t talk about data in Clojure without talking about clojure.spec!

I believe there’s a drawback to the ambiguities of working with data in Clojure, and one that clojure.spec can address: sometimes you really do need a rigid specification of data. With clojure.spec you can document the shape of data (in a machine-readable format), then use that to verify properties of functions of that data (via generative and property-based testing), and a whole lot more. I’ve even leveraged clojure.spec to generate API documentation and sample code. Watch this excellent talk for more on clojure.spec.

Spec solves many problems but what interested me at first was using it as a sort of opt-in gradual typing system for my projects. I could write specs for only the data structures and functions I wanted. With function specs, I could assert the inputs and outputs to specific functions at runtime; not quite as reassuring as compile-time but better than nothing!

I hope clojure.spec adoption continues and this kind of opt-in type checking becomes more prevalent. I think there’s a very sweet spot to occupy between everything must be typed and total dynamic ambiguity. The new Clojure JDBC library shipping with specs is hopefully a good sign of what’s to come.

Code as Data

Whatever superlative or diss you might use to describe a Lisp’s syntax or lack thereof, it makes one thing evident: your code itself is just data, and data can be manipulated. A function at rest is just a list containing other lists and data structures. Code as data was one of the profound enlightenment moments I’ve had as a programmer, and one that can’t be realized in most popular languages.

Clojure’s macros allow you to work with code as data. Much of Clojure’s core functionality is written as macros that get expanded into final forms before evaluation. F# has some metaprogramming facilities but they’re clunky (and fragmented depending on what kind of syntax tree you want).

I don’t often write new macros, and it’s not advised to write macros where functions work just as well, but they’re invaluable in certain contexts.

A few times I’ve been able to do some trivial refactorings of code programmatically in the REPL. Of course .NET has Roslyn but it’s a totally separate facility; in Clojure this ability is inherited more or less by virtue of being a Lisp.

Data as Code

Some core Clojure data structures can be used as functions. I didn’t know what to make of this at first but I love it now. For example, a map is a function of one (key) argument. A set is a function of one (member) argument.

(def my-set #{1 2 3})       ;; a set of integers
(my-set 1) => 1             ;; returns the member value
(my-set 3) => 3             ;; returns the member value
(my-set "no") => nil        ;; returns nil 
(map my-set [3 2]) => (3 2) ;; mapping the set over a vector

Lately I find myself thinking more in terms of data than procedural code. For example, sometimes it’s nice to express program logic declaratively with data as opposed to control-flow/case/cond statements. Consider this case:

(case x
  "High"   100
  "Medium" 50
  "Low"    0)

This can also be expressed as a map, and that map can be used as a function that returns the matched key’s value. This code has the same effect as the case above:

(def foo
  {"High"   100
   "Medium" 50
   "Low"    0})
(some foo "Medium") => 50

Here we’re passing the map foo as a function to some, which will return the first value of the matching key.

This approach is nice when you want to define some “rules” in one place and it use them in many, or maybe it’s loaded from a database or configuration file.

Nil

Some functional languages go to lengths to cordon null/nil values. Clojure embraces nil (in most cases). I remember thinking “there should be an Option type” a lot in my first days. Now I embrace the nil. I rarely find myself needing to write defensive nil-check code.

I do miss nullable comparison operators and I wish there was something like F#’s Option module, but it’s just a matter of defining your own functions to do the same.

There are many nil-friendly core functions, and some nil-adverse functions too. It’s not something I have to worry about much, but a nil-propagation bug can be hard to debug when it happens.

REPL-driven Development

I didn’t use the F# REPL that much for interactive development as much as I did for interactive execution. The Clojure REPL experience is far more ingrained into my development workflow, and I like it. I feel I’m more of an inhabitant of the program as I write it rather than an observant designer. I also get more immediate feedback about the runtime behavior of code, which can be hard to reason about in functional languages.

I think there’s another reason for the REPL being so integral to the Clojure development experience: discovery through interaction. Clojure just doesn’t have the information I would normally get via type definitions (and IntelliSense/auto-complete). In Clojure it’s more natural to get that information by playing with the code in hand. Doc strings help but they aren’t schematics. Again, another area where clojure.spec may change the landscape.

I haven’t tried this, but I’ve heard you can be so bold as to have a deployed application host a network REPL which can be used to remotely inspect (and monkey with) the running application’s code and state. 😮

Debugging

I used the interactive debugger much more often with F#. I’ve only tried to interactively debug Clojure once or twice. It’s possible but I find it’s just easier to rely on REPL evaluation and printf debugging in most scenarios. I don’t get the feeling interactive debuggers are a hot topic in the Clojure community.

Invariants

One of my favorite things about more strongly-typed functional languages is the ability to make illegal states unrepresentable. In Clojure it’s very hard to make illegal states unrepresentable, in fact I’d say it’s just not worth the effort. In ML variants it’s harder to write a program that compiles but doesn’t work; I quickly found in Clojure it’s quite easy to write a program that compiles and doesn’t work.

Clojure feels very open and permissive when it comes to data and passing it around. The thought that a caller could pass anything as an argument was unsettling at first.

Clojure does allow you to define pre- and post-condition metadata on functions for call assertions, but they don’t seem common in practice and they only work at runtime. clojure.spec allows much more powerful assertions, but again only at runtime and strictly opt-in.

As I embraced this laissez faire approach, the Clojure lifestyle started to make sense. Clojure codebases are driven more by convention than contract. In F# I mostly thought about problems in terms of the types I’d use to model them, essentially defining a contract. If my program compiled I could be fairly certain it’d do at least something like what I intended. This meant putting a lot of upfront thought into the types I’d need to model the problem.

In Clojure I find myself doing the opposite—just diving into the REPL and immediately writing code that operates on data. That same code can be sculpted into tests (if I’m not too lazy) and ultimately end up in production.

Parenthetical Gestalt!

It’s the carefully and conservatively curated confluence (alliteration achievement unlocked 🏅) of Clojure’s design decisions and philosophies, especially those that were hard for me to embrace at first, that make it such an enjoyable programming experience for me.

I pre-judged Clojure for being a dynamic language because I’d never enjoyed working in a dynamic language before. As I look back on the things I thought I’d miss from a strongly typed language, I can easily say I’ve faired just as well without them.

There were a bunch of other cool Clojure tidbits I wanted to mention but this post was getting too long!