I first read about the legendary lisp macros in Paul Graham’s essay, and since then my whole journey of learning lisp has been driven by a desire to fully understand the power of macros.

If you have not heard about the power of lisp macros, here’s a quote from one of the creators of Scheme (a lisp) that captures the essence of this power.

If you give someone Fortran, he has Fortran. If you give someone Lisp, he has any language he pleases. - Guy Steele

It’s said that Macros are a way to extend the language and that macros let you write programs more suitable to your problem domain. But none of this will make sense to you if you don’t understand macros. I didn’t understand that when I first heard of them.

So, how about we start simple, forget all about the legendary powers of macros.

Macroland: Where normal rules don’t apply.

We are going to look at the behaviour of macros and try to understand their internal mechanics. Macro definitions, (atleast the trivial ones) look like function definitions.

Let’s look at a function and a macro.

(defn identity-fn [x] x)

(defmacro identity-macro [x] x)

They look similar, right? The real question is do they behave the same? Let’s test that out.

user=> (identity-fn 10)
user=> (identity-macro 10)

user=> (identity-fn (+ 1 2))
user=> (identity-macro (+ 1 2)) 

Hmm, they seem to look and behave the same.

Lets tweak the definitions a little bit

(defn identity-fn [x]
  (println "finding the identity of x")

(defmacro identity-macro [x] 
  (println "finding the identity of x")
user=> (identity-fn 10)
finding the identity of x
user=> (identity-macro 10)
finding the identity of x

Hmm, still nothing different. Let’s tweak the arguments we pass.

user=> (identity-fn (println "I return nil"))
I return nil
finding the identity of x

; something interesting is about to happen
user=> (identity-macro (print "I return nil"))
finding the identity of x
I return nil

If you didn’t spot the difference, I’d suggest you look at it again.

Here is the output side by side

; Macro                     | Function
; finding the identity of x | I return nil               
; I return nil              | finding the identity of x
; nil                       | nil

The macro does something that is similar to a function that looks like this.

(defn identity-macro-fn []
    (println "finding the identity of x")
    (println "I return nil"))

user=> (identity-macro-fn)           
finding the identity of x
I return nil

okay… that was a bit of a stretch, but not completely useless.

try this

user=> (macroexpand-1 '(identity-macro (println "I return nil")))
finding the identity of x
(println "I return nil")

that oddly resembles identity-macro-fn. for now, don’t worry about the ' in '(identity-macro (println "I return nil")) I’ll explain it later.

The evaluation steps in identity-fn are -

  1. The argument (println "I return nil") is evaluated first, triggering the side effect of printing that string.
  2. The result of (println "I return nil") i.e. nil is passed to identity-fn
  3. identity-fn then evaluates this (println "finding the identity of x") triggering the side effect.
  4. Then finally returns the value passed to it. i.e. nil (see #2)

The evaluation steps in identity-macro are -

Here’s one hypothesis as to how macros are evaluated.

  1. (println "I return nil") is NOT evaluated and passed to the body directly.
  2. The macro returns a body that might look like this
(do (println "finding the identity of x") 
    (println "I return nil"))
  1. Then clojure evaluates body, calling the first println and printing "finding the identity of x"
  2. then calling the second println and printing "finding the identity of x"
  3. Finally returning the value nil

Phew! that was quite a bit to take in. Now we know enough to formulate the first rule of macroland.

Macroland rule #1: Arguments are not evaluated before being passed to the macro body.

If you already know about macros, something is probably bothering you (hold on to that thought).

Let’s actually test out my hypothesis about the evaluation of identity-macro. Let’s write a macro similar to the identity-macro.

(defmacro id-mac [x]
    "I'm in a macro!"  

user=> (macroexpand-1 '(id-mac (println "str")))
(println "str")

hmm, "I'm in a macro!" is nowhere to be seen, so, like a normal function, the last form is returned. which means when we called identity-macro the form (println "finding the identity of x") was evaluated before the last form was returned.

;for reference
(defmacro identity-macro [x] 
  (println "finding the identity of x")

Which brings us to the next rule-

Macroland rule #2: Macro bodies are evaluated according to normal clojure rules.

Who said side effects aren’t hepful?

Here’s another macro & function pair.

(defn triplet-fn [a b c]
  (list a b c))

(defmacro triplet-macro [a b c]
  (list a b c))

Let’s test it out.

user=> (triplet-fn 1 2 3)
(1 2 3)

user=> (triplet-macro 1 2 3)
java.lang.Exception: Cannot call 1 as a function.

ack! why does this happen? any ideas? lets call macroexpand and see.

user=> (macroexpand '(triplet-macro 1 2 3))
(1 2 3)

That looks about right. Let’s try this.

user=> (eval (macroexpand '(triplet-macro 1 2 3)))
java.lang.Exception: Cannot call 1 as a function.

You get the same error as calling triplet-macro directly. Which means that Clojure tries to evaluate (1 2 3) which is an illegal form. Bringing us to rule #3

Macroland rule #3: Data returned by macros are immediately evaluated and the result of that evaluation is returned when the macro is called.

(P.S rule #3 isn’t entirely accurate but is a useful mental model for now)

A helpful way to think about it is that data is evaluated twice when writing macros, and only once with functions.

This is a source of confusion and much of the power of macros.

That’s it, there are 3 rules of macroland, and you know them now. To recap

Macroland rule #1: Arguments are not evaluated before being passed to the macro body.
Macroland rule #2: Macro bodies are evaluated according to normal clojure rules.
Macroland rule #3: Data returned by macros are immediately evaluated and the result of that evaluation is returned when the macro is called.

Datalchemy: Transforming data into code

If you’ve programmed in lisp for sometime, you’ve heard that in lisp, code is data and data is code. While that statement is true, it seems vague at first.

Aren’t .py and .java files binary data that is also code? how is lisp different?

Here’s a python function

def func(foo):
    return do_something_with(foo)

here’s the same function in clojure

(defn func [foo]
    (do-something-with foo))

The syntax is almost the same. The only difference is the parentheses, you are writing a lisp program inside a list. Lisp doesn’t differentiate between (func arg1 arg2) and (1 2 3). Instruction-wise the former is a function call while the latter is a list but semantics-wise, both of them are lists containing three elements! Let me prove it to you.

; you don't even need to define func, arg1, and arg2
user=> (count '(func arg1 arg2))
user=> (count '(1 2 3))          

But you can’t do this in python.

>>> len(func(arg1, arg2))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'func' is not defined

Another way to think of it is, python and java programs are instructions written inside a file. Lisp programs are instructions writting inside lists that happen to be stored in files. If you truly want to embrace the philosophy of lisp, you need to stop differentiating between instructions and data, they are the same thing.

What the heck is the purpose of a macro?

Simply put, macros build valid lisp forms (lists) for evaluation. You can think of macros as lisp translators; you give it some data, and it translates it into valid lisp data (code).

Symbols and values

It’s key to understand the difference between a symbol and its value. Normally, we are used to thinking about symbols in terms of their values. for e.g

foo = 10

We think of foo as 10, generally when we talk about foo, we’re talking about the value it represents, not the symbol foo itself.

(def one 7)
; here one is the symbol, and 7 is its value.
user=> one

In lisp, it is possible to refer to a symbol without referring to its value. This isn’t possible in most languages, so its likely to be new to you. I’ll talk more in depth about this but if you want to refer to a symbol and not its value, attach a ' to it. for e.g.

; referring to the value of one
user=> one

; referring to the symbol one
user=> 'one

Brewing some macros

Infix notation.

Lisp uses prefix notation for math functions like +, let’s write macros to support prefix. Think about what infix notation would look like. (1 + 2)

Treat this like a list of elements. (yes + is also an element) Then think how we could rearrange this data to form a valid lisp expression. We want to do this-
(1 + 2) -> (+ 1 2)
Take a minute to think it through. We need to swap the second element with the first element.

(defmacro infix-notation [[left operator right]]
  (list operator left right))

user=> (infix-notation (1 + 2))

(1 + 2) is NOT valid code, but the macro translates it into (+ 1 2) which is valid lisp code that lisp can evaluate. Now, try to implement a macro for postfix notation.


Let’s say we have a list of names what we want to preserve in a namespace. So we only let someone define a function if it is not in a list of names.

Think about the args that will be passed to the macro. It’ll be the same as defn. To keep things simple, let’s use name, args and body.

A normal function definition is a list. (defn name args body) So if we want to define a function using a macro we need to return a list like the one above. The only difference is that special-defn must check if the name exists in some blacklist, before allowing the user to define a function.

Let’s try to solve this.

(def black-list #{(symbol "java") (symbol "is") (symbol "evil")})
; symbol takes a string and returns a symbol

user=>(defmacro special-defn [name args body]
          (if-not (contains? black-list name)
            (list defn name args body)
              "you can't define this function"))
; Syntax error compiling 
; Can't take value of a macro: #'clojure.core/defn

This happens because lisp tries resolve the symbol defn to a value. You can’t take a value of a macro. In fact, you don’t need the value of a macro, you want the symbol defn not its value. Remember that you want to return a list that looks like this (defn name args body). let’s use a quote now.

Macrobrew toolkit tool #1: The quote

The ' quote operator is the first tool the young macro-brewer learns about. It is used to tell lisp to skip evaluating a symbol. Here’s an example.

user=> (defn gimme-foo [] 'foo)     

Even though foo is not defined, gimme-foo returns foo. If you evaluate foo you’ll get an error.


If you want to refer to a symbol use a quote ' otherwise lisp will resolve it to a value.

If ' is placed before a parenthesis it will recursively quote all elements inside the parentheses.

user=> '(foo bar baz)    
(foo bar baz)
user=> '(foo (bar tavern pub) baz)
(foo (bar tavern pub) baz)

This can save you from quoting each element in the list. Let’s use this knowledge to solve our problem.

(def black-list #{(symbol "java") (symbol "is") (symbol "evil")})
; symbol takes a string and returns a symbol

user=>(defmacro special-defn [name args body]
          (if-not (contains? black-list name)
            '(defn name args body)
              "you can't define this function"))


Progress! it compiled. Let’s test it out.

;; This works fine
=>(special-defn java [a] a)
"you can't define this function"

=>(special-defn coffee [a] a)
; Syntax error macroexpanding clojure.core/defn
; args - failed: vector? at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
; args - failed: (or (nil? %) (sequential? %)) at: [:fn-tail :arity-n :bodies] spec: :clojure.core.specs.alpha/params+body

Something is still wrong.

MacroBrew toolkit tool #2: macroexpand, macroexpand-1

You’ve used macroexpand before, macroexpand-1 is similar. These are very useful functions to see the list that your macros return. It’s good practice to use them to debug your macros.

(macroexpand '(special-defn coffee [a] a))
; Syntax error macroexpanding clojure.core/defn at (c:\Users\abhin\Downloads\programming\clojure\temp\.calva\output-window\output.calva-repl:244:1).
; args - failed: vector? at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
;args - failed: (or (nil? %) (sequential? %)) at: [:fn-tail :arity-n :bodies] spec: :clojure.core.specs.alpha/params+body

Not very helpful, since the error is still there. Let’s try macroexpand-1.

(macroexpand-1 '(special-defn coffee [a] a))
(defn name args body)

There’s a lot going on here. defn looks fine! But what is name args and body? shouldn’t those be coffee, [a] and a ?

The list we expected was this (defn coffee [a] a) The macro returned the symbols name args and body instead of their values.

(you see what I meant when I said symbols and values get confusing).

Before we fix that, ask yourself why macroexpand-1 worked but not macroexpand. The reason is macroexpand recursively calls macroexpand-1 until a normal lisp form is returned. Since we return defn in this case, and defn is a macro, macroexpand expanded defn resulting in an error.

Let’s get back to fixing special-defn. To recap, we want to return a list containing the symbol defn and the values that name, args and body represent. Which means we should only quote defn and not the whole list.

(def black-list #{(symbol "java") (symbol "is") (symbol "evil")})
; symbol takes a string and returns a symbol

(defmacro special-defn [name args body]
  (if-not (contains? black-list name)
    (list 'defn name args body)
    "you can't define this function"))

(special-defn java [a b c]
              (list a b c))
"you can't define this function"

=>(special-defn coffee [a] a)

Time for a contrived example

Our contrived macro will take a list of integers, add all of them, then subtract the list of integers from that result. I’ll show you what the function looks like

(defn contrived-fn [coll]
  (apply - (apply + coll) coll))

If we want this to be a macro, the macro must return the last form. Since we know that we want the symbol apply, -, and + and not their value. We’ll quote the last form.

(defmacro contrived-mac [coll]
  '(apply - (apply + coll) coll))

user=>(contrived-mac [1 4 2 4])
Could not resolve symbol: coll

Ah, we want the value coll refers to. But how do we get the value when the whole form is quoted?

MacroBrew toolkit tool #3: The syntax quote

Besides using ' for quoting you can use a more powerful operator called syntax quote.

user=> `(+ 1 2 3)
(clojure.core/+ 1 2 3)

The beauty of the syntax quote is that it lets you unquote an element in the form using ~. Just like how quoting tells clojure not to evaluate something, unquoting tells clojure to evaluate it. Also, notice how the + is a fully-qualified symbol. This helps prevent name collisions (we’ll cover that later).

Use unquote ~ when you want the value of a symbol that is inside a syntax-quoted list.

Let’s use this in contrived-mac.

(defmacro contrived-mac [coll]
  `(apply - (apply + ~coll) ~coll))

user=> (contrived-mac [1 4 2 4])           

Convenient, right?

Another contrived example

I know I know, I’ve reached the quota of contrived examples but this is a longer post, bear with me.

We’ll define a macro that prints all elements in a list and returns the list. Try it yourself, before you read ahead.

(defmacro print-els [coll]
`(do ~(map println coll)
user=>(print-els [1 2 3])
; Syntax error (IllegalArgumentException)
; Can't call nil, form: (nil nil nil)

Let’s expand the macro to see what is happening

(macroexpand-1 '(print-els [1 2 3]))
(do (nil nil nil) [1 2 3])

So ~(map println coll) returned (nil nil nil), since nil isn’t callable you get an error. BUT BUT BUT. here’s the thing, this piece of code works.

=>(do (map println [1 2 3]))
(nil nil nil)

;; and so does this.
(do nil nil nil [1 2 3])
[1 2 3]

In fact we want our macroexapansion to resemble the list above.

MacroBrew toolkit tool #4: Unquote splicing.

Unquote splicing was created for the above problem. Sometimes you want a list’s contents (not the list) in a clojure form. The unquote splice operator looks like this ~@.

(defmacro print-els [coll]
`(do ~@(map println coll)
user=>(print-els [1 2 3])
[1 2 3]

user=>(macroexpand-1 '(print-els [1 2 3]))
(do nil nil nil [1 2 3])

Can’t we just return the data and evaluate it outside the macro?

You could define print-els like this

(defmacro print-els* [coll]
`(do (map println ~coll)

But what if in the namespace you defined this macro, map means something else? what if map was a hash-map ? try this.

; redefining map
(def map #{:a 1})

; 1 2 3 doesn't get printed
user=>(print-els* [1 2 3])
[1 2 3]

Remember Macroland rule #3?

Data returned by macros are immediately evaluated and the result of that evaluation is returned when the macro is called.

Here you can see that the data returned by the macros are evaluated in the context of the namespace it is called. Let’s amend rule #3

Macroland rule #3 Data returned by macros are evaluated in the context of the namespace it is called in.

Hygenic brews

Nobody likes a stale brew. People fall sick, and it is bad for business. The same goes for macros. What do I mean by hygene in macros?

(defmacro return-some-list [x]
  (list 'let ['one 5]
  ['one x]) ) 

user=>(return-some-list 1)
[5 1]

works fine, let’s say we get a var in the namespace called one and we pass that to the macro.

(def one 1)
user=>(return-some-list one)
[5 5]

Not what expected, right? Remember that args passed to macros are not evaluated, and since we’re returning the symbol one in the context of a let binding, the value of one is actually 5. See the macroexpansion

user=>(macroexpand-1 '(return-some-list one))
(let [one 5] [one one]) ; one is 5 here

This is called a symbol capture, and is a common macro pitfall.

You could try to return syntax quoted symbols (which are fully qualified symbols) but you’ll get an exception. try it.

(defmacro return-some-list [x]
  `(let [one 5]
     [one ~x]))
(return-some-list one)
; Syntax error macroexpanding clojure.core/let .
; user/one - failed: simple-symbol? at: [:bindings :form :local-symbol] spec: :clojure.core.specs.alpha/local-name

This happens because let only accepts simple symbols. fully-qualified symbols are not simple symbols. While the source of the exception is cryptic and you might spend a lot of time trying to figure out what’s going on, the exception is for your own good.

=>(simple-symbol? 'one)

=>(simple-symbol? 'user/one)

MacroBrew toolkit tool #5: Gensym and autogensym aka #

You can solve both these problems by using a unique name for the symbol one that is inside the macro. Thankfully clojure provides a function called gensym which does this.

G__7429 ; your output will be different.

In fact, this is a common task, and clojure provides syntactic sugar to solve this. It’s called the autogensym or #


=>(simple-symbol? `one#)

Let’s use that in our macro.

(defmacro return-some-list [x]
  `(let [one# 5]
      [one# ~x]) )

user=> (return-some-list one)
[5 1]

Those are almost all the tools you need to write macros.

Bespoke Brews: A simple 3-step approach to writing a macro.

Just like knowing the rules of Go will not make you a brilliant Go player, knowing the rules of macros doesn’t mean you know how to write your macros. Here’s a 3-step process to help you get started.

  1. Think about the data that will be passed to the macro.
  2. Think about the data the macro will return.
  3. Think about how you’ll transform the passed data to the returned data.

Sometimes, step #2 will occur before step #1 and that’s fine, this is just a guideline to help you get started.

Let’s implement postfix notation with this process.

Step 1 think about the raw data.

An example input for postfix will be (2 3 +). The general shape is (operand1 operand2 operator)

Step 2 think about the data that will have to be returned

We know that in lisp the operator comes first, followed by the operands. So the returned data must look like this (operator operand1 operand2)

Step 3 think about how you will make the transformation happen

Work backwards in tiny steps till you get the macro definition. I like to work with concrete examples.

Here’s the transformation

; example data => data returned by macro
       (1 2 +) => (+ 1 2) 

so the macro returns a list of 3 elements.

; example data => inside macro => data returned by macro
       (1 2 +) => (list + 1 2) => (+ 1 2) 
;; with arg names
;       raw data          =>          inside macro               =>    data returned by macro
 (op1_sym op2_sym op_sym) => (list op_value op1_value op2_value) => (op_value op1_value op2_value)

It’s obvious that we want the values of the symbols so no need to syntax quote anything. Here’s the implementation.

(defmacro postfix [[op1 op2 operator]]
  (list operator op1 op2))

user=>(postfix (1 2 +))
user=>(postfix (1 2 -))

When in doubt always work backwards in tiny incremental steps, like I’ve shown, and experiment with the repl. The repl lets you build macros incrementally like this. Use it.

Going from apprentice to Brew master.

The downsides of macros.

Macros are not values.

A function is a value, but a macro is not. Which means you can’t pass macros to higher order functions.

(defmacro my-odd [x] `(odd? ~x))

(filter my-odd [1 2 3 4 5 6])
; Syntax error compiling 
; Can't take value of a macro: #'user/my-odd

If you want to use the macro you will have to wrap it in a function

(filter #(my-odd %) [1 2 3 4 5 6])

But this will only work in simple cases like this.

Macroexpansion happens during compile time.

This is what I meant when I said macroland rule #3 isn’t entirely accurate. When you call a macro in your code, the macro call is replaced by the list returned by the macro when you compile your program. The list returned by the macro is evaluated when you run your program.

You won’t realize this when you experiment at the REPL since it’s compiled and run immediately. But the implications of this fact are not obvious at first. So let’s explore this a little.

(defmacro mul-2 [xs] 
  `(* 2 ~@xs))

Let’s use the macro in a file.

; file.clj
(mul-2 [1 2])

when you compile the file, that line of code is replaced with this.

(* 2 1 2)

This is only evaluated when you run the file. Since the macro needs to know the value of xs during compilation, (because of ~@xs) we can’t pass a var in its place.

(defn multiply-by-2 [nums]
  (mul-2 nums))
; Syntax error (IllegalArgumentException) compiling
; Don't know how to create ISeq from: clojure.lang.Symbol

Since nums is passed at RUNTIME the macro doesn’t know the value of nums, Here, the macro receieves the symbol nums and not the value of nums.

This is another reason, why you can’t pass macros to higher order functions.

Macros attract more macros.

Let’s define add as a macro

(defmacro add [& args] 
`(+ ~@args))

user=>(add 1 2 3)

What if we wanted to find the sum of a sequence of vectors containing integers? for e.g [[1 2 3] [2 4]] => [6 6] Normally we would use map and do something like this.

=>(map #(apply + %) [[1 2 3] [2 4]])
(6 6)

But as you know we can’t pass a macro to map.

=>(map #(apply add %) [[1 2 3] [2 4]])
(map #(apply add %) [[1 2 3] [2 4]])
; Syntax error compiling at 
; Can't take value of a macro: #'user/add

the only way we can do this is by defining another macro.

(defmacro add-vecs [vecs]
  (loop [f (first vecs)
         r (rest vecs)
         res `(list)]
    (if (seq r)
      (recur (first r) (rest r) (concat res `((add ~@f))))
      (concat res `((add ~@f))))))

user=> (add-vecs [[1 2] [3 4]])
(3 7)

So, things can get complicated real fast.


This is a mix of things I’ve learned and advice from the community.

Always use a syntax quote

The syntax quote is there to protect you from common macro pitfalls like symbol capture in let bindings. Always, always use a syntax quote. Unless you absolutely need to return an unqualified symbol(are you certain this is what you want?) you should always use a syntax quote. .

Macros are hard to read, write, and understand.

Because of the “double evaluation” macros are hard to understand (and you know how important readability is) They are hard to write, (it took me quite a while to get add-vecs right).

Your best friends are macroexpand and macroexpand-1.

Use them liberally to understand and debug your macros. Use them to see how other people’s macros expand. It’s a good learning exercise to see what clojure.core macros expand to.

Read macros written by macro masters…

…and then try implementing them on your own. Clojure.core macros are a good place to start.
Some other macro libraries are swiss-arrows, and potpuri (thanks to @Remy_sajoR for suggesting these).

Don’t write Macros if a function will do

“Macros are harder to write than ordinary Lisp functions, and it’s considered to be bad style to use them when they’re not necessary.” -Paul Graham, “Beating the Averages”

The only exception to this is when a macro call will be more convenient compared to a function call. The threading macros are an excellent example of this.

; This is valid code, but arguably a bit ugly.
(map str 
     (map #(* % %) 
          (map inc 
               (filter even? (range 25)))))

(->> (range 25)
     (filter even?)
     (map inc)
     (map #(* % %))
     (map str))

This is not only convenient to write, it is beautiful to read. There is another subtle benefit of this macro, it encourages you to write code in terms of transformations to data, because of how pleasureable it is when you see code likey this (this is true for me atleast).

Macros are not the only powerful tools clojure has.

Macros aren’t composable and result in code that is harder to reason about.

Write macros when functions just won’t do

Have you tried solving it with a function? Have you really tried? have you tried throwing a monad at the problem? Then you probably should use a macro.

Write macros when you find repetitive patterns.

Another example is testing.

We have an expression, we want to test whether it returns a value or not. Internally, there is a repetitive task that goes on.

  1. Try an expression.
  2. If it doesn’t raise an exception check if the assertion is true or false and then report.
  3. Else catch an exception if raised and report exception.

This pattern is encapsulated in the is macro.

(is (= (func arg) expected-value))

(is (thrown? Exception (code-that-raises-exception)))

When possible delegate to functions

Functions are easy to reason about (did I tell you that functions are easy to reason about?) So extracting code from the macro body to functions helps you and the reader of your code. When do you extract to a function though? If you find yourself with a big body of expressions that are evaluated inside the context of a macro, that might be a sign that you need to extract it to a function. But at times it’s not as clear cut as that, you will learn by reading other people’s macros. is would be a good place to start. Inside is there’s a delegation to another macro try-expr which in turn delegates parts to a multimethod assert-expr and function do-report.

Closing thoughts

That was the a very long post, I hope you enjoyed reading it, and learnt something. If you find any mistakes, have suggestions, or questions tweet at me @the_lazy_folder I respond to all tweets.