Kotlin DSL - Type safe builders

A (D)omain (S)pecific (L)anguage is a format for describing things in a specific domain. A very basic example is a shopping list: a list with items and optional item count. Very useful for shopping. But a goal of a DSL in this context is to help developers. Well known examples are HTML and CSS, formats for describing web pages and styling. For us humans it’s easier to read and write HTML and CSS than it is to achieve the same results with regular programming. But you do need a dedicated parser for your DSL. And it would be great if there is a little bit of editor support for helping you when you are writing in a DSL.

In Kotlin you can achieve this using type safe builders. They are used for things like the Gradle DSL and setting up a Ktor server. But they can also be used by developers to write a DSL for their specific domain. They provide type safety and editor support (code completion), so using them is mostly painless.

This is nice and all, but as a developer you want to understand what you are doing and why this works. Kotlin DSLs are not intuitive, they rely on some advanced Kotlin concepts. Unfortunately, the standard docs (https://kotlinlang.org/docs/reference/type-safe-builders.html) go over these concepts fairly quickly.

This tutorial aims at thoroughly explaining the basic concept behind the type safe builder, and then explain how to use that to create this DSL:

data class Person(val age: Int, val greeting: Greeting)
data class Greeting(val specific: Map<String, String>, val default: String)

fun main() {
    val person = buildPerson {
        age = 42
        greeting {
            "Hello" to "father"
            "Hi" to "mother"
            default = "What's up"
        }
    }
    println(person)
}

Which outputs:

Person(age=42, greeting=Greeting(
  specific={mother=Hi, father=Hello}, default=What's up))

We will be taking baby steps when explaining the fundamental concept, to make sure that you can follow along. Then we will speed up a bit to explain how to craft our DSL. By the end of this tutorial you will understand how and why this works and how to write the necessary builders yourself.

Links to the code in the Kotlin Playground are provided. It really helps to play around with the concepts yourself, so if you are confused, just play around with the code for a bit.

Part 1 - The basics

Extension functions

Kotlin DSLs are built using extension functions, so let's go over the basics of normal extension functions first. See https://kotlinlang.org/docs/reference/extensions.html for more details.

data class Person ( var age: Int = 0 )

fun Person.defineAge() { // this: Person
    this.age = 42
}

fun main() {
    val person = Person()
    person.defineAge()
    println(person)
}

This code outputs:

Person(age=42)

Play around with this code: https://pl.kotl.in/91ERUDeaK

Inside the extension function fun Person.setAge() the this is the Person object. We can use this to access any public functions and properties of the person object. While it seems that we are somehow adding functionality inside the Person class, we are not. We can only access the public functions and properties because we are handed the object (under the name this) as any other piece of code residing outside the Person class.

It's all just syntactic sugar offered by the compiler. Another way to write the same functionality would be:

fun normalDefineAge(person: Person) {
    person.age = 42
}

normalDefineAge(person)

Extension function are easier to read, so that is one big advantage of them. Another is that this is special in the sense that it can be omitted. So we could also write:

fun Person.defineAge() { // this: Person
    age = 42
}

Passing an extension function to another function

Functions are first class citizens in Kotlin, and so are extension functions. So, we can pass them around as parameters to other functions. What would that look like for an extension function?

data class Person ( var age: Int = 0 )

fun Person.defineAge() { // this: Person
    this.age = 42
}

fun buildPerson(init: Person.() -> Unit): Person {
    val person = Person()
    person.init()
    return person
}

fun main() {
    val person = buildPerson(Person::defineAge)
    println(person)
}

(Play with it: https://pl.kotl.in/1rw44-k5n)

In buildPerson you see that the function type of the extension function is Person.() -> Unit. Which makes sense: it is a function that extends Person, has no parameters and does not return anything. The init function cannot be called by itself, it must be called on a Person.

Whilst it might look a bit odd, it is still just the natural consequence of being able to pass an (extension) function to another function. And the result is awesome: by calling person.init() we can change the contents of person, as defined outside of the builder function. This is the foundation that the Kotlin DSL is built upon.

Using a lambda as an extension function

The extension function fun Person.defineAge() {} from the previous section can be assigned to a value by writing it as an anonymous function:

val defineAgeFunction: Person.() -> Unit = fun Person.() { this.age = 42 }

Using Kotlin type inference, we can abbreviate this to:

val defineAgeFunction = fun Person.() { this.age = 42 } 

If we rewrite this slightly, we get the lambda syntax:

val defineAgeLambda: Person.() -> Unit = { this.age = 42 }

To me, this last line looks weird: I think of type inference as working from right to left. As in: the type of setAge derives from what is on the right side of our equals sign (our lambda function). If that were all there was to it, we would get a compile error. The thing on the right side does not make sense.

For lambdas it works from left to right: We specify the type we need on the left side of the equals sign, and this influences whatever is written on its right side. To borrow from the duck typing metaphor, the compiler is telling us: you are a duck, so feel free to walk and quack like one.

Maybe you can see where we are going with this. We can pass the defineAgeLambda value to buildPerson(), but we can also inline the lambda expression to avoid having a separate defineAgeLambda. If the last parameter of a function is a lambda, we can move the expression to behind the parenthesis. And if there are no other parameters, we can omit the parenthesis altogether.

data class Person ( var age: Int = 0 )

val defineAgeLambda: Person.() -> Unit = { this.age = 1 }

fun main() {
    // pass the extension function as a lambda value
    val person1 = buildPerson(defineAgeLambda)

    // write the argument as an inline lambda
    val person2 = buildPerson({ this.age = 2 })

    // move the lambda expression outside the buildPerson parenthesis  
    val person3 = buildPerson() { this.age = 3 }

    // omit the empty parenthesis
    val person4 = buildPerson { this.age = 4 }  

    // omit `this`
    val person5 = buildPerson {  
        age = 5
    }

    println("$person1, $person2, $person3, $person4, $person5")
}

fun buildPerson(init: Person.() -> Unit): Person {
    val person = Person()
    person.init()
    return person
}

Which outputs:

Person(age=1), Person(age=2), Person(age=3), Person(age=4), Person(age=5)

(Play with it: https://pl.kotl.in/zDFFBb-3Q)

And there we have it: the first part of the DSL. It is no longer visible that the age = 5 is actually setting the age of a Person object via its this using an extension function written as an inline lambda.

If you understand this, then you understand how DSLs work. We can make the DSL a lot more useful and its implementation cleaner, but the basics stay the same. So play around with this code!

Part 2 - Making our DSL

Now that we have the fundamental concept down, we can work on how to create the full DSL shown in the beginning.

Mutability

The examples from part 1 relied on age being a var. This is generally not what we want, but we need that to be able to set its value. The solution is to have a builder version of the Person (with var age) and a final version of Person with val age as follows:

data class Person(val age: Int)

fun main() {
    val person: Person = buildPerson { // this: PersonBuilder
        age = 42
    }
    println(person)
}


class PersonBuilder {
    var age = 0

    fun build(): Person { return Person(this.age) }
}

fun buildPerson(init: PersonBuilder.() -> Unit): Person {
    val builder = PersonBuilder()
    builder.init()
    return builder.build()
}

(Play with it: https://pl.kotl.in/mpJEWt57E)

Nesting

While having immutable values is what we want, the builders class makes the DSL code harder to read. Our lambda no longer extends the Person class (that we know and understand), but it extends an intermediate class hidden inside the DSL code.

But the builder classes do have another reason to exist. They allow us to add additional functions, which we need to create nested properties. Let's look at code that adds a Greeting to a person, and just limit ourselves to the default greeting for now.

data class Person(val age: Int, val greeting: Greeting)
data class Greeting(val default: String)

fun main() {
    val person1: Person = buildPerson { // this: PersonBuilder
        this.age = 1
        this.greeting({ // this: GreetingBuilder
            this.default = "How are you"
        })
    }
    val person2: Person = buildPerson {
        age = 2
        greeting {
            default = "What's up"
        }
    }

    println("$person1, $person2")
}

class GreetingBuilder {
    var default = ""
    fun build(): Greeting { return Greeting(default) }
}

class PersonBuilder {
    var age = 0
    private val greetingBuilder = GreetingBuilder()
    
    fun greeting(init: GreetingBuilder.() -> Unit) {
        greetingBuilder.init()
    }

    fun build(): Person { return Person(this.age, this.greetingBuilder.build()) }
}

fun buildPerson(init: PersonBuilder.() -> Unit): Person {
    val builder = PersonBuilder()
    builder.init()
    return builder.build()
}

(Play with it: https://pl.kotl.in/D_XqPdmHm)

The PersonBuilder is re-using our fundamental DSL building block -the extension function lambda- to create a nested Greeting property.

Useful tricks with extension functions

In order to make our DSL look nice, we can use the extension function in another way. For example, we can declare an extension function to String class as a member function inside the builder class. This is very similar to just creating an extension function to String, but we limit the scope to the builder class.

class GreetingBuilder {
    var default = ""
    
    fun String.useAsDefault () { // this: String
        default = this
    }
    
    fun useFormal() {
        "How do you do?".useAsDefault()
    }
    
    fun build(): Greeting { return Greeting( default) }
}

The function useAsDefault() cannot be used in the global scope, but must be used in the context of the GreetingBuilder, like it is in fun useFormal().

However, it is still a public part of the GreetingBuilder class, so it can also be used in extension functions to the GreetingBuilder class. Meaning that we can use it in our DSL lambda. For example:

data class Greeting(val default: String)

fun main() {
    val greeting: Greeting = buildGreeting { // this: GreetingBuilder
        "What's up".useAsDefault()
    }
    println(greeting)
}

fun buildGreeting(init: GreetingBuilder.() -> Unit): Greeting {
    val builder = GreetingBuilder()
    builder.init()
    return builder.build()
}

Using this construct, we can create the last part of our DSL. We use a HashMap to store the specific greetings, and use a String extension function to make filling the Map look nice. For polish, we use the infix notation to make it look even nicer.

data class Greeting(val specific: Map<String, String>, val default: String)

fun main() {
    
    val greeting: Greeting = buildGreeting { // this: GreetingBuilder
        
        "Hello".to("father")  // using String extension member function
        "Hi" to "mother"      // also making use of infix notation
        
        default = "What's up"
    }

    println(greeting)
}

class GreetingBuilder {
    var default = ""
    private val specific = HashMap<String, String>()
    
    infix fun String.to (target: String) {
        specific[target] = this
    }
    
    fun build(): Greeting { return Greeting(specific, default) }
}

fun buildGreeting(init: GreetingBuilder.() -> Unit): Greeting {
    val builder = GreetingBuilder()
    builder.init()
    return builder.build()
}

(Play with it: https://pl.kotl.in/satygTQzb)

@DslMarker

It looks like we are all done, we achieved what we set out to. However, there is one more piece to the puzzle. Consider again the case of the nested builders:

fun main() {
    val person: Person = buildPerson {  // this: PersonBuilder
        age = 2
        greeting { // this: GreetingBuilder
            default = "What's up"
        }
    }
    println(person)
}

If we play around a bit in the Kotlin Playground, we can see that code completion is telling us that we can access age as part of the greeting.

And in fact, it is true. We can access the PersonBuilder's properties inside the GreetingBuilder. The reason is that we are still in the scope of the PersonBuilder. So from a compiler point of view it makes sense that we can access its properties.

It might feel strange to have multiple this in one scope, but that's a consequence of using extension functions. We already used it when writing an extension function as a member function.

class GreetingBuilder {
    var default = ""
    
    fun String.useAsDefault () { // this: String
        // access to GreetingBuilder.default via implicit this
        default = this
                          
        // access to GreetingBuilder.default via explicit this
        this@GreetingBuilder.default = this     
    }
}

As you see, we can also make the access to the outer this explicit.

Fortunately there exists a mechanism in Kotlin to tell the compiler to prevent the implicit this from being available: @DslMarker. With it we define an annotation that is used to mark our code as our DSL language.

@DslMarker
annotation class PersonDsl

Now we can annotate our PersonBuilder and HelloBuilder classes to be part of PersonDSL.

data class Person(val age: Int, val greeting: Greeting)
data class Greeting(val specific: Map<String, String>, val default: String)

@DslMarker
annotation class PersonDsl

fun main() {
    val person = buildPerson {
        age = 42
        greeting {
            "Hello" to "father"
            "Hi" to "mother"
            default = "What's up"

            // age = 41
            // Uncommenting the line above will yield a compile error.  
			
			this@buildPerson.age = 43  // But we can still be explicit
        }
    }
    println(person)
}

@PersonDsl
class GreetingBuilder {
    var default = ""
    private val specific = HashMap<String, String>()
    
    infix fun String.to (target: String) { specific[target] = this }
   
    fun build(): Greeting { return Greeting(specific, default) }
}

@PersonDsl
class PersonBuilder {
    var age = 0
    private val greetingBuilder = GreetingBuilder()
    
    fun greeting(init: GreetingBuilder.() -> Unit) { greetingBuilder.init() }

    fun build(): Person { return Person(this.age, this.greetingBuilder.build()) }
}

fun buildPerson(init: PersonBuilder.() -> Unit): Person {
    val builder = PersonBuilder()
    builder.init()
    return builder.build()
}

(Play with it: https://pl.kotl.in/ZH8nSGB-Q)

There we have it: the DSL we set out to write. And our @PersonDsl is preventing that we accidentally reference age inside the greeting {} block.

You might wonder why we have to define our DSL language annotation PersonDsl and why we could not just slap the @DslMarker on our builder classes. The reason is that this way multiple DSL languages can coexist. However, don't ask me to come up with an example where this is useful ;)

Conclusion

DSL languages are a nice tool to help you write correct code or configuration. Hopefully this guide has helped to de-mystify how it works under the hood. This will help you to troubleshoot problems when using DSLs, and perhaps it will inspire you to write your own Kotlin DSLs!