Gradle Project Layout

30 June 2020

Most build systems use the file system as the project layout. It’s most notably the case for Make-based build system. [1] The file system-based project modelling approach is quick and easy to implement. It relies on a simple check with the file system, and off the build goes.

However, it’s not a magic solution as it can be. What happens when projects are deeply nested? Or when we move projects around? Or when we deal with legacy tools assuming a specific layout? The solutions are often unimaginative and simply work around the core issue. For example, native projects often prefer a flat layout. Developers split the includes, binaries, and source codes into top-level folders.

Gradle has a distinct model for the project layout and on-disk layout. By convention, both follow the same layout, e.g. project :foo is in folder foo. However, we can configure Gradle to use two different layouts. The Nokee project remap all projects inside the subprojects folders. The project folders follow a kebab convention, which differs from the project name, following a lowerCamelCase convention.

Gradle doesn’t stop there. We can change the source layout convention, from the well-known Maven layout to something completely different. We can relocate the build directory anywhere we want. It renders the argument of in-source build vs out-source build irrelevant as Gradle can do both, or an hybrid or something completely crazy. The point is, Gradle is much more flexible than everyone is using it.

With all this flexibility offered by Gradle, there isn’t an optimal project layout. It depends on your projects. Here are some guidelines to help you choose the right project layout.

Focus on what you are building instead of how

Identifying the building blocks inside your repository will help create the project layout. Whether your organization follows a mono-repo or multi-repo or whatever the on-disk source layout may be, there are always holistic building blocks to your projects. Each of those blocks is a project. Some of those blocks may only make sense together; make a note of them.

Some questions that can help you are:

  • What kind of component is your build producing? Application? Library? Something else?

  • What are you distributing to your clients? RPM? Zip archives?

  • What is the relationship between your components? 3rd party libraries build from sources? Helper tools?

  • What are the artifacts produced? For other projects? For testing?

Make your developers happy

Identifying which building blocks are essential to your developers will help establish an efficient project layout. The developers are the users of the build. The build system needs to work with them. They also know which projects as well as which actions they require for their daily work.

For example, the holistic project layout may favour deep nesting of the projects. However, the developer may only care about executing test coverage for a project subset. It’s not about solving the deep nesting of projects but rather providing tasks to allow smooth test coverage execution. Or maybe offer an umbrella project that exposes an aggregated set of actions for a subset of projects.

Did you know Gradle creates intermediate projects in your hierarchy? For example, include 'a:b:c' creates a project :a, project :a:b and project :a:b:c. Project :a could contain convenient tasks that act on its subprojects. Then developer could execute ./gradlew :a:foo which would apply to project :a:b and project :a:b:c.

Not all projects needs a build script

Some build systems identify projects by the presence of a build script. Gradle configures projects as declared in the settings.gradle[.kts] file. Whether the project build script exists or not, the project still exists and is configurable.

Did you know Gradle can inject configuration code inside any projects? project(path, Action), subprojects(Action) and allprojects(Action) are great ways to configure other projects than the currently configuring projects. For example, you could use a single build.gradle to configure multiple projects:
include 'a:b1'
include 'a:b2'
include 'a:b3'
project('b1') {
    //configure b1
}
project('b2') {
    //configure b2
}
project('b3') {
    // configure b3
}

You could even model the b* projects inside project :a with an extension.

Prefer convention-over-configuration

The convention is always much more powerful than configuring each project individually. Suppose a C library project is defined by the presence of C source files located by convention in src/main/cpp. We could have something like this in the root build script:

allprojects {
    if (file('src/main/cpp').exists) {
        apply plugin: 'dev.nokee.cpp-library'
    }
}

These four lines of code may apply to 100s of projects. There was no need to create build.gradle files in those 100s of projects.

Now suppose some of those C++ library projects are in fact applications, we could build a look aside list of those exceptions and barely make the configuration anymore complicated:

def applicationProjects = [':foo', ':bar']
allprojects {
    if (file('src/main/cpp').exists) {
        if (applicationProjects.contains(path)) {
            apply plugin: 'dev.nokee.cpp-application'
        } else {
            apply plugin: 'dev.nokee.cpp-library'
        }
    }
}

If by convention, application projects have an App suffix, we don’t need the lookaside list. It may be tempting to nest applications and libraries under umbrella projects to allow easy configurations such as project(':lib').subprojects { …​ }. Experience has shown it’s usually better to avoid this kind of configuration. There are some cases where it makes sense.

Did you know each build script is a Turin-complete language? You can write modelling code as you see fit. For example, suppose your project has 1000s of projects, but they all follow a similar on-disk convention layout. You could write a file system walker to discover every project. Since Gradle is JVM-based, you can use the https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#walkFileTree-java.nio.file.Path-java.nio.file.FileVisitor-(Files#walkFileTree API) for this purpose. There are some performance considerations, but we can mitigate them one way or another.

1. Technically, Make don’t have a concept of projects, but most developers write their project modelling on top, which are mostly file-based.