Introducing Dagger Functions

Developers use Dagger to transform their messy CI scripts into clean code. This makes their CI simpler, faster, more reliable, more portable - and more fun to write.

Today we’re taking the fun to a new level, with a feature we call Dagger Functions, available in Dagger 0.10.

If you want to jump in directly, get started! Otherwise, read on for the details.

What are Dagger Functions?

Dagger Functions are a core feature of the Dagger Engine, which we are expanding and making available to all developers.

At the heart of Dagger is a next-generation container engine controlled by a GraphQL API. Using this API, a workflow that once required hundreds of lines of bash and YAML can be distilled to a few dozen lines of Go, Python or Typescript. How can so little code accomplish so much? By calling Dagger Functions.

Functions are the fundamental unit of computing in the Dagger Engine. Each core engine operation, for example “pull a container image”, “copy a file”, “forward a TCP port”, is exposed to API clients as a function that they can call. Calling functions over HTTP is a well-known way to standardize API clients across many programming languages, but Dagger adds a crucial innovation: instead of calling one function at a time, clients can use the GraphQL query language to chain multiple functions together into a dynamic pipeline, with a single query.

Thanks to this combination of powerful core functions, a GraphQL API to compose them, and native SDKs to do so in a familiar language, Dagger has made CI-as-code a reality.

Today we are expanding this core capability of the engine in three major ways:

• First, you can now write your own Dagger Functions - effectively making the Dagger API infinitely extensible.

• Second, you can package your Dagger Functions into reusable modules - unlocking a whole new world of cross-language collaboration for the devops community

• Third, you can now call Dagger Functions directly from the CLI - making them immediately usable without having to write code

Write your own Dagger Functions

Dagger lets you distill complex CI into clean code, by querying the Dagger API, and composing its core functions into powerful pipelines. But wouldn’t it be great if you could also encapsulate your own code into Dagger Functions, to be composed into even more powerful pipelines? Imagine if Dagger’s GraphQL API and native SDKs could magically be used to call your functions from any language?

As of Dagger 0.10, all of this is now possible.

Dagger Functions are just regular code, written in your usual language using a Dagger SDK. You can find developer guides for writing Dagger Functions in Go, Python and TypeScript, with more coming soon. At runtime, the Dagger Engine compiles each of your functions into a special container, and exposes a custom GraphQL API for calling them. Clients can compose your functions into dynamic pipelines, just like with core functions.

Functions can call other functions, across languages. For example, a Python function can call a Go function, which can call a TypeScript function, and so on. Each Dagger SDK generates native code-bindings for all dependencies, which abstract away the underlying GraphQL queries. This gives you all the benefits of type-checking, code completion and other IDE features when developing Dagger Functions.

Here’s an example of a simple Dagger Function written in TypeScript.

@func()
hello(name: string): Promise<string> {
    return dag.container()
        .from("alpine:latest")
        .withExec(["echo", `Hello ${name}!`])
        .stdout()
  }

Here are 3 Dagger Functions, written in Go, calling each other to run integration tests in a Drupal application, using the “testcontainers” pattern.

func (m *Drupal) Base() *Container {
    return dag.Container().
        From("drupal:10.2.3-php8.3-fpm").
        WithExec([]string{"composer", "require", "drupal/core-dev", "--dev", "--update-with-all-dependencies"})
}

func (m *MariaDb) AsService() *Service {
    return dag.Container().
        From("mariadb:10.11.4").
        WithEnvVariable("MARIADB_USER", "user").
        WithEnvVariable("MARIADB_PASSWORD", "password").
        WithEnvVariable("MARIADB_DATABASE", "drupal").
        WithEnvVariable("MARIADB_ROOT_PASSWORD", "root").
        WithExposedPort(3306).
        AsService()
}

func (m *DrupalTest) Run(ctx context.Context) (string, error) {
    drupal := dag.Drupal().Base()

    mariaSvc := dag.MariaDb().AsService()

    test, err := drupal.
        WithServiceBinding("db", mariaSvc).
        WithEnvVariable("SIMPLETEST_DB", "mysql://user:password@db/drupal").
        WithEnvVariable("SYMFONY_DEPRECATIONS_HELPER", "disabled").
        WithWorkdir("/opt/drupal/web/core").
        WithExec([]string{"../../vendor/bin/phpunit", "-v", "--group", "KernelTests"}).
        Stdout(ctx)

    if err != nil {
        return "", err
    }
    return test, err

Here’s a more abstract example, where a Dagger Function written in Python calls both core functions, and functions from other modules written in Go!

    @function
    async def ci(self, source: dagger.Directory) -> str:
        # Use Golang module to configure project
        go_project = dag.golang().with_project(source)

        # Run Go tests using Golang module
        await go_project.test()

        # Get container with built binaries using Golang module
        image = await go_project.build_container()

        # Push image to a registry using core Dagger API
        ref = await image.publish("ttl.sh/demoapp:1h")

        # Scan image for vulnerabilites using Trivy module
        return await dag.trivy().scan_container(dag.container().from_(ref))

Reusable Modules

Dagger Functions are designed to be safely reusable. This requires a way to package your functions and share them with others. Enter Dagger Modules.

Dagger Modules are simply a collection of Dagger Functions, packaged together for easy sharing and consumption. Modules are language-agnostic, but their design is inspired by Go modules:

• Modules are just source code. Binary artifacts are built locally, and aggressively cached.

• Git is the source of truth. If you can host a git repository, you can host a Dagger Module.

• No central registry. Git is decentralized by nature, and so is the “Daggerverse”, as we like to call the decentralized ecosystem of Dagger Modules.

• No name squatting. Since there is no registry, there is no reserved name. The address of your module’s source code is its authoritative name.

• Dependencies are pinned by default. So you can trust that the version you installed is the version that will run.

• Semantic versioning using git tags. If you can tag a git repository, you can publish new versions of your module.

Other useful features of Dagger Modules include:

• No dependency hell. Since Dagger Functions are containerized, their dependencies are naturally scoped. Different modules can require different versions of the same dependency, and everything will just work.

• First-class monorepo support. Dagger is agnostic to repository layout, and any number of Dagger Modules can peacefully coexist in a monorepo. It’s up to you how to organize your module’s source code in git. Some like to publish each module as a dedicated repository; others like to organize all their modules together, with the git repository acting as a “catalog”. These repositories are often named “daggerverse” by convention. For functions that are specific to an application, it’s common to embed a module directly in a subdirectory of the application repository. 

Because modules are reusable across languages, we believe they will unlock a kind of software ecosystem for the devops community, where best practices can be shared widely, pipelines are easier to improve and refactor, and software teams can standardize their platform across language silos.

Call Functions from the CLI

So you’ve replaced that 500-line franken-CI with a clean Dagger Function. Wonderful! Now all you need to do is call it. Since functions can call other functions, you can always write a higher-level function, but how will you call that function?

The answer is that all Dagger Functions can be called from the CLI. The dagger tool can call any function from any module, either from your local filesystem or straight from a git repository. No other infrastructure is required. It can be used interactively, from a shell script, or from a CI configuration.

The Dagger CLI introspects the target module’s GraphQL API, and exposes all available functions and their arguments as subcommands and flags, respectively. All of it can be explored with the familiar --help 

For example, this command will call a function called “hello” from a remote repository:

dagger -m github.com/shykes/daggerverse/hello@v0.1.2 call hello --giant --name

This command will scan a container image for vulnerabilities using Trivy:

dagger -m github.com/jpadams/daggerverse/trivy@v0.3.0 call scan-image --image-ref

This command will build a container with Wolfi Linux, and open an interactive terminal into it:

dagger -m github.com/shykes/daggerverse/wolfi@v0.1.2 call container --packages

Notice that you didn’t need to install any dependencies on your local machine. Dagger did it all for you with one single command.

Beyond CI?

Over several months of testing these features with the community, we noticed something interesting: Dagger Functions are great for CI, but they can be applied to other problems as well. From test data management, to local AI development, to dev environments, to just-in-time artifacts, to SaaS integration, to OpenAI development… The world is full of pipelines, just waiting to be Daggerized. We look forward to seeing what you build next!

Get Started!

There’s never been a better time to Daggerize your project! Remember, Dagger is designed to meet you where you are, so you don’t have to do it all at once. Just pick a script that causes pain, and write a Dagger Function to replace it. No need to change your development tools, CI platform, or anything else.

If you’re new to Dagger, the best place to start is with our Quickstart Guide. If you’re an experienced Daggernaut already familiar with our SDKs, take our module developer guide here.

Make sure to join our Discord server! It is the beating heart of our community, where Daggernauts from around the world share ideas and help each other. We love answering questions and are always happy to help.

To our amazing Daggernauts: thank you for your support. We couldn’t have shipped this release without the time and energy you invested with us, every step of the way. We appreciate you!

The Dagger team