Introducing the Dagger GraphQL API

October 24th, 2024 Update - The Dagger Playground mentioned in the blog post has been decommissioned. If you have questions or need help, please contact us on Discord.
_______________________________________________________________________________________________________

Here at Dagger, we are on a mission to make your CI/CD pipelines programmable in the language of your choice. In the last few weeks, we have made great progress towards that goal by releasing SDKs for Go, Python, Javascript and Typescript in rapid succession. So rapid in fact, that some of you wondered how exactly we managed to pull it off!

Here’s the answer: we were cheating. Our SDKs don’t actually run your pipelines! Instead they send pipeline definitions to the Dagger Engine, which runs them and returns the result in a language-agnostic way.

Under the hood, we use a powerful API technology called GraphQL. And as of today, so can you. We are excited to publicly release the Dagger GraphQL API: a unified interface for programming the Dagger Engine using any standard GraphQL client.

This release includes:

• API documentation, including guides, examples and a complete reference.

• An API playground to run pipelines straight from your browser!

• A new CLI to run pipelines directly from the command-line

What does this mean for me?

First, and most importantly, if you are currently using a Dagger SDK, you can safely ignore this announcement and continue using Dagger exactly as before.

However, this release creates exciting new possibilities for the Dagger platform:

1. Developers can now use Dagger to program their CI/CD pipelines in over 20 languages, thanks to GraphQL’s large and growing list of client implementations. Note that the experience of using a standard GraphQL client is not the same as an official Dagger SDK. This is because “vanilla” clients follow the current state of the art in GraphQL development, which requires developers to write their queries manually (ideally with the help of an interactive playground). Dagger SDKs on the other hand automatically generate client libraries to hide the underlying GraphQL language, for a more native and polished experience. Developers that already know and love GraphQL may prefer the “vanilla” method; others will prefer official SDKs when they are available. Besides the aesthetic preferences, both are equally robust and full-featured. Going forward, we will continue to expand the list of available SDKs, and make it easier for the community to contribute new SDKs.

2. Devops engineers can use the new CLI to run pipelines on-the-fly from the terminal or a shell script. This can be used for experimentation, troubleshooting, and to retrofit Dagger pipelines into any system that can run shell commands, including: Makefiles, Jenkinsfiles, Github workflows, Gitlab pipelines, CircleCI configurations, Ansible playbooks, and many more.

3. There are many ways to develop a Dagger pipeline, but only one way to run them: by sending GraphQL queries to the Dagger Engine. This gives infrastructure engineers a natural insertion point to control every aspect of how pipelines are run, regardless of how they were developed. Whether it’s engine provisioning, logging, monitoring, authentication, authorization, networking, artifact management, or any other aspect of a robust and secure CI infrastructure: Dagger’s architecture allows for all of it to be configured or overridden by the operator. This will be the primary focus of our roadmap in the coming weeks.

Why GraphQL?

Engineers are generally not known for agreeing on things - we like to argue! But one thing we all seem to agree on is that the CI/CD experience is often frustrating, and should be improved. But how?

We believe the answer is to decouple pipeline logic from pipeline infrastructure, which in turn allows decoupling the concerns of application and infrastructure teams. This can be achieved with a powerful architectural pattern called the hourglass model: the same pattern that enabled the Internet, Unix, and more recently, Docker.

To apply the hourglass pattern to CI/CD we need an API expressive enough to connect the two parts of the hourlgass: run any pipeline logic, on any pipeline infrastructure - and do it well.

That is where GraphQL comes in. GraphQL is a perfect fit for the CI/CD “hourglass”, for three reasons:

1. Technical: GraphQL is fundamentally a graph traversal language. And it turns out that pipeline execution is fundamentally a graph traversal problem - it’s no coincidence that “Dagger” starts with “DAG”! Our early prototypes ended up looking so much like GraphQL, that it quickly became obvious that we should just build on top of it instead of reinventing the wheel.

2. Human: GraphQL was designed to decouple concerns between frontend and backend teams, by enabling frontend teams to compose queries themselves from building blocks provided by the backend team. Meanwhile, Dagger seeks to decouple concerns between application and infrastructure teams, by enabling application teams to compose pipelines themselves from building blocks provided by the infrastructure team. In both case, decoupling of concerns allows everyone to be happier and more productive.

3. Ecosystem: no one company can solve this problem alone. We are already standing on the shoulders of giants by leveraging the container ecosystem. Thanks to the GraphQL ecosystem we are now standing on the shoulders of TWO GIANTS! Every time anyone in the GraphQL and container ecosystem solves a problem for themselves, they are also solving it for everyone in the Dagger ecosystem.

Ready to give it a try? You can follow our introduction guide, browse the API reference, or experiment with the API playground.

As always, we love hearing from you, let us know how you enjoy using the API, and what cool things you build with it.

Happy coding!