OpenMeter is an open-source metering service that helps engineers to monetize consumption data. Under the hood OpenMeter uses stream processing and combines the power of Go, Kafka, and ClickHouse to provide scalable data ingestion and rapid usage aggregations.
Key technologies:
Go, Github Actions, Depot.dev Runners
How do they use Dagger?
OpenMeter wanted a toolchain that eliminated push and pray for developers, made it easy to create standardized environments, and was compatible with DRY (Don't Repeat Yourself) principles so that they could quickly leverage existing pipelines to create new ones:
"At OpenMeter, we use Dagger to run our programmable CI/CD pipelines in containers. The process includes building and publishing various types of artifacts like Docker images to OCI-compliant registries (GitHub Container registry, etc.)"
How can I learn more?
Read a case study
Read about accelerating CI on OpenMeter's Blog
Take a look at their Go module in the Daggerverse
Airbyte is an open-source data integration platform designed to help you consolidate data from various sources into your data warehouses, lakes, and databases.
Key technologies:
Python, Github Actions
How do they use Dagger?
Airbyte has an extensive ecosystem of 350+ containerized connectors, all of which need to be tested regularly. On average, Airbyte publishes 17 connectors per day and tests 100+ connectors per night, both as part of regularly scheduled testing cycles and in response to incoming pull requests.
Airbyte not only leverages Dagger internally but also enables connector contributors to get a consistent testing environment both locally and in CI thanks to Dagger. "We decided on Dagger because of the expressiveness of being able to do things completely in code. Being able to write things in a more simple, imperative style makes it a lot easier to understand and test."
How can I learn more?
Take a look at Airbyte's Daggerized CI on Github
Download a Case Study on Airbyte
Learn how they test connectors: blog post on Airbyte.com
Flipt is an open-source, self-hosted, feature flag application that enables a team to follow modern DevOps best practices such as trunk-based development, continuous delivery, and software kill switches all without needing to restart the applications. Flipt Cloud is a fully managed feature flag solution built on top of Flipt open source.
Key technologies:
Go, Github Actions
How do they use Dagger?
The Flipt team currently uses Dagger for both build and test phases of their pipelines, handing off to GoReleaser for the release phase.
In the build phase, the Dagger pipeline downloads the application source code, adds required dependencies and build tools, and builds the application. The end result of the build phase is two artifacts: a "test" image with the source code plus build and test dependencies, and a "production" image optimized for distribution.
In the test phase, the Dagger pipeline runs Flipt's complete suite of unit and integration tests, validating the test image against a common set of Flipt operations. In addition to performing complete end-to-end API testing, the pipeline also performs additional tests related to importing, validating and exporting feature flag configurations. These tests are performed against a variety of databases (including Postgres, MySQL and CockroachDB).
How can I learn more?
Check out their Go module
Read a Case Study on Flipt's use of Dagger
The Prometheus project is an open-source monitoring and alerting toolkit.
Prometheus client_golang is the official Go client library for Prometheus, enabling Go applications to instrument and expose metrics for monitoring. This library provides Go-specific tools for creating custom metrics, such as counters, gauges, histograms, and summaries, which Prometheus can scrape
Key technologies:
Go, Github Actions
How do they use Dagger?
The Golang client team was looking for "vendor-independent and locally reproducible CI/CD pipelines so that our contributors can ensure quality before opening the pull requests." One challenge is that they test with several Go versions. Like many users while they were curious about using Dagger, which seemed to fit their needs, they were wary of embarking on a huge all-encompassing change. They decided to take an incremental approach, Daggerizing a few simpler pipelines first, getting feedback from developers and expanding from there.
How can I learn more?
Look at a simple Daggerized Golangci-lint pipeline
Examine the full Prometheus Golang client main pipeline
Tracetest is a trace-based testing tool for integration and end-to-end testing using OpenTelemetry traces. Tracetest allows you to verify end-to-end transactions and side-effects across microservices & event-driven apps by using trace data as test specs.
Key technologies:
Go
How do they use Dagger?
Tracetest was looking for a better way to onboard their users given that it previously required them to install multiple tools and go through a lengthy and YAML based flow. Additionally, they were also eager to give their users the ability to run their product in a more controlled and unified way across different providers as each of them has its own custom settings that they needed to maintain and test each time they performed a release.
Dagger Modules enabled them to not only deliver a better onboarding flow to their users, but also to encapsulate all the complexity into reusable functions that can now be consistently distributed across multiple runtime environments.
How can I learn more?
Checkout this Tracetest module to run assertions in a pipeline
Take a look at the GitHub Actions definition
A swiss army knife of container debugging:
Troubleshoot containers and pods lacking shell and/or debugging tools.
Forward unpublished or even localhost ports to your host system.
Expose endpoints from the host system to containers & Kubernetes networks.
Handily export image's and/or container's filesystem to local folders.
Key technologies:
Go, Github Actions
How do they use Dagger?
Like many other infra related projects, cdebug was having a hard time coming up with automated tests as it relies on low level infrastructure dependencies like Containerd and Kubernetes which are not usually straightforward to setup reliably in a CI/CD pipeline. All the testing work had to be done manually in the developer's machine, which was error prone and directly affected the speed of iteration of the overall project.
Leveraging Dagger Services, they were able to structure their tests in a way that they can now run complex scenarios both locally and in CI with confidence and delivering the required performance
How can I learn more?
Look cdebug's main Dagger pipeline
Examine cdebug's GitHub Actions definition