Dagger 0.14: Git credential helpers, exit code API, OCI annotations, and more
November 12, 2024
Nov 12, 2024
None
Today we are introducing version 0.14 of the Dagger Engine. In this release we introduce a powerful new way to authenticate against your existing Git infrastructure; an API to better integrate with test tooling; better OCI interop; a more flexible networking model; performance improvements; CPU metrics; public traces; and more.
Let’s dive in!
Git credential helpers
Dagger already supported loading modules from private Git servers, making it easier for engineering teams to leverage Dagger in proprietary codebases. With Dagger 0.14, we take this to the next level, with native support for Git's native credential management system, known as “Git credential helpers.” This standard is ubiquitous in enterprise organizations, and Dagger now supports it out-of-the-box. Whether you use Personal Access Tokens (PATs), Git Credential Manager, or other authentication methods, Dagger can now load modules from any Git repository that you have access to, with no additional configuration.
Here’s a video showing how to do this on GitHub using the official Git Credential Manager:
Exit code API
With this release, Dagger introduces an API for better handling the exit code of commands executed in containers. This is especially useful when executing tests, or other commands that are expected to fail. This new API makes it easier to process test failures in a custom way - for example, to detect and retry flaky tests, or upload a test report.
The exit code API has two parts: a way to specify expected exit codes before execution, and a way to retrieve the exit code after execution.
To specify expected exit codes,
Container.withExechas a new optional argument:expectTo retrieve the exit code,
Container.exitCodecan be called after executing the command
By defining expected return codes, you can manage command outcomes with greater flexibility, simplifying error handling and enabling custom workflows that depend on non-standard exit codes. Additionally, the new Container.exitCode function lets you capture and inspect the exact exit code of a command, making it particularly useful for generating reports from commands that might fail. This powerful combination provides finer control over execution scenarios and error management.
Here’s an example, using Go, of how to use expect and retrieve the exit code:
package main
import (
"context"
"dagger.io/dagger"
"fmt"
)
func main() {
ctx := context.Background()
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
// Run a command with a custom expectation for its exit status
container := client.Container().
From("alpine:latest").
WithExec([]string{"sh", "-c", "exit 42"}, dagger.ContainerWithExecOpts{
Expect: dagger.ReturnTypeAny,
})
// Get the precise exit code
exitCode, err := container.ExitCode(ctx)
if err != nil {
panic(err)
}
// Output the exit code, potentially useful for reporting
fmt.Printf("Command exited with code: %d\n", exitCode
In this example, expect is set to allow any exit codes, so the command won’t trigger an error even if it exits with 42. By calling ExitCode, you can retrieve the exact exit code of the command, which is useful for logging or generating reports from commands that might otherwise be treated as failures.
The PR can be found here: https://github.com/dagger/dagger/pull/8466
OCI annotations
Dagger now supports setting custom OCI annotations on container images, offering developers greater control over image metadata. With the new Container.withAnnotation API, you can add standard metadata fields like authors (org.opencontainers.image.authors) and titles (org.opencontainers.image.title), as well as custom key-value pairs to tag built images. This flexibility enhances the traceability and organization of images, making it easier to manage metadata across environments.
Additionally, this functionality has been extended to Container.Export and Container.AsTarball, ensuring that annotations are preserved when images are exported as OCI tarballs. This comprehensive support for annotations provides a streamlined way to embed metadata directly within images for more robust workflow management.
Example:
package main
import (
"context"
"fmt"
"math"
"math/rand/v2"
)
type CustomImage struct{}
func (m *CustomImage) Build(ctx context.Context) (string, error) {
address, err := dag.Container().
From("alpine:latest").
WithExec([]string{"apk", "add", "git"}).
WithWorkdir("/src").
WithExec([]string{"git", "clone", "https://github.com/dagger/dagger", "."}).
WithAnnotation("org.opencontainers.image.authors", "John Doe").
WithAnnotation("org.opencontainers.image.title", "Dagger source image viewer").
Publish(ctx, fmt.Sprintf("ttl.sh/custom-image-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
if err != nil {
return "", err
}
return address, nil
$ dagger call build
✔ connect 0.2s
✔ initialize 4.0s
✔ prepare 0.0s
✔ customImage: CustomImage! 0.0s
✔ CustomImage.build: String! 1m7.9s
ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
$ docker buildx imagetools inspect --raw ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:a8ef33da285c3b91ceb6144afd0b3ee99ef4e80eb7ed88142918a21895efdef0",
"size": 992
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:43c4264eed91be63b206e17d93e75256a6097070ce643c5e8f0379998b44f170",
"size": 3623807
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:73064da859e9304bec448cc2bddd43c9a2645e6a548f8657792116fb476d17bc",
"size": 8495103
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:88d251340f12d6616bcb2f6a1af5acb35e864fee2bbe5dba6d7271031d8a7acb",
"size": 110909343
}
],
"annotations": {
"org.opencontainers.image.authors": "John Doe",
"org.opencontainers.image.title": "Dagger source image viewer"
This example builds a custom container image with Dagger, starting from an Alpine base, adding Git, and cloning the Dagger repository. It then sets metadata annotations for the author and title using Container.withAnnotation and publishes the image to ttl.sh. The docker buildx imagetools inspect command confirms the annotations, showing how metadata can be embedded directly within the image for improved traceability.
The PRs can be found here:
Other improvements
Global service hostnames
Developers can now assign global hostnames to their services. This feature is especially valuable for complex networking configurations, such as circular dependencies between services, by allowing services to reference each other by predefined hostnames, without requiring an explicit service binding. Custom hostnames follow a structured format (<host>.<module id>.<session id>.dagger.local), ensuring unique identifiers across modules and sessions. This addition provides enhanced flexibility for managing service-to-service communication, particularly in modular, interconnected workflows.
For example, you can now run two services that depend on each other, each using WithHostname to refer to the other by name:
func (m *Lala) Test(ctx context.Context) (*dagger.Service, error) {
svcA := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svcb:80 && sleep 1; done
`}).
AsService().WithHostname("svca")
_, err := svcA.Start(ctx)
if err != nil {
return nil, err
}
svcB := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svca:80 && sleep 1; done
`}).
AsService().WithHostname("svcb")
svcB, err = svcB.Start(ctx)
if err != nil {
return nil, err
}
return svcB, nil
In this example, svcA and svcB are set up with custom hostnames svca and svcb, allowing each service to communicate with the other by hostname. This capability provides enhanced flexibility for managing service dependencies and interconnections within modular workflows, making it easier to handle complex setups in Dagger.
For more information, check out the documentation here.
Support for public traces
Dagger has introduced Public Traces for open-source repositories, allowing project teams to share trace data from Dagger pipelines publicly. This enables you to give community contributors visibility into execution steps, errors, and caching states, simplifying troubleshooting and collaboration on CI issues. By sharing trace URLs, developers can review detailed pipeline data without requiring special permissions. You can read all the details here.
Performance improvements
Performance has been improved in a few areas, with more coming soon in subsequent releases.
Module `initialize` is faster when the engine starts with an empty cache
The engine skips registry lookups for locally cached images that are pinned with a digest
New disk usage policy
We’ve improved how Dagger manages its disk usage by default. The old policy was “use 75% of the disk.” The new policy is:
Always allow use of 10% or 10GB (whichever is smaller, to handle disks that are very small, like docker-desktop's defaults) - Dagger's cache will never be forced to reduce below this amount
Never consume more than 75% of the total disk size - Dagger's cache will also be gc-ed at this amount
Try to keep 20% of the disk free at all times - but never go below the reserved storage of 10%/10GB.
You can see the PR here: https://github.com/dagger/dagger/pull/8725
New file structure when creating a Python module
Dagger’s Python SDK modules can now use custom package names instead of being limited to “main.” With this update, developers have more flexibility in organizing module structures and can specify the exact location of the main object via entry points in pyproject.toml. This change enhances modularity, letting users customize their Python modules’ layout and import paths, making it easier to manage complex module configurations within Dagger. Existing modules are unaffected by this change, preserving backward compatibility.
The PR can be found here: https://github.com/dagger/dagger/pull/8709
Metrics in the CLI
We have added CPU stat and pressure metrics to the Dagger Engine, enabling the monitoring of CPU resource usage and contention within containers. These metrics help developers identify when CPU contention occurs, such as when all threads are blocked waiting for CPU access, which can impact performance. Initially, only CPU pressure time is displayed in the TUI (text user interface), but other CPU usage data is also available. This feature aids in diagnosing performance bottlenecks related to CPU resource constraints.
The PR can be found here: https://github.com/dagger/dagger/pull/8750
Module references are now stored unchanged in dagger.json
We have added module source pins to Dagger’s configuration, allowing dependencies to retain both their original source (e.g. main) and a specific commit hash as a pin for reference. This update enhances version control by preserving the exact version of each module installed with dagger install, ensuring reproducibility in builds. This change is backward-compatible, as it keeps the existing source field unchanged for older configurations.
The PR can be found here: https://github.com/dagger/dagger/pull/8587
Scoped enum values in Go
We have added scoped enum values in the Go SDK to prevent naming conflicts between different enums that may share the same value names. Previously, unscoped enum values could easily clash due to Go’s top-level package structure. With this change, enums are now prefixed with their type, following standard Go practices.
Enum values should now be accessed with the name prefixed by the name of the enum type - for example, Locked becomes CacheSharingModeLocked. The old unscoped constants are still available, but marked as deprecated, and are set to be removed in version 0.15.0.
The PR can be found here:https://github.com/dagger/dagger/pull/8669
What’s next?
Thanks to our amazing community of Daggernauts, we continue to see a lot of great feedback, bug reports, and feature requests. Keep it coming!
If you have requests, feedback, or want to contribute, don’t hesitate to join our Discord server.
Thank you for your support, as we continue our mission to transform the world of CI with you!
The Dagger team
Today we are introducing version 0.14 of the Dagger Engine. In this release we introduce a powerful new way to authenticate against your existing Git infrastructure; an API to better integrate with test tooling; better OCI interop; a more flexible networking model; performance improvements; CPU metrics; public traces; and more.
Let’s dive in!
Git credential helpers
Dagger already supported loading modules from private Git servers, making it easier for engineering teams to leverage Dagger in proprietary codebases. With Dagger 0.14, we take this to the next level, with native support for Git's native credential management system, known as “Git credential helpers.” This standard is ubiquitous in enterprise organizations, and Dagger now supports it out-of-the-box. Whether you use Personal Access Tokens (PATs), Git Credential Manager, or other authentication methods, Dagger can now load modules from any Git repository that you have access to, with no additional configuration.
Here’s a video showing how to do this on GitHub using the official Git Credential Manager:
Exit code API
With this release, Dagger introduces an API for better handling the exit code of commands executed in containers. This is especially useful when executing tests, or other commands that are expected to fail. This new API makes it easier to process test failures in a custom way - for example, to detect and retry flaky tests, or upload a test report.
The exit code API has two parts: a way to specify expected exit codes before execution, and a way to retrieve the exit code after execution.
To specify expected exit codes,
Container.withExechas a new optional argument:expectTo retrieve the exit code,
Container.exitCodecan be called after executing the command
By defining expected return codes, you can manage command outcomes with greater flexibility, simplifying error handling and enabling custom workflows that depend on non-standard exit codes. Additionally, the new Container.exitCode function lets you capture and inspect the exact exit code of a command, making it particularly useful for generating reports from commands that might fail. This powerful combination provides finer control over execution scenarios and error management.
Here’s an example, using Go, of how to use expect and retrieve the exit code:
package main
import (
"context"
"dagger.io/dagger"
"fmt"
)
func main() {
ctx := context.Background()
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
// Run a command with a custom expectation for its exit status
container := client.Container().
From("alpine:latest").
WithExec([]string{"sh", "-c", "exit 42"}, dagger.ContainerWithExecOpts{
Expect: dagger.ReturnTypeAny,
})
// Get the precise exit code
exitCode, err := container.ExitCode(ctx)
if err != nil {
panic(err)
}
// Output the exit code, potentially useful for reporting
fmt.Printf("Command exited with code: %d\n", exitCode
In this example, expect is set to allow any exit codes, so the command won’t trigger an error even if it exits with 42. By calling ExitCode, you can retrieve the exact exit code of the command, which is useful for logging or generating reports from commands that might otherwise be treated as failures.
The PR can be found here: https://github.com/dagger/dagger/pull/8466
OCI annotations
Dagger now supports setting custom OCI annotations on container images, offering developers greater control over image metadata. With the new Container.withAnnotation API, you can add standard metadata fields like authors (org.opencontainers.image.authors) and titles (org.opencontainers.image.title), as well as custom key-value pairs to tag built images. This flexibility enhances the traceability and organization of images, making it easier to manage metadata across environments.
Additionally, this functionality has been extended to Container.Export and Container.AsTarball, ensuring that annotations are preserved when images are exported as OCI tarballs. This comprehensive support for annotations provides a streamlined way to embed metadata directly within images for more robust workflow management.
Example:
package main
import (
"context"
"fmt"
"math"
"math/rand/v2"
)
type CustomImage struct{}
func (m *CustomImage) Build(ctx context.Context) (string, error) {
address, err := dag.Container().
From("alpine:latest").
WithExec([]string{"apk", "add", "git"}).
WithWorkdir("/src").
WithExec([]string{"git", "clone", "https://github.com/dagger/dagger", "."}).
WithAnnotation("org.opencontainers.image.authors", "John Doe").
WithAnnotation("org.opencontainers.image.title", "Dagger source image viewer").
Publish(ctx, fmt.Sprintf("ttl.sh/custom-image-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
if err != nil {
return "", err
}
return address, nil
$ dagger call build
✔ connect 0.2s
✔ initialize 4.0s
✔ prepare 0.0s
✔ customImage: CustomImage! 0.0s
✔ CustomImage.build: String! 1m7.9s
ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
$ docker buildx imagetools inspect --raw ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:a8ef33da285c3b91ceb6144afd0b3ee99ef4e80eb7ed88142918a21895efdef0",
"size": 992
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:43c4264eed91be63b206e17d93e75256a6097070ce643c5e8f0379998b44f170",
"size": 3623807
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:73064da859e9304bec448cc2bddd43c9a2645e6a548f8657792116fb476d17bc",
"size": 8495103
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:88d251340f12d6616bcb2f6a1af5acb35e864fee2bbe5dba6d7271031d8a7acb",
"size": 110909343
}
],
"annotations": {
"org.opencontainers.image.authors": "John Doe",
"org.opencontainers.image.title": "Dagger source image viewer"
This example builds a custom container image with Dagger, starting from an Alpine base, adding Git, and cloning the Dagger repository. It then sets metadata annotations for the author and title using Container.withAnnotation and publishes the image to ttl.sh. The docker buildx imagetools inspect command confirms the annotations, showing how metadata can be embedded directly within the image for improved traceability.
The PRs can be found here:
Other improvements
Global service hostnames
Developers can now assign global hostnames to their services. This feature is especially valuable for complex networking configurations, such as circular dependencies between services, by allowing services to reference each other by predefined hostnames, without requiring an explicit service binding. Custom hostnames follow a structured format (<host>.<module id>.<session id>.dagger.local), ensuring unique identifiers across modules and sessions. This addition provides enhanced flexibility for managing service-to-service communication, particularly in modular, interconnected workflows.
For example, you can now run two services that depend on each other, each using WithHostname to refer to the other by name:
func (m *Lala) Test(ctx context.Context) (*dagger.Service, error) {
svcA := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svcb:80 && sleep 1; done
`}).
AsService().WithHostname("svca")
_, err := svcA.Start(ctx)
if err != nil {
return nil, err
}
svcB := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svca:80 && sleep 1; done
`}).
AsService().WithHostname("svcb")
svcB, err = svcB.Start(ctx)
if err != nil {
return nil, err
}
return svcB, nil
In this example, svcA and svcB are set up with custom hostnames svca and svcb, allowing each service to communicate with the other by hostname. This capability provides enhanced flexibility for managing service dependencies and interconnections within modular workflows, making it easier to handle complex setups in Dagger.
For more information, check out the documentation here.
Support for public traces
Dagger has introduced Public Traces for open-source repositories, allowing project teams to share trace data from Dagger pipelines publicly. This enables you to give community contributors visibility into execution steps, errors, and caching states, simplifying troubleshooting and collaboration on CI issues. By sharing trace URLs, developers can review detailed pipeline data without requiring special permissions. You can read all the details here.
Performance improvements
Performance has been improved in a few areas, with more coming soon in subsequent releases.
Module `initialize` is faster when the engine starts with an empty cache
The engine skips registry lookups for locally cached images that are pinned with a digest
New disk usage policy
We’ve improved how Dagger manages its disk usage by default. The old policy was “use 75% of the disk.” The new policy is:
Always allow use of 10% or 10GB (whichever is smaller, to handle disks that are very small, like docker-desktop's defaults) - Dagger's cache will never be forced to reduce below this amount
Never consume more than 75% of the total disk size - Dagger's cache will also be gc-ed at this amount
Try to keep 20% of the disk free at all times - but never go below the reserved storage of 10%/10GB.
You can see the PR here: https://github.com/dagger/dagger/pull/8725
New file structure when creating a Python module
Dagger’s Python SDK modules can now use custom package names instead of being limited to “main.” With this update, developers have more flexibility in organizing module structures and can specify the exact location of the main object via entry points in pyproject.toml. This change enhances modularity, letting users customize their Python modules’ layout and import paths, making it easier to manage complex module configurations within Dagger. Existing modules are unaffected by this change, preserving backward compatibility.
The PR can be found here: https://github.com/dagger/dagger/pull/8709
Metrics in the CLI
We have added CPU stat and pressure metrics to the Dagger Engine, enabling the monitoring of CPU resource usage and contention within containers. These metrics help developers identify when CPU contention occurs, such as when all threads are blocked waiting for CPU access, which can impact performance. Initially, only CPU pressure time is displayed in the TUI (text user interface), but other CPU usage data is also available. This feature aids in diagnosing performance bottlenecks related to CPU resource constraints.
The PR can be found here: https://github.com/dagger/dagger/pull/8750
Module references are now stored unchanged in dagger.json
We have added module source pins to Dagger’s configuration, allowing dependencies to retain both their original source (e.g. main) and a specific commit hash as a pin for reference. This update enhances version control by preserving the exact version of each module installed with dagger install, ensuring reproducibility in builds. This change is backward-compatible, as it keeps the existing source field unchanged for older configurations.
The PR can be found here: https://github.com/dagger/dagger/pull/8587
Scoped enum values in Go
We have added scoped enum values in the Go SDK to prevent naming conflicts between different enums that may share the same value names. Previously, unscoped enum values could easily clash due to Go’s top-level package structure. With this change, enums are now prefixed with their type, following standard Go practices.
Enum values should now be accessed with the name prefixed by the name of the enum type - for example, Locked becomes CacheSharingModeLocked. The old unscoped constants are still available, but marked as deprecated, and are set to be removed in version 0.15.0.
The PR can be found here:https://github.com/dagger/dagger/pull/8669
What’s next?
Thanks to our amazing community of Daggernauts, we continue to see a lot of great feedback, bug reports, and feature requests. Keep it coming!
If you have requests, feedback, or want to contribute, don’t hesitate to join our Discord server.
Thank you for your support, as we continue our mission to transform the world of CI with you!
The Dagger team
Today we are introducing version 0.14 of the Dagger Engine. In this release we introduce a powerful new way to authenticate against your existing Git infrastructure; an API to better integrate with test tooling; better OCI interop; a more flexible networking model; performance improvements; CPU metrics; public traces; and more.
Let’s dive in!
Git credential helpers
Dagger already supported loading modules from private Git servers, making it easier for engineering teams to leverage Dagger in proprietary codebases. With Dagger 0.14, we take this to the next level, with native support for Git's native credential management system, known as “Git credential helpers.” This standard is ubiquitous in enterprise organizations, and Dagger now supports it out-of-the-box. Whether you use Personal Access Tokens (PATs), Git Credential Manager, or other authentication methods, Dagger can now load modules from any Git repository that you have access to, with no additional configuration.
Here’s a video showing how to do this on GitHub using the official Git Credential Manager:
Exit code API
With this release, Dagger introduces an API for better handling the exit code of commands executed in containers. This is especially useful when executing tests, or other commands that are expected to fail. This new API makes it easier to process test failures in a custom way - for example, to detect and retry flaky tests, or upload a test report.
The exit code API has two parts: a way to specify expected exit codes before execution, and a way to retrieve the exit code after execution.
To specify expected exit codes,
Container.withExechas a new optional argument:expectTo retrieve the exit code,
Container.exitCodecan be called after executing the command
By defining expected return codes, you can manage command outcomes with greater flexibility, simplifying error handling and enabling custom workflows that depend on non-standard exit codes. Additionally, the new Container.exitCode function lets you capture and inspect the exact exit code of a command, making it particularly useful for generating reports from commands that might fail. This powerful combination provides finer control over execution scenarios and error management.
Here’s an example, using Go, of how to use expect and retrieve the exit code:
package main
import (
"context"
"dagger.io/dagger"
"fmt"
)
func main() {
ctx := context.Background()
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
// Run a command with a custom expectation for its exit status
container := client.Container().
From("alpine:latest").
WithExec([]string{"sh", "-c", "exit 42"}, dagger.ContainerWithExecOpts{
Expect: dagger.ReturnTypeAny,
})
// Get the precise exit code
exitCode, err := container.ExitCode(ctx)
if err != nil {
panic(err)
}
// Output the exit code, potentially useful for reporting
fmt.Printf("Command exited with code: %d\n", exitCode
In this example, expect is set to allow any exit codes, so the command won’t trigger an error even if it exits with 42. By calling ExitCode, you can retrieve the exact exit code of the command, which is useful for logging or generating reports from commands that might otherwise be treated as failures.
The PR can be found here: https://github.com/dagger/dagger/pull/8466
OCI annotations
Dagger now supports setting custom OCI annotations on container images, offering developers greater control over image metadata. With the new Container.withAnnotation API, you can add standard metadata fields like authors (org.opencontainers.image.authors) and titles (org.opencontainers.image.title), as well as custom key-value pairs to tag built images. This flexibility enhances the traceability and organization of images, making it easier to manage metadata across environments.
Additionally, this functionality has been extended to Container.Export and Container.AsTarball, ensuring that annotations are preserved when images are exported as OCI tarballs. This comprehensive support for annotations provides a streamlined way to embed metadata directly within images for more robust workflow management.
Example:
package main
import (
"context"
"fmt"
"math"
"math/rand/v2"
)
type CustomImage struct{}
func (m *CustomImage) Build(ctx context.Context) (string, error) {
address, err := dag.Container().
From("alpine:latest").
WithExec([]string{"apk", "add", "git"}).
WithWorkdir("/src").
WithExec([]string{"git", "clone", "https://github.com/dagger/dagger", "."}).
WithAnnotation("org.opencontainers.image.authors", "John Doe").
WithAnnotation("org.opencontainers.image.title", "Dagger source image viewer").
Publish(ctx, fmt.Sprintf("ttl.sh/custom-image-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
if err != nil {
return "", err
}
return address, nil
$ dagger call build
✔ connect 0.2s
✔ initialize 4.0s
✔ prepare 0.0s
✔ customImage: CustomImage! 0.0s
✔ CustomImage.build: String! 1m7.9s
ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
$ docker buildx imagetools inspect --raw ttl.sh/custom-image-4527543@sha256:7cd129081fe0580c57eee12af7936ace72e6babc3c318dabf705ef1739b44e7b
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:a8ef33da285c3b91ceb6144afd0b3ee99ef4e80eb7ed88142918a21895efdef0",
"size": 992
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:43c4264eed91be63b206e17d93e75256a6097070ce643c5e8f0379998b44f170",
"size": 3623807
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:73064da859e9304bec448cc2bddd43c9a2645e6a548f8657792116fb476d17bc",
"size": 8495103
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:88d251340f12d6616bcb2f6a1af5acb35e864fee2bbe5dba6d7271031d8a7acb",
"size": 110909343
}
],
"annotations": {
"org.opencontainers.image.authors": "John Doe",
"org.opencontainers.image.title": "Dagger source image viewer"
This example builds a custom container image with Dagger, starting from an Alpine base, adding Git, and cloning the Dagger repository. It then sets metadata annotations for the author and title using Container.withAnnotation and publishes the image to ttl.sh. The docker buildx imagetools inspect command confirms the annotations, showing how metadata can be embedded directly within the image for improved traceability.
The PRs can be found here:
Other improvements
Global service hostnames
Developers can now assign global hostnames to their services. This feature is especially valuable for complex networking configurations, such as circular dependencies between services, by allowing services to reference each other by predefined hostnames, without requiring an explicit service binding. Custom hostnames follow a structured format (<host>.<module id>.<session id>.dagger.local), ensuring unique identifiers across modules and sessions. This addition provides enhanced flexibility for managing service-to-service communication, particularly in modular, interconnected workflows.
For example, you can now run two services that depend on each other, each using WithHostname to refer to the other by name:
func (m *Lala) Test(ctx context.Context) (*dagger.Service, error) {
svcA := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svcb:80 && sleep 1; done
`}).
AsService().WithHostname("svca")
_, err := svcA.Start(ctx)
if err != nil {
return nil, err
}
svcB := dag.Container().From("nginx").
WithExposedPort(80).
WithExec([]string{"sh", "-c", `
nginx & while true; do curl svca:80 && sleep 1; done
`}).
AsService().WithHostname("svcb")
svcB, err = svcB.Start(ctx)
if err != nil {
return nil, err
}
return svcB, nil
In this example, svcA and svcB are set up with custom hostnames svca and svcb, allowing each service to communicate with the other by hostname. This capability provides enhanced flexibility for managing service dependencies and interconnections within modular workflows, making it easier to handle complex setups in Dagger.
For more information, check out the documentation here.
Support for public traces
Dagger has introduced Public Traces for open-source repositories, allowing project teams to share trace data from Dagger pipelines publicly. This enables you to give community contributors visibility into execution steps, errors, and caching states, simplifying troubleshooting and collaboration on CI issues. By sharing trace URLs, developers can review detailed pipeline data without requiring special permissions. You can read all the details here.
Performance improvements
Performance has been improved in a few areas, with more coming soon in subsequent releases.
Module `initialize` is faster when the engine starts with an empty cache
The engine skips registry lookups for locally cached images that are pinned with a digest
New disk usage policy
We’ve improved how Dagger manages its disk usage by default. The old policy was “use 75% of the disk.” The new policy is:
Always allow use of 10% or 10GB (whichever is smaller, to handle disks that are very small, like docker-desktop's defaults) - Dagger's cache will never be forced to reduce below this amount
Never consume more than 75% of the total disk size - Dagger's cache will also be gc-ed at this amount
Try to keep 20% of the disk free at all times - but never go below the reserved storage of 10%/10GB.
You can see the PR here: https://github.com/dagger/dagger/pull/8725
New file structure when creating a Python module
Dagger’s Python SDK modules can now use custom package names instead of being limited to “main.” With this update, developers have more flexibility in organizing module structures and can specify the exact location of the main object via entry points in pyproject.toml. This change enhances modularity, letting users customize their Python modules’ layout and import paths, making it easier to manage complex module configurations within Dagger. Existing modules are unaffected by this change, preserving backward compatibility.
The PR can be found here: https://github.com/dagger/dagger/pull/8709
Metrics in the CLI
We have added CPU stat and pressure metrics to the Dagger Engine, enabling the monitoring of CPU resource usage and contention within containers. These metrics help developers identify when CPU contention occurs, such as when all threads are blocked waiting for CPU access, which can impact performance. Initially, only CPU pressure time is displayed in the TUI (text user interface), but other CPU usage data is also available. This feature aids in diagnosing performance bottlenecks related to CPU resource constraints.
The PR can be found here: https://github.com/dagger/dagger/pull/8750
Module references are now stored unchanged in dagger.json
We have added module source pins to Dagger’s configuration, allowing dependencies to retain both their original source (e.g. main) and a specific commit hash as a pin for reference. This update enhances version control by preserving the exact version of each module installed with dagger install, ensuring reproducibility in builds. This change is backward-compatible, as it keeps the existing source field unchanged for older configurations.
The PR can be found here: https://github.com/dagger/dagger/pull/8587
Scoped enum values in Go
We have added scoped enum values in the Go SDK to prevent naming conflicts between different enums that may share the same value names. Previously, unscoped enum values could easily clash due to Go’s top-level package structure. With this change, enums are now prefixed with their type, following standard Go practices.
Enum values should now be accessed with the name prefixed by the name of the enum type - for example, Locked becomes CacheSharingModeLocked. The old unscoped constants are still available, but marked as deprecated, and are set to be removed in version 0.15.0.
The PR can be found here:https://github.com/dagger/dagger/pull/8669
What’s next?
Thanks to our amazing community of Daggernauts, we continue to see a lot of great feedback, bug reports, and feature requests. Keep it coming!
If you have requests, feedback, or want to contribute, don’t hesitate to join our Discord server.
Thank you for your support, as we continue our mission to transform the world of CI with you!
The Dagger team
