Dagger 0.4: Service containers, secrets scrubbing, and more
March 8, 2023
Mar 8, 2023
None
Today we are delighted to introduce Dagger 0.4.0, featuring service containers, secrets scrubbing, and more.
Service containers
Service containers have been one of the most requested features. They are now finally available. This feature enables you to run network services - like databases or webapps - as ephemeral containers, directly inside a Dagger pipeline.
Some common use cases for this new feature are:
Run a test database
Run end-to-end integration tests
Run sidecar services
Service containers come with the following built-in features:
Service containers are started just-in-time, de-duplicated, and stopped when no longer needed
Service containers are health checked prior to running clients
Service containers are given an alias for the client container to use as its hostname
Using service containers
Binding a service to a container expresses a dependency: the service container needs to be running when the client container runs. The bound service container is started automatically whenever its client container runs.
Here's an example of creating an HTTP service and fetching from it:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()
// create HTTP service container with exposed port 8080
httpSrv := client.Container().
From("python").
WithDirectory("/srv", client.Directory().WithNewFile("index.html", "Hello, world!")).
WithWorkdir("/srv").
WithExec([]string{"python", "-m", "http.server", "8080"}).
WithExposedPort(8080)
// create client container with service binding
// access HTTP service and print result
val, err := client.Container().
From("alpine").
WithServiceBinding("www", httpSrv).
WithExec([]string{"wget", "-O-", "http://www:8080"}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(val)
}Here, the WithExposedPort() method sets the ports that the container will listen on. Dagger checks the health of each exposed port prior to running any clients that use the service, so that clients don't have to implement their own polling logic.
Want to learn more about how services work in Dagger? Check out our documentation.
Secrets scrubbing
Dagger now automatically scrubs secrets from its various logs and output streams. This ensures that sensitive data does not leak - for example, in the event of a crash. This applies to secrets stored in both environment variables and file mounts.
Here's an example that demonstrates this in action:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
cleanupEnv := setDemoHostEnv("my_secret_file")
defer cleanupEnv()
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
secretEnv := client.Host().EnvVariable("MY_SECRET_ENV").Secret()
secretFile := client.Host().Directory(".").File("my_secret_file").Secret()
output, err := client.Container().
From("alpine@sha256:e2e16842c9b54d985bf1ef9242a313f36b856181f188de21313820e177002501").
WithSecretVariable("MY_SECRET_ENV", secretEnv).
WithMountedSecret("/my_secret_file", secretFile).
WithExec([]string{"sh", "-c", `echo -e "env secret data: $MY_SECRET_ENV || secret file data: "; cat /my_secret_file`}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(output)
}
func setDemoHostEnv(hostFilePath string) (cleanup func() error) {
// Setting a test host environment variable
const envName = "MY_SECRET_ENV"
err := os.Setenv(envName, "Secret Env Content")
if err != nil {
panic(err)
}
// Setting a test host file
const hostFileContent = "Secret File Content"
err = os.WriteFile(hostFilePath, []byte(hostFileContent), 0o644)
if err != nil {
panic(err)
}
return func() error {
return os.Remove(hostFilePath)
}
}The output will be:
env secret data: *** || secret file data:
*
Instead of
env secret data: Secret Env Content || secret file data:
Secret File ContentSDK Updates
We've also released new versions of our SDKs with support for all the new features in Dagger 0.4.0, plus various SDK-specific bug fixes and improvements.
For a complete list of improvements, refer to the changelog for each SDK:
Go SDK 0.5.0 changelog and documentation
Node.js SDK 0.4.0 changelog and documentation
Python SDK 0.4.0 changelog and documentation
We look forward to releasing more features soon! Meanwhile, if you have feedback or would like to suggest new features or documentation, let us know on Discord or create a GitHub issue.
Today we are delighted to introduce Dagger 0.4.0, featuring service containers, secrets scrubbing, and more.
Service containers
Service containers have been one of the most requested features. They are now finally available. This feature enables you to run network services - like databases or webapps - as ephemeral containers, directly inside a Dagger pipeline.
Some common use cases for this new feature are:
Run a test database
Run end-to-end integration tests
Run sidecar services
Service containers come with the following built-in features:
Service containers are started just-in-time, de-duplicated, and stopped when no longer needed
Service containers are health checked prior to running clients
Service containers are given an alias for the client container to use as its hostname
Using service containers
Binding a service to a container expresses a dependency: the service container needs to be running when the client container runs. The bound service container is started automatically whenever its client container runs.
Here's an example of creating an HTTP service and fetching from it:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()
// create HTTP service container with exposed port 8080
httpSrv := client.Container().
From("python").
WithDirectory("/srv", client.Directory().WithNewFile("index.html", "Hello, world!")).
WithWorkdir("/srv").
WithExec([]string{"python", "-m", "http.server", "8080"}).
WithExposedPort(8080)
// create client container with service binding
// access HTTP service and print result
val, err := client.Container().
From("alpine").
WithServiceBinding("www", httpSrv).
WithExec([]string{"wget", "-O-", "http://www:8080"}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(val)
}Here, the WithExposedPort() method sets the ports that the container will listen on. Dagger checks the health of each exposed port prior to running any clients that use the service, so that clients don't have to implement their own polling logic.
Want to learn more about how services work in Dagger? Check out our documentation.
Secrets scrubbing
Dagger now automatically scrubs secrets from its various logs and output streams. This ensures that sensitive data does not leak - for example, in the event of a crash. This applies to secrets stored in both environment variables and file mounts.
Here's an example that demonstrates this in action:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
cleanupEnv := setDemoHostEnv("my_secret_file")
defer cleanupEnv()
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
secretEnv := client.Host().EnvVariable("MY_SECRET_ENV").Secret()
secretFile := client.Host().Directory(".").File("my_secret_file").Secret()
output, err := client.Container().
From("alpine@sha256:e2e16842c9b54d985bf1ef9242a313f36b856181f188de21313820e177002501").
WithSecretVariable("MY_SECRET_ENV", secretEnv).
WithMountedSecret("/my_secret_file", secretFile).
WithExec([]string{"sh", "-c", `echo -e "env secret data: $MY_SECRET_ENV || secret file data: "; cat /my_secret_file`}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(output)
}
func setDemoHostEnv(hostFilePath string) (cleanup func() error) {
// Setting a test host environment variable
const envName = "MY_SECRET_ENV"
err := os.Setenv(envName, "Secret Env Content")
if err != nil {
panic(err)
}
// Setting a test host file
const hostFileContent = "Secret File Content"
err = os.WriteFile(hostFilePath, []byte(hostFileContent), 0o644)
if err != nil {
panic(err)
}
return func() error {
return os.Remove(hostFilePath)
}
}The output will be:
env secret data: *** || secret file data:
*
Instead of
env secret data: Secret Env Content || secret file data:
Secret File ContentSDK Updates
We've also released new versions of our SDKs with support for all the new features in Dagger 0.4.0, plus various SDK-specific bug fixes and improvements.
For a complete list of improvements, refer to the changelog for each SDK:
Go SDK 0.5.0 changelog and documentation
Node.js SDK 0.4.0 changelog and documentation
Python SDK 0.4.0 changelog and documentation
We look forward to releasing more features soon! Meanwhile, if you have feedback or would like to suggest new features or documentation, let us know on Discord or create a GitHub issue.
Today we are delighted to introduce Dagger 0.4.0, featuring service containers, secrets scrubbing, and more.
Service containers
Service containers have been one of the most requested features. They are now finally available. This feature enables you to run network services - like databases or webapps - as ephemeral containers, directly inside a Dagger pipeline.
Some common use cases for this new feature are:
Run a test database
Run end-to-end integration tests
Run sidecar services
Service containers come with the following built-in features:
Service containers are started just-in-time, de-duplicated, and stopped when no longer needed
Service containers are health checked prior to running clients
Service containers are given an alias for the client container to use as its hostname
Using service containers
Binding a service to a container expresses a dependency: the service container needs to be running when the client container runs. The bound service container is started automatically whenever its client container runs.
Here's an example of creating an HTTP service and fetching from it:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()
// create HTTP service container with exposed port 8080
httpSrv := client.Container().
From("python").
WithDirectory("/srv", client.Directory().WithNewFile("index.html", "Hello, world!")).
WithWorkdir("/srv").
WithExec([]string{"python", "-m", "http.server", "8080"}).
WithExposedPort(8080)
// create client container with service binding
// access HTTP service and print result
val, err := client.Container().
From("alpine").
WithServiceBinding("www", httpSrv).
WithExec([]string{"wget", "-O-", "http://www:8080"}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(val)
}Here, the WithExposedPort() method sets the ports that the container will listen on. Dagger checks the health of each exposed port prior to running any clients that use the service, so that clients don't have to implement their own polling logic.
Want to learn more about how services work in Dagger? Check out our documentation.
Secrets scrubbing
Dagger now automatically scrubs secrets from its various logs and output streams. This ensures that sensitive data does not leak - for example, in the event of a crash. This applies to secrets stored in both environment variables and file mounts.
Here's an example that demonstrates this in action:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
cleanupEnv := setDemoHostEnv("my_secret_file")
defer cleanupEnv()
ctx := context.Background()
// create Dagger client
client, err := dagger.Connect(ctx)
if err != nil {
panic(err)
}
defer client.Close()
secretEnv := client.Host().EnvVariable("MY_SECRET_ENV").Secret()
secretFile := client.Host().Directory(".").File("my_secret_file").Secret()
output, err := client.Container().
From("alpine@sha256:e2e16842c9b54d985bf1ef9242a313f36b856181f188de21313820e177002501").
WithSecretVariable("MY_SECRET_ENV", secretEnv).
WithMountedSecret("/my_secret_file", secretFile).
WithExec([]string{"sh", "-c", `echo -e "env secret data: $MY_SECRET_ENV || secret file data: "; cat /my_secret_file`}).
Stdout(ctx)
if err != nil {
panic(err)
}
fmt.Println(output)
}
func setDemoHostEnv(hostFilePath string) (cleanup func() error) {
// Setting a test host environment variable
const envName = "MY_SECRET_ENV"
err := os.Setenv(envName, "Secret Env Content")
if err != nil {
panic(err)
}
// Setting a test host file
const hostFileContent = "Secret File Content"
err = os.WriteFile(hostFilePath, []byte(hostFileContent), 0o644)
if err != nil {
panic(err)
}
return func() error {
return os.Remove(hostFilePath)
}
}The output will be:
env secret data: *** || secret file data:
*
Instead of
env secret data: Secret Env Content || secret file data:
Secret File ContentSDK Updates
We've also released new versions of our SDKs with support for all the new features in Dagger 0.4.0, plus various SDK-specific bug fixes and improvements.
For a complete list of improvements, refer to the changelog for each SDK:
Go SDK 0.5.0 changelog and documentation
Node.js SDK 0.4.0 changelog and documentation
Python SDK 0.4.0 changelog and documentation
We look forward to releasing more features soon! Meanwhile, if you have feedback or would like to suggest new features or documentation, let us know on Discord or create a GitHub issue.
