feat: provision TreatVault service and manage PawSQL secrets through it
All checks were successful
Build and Push Image / docker-build-and-push (push) Successful in 6m30s
Test and Release Barkstack / test (push) Successful in 42s
Test and Release Barkstack / release (push) Successful in 9s

This commit is contained in:
2026-09-16 16:06:41 -04:00
parent 6652a1f62c
commit 52f4e373c5
6 changed files with 254 additions and 41 deletions

View File

@@ -2,7 +2,7 @@
Barkstack is the control-plane CLI for Barkstack services. Its first backend is Docker Swarm.
`init` validates the Barkfile, then ensures Docker Swarm, the `barkstack` overlay network, PawSQL, and the Barkstack Console:
`init` validates the Barkfile, then ensures Docker Swarm, the `barkstack` overlay network, PawSQL, the Barkstack Console, and — when the Barkfile configures a `treatvault` block — TreatVault:
```sh
go run ./cmd/barkstack init --config ./Barkfile
@@ -12,9 +12,9 @@ go run ./cmd/barkstack init --config ./Barkfile
1. initializes Docker Swarm when this Docker node is inactive;
2. requires an active Swarm manager before proceeding;
3. verifies every Docker secret referenced by the Barkfile exists;
3. with a `treatvault` block: verifies the TreatVault identity Docker secret exists and creates or refreshes the `barkstack-treatvault` service; without one: verifies every Docker secret referenced by `password_secret` exists;
4. creates the attachable `barkstack` overlay network when it is absent;
5. pulls the PawSQL and Barkstack Console images;
5. pulls the PawSQL, Barkstack Console, and TreatVault images;
6. creates or refreshes the manager-constrained `barkstack-pawsql` service;
7. mounts referenced Docker secrets into PawSQL and publishes PostgreSQL port `5432` through Swarm ingress;
8. creates or refreshes the `barkstack-console` service with PawSQL registered at `http://barkstack-pawsql:9090`;
@@ -22,14 +22,27 @@ go run ./cmd/barkstack init --config ./Barkfile
PawSQL bind-mounts the Barkfile at `/etc/barkstack/Barkfile` and bind-mounts the configured TLS certificate and key at their configured paths. PawSQL is constrained to managers because those local files must be present where it runs. The console needs no host bind mounts and reaches PawSQL over the overlay network.
Managed PostgreSQL routes use a short `password_secret` reference. Barkstack prefixes it with `barkstack_`, verifies that the resulting Docker Swarm secret exists, and mounts it into PawSQL. For example, `password_secret application_postgres_password` requires:
Managed PostgreSQL routes use a short `password_secret` reference. Without TreatVault, Barkstack prefixes the reference with `barkstack_`, verifies the Docker Swarm secret exists, and mounts it into PawSQL:
```sh
docker secret create barkstack_application_postgres_password /secure/path/application-postgres-password
barkstack init --config ./Barkfile
```
Secret values never belong in the Barkfile. Docker secrets are immutable; rotate a password under a new secret name, update `password_secret`, and run `barkstack init` again.
With a `treatvault` block, TreatVault manages the namespaced secrets instead: `barkstack init` provisions `barkstack-treatvault` with the encrypted source-of-truth file bind-mounted read-write, the Docker socket, and the identity secret, then labels PawSQL as a TreatVault consumer. TreatVault watches the encrypted file and creates, rotates, and mounts versioned Docker Swarm secrets at stable `barkstack_<name>` targets. The console gains a TreatVault page for creating, rotating, and deleting secrets; values are write-only and never displayed again.
Bootstrap TreatVault before the first `init`:
```sh
go run ./cmd/treatvault keygen --identity ./treatvault-identity.txt # keep this file out of source control
docker secret create barkstack_treatvault_identity ./treatvault-identity.txt
go run ./cmd/treatvault init --file ./secrets.age --identity ./treatvault-identity.txt
go run ./cmd/treatvault set --name application_postgres_password --file ./secrets.age --identity ./treatvault-identity.txt
```
The encrypted `secrets.age` file is safe to commit and back up; only the identity file must stay secret. The TreatVault image and service name can be overridden with `--treatvault-image` and `--treatvault-service`.
Secret values never belong in the Barkfile. With TreatVault, rotate through the console or `treatvault set`; consumers are re-mounted automatically on the next sync. Without TreatVault, Docker secrets are immutable, so rotate under a new secret name, update `password_secret`, and run `barkstack init` again.
Override defaults when necessary:
@@ -41,7 +54,9 @@ barkstack init \
--docker-socket /var/run/docker.sock \
--network barkstack \
--service barkstack-pawsql \
--console-service barkstack-console
--console-service barkstack-console \
--treatvault-image registry.campbellwireless.net/barkstack/treatvault:latest \
--treatvault-service barkstack-treatvault
```
PawSQL also bind-mounts `/var/run/docker.sock`. It must run managed `postgres` containers itself — creating, starting, and stopping them on idle timeouts — so it needs Docker Engine access. The Docker socket grants PawSQL root-equivalent control of the host; mount it only for trusted Barkfiles and administrators. Override the path with `--docker-socket`.
@@ -50,13 +65,14 @@ The Docker CLI must point to the intended Swarm manager. The current implementat
## Management Console
The embedded Svelte console listens on plain HTTP and discovers independently shipped service UIs at runtime:
With TreatVault configured, register its write-only plugin the same way; `barkstack init` does this automatically for the Swarm deployment. The embedded Svelte console listens on plain HTTP and discovers independently shipped service UIs at runtime:
```sh
cd ui && bun install && bun run build && cd ..
go build -o barkstack ./cmd/barkstack
./barkstack console --listen :8080 \
--plugin pawsql=http://localhost:9090
--plugin pawsql=http://localhost:9090 \
--plugin treatvault=http://localhost:9090
```
Open `http://localhost:8080`. Repeat `--plugin id=http://internal-service:port` for additional plugins.

View File

@@ -43,10 +43,12 @@ func runInit(args []string, stdout, stderr io.Writer) error {
configPath := flags.String("config", "Barkfile", "Barkfile path")
image := flags.String("pawsql-image", swarm.DefaultPawSQLImage, "PawSQL image")
consoleImage := flags.String("console-image", swarm.DefaultConsoleImage, "Barkstack Console image")
treatVaultImage := flags.String("treatvault-image", swarm.DefaultTreatVaultImage, "TreatVault image")
network := flags.String("network", swarm.DefaultNetwork, "Swarm overlay network")
service := flags.String("service", swarm.DefaultService, "PawSQL service name")
consoleService := flags.String("console-service", swarm.DefaultConsoleService, "Barkstack Console service name")
dockerSocket := flags.String("docker-socket", swarm.DefaultDockerSocket, "Docker socket mounted into PawSQL")
treatVaultService := flags.String("treatvault-service", swarm.DefaultTreatVaultService, "TreatVault service name")
dockerSocket := flags.String("docker-socket", swarm.DefaultDockerSocket, "Docker socket mounted into PawSQL and TreatVault")
if err := flags.Parse(args); err != nil {
return err
}
@@ -56,13 +58,15 @@ func runInit(args []string, stdout, stderr io.Writer) error {
provisioner := swarm.Provisioner{Runner: swarm.ExecRunner{}}
if err := provisioner.Init(context.Background(), swarm.Options{
BarkfilePath: *configPath,
PawSQLImage: *image,
ConsoleImage: *consoleImage,
NetworkName: *network,
ServiceName: *service,
ConsoleServiceName: *consoleService,
DockerSocketPath: *dockerSocket,
BarkfilePath: *configPath,
PawSQLImage: *image,
ConsoleImage: *consoleImage,
TreatVaultImage: *treatVaultImage,
NetworkName: *network,
ServiceName: *service,
ConsoleServiceName: *consoleService,
TreatVaultServiceName: *treatVaultService,
DockerSocketPath: *dockerSocket,
}); err != nil {
return err
}

2
go.mod
View File

@@ -2,4 +2,4 @@ module cloud.campbellwireless.net/git/barkstack/barkstack
go 1.24
require cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.0.0
require cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.1.0

4
go.sum
View File

@@ -1,2 +1,2 @@
cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.0.0 h1:oXVNd8wwR06OaBMDPCVr4zAWZw6L0VBCODkXaHeMI/I=
cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.0.0/go.mod h1:um5p/9XtFqZqhh087AlPB2nr4NxhrQxw5sxe32uMbzw=
cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.1.0 h1:kqEalBOysM2w2ET7ISh1LblIsvo6dT2rOk9dqaQjDMQ=
cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2 v2.1.0/go.mod h1:um5p/9XtFqZqhh087AlPB2nr4NxhrQxw5sxe32uMbzw=

View File

@@ -16,23 +16,27 @@ import (
)
const (
DefaultNetwork = "barkstack"
DefaultPawSQLImage = "registry.campbellwireless.net/barkstack/pawsql:latest"
DefaultConsoleImage = "registry.campbellwireless.net/barkstack/barkstack:latest"
DefaultService = "barkstack-pawsql"
DefaultConsoleService = "barkstack-console"
DefaultDockerSocket = "/var/run/docker.sock"
serviceConfigPath = "/etc/barkstack/Barkfile"
DefaultNetwork = "barkstack"
DefaultPawSQLImage = "registry.campbellwireless.net/barkstack/pawsql:latest"
DefaultConsoleImage = "registry.campbellwireless.net/barkstack/barkstack:latest"
DefaultTreatVaultImage = "registry.campbellwireless.net/barkstack/treatvault:latest"
DefaultService = "barkstack-pawsql"
DefaultConsoleService = "barkstack-console"
DefaultTreatVaultService = "barkstack-treatvault"
DefaultDockerSocket = "/var/run/docker.sock"
serviceConfigPath = "/etc/barkstack/Barkfile"
)
type Options struct {
BarkfilePath string
NetworkName string
PawSQLImage string
ConsoleImage string
ServiceName string
ConsoleServiceName string
DockerSocketPath string
BarkfilePath string
NetworkName string
PawSQLImage string
ConsoleImage string
TreatVaultImage string
ServiceName string
ConsoleServiceName string
TreatVaultServiceName string
DockerSocketPath string
}
// Runner executes Docker CLI commands.
@@ -95,9 +99,25 @@ func (p Provisioner) Init(ctx context.Context, options Options) error {
if err := p.ensureSwarmManager(ctx); err != nil {
return err
}
secrets := configuredDockerSecrets(config)
if err := p.ensureSecrets(ctx, secrets); err != nil {
return err
if config.TreatVault != nil {
if _, err := p.Runner.Run(ctx, "pull", options.TreatVaultImage); err != nil {
return fmt.Errorf("pull TreatVault image: %w", err)
}
identitySecret := barkfile.DockerSecretName(config.TreatVault.IdentitySecret)
if err := p.ensureSecrets(ctx, []string{identitySecret}); err != nil {
return err
}
vaultPath, err := existingPath(config.TreatVault.File, filepath.Dir(configPath))
if err != nil {
return fmt.Errorf("TreatVault encrypted file: %w", err)
}
if err := p.ensureTreatVault(ctx, options, configPath, socketPath, vaultPath, identitySecret); err != nil {
return err
}
} else {
if err := p.ensureSecrets(ctx, configuredDockerSecrets(config)); err != nil {
return err
}
}
if err := p.ensureNetwork(ctx, options.NetworkName); err != nil {
return err
@@ -108,11 +128,10 @@ func (p Provisioner) Init(ctx context.Context, options Options) error {
if _, err := p.Runner.Run(ctx, "pull", options.ConsoleImage); err != nil {
return fmt.Errorf("pull Barkstack Console image: %w", err)
}
if err := p.ensurePawSQL(ctx, options, configPath, certificatePath, keyPath, socketPath, secrets); err != nil {
if err := p.ensurePawSQL(ctx, options, configPath, certificatePath, keyPath, socketPath, config); err != nil {
return err
}
if err := p.ensureConsole(ctx, options); err != nil {
if err := p.ensureConsole(ctx, options, config.TreatVault != nil); err != nil {
return err
}
return nil
@@ -131,12 +150,18 @@ func (o *Options) applyDefaults() {
if o.ConsoleImage == "" {
o.ConsoleImage = DefaultConsoleImage
}
if o.TreatVaultImage == "" {
o.TreatVaultImage = DefaultTreatVaultImage
}
if o.ServiceName == "" {
o.ServiceName = DefaultService
}
if o.ConsoleServiceName == "" {
o.ConsoleServiceName = DefaultConsoleService
}
if o.TreatVaultServiceName == "" {
o.TreatVaultServiceName = DefaultTreatVaultService
}
if o.DockerSocketPath == "" {
o.DockerSocketPath = DefaultDockerSocket
}
@@ -237,7 +262,17 @@ func (p Provisioner) ensureSecrets(ctx context.Context, secrets []string) error
return nil
}
func (p Provisioner) ensurePawSQL(ctx context.Context, options Options, barkfilePath, certificatePath, keyPath, socketPath string, secrets []string) error {
func (p Provisioner) ensurePawSQL(ctx context.Context, options Options, barkfilePath, certificatePath, keyPath, socketPath string, config barkfile.Config) error {
var secrets []string
labels := []string{}
if config.TreatVault != nil {
labels = append(labels,
"--label-add", barkfile.TreatVaultConsumerLabel+"=true",
"--label-add", barkfile.TreatVaultNamesLabel+"="+strings.Join(managedSecretNames(config), ","),
)
} else {
secrets = configuredDockerSecrets(config)
}
output, err := p.Runner.Run(ctx, "service", "inspect", "--format", "{{json .Spec.TaskTemplate.ContainerSpec.Mounts}}", options.ServiceName)
if err == nil {
mounts, decodeErr := serviceMounts(output)
@@ -256,6 +291,7 @@ func (p Provisioner) ensurePawSQL(ctx context.Context, options Options, barkfile
if !hasMount(mounts, socketPath, DefaultDockerSocket) {
args = append(args, "--mount-add", writableBindMount(socketPath, DefaultDockerSocket))
}
args = append(args, labels...)
desiredSecrets := make(map[string]struct{}, len(secrets))
for _, secret := range secrets {
desiredSecrets[secret] = struct{}{}
@@ -296,6 +332,9 @@ func (p Provisioner) ensurePawSQL(ctx context.Context, options Options, barkfile
for _, secret := range secrets {
args = append(args, "--secret", dockerSecretMount(secret))
}
for index := 0; index < len(labels); index += 2 {
args = append(args, "--label", labels[index+1])
}
args = append(args, options.PawSQLImage, "--config", serviceConfigPath)
if _, err := p.Runner.Run(ctx, args...); err != nil {
return fmt.Errorf("create PawSQL service: %w", err)
@@ -303,9 +342,81 @@ func (p Provisioner) ensurePawSQL(ctx context.Context, options Options, barkfile
return nil
}
func (p Provisioner) ensureConsole(ctx context.Context, options Options) error {
func managedSecretNames(config barkfile.Config) []string {
unique := make(map[string]struct{})
for _, database := range config.Databases {
if database.Postgres != nil {
unique[database.Postgres.PasswordSecret] = struct{}{}
}
}
names := make([]string, 0, len(unique))
for name := range unique {
names = append(names, name)
}
sort.Strings(names)
return names
}
func (p Provisioner) ensureTreatVault(ctx context.Context, options Options, barkfilePath, socketPath, vaultPath, identitySecret string) error {
args := []string{"serve", "--file", vaultPath, "--identity", "/run/secrets/" + identitySecret, "--listen", ":9090"}
output, err := p.Runner.Run(ctx, "service", "inspect", "--format", "{{json .Spec.TaskTemplate.ContainerSpec.Mounts}}", options.TreatVaultServiceName)
if err == nil {
mounts, decodeErr := serviceMounts(output)
if decodeErr != nil {
return fmt.Errorf("decode TreatVault service mounts: %w", decodeErr)
}
secretOutput, secretErr := p.Runner.Run(ctx, "service", "inspect", "--format", "{{json .Spec.TaskTemplate.ContainerSpec.Secrets}}", options.TreatVaultServiceName)
if secretErr != nil {
return fmt.Errorf("inspect TreatVault service secrets: %w", secretErr)
}
mountedSecrets, decodeErr := serviceSecrets(secretOutput)
if decodeErr != nil {
return fmt.Errorf("decode TreatVault service secrets: %w", decodeErr)
}
update := []string{"service", "update", "--with-registry-auth", "--image", options.TreatVaultImage, "--args", strings.Join(args, " "), "--force"}
if !hasMount(mounts, vaultPath, vaultPath) {
update = append(update, "--mount-add", writableBindMount(vaultPath, vaultPath))
}
if !hasMount(mounts, socketPath, DefaultDockerSocket) {
update = append(update, "--mount-add", writableBindMount(socketPath, DefaultDockerSocket))
}
if !hasSecret(mountedSecrets, identitySecret) {
update = append(update, "--secret-add", dockerSecretMount(identitySecret))
}
update = append(update, options.TreatVaultServiceName)
if _, err := p.Runner.Run(ctx, update...); err != nil {
return fmt.Errorf("update TreatVault service: %w", err)
}
return nil
}
if !isNotFound(err) {
return fmt.Errorf("inspect TreatVault service: %w", err)
}
create := []string{
"service", "create",
"--name", options.TreatVaultServiceName,
"--with-registry-auth",
"--constraint", "node.role==manager",
"--network", options.NetworkName,
"--mount", bindMount(barkfilePath, serviceConfigPath),
"--mount", writableBindMount(vaultPath, vaultPath),
"--mount", writableBindMount(socketPath, DefaultDockerSocket),
"--secret", dockerSecretMount(identitySecret),
options.TreatVaultImage,
}
create = append(create, args...)
if _, err := p.Runner.Run(ctx, create...); err != nil {
return fmt.Errorf("create TreatVault service: %w", err)
}
return nil
}
func (p Provisioner) ensureConsole(ctx context.Context, options Options, treatVaultEnabled bool) error {
output, err := p.Runner.Run(ctx, "service", "inspect", "--format", "{{json .Endpoint.Ports}}", options.ConsoleServiceName)
pluginEndpoint := "pawsql=http://" + options.ServiceName + ":9090"
if treatVaultEnabled {
pluginEndpoint += " --plugin treatvault=http://" + options.TreatVaultServiceName + ":9090"
}
if err == nil {
ports, decodeErr := servicePorts(output)
if decodeErr != nil {

View File

@@ -11,6 +11,8 @@ import (
barkfile "cloud.campbellwireless.net/git/barkstack/barkfile-parser/v2"
)
const treatVaultBarkfile = "treatvault {\n file ./secrets.age\n identity_secret treatvault_identity\n}\n"
func TestInitCreatesSwarmNetworkAndCoreServices(t *testing.T) {
paths := writeBarkfile(t)
runner := &fakeRunner{responses: []response{
@@ -293,6 +295,71 @@ func serviceMountsJSON(socketPath string) string {
return `[{"Type":"bind","Source":"` + socketPath + `","Target":"` + DefaultDockerSocket + `"}]`
}
func TestInitProvisionsTreatVaultAndConsumerPawSQL(t *testing.T) {
paths := writeTreatVaultBarkfile(t)
runner := &fakeRunner{responses: []response{
{output: "active true"},
{}, // pull treatvault image
{}, // secret inspect identity
{err: errors.New("service barkstack-treatvault not found")},
{}, // TreatVault create
{err: errors.New("network barkstack not found")},
{}, // network create
{}, // pull pawsql
{}, // pull console
{err: errors.New("service barkstack-pawsql not found")},
{}, // pawsql create
{err: errors.New("service barkstack-console not found")},
{}, // console create
}}
if err := (Provisioner{Runner: runner}).Init(context.Background(), Options{BarkfilePath: paths.barkfile, DockerSocketPath: paths.socket}); err != nil {
t.Fatal(err)
}
runner.requireCall(t, "pull "+DefaultTreatVaultImage)
runner.requireCall(t, "secret inspect barkstack_treatvault_identity")
treatVault := runner.callContaining(t, "service create --name "+DefaultTreatVaultService)
for _, want := range []string{
"--constraint node.role==manager",
"--network barkstack",
writableBindMount(paths.vault, paths.vault),
writableBindMount(paths.socket, DefaultDockerSocket),
"--secret " + dockerSecretMount("barkstack_treatvault_identity"),
DefaultTreatVaultImage,
"serve --file " + paths.vault + " --identity /run/secrets/barkstack_treatvault_identity --listen :9090",
} {
if !strings.Contains(treatVault, want) {
t.Errorf("TreatVault service create = %q, missing %q", treatVault, want)
}
}
pawSQL := runner.callContaining(t, "service create --name "+DefaultService)
if !strings.Contains(pawSQL, "--label "+barkfile.TreatVaultConsumerLabel+"=true") {
t.Errorf("PawSQL service create = %q, missing consumer label", pawSQL)
}
if !strings.Contains(pawSQL, "--label "+barkfile.TreatVaultNamesLabel+"=database_password") {
t.Errorf("PawSQL service create = %q, missing names label", pawSQL)
}
if strings.Contains(pawSQL, "--secret") {
t.Errorf("PawSQL service create mounted password secrets directly: %q", pawSQL)
}
console := runner.callContaining(t, "service create --name "+DefaultConsoleService)
if want := "--plugin treatvault=http://" + DefaultTreatVaultService + ":9090"; !strings.Contains(console, want) {
t.Errorf("console service create = %q, missing %q", console, want)
}
}
func TestInitRejectsMissingTreatVaultIdentitySecret(t *testing.T) {
paths := writeTreatVaultBarkfile(t)
runner := &fakeRunner{responses: []response{
{output: "active true"},
{},
{err: errors.New("secret not found")},
}}
err := (Provisioner{Runner: runner}).Init(context.Background(), Options{BarkfilePath: paths.barkfile, DockerSocketPath: paths.socket})
if err == nil || !strings.Contains(err.Error(), `required Docker secret "barkstack_treatvault_identity" does not exist`) {
t.Fatalf("Init() error = %v", err)
}
}
func servicePortsJSON(includeConsole bool) string {
if !includeConsole {
return `[]`
@@ -300,11 +367,26 @@ func servicePortsJSON(includeConsole bool) string {
return `[{"Protocol":"tcp","TargetPort":8080,"PublishedPort":8080,"PublishMode":"ingress"}]`
}
func writeTreatVaultBarkfile(t *testing.T) barkfilePaths {
t.Helper()
paths := writeBarkfile(t)
contents := treatVaultBarkfile + "pawsql {\n listen :5432\n database app {\n postgres {\n image postgres:18\n volume application-data\n password_secret database_password\n }\n }\n}\n"
if err := os.WriteFile(paths.barkfile, []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
paths.vault = filepath.Join(filepath.Dir(paths.barkfile), "secrets.age")
if err := os.WriteFile(paths.vault, []byte("placeholder"), 0o600); err != nil {
t.Fatal(err)
}
return paths
}
type barkfilePaths struct {
barkfile string
certificate string
key string
socket string
vault string
}
func writeBarkfile(t *testing.T) barkfilePaths {