Deploying to Staging Environment

Overview

The staging environment deployment is a coordinated multi-repo process that synchronizes code across three repositories (relay, relay-dashboard, relay-cloud) and automatically triggers deployment via GitHub Actions. Staging deployments ensure feature verification before production while keeping main branch clean during development.

When to Use

Integrating features across repos - Multiple components depend on changes in different repos
Testing feature branches together - Verify feature branch changes don't break integration
Promoting main to staging - Standard sync to keep staging up-to-date with latest main
Verifying deployment readiness - Test infrastructure changes or deployment workflows
Cross-team coordination - Share feature branches for integration testing

When NOT to use:

Emergency hotfixes (use production deployment instead)
Single-repo changes only (push directly if not blocking other repos)
Testing without intent to deploy (use local environment instead)

Architecture

The staging deployment system consists of:

Three Repos (relay, relay-dashboard, relay-cloud)
    ↓
Staging Branches (synced via git push)
    ↓
relay-cloud staging push triggers GitHub Actions
    ↓
Deploy-Staging Workflow (deploy-staging.yml)
    ↓
Fly.io Staging Environment (agent-relay-staging)
    ↓
Automatic health check verification

Key details:

Each repo has independent staging branch
relay-cloud staging branch push triggers automatic deployment
Workflow accepts optional relay/dashboard branch overrides
Falls back to main if specified branch doesn't exist
Workspace image builds in parallel with API deployment

Quick Reference

Standard Workflow (All Repos)

# 1. Create git worktree for staging work (BEST PRACTICE)
cd /data/repos/relay
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push

# 2. Push latest main to staging (fetch fresh)
git fetch origin main:main
git push origin main:staging

# 3. Or push current feature branch to staging (if desired)
git fetch origin feature/your-branch:feature/your-branch
git push origin feature/your-branch:staging

# 4. Clean up worktree
cd /data/repos/relay
git worktree remove .worktrees/staging-push

Relay-Dashboard

cd /data/repos/relay-dashboard
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin main:main
git push origin main:staging
cd /data/repos/relay-dashboard
git worktree remove .worktrees/staging-push

Relay-Cloud (Triggers Deployment)

cd /data/repos/relay-cloud
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin main:main
git push origin main:staging
# ⚠️ This push triggers deploy-staging.yml workflow
cd /data/repos/relay-cloud
git worktree remove .worktrees/staging-push

Implementation

Why Git Worktrees?

Git worktrees provide several benefits for staging workflows:

Isolation - Work on staging without changing main working directory state
Safety - Feature branch checked out in worktree won't affect your current work
Cleanliness - No lingering state changes after pushing
Best practice - Standard DevOps approach for multi-branch operations

Step-by-Step Deployment Process

Prerequisites

Access to all three repos with push permission
Current working directory: one of the repos
No uncommitted changes blocking worktree creation

Execute Staging Push Across All Repos

Step 1: Relay

cd /data/repos/relay
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin main:main
git push origin main:staging
cd /data/repos/relay
git worktree remove .worktrees/staging-push

Expected output:

Updated 'main' to 'origin/main'
remote: Create pull request for staging...
To github.com:AgentWorkforce/relay.git
 [new branch]      main -> staging
or
 * [up-to-date]    main -> staging

Step 2: Relay-Dashboard

cd /data/repos/relay-dashboard
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin main:main
git push origin main:staging
cd /data/repos/relay-dashboard
git worktree remove .worktrees/staging-push

Step 3: Relay-Cloud (Triggers Deployment)

cd /data/repos/relay-cloud
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin main:main
git push origin main:staging
# Deployment starts automatically!
cd /data/repos/relay-cloud
git worktree remove .worktrees/staging-push

Verify Deployment

After pushing relay-cloud, GitHub Actions starts automatically:

Watch workflow progress:
- Go to: https://github.com/AgentWorkforce/relay-cloud/actions
- Find "Deploy (Staging)" workflow run
- Check that both jobs pass:
  - "Deploy to Staging" (Fly.io deployment)
  - "Build Staging Workspace" (Docker image build)
Verify health check:
- Workflow runs curl https://agent-relay-staging.fly.dev/health
- Expected: 200 response within 150 seconds
- Retries 30 times with 5-second intervals
Check deployment summary:
- Click workflow run
- View "Deployment Summary" in step summary
- Confirms deployment to agent-relay-staging

Pushing Feature Branches to Staging

Instead of main, push a specific feature branch:

cd /data/repos/relay
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin feature/your-feature:feature/your-feature
git push origin feature/your-feature:staging
cd /data/repos/relay
git worktree remove .worktrees/staging-push

For relay-cloud with custom relay/dashboard branches:

The deploy-staging.yml workflow accepts optional inputs to specify exact branches:

# Push relay-cloud staging (will trigger GitHub Actions with inputs)
cd /data/repos/relay-cloud
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git fetch origin feature/custom-branch:feature/custom-branch
git push origin feature/custom-branch:staging
cd /data/repos/relay-cloud
git worktree remove .worktrees/staging-push

Then manually trigger with specific branches:

Go to: https://github.com/AgentWorkforce/relay-cloud/actions/workflows/deploy-staging.yml
Click "Run workflow"
Enter: relay_branch and dashboard_branch (optional)
Confirm run starts

Common Mistakes

Mistake	Problem	Fix
Pushing from dirty working tree	Worktree creation fails with error	Commit or stash changes before creating worktree
Forgetting to remove worktree	Accumulates worktree directories	Always run `git worktree remove .worktrees/staging-push` after pushing
Pushing staging without relay/dashboard	Deployment uses outdated relay/dashboard	Coordinate all three repos or use GitHub Actions inputs to override
Not fetching fresh ref	Pushes stale local branch state	Always `git fetch origin branch:branch` before pushing
Confusing staging branch direction	Push main→staging, not staging→main	Remember: feature work → staging branch, for promotion/testing
Assuming all repos auto-sync	relay/dashboard changes don't auto-trigger	Only relay-cloud staging push triggers deployment
Not checking health status	Deployment may fail silently	Always verify workflow completes and health check passes
Creating worktrees in wrong directory	Path confusion for multiple repos	Each repo is separate; create worktrees within that repo's .worktrees/

Workflow Details

deploy-staging.yml Behavior

Triggers:

Manual push to staging branch (automatic)
GitHub Actions workflow_dispatch (manual with optional inputs)

Optional Workflow Inputs:

relay_branch - Override which relay branch to deploy (defaults to current staging branch)
dashboard_branch - Override which relay-dashboard branch to deploy (defaults to current staging branch)
Falls back to main if specified branch doesn't exist in remote

Deployment Steps:

Checkout relay-cloud repository
Determine relay branch (input or current, fallback to main)
Determine dashboard branch (input or current, fallback to main)
Setup Fly CLI
Deploy to Fly.io with flyctl deploy
Verify health check endpoint
Create deployment summary

Environment: staging (requires GitHub Actions environment secrets)

Deployment Success Criteria

✅ All checks passed:

"Deploy to Staging" job completes
"Build Staging Workspace" job completes
Health check succeeds (HTTP 200 from /health endpoint)
Deployment summary shows all expected values

❌ Deployment failed:

Workflow run shows red X
Check step that failed in workflow logs
Common failures: auth (secrets), health check timeout, Docker build error

Real-World Workflow

Scenario: Feature across all three repos ready for integration testing

# 1. Ensure local main branches are up-to-date
cd /data/repos/relay
git fetch origin main

cd /data/repos/relay-dashboard
git fetch origin main

cd /data/repos/relay-cloud
git fetch origin main

# 2. Push relay main to staging (using worktree)
cd /data/repos/relay
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git push origin main:staging
cd ..
git worktree remove .worktrees/staging-push

# 3. Push relay-dashboard main to staging
cd /data/repos/relay-dashboard
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git push origin main:staging
cd ..
git worktree remove .worktrees/staging-push

# 4. Push relay-cloud main to staging (triggers deployment)
cd /data/repos/relay-cloud
git worktree add .worktrees/staging-push main
cd .worktrees/staging-push
git push origin main:staging
cd ..
git worktree remove .worktrees/staging-push

# 5. Monitor deployment
echo "Check: https://github.com/AgentWorkforce/relay-cloud/actions"
# Wait for "Deploy (Staging)" workflow to complete
# Verify health check passes
# Access staging at: https://agent-relay-staging.fly.dev

Troubleshooting

Worktree Errors

Error: "fatal: 'main' already exists"

Solution: git worktree remove .worktrees/staging-push
         (if worktree from previous run still exists)

Error: "fatal: cannot checkout branch into locked working tree"

Solution: Ensure main branch isn't checked out in primary working tree
         git status (confirm different branch is checked out)

Deployment Failures

Health check timeout after 150s

Problem: Staging environment took too long to become healthy
Solution: - Check Fly.io logs: flyctl logs --app agent-relay-staging
         - Restart app: flyctl restart --app agent-relay-staging
         - Check for build errors in workflow logs
         - Look for startup errors in deployment step

Docker build fails

Problem: "Build Staging Workspace" job failed
Solution: - Check docker-staging.yml workflow logs
         - Verify Dockerfile exists and is valid
         - Check for resource/dependency issues
         - Rebuild locally to verify docker configuration

Branch not found fallback to main

Problem: Workflow says "using main" for your custom branch
Reason: Specified branch doesn't exist in relay/relay-dashboard
Solution: - Verify branch name is exact (case-sensitive)
         - Ensure branch exists in remote (git ls-remote origin)
         - Use GitHub Actions workflow_dispatch with correct inputs

Dependencies & Requirements

Git: Worktree feature (Git 2.5+, standard on all modern systems)

Permissions: Push access to all three repos:

github.com:AgentWorkforce/relay
github.com:AgentWorkforce/relay-dashboard
github.com:AgentWorkforce/relay-cloud

GitHub Secrets (relay-cloud):

CROSS_REPO_TOKEN - PAT with repo access across all three repos
FLY_API_TOKEN or FLY_API_TOKEN_STAGING - Fly.io deployment token

Environment: staging (configured in relay-cloud)

Key Principle: Use git worktrees for clean, isolated branch operations. Push coordination across repos, with relay-cloud staging push triggering automatic deployment.

deploying-to-staging-environment

details