In the evolving landscape of software quality assurance, AI-driven testing is no longer a luxury—it’s a necessity for speed and scale. TestSprite, an AI-powered testing platform, offers a way to generate and execute end-to-end tests using agentic workflows. However, the real power of these tools is unlocked when they are integrated directly into your CI/CD pipeline.
In this article, I will demonstrate how to bridge the gap between containerized applications and cloud-based AI testing agents which is TestSprite in our case and using it with GitHub Actions.
The Challenge: Testing Private Containers from the Cloud
When your application runs in a managed environment like Vercel or Netlify, integration is straightforward. But what if your app is running in a local Docker container or a private GitHub runner?
Because the TestSprite cloud needs to "see" your application to test it, you face a networking hurdle. You cannot simply point the AI to localhost. To solve this, the video introduces a three-tier architecture: Build, Tunnel, and Execute.
Phase 1: The Build
The process starts with a standard GitHub Actions runner. Using Docker Compose, the workflow spins up both the front-end (e.g., Vite) and the back-end (e.g., Node/Express) services. This ensures the environment is a perfect replica of your production stack.
Phase 2: The Tunnel (The Secret Sauce)
To make the private container accessible to TestSprite without exposing it permanently to the internet, the tutorial utilizes the Cloudflare Tunneling library (cloudflared).
Temporary URL: The tunnel creates a random, dynamic URL (e.g.,
*.trycloudflare.com).Dynamic Routing: This URL is passed to the TestSprite agent, serving as the
BASE_URLfor all test execution.Security: The tunnel is temporary and only exists for the duration of the test execution phase.
Phase 3: Integration and Execution
To trigger the tests, you need to configure your GitHub repository with a few essential pieces:
API Key: Securely store your TestSprite API key in GitHub Repository Secrets as
TESTSPRITE_API_KEY.Workflow YAML: Define a
.github/workflows/ci.ymlfile that calls thetest-sprite/run-action@v1.
Once pushed, the pipeline automatically triggers on every Pull Request.
GitHub Action YAML file
Here is how the GH Action Pipeline will look like
name: CI
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build-and-test:
runs-on: ubuntu-latest
permissions:
pull-requests: write
issues: write
contents: read
steps:
- uses: actions/checkout@v4
- name: Build and start app containers
run: docker compose up -d --build
- name: Wait for frontend to be ready
run: |
echo "Waiting for frontend on port 5173..."
timeout 60 bash -c 'until curl -s http://127.0.0.1:5173 > /dev/null 2>&1; do sleep 2; done'
echo "Frontend is up!"
- name: Wait for backend to be ready
run: |
echo "Waiting for backend on port 4000..."
timeout 60 bash -c 'until curl -s http://127.0.0.1:4000/employees > /dev/null 2>&1; do sleep 2; done'
echo "Backend is up!"
- name: Install and start Cloudflare Tunnel
run: |
# Install cloudflared
curl -sL https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb -o cloudflared.deb
sudo dpkg -i cloudflared.deb
# Start tunnel for frontend and capture the public URL
cloudflared tunnel --url http://127.0.0.1:5173 --no-autoupdate > /tmp/cloudflared.log 2>&1 &
sleep 10
# Extract the tunnel URL from the logs
TUNNEL_URL=$(grep -o 'https://.*\.trycloudflare\.com' /tmp/cloudflared.log | head -1)
echo "Tunnel URL: $TUNNEL_URL"
echo "TUNNEL_URL=$TUNNEL_URL" >> $GITHUB_ENV
- name: Verify tunnel is working
run: |
echo "Testing tunnel URL: $TUNNEL_URL"
curl -sI "$TUNNEL_URL" | head -5
- name: Rewrite hardcoded URLs in test scripts
run: |
echo "Replacing http://127.0.0.1:5173 with $TUNNEL_URL in test files..."
find testsprite_tests -name '*.py' -exec sed -i "s|http://127.0.0.1:5173|$TUNNEL_URL|g" {} +
echo "Replacing http://localhost:5173 with $TUNNEL_URL in test files..."
find testsprite_tests -name '*.py' -exec sed -i "s|http://localhost:5173|$TUNNEL_URL|g" {} +
# Show a sample to verify
grep -r 'trycloudflare' testsprite_tests/*.py | head -3 || echo "No replacements found - check URL pattern"
- name: Run Testsprite Action
uses: TestSprite/run-action@v1
with:
testsprite-api-key: ${{ secrets.TESTSPRITE_API_KEY }}
github-token: ${{ secrets.GITHUB_TOKEN }}
base_url: ${{ env.TUNNEL_URL }}
blocking: 'true'
Real-World Results: AI-Powered PR Reports
The most impressive part of this integration is the feedback loop. Once the tests are complete:
PR Blocking: If tests fail, the Pull Request is automatically blocked from merging, ensuring code quality.
Comprehensive Dashboards: TestSprite provides a detailed dashboard within the PR, showing exactly which cases passed or failed.
Video Evidence: Perhaps most helpfully, the platform provides video recordings of the AI agent interacting with the browser, making it incredibly easy to debug failures.
The most impressive part of this integration is the feedback loop. Once the tests are complete:
PR Blocking: If tests fail, the Pull Request is automatically blocked from merging, ensuring code quality.
Comprehensive Dashboards: TestSprite provides a detailed dashboard within the PR, showing exactly which cases passed or failed.
Video Evidence: Perhaps most helpfully, the platform provides video recordings of the AI agent interacting with the browser, making it incredibly easy to debug failures.
Conclusion
Integrating AI testing into GitHub Actions transforms QA from a bottleneck into a seamless part of the development cycle. By combining Docker, Cloudflare tunneling, and TestSprite, teams can achieve high-confidence deployments with minimal manual intervention.
For a detailed walkthrough of the code and configuration, check out the full video by Execute Automation:
