Getting Started

Quickstart: Run your first test suite

Install TestCo, point it at your repo, and see your first test results in the dashboard — no YAML guesswork, no CI pipeline required for the first run. Five minutes, start to finish.

Step 1: Install the CLI

Pick the install path for your platform. All four are first-class and stay in sync.

macOS (Homebrew)

brew install testco/tap/testco

Linux (apt)

echo "deb [signed-by=/usr/share/keyrings/testco.gpg] https://apt.testco.com stable main" | sudo tee /etc/apt/sources.list.d/testco.list
sudo apt update && sudo apt install testco

Linux (curl — any distro)

curl -fsSL https://cli.testco.com/install.sh | sh

Windows (PowerShell)

winget install TestCo.TestCoCLI

Verify the install:

testco version
# → testco 1.2.0 (commit abc123)

Step 2: Authenticate

Create an API key in the TestCo dashboard at Settings > API Keys and set it:

testco auth login
# Opens your browser — or paste your key manually:
testco auth set-key tc_key_abc123def456

Verify your auth:

testco auth whoami
# → Logged in as you@company.com — Project: my-project

Step 3: Run your first suite

From the root of your repository, run:

testco run

TestCo auto-detects common test frameworks (Jest, pytest, RSpec, Playwright, Cypress, Go test) and runs your default suite. You'll see live output:

testco run — scanning for test framework...
  Found jest@29.7.0 — running `npx jest`
  ✓ 47 passed  —  3 failed  —  0 flaky
  View results: https://app.testco.com/runs/run_xyz789

Step 4: See your results

Open the URL printed after the run. The signal dashboard shows:

• Pass rate — tests that passed.
• Failures — tests that failed, with full output and stack traces.
• Flaky flag — tests that passed on retry (auto-detected after 2+ runs).

No dashboard? Run testco open to open it in your browser.

Open the dashboard

What's next

• Connect your repository so TestCo runs on every push.
• Add TestCo to CI — GitHub Actions, GitLab CI, or any CI.
• Configure a suite — set timeouts, retries, environment variables.

Getting Started

Install & configure the CLI

Platform packages, auth token setup, and the defaults TestCo assumes until you add a CI pipeline.

Platform packages

The TestCo CLI is distributed through native package managers and a universal curl installer. Pick one — they are kept in lockstep with every release.

Authenticate

Create an API key at Settings > API Keys in the dashboard, then sign in:

testco auth login

For CI and headless environments, set the key from an environment variable instead of a browser flow:

export TESTCO_API_KEY=tc_key_abc123def456
testco auth whoami

Pointing TestCo at CI

Until you wire TestCo into CI, runs are manual (testco run). The first CI integration page covers GitHub Actions and GitLab CI; the rest of the CI/CD Integration category handles CircleCI, Jenkins, and generic providers.

Getting Started

Connect your repository

Link a GitHub, GitLab, or Bitbucket repo so TestCo tracks every push and associates runs with branches and commits automatically.

OAuth connection

From the dashboard, open Settings > Repositories > Add repository and authorize the TestCo GitHub, GitLab, or Bitbucket app. TestCo requests read access to repository metadata and commit status only — it does not need write access to source.

Once connected, TestCo receives push webhooks and queues a run for each push to a tracked branch. The dashboard shows the repo, default branch, and last seen commit.

Manual webhook (self-hosted or restricted orgs)

If your org blocks OAuth apps, configure the webhook manually:

testco repo connect --owner myorg --repo my-service \
  --webhook-url https://api.testco.com/v1/hooks/push

After you connect

TestCo begins tracking immediately. The first push triggers a run; subsequent pushes on tracked branches queue automatically. See Branch and commit tracking for how runs are associated.

Getting Started

Your first CI integration

Run TestCo on every push and pull request. Two examples cover the common cases; the rest live in the CI/CD Integration category.

GitHub Actions

Add this reusable workflow as .github/workflows/testco-run.yml. Store your API key as a repository secret named TESTCO_API_KEY.

name: TestCo — on push
on:
  push:
    branches: [main]
  pull_request:
jobs:
  testco:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: testco/action-run@v1
        with:
          api-key: ${{ secrets.TESTCO_API_KEY }}
          # Optional: suite identifier
          suite: unit-tests

GitLab CI

Add a job to your .gitlab-ci.yml. Define TESTCO_API_KEY as a CI/CD variable in your project settings.

testco-run:
  image: testco/cli:latest
  script:
    - testco run --suite integration-tests
  variables:
    TESTCO_API_KEY: $TESTCO_API_KEY
  artifacts:
    reports:
      junit: testco-report.xml

What's next

• Configure a suite to control which tests run and how.
• GitLab CI setup for artifact handling and parallel jobs.
• Other CI (generic) for any provider via the CLI and API.

Core Concepts

How TestCo works

Three pieces — the agent, the runner, and the signal layer — turn your existing test suite into a dashboard of useful signals without changing how you write tests.

Architecture overview

TestCo does not host your tests. It runs your existing test command in the environment you already use (your laptop, CI, or a TestCo runner) and reads the structured output to produce signals.

• Agent — the testco CLI. Detects your framework, invokes your test command, and emits structured results.
• Runner — where the command executes. In CI, that's the CI job; locally, it's your shell. TestCo also offers hosted runners for scheduled suites.
• Signal layer — ingests results, computes pass/fail/flaky/degraded/timeout, and renders the dashboard.

TestCo reads JUnit XML, Jest JSON, pytest output, and Playwright reports natively. If your framework emits any of these, you do not need a custom adapter.

Data flow

1. You run testco run (or a push triggers it).
2. The agent detects the framework and invokes your test command.
3. Results stream to TestCo and are associated with your branch and commit.
4. The signal dashboard updates in near real time.

Continue with Suites, runs, and results for the data model, and The signal dashboard for the UI.

Core Concepts

Suites, runs, and results

The core data model: a suite is a named test command; a run is one execution of it; a result is the outcome of a single test.

Suites

A suite binds a name to a test command, a framework, file paths, a timeout, retries, and environment. You define suites in .testco.yml at your repo root. One repo can hold many suites — see monorepos for multi-suite patterns.

Runs

A run is a single execution of a suite. Each run has an id (run_xyz789), a status (queued, running, passed, failed, timed_out), and is pinned to a branch and commit. Runs are immutable once finished.

Results

A result is the outcome of one test: pass, fail, flaky, or skipped, with duration, output, and stack trace. TestCo rolls results up into run-level signals — see Test signals.

Core Concepts

Branch and commit tracking

How TestCo associates each run with a branch and commit so you can compare signal across time and pinpoint regressions.

Automatic association

When TestCo runs inside a git checkout, the agent reads the current branch and HEAD commit from git and attaches them to the run. No configuration needed — testco run from a branch tags the run with that branch.

Explicit association

In CI where the checkout may be detached, pass the branch and commit explicitly:

testco run --branch "$CI_COMMIT_BRANCH" --commit "$CI_COMMIT_SHA"

Pinning a suite to a branch

You can restrict a suite so it only runs on specific branches. See Pin a suite to a branch for the full guide.

Core Concepts

Test signals — what we surface

TestCo reduces a run to five signals. Each one is actionable, not decorative.

Core Concepts

The signal dashboard

The UI that turns runs into decisions. Every panel answers one question.

Overview panel

At the top: pass rate, failure count, flaky count, and the latest run's duration. Below it, a sparkline of pass rate over the last 30 runs so you can spot drift at a glance.

Run list

Each row is a run: suite, branch, commit (short SHA), signal counts, duration, and status. Click a row to open run details with full test output.

Filters

Filter by suite, branch, status, and signal. Saved views persist per user. Use the List runs API to pull the same data into your own tooling.

Guides (How-To)

Configure a test suite

Define suites in .testco.yml at your repo root. One file, many suites, full control over timeouts, retries, and environment.

Basic structure

# .testco.yml — project root
version: 1
suites:
  - name: unit-tests
    framework: jest
    command: npx jest --ci --maxWorkers=4
    paths:
      - src/**/*.test.ts
    timeout: 10m
    retries: 2
    env:
      CI: "true"
      NODE_ENV: test

  - name: integration-tests
    framework: playwright
    command: npx playwright test
    paths:
      - tests/e2e/**
    timeout: 30m
    retries: 1
    env:
      BASE_URL: http://localhost:3000

  - name: api-tests
    framework: pytest
    command: pytest tests/api/ -v
    paths:
      - tests/api/**
    timeout: 15m
    retries: 0

Field reference

For the full field list and defaults, see CLI configuration. The most-tuned fields are timeout, retries, and env — set environment variables here rather than exporting them in CI so the suite is self-describing.

Environment variables

Sensitive values should come from environment variables referenced by name, not hard-coded. TestCo resolves env entries that look like $VAR from the surrounding environment at run time.

Guides (How-To)

Pin a suite to a branch

Restrict a suite so it only runs on the branches that matter — keep main fast, run heavy e2e only on release branches.

Via the CLI

# Pin a suite to a specific branch
testco config set suite.unit-tests.branch main

# Run only suites pinned to the current branch
testco run --pinned

# List pinned suites
testco config get suite.*.branch

Via .testco.yml

Set branch on a suite to restrict it. Omit the field to run the suite on all branches.

suites:
  - name: release-e2e
    framework: playwright
    command: npx playwright test
    branch: release/*
    timeout: 30m

Guides (How-To)

Rerun a failed test

One-click rerun from the dashboard, or scripted rerun from the CLI. Both create a new run linked to the original.

From the dashboard

On any failed run, the Rerun failures button creates a new run that executes only the tests that failed in the original. The new run is linked to the source run in the timeline.

From the CLI

# Rerun all failed tests from the last run
testco rerun

# Rerun from a specific run
testco rerun run_xyz789

# Rerun a single test by name
testco rerun run_xyz789 --test "User API - creates new user"

For programmatic reruns, use the Rerun failed tests API.

Guides (How-To)

Set up flaky-test detection

TestCo flags a test as flaky when it fails on the first attempt and passes on retry repeatedly within a history window. Tune the window and thresholds to match your tolerance.

Configuration

# In .testco.yml under a suite
flaky:
  enabled: true
  # A test is flaky if it fails on the first run and passes on retry
  # at least N times in the last M runs
  history_window: 10        # last 10 runs
  fail_threshold: 3        # flagged after 3 failure-then-pass cycles
  retries_on_flaky: 2      # auto-retry flagged tests up to 2 times

Tuning

• history_window — how many recent runs to consider. Lower for high-traffic repos, higher for stable ones.
• fail_threshold — how many fail-then-pass cycles before flagging. 1 is noisy; 3 is the default.
• retries_on_flaky — auto-retry count for already-flagged tests, to keep your suite green while you investigate.

Guides (How-To)

Manage team notifications

Route test signals to Slack, email, or a webhook. Subscribe per suite and per event so the right people see only what matters.

Slack webhook

# Add a Slack webhook for test notifications
testco integrations add slack \
  --webhook https://hooks.slack.com/services/T00/B00/abc123 \
  --events failed,flaky

# Test the notification
testco integrations test slack

Email and generic webhooks

Replace slack with email or webhook and supply the matching destination. --events accepts any combination of failed, flaky, degraded, timeout, and passed.

Guides (How-To)

Use TestCo with monorepos

Split a monorepo into independent suites with per-suite watch paths so a frontend change doesn't run backend tests.

Multi-suite strategy

# .testco.yml in a monorepo root
version: 1
suites:
  - name: frontend-unit
    framework: jest
    command: cd frontend && npx jest
    paths:
      - frontend/**
    watch:
      - frontend/src/**
      - frontend/package.json

  - name: backend-unit
    framework: pytest
    command: cd backend && pytest
    paths:
      - backend/**
    watch:
      - backend/**/*.py
      - backend/requirements.txt

  - name: e2e
    framework: playwright
    command: npx playwright test
    paths:
      - e2e/**
    watch:
      - frontend/src/**
      - backend/src/**

The watch field tells TestCo which path changes should trigger a suite. A change to frontend/ runs frontend-unit and e2e, but not backend-unit.

CI/CD Integration

GitHub Actions setup

Step-by-step YAML and a reusable workflow. Store your API key as a repository secret and reference it by name.

Reusable workflow

name: TestCo — on push
on:
  push:
    branches: [main]
  pull_request:
jobs:
  testco:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: testco/action-run@v1
        with:
          api-key: ${{ secrets.TESTCO_API_KEY }}
          # Optional: suite identifier
          suite: unit-tests

Adding the secret

In your repo, go to Settings > Secrets and variables > Actions > New repository secret, name it TESTCO_API_KEY, and paste your key from the dashboard API keys page.

Related

• Configure a suite — define what the workflow runs.
• REST API overview — trigger runs from custom actions.

CI/CD Integration

GitLab CI setup

Job definition, artifact handling, and passing the branch and commit so TestCo can associate the run.

Job definition

testco-run:
  image: testco/cli:latest
  script:
    - testco run --suite integration-tests
  variables:
    TESTCO_API_KEY: $TESTCO_API_KEY
  artifacts:
    reports:
      junit: testco-report.xml

Artifact handling

The artifacts.reports.junit entry makes the report downloadable from the GitLab job UI and keeps it for the job's retention window. For non-JUnit frameworks, point TestCo at the right report format with --report-path.

Branch and commit

The testco/cli image reads CI_COMMIT_BRANCH and CI_COMMIT_SHA automatically. For detached checkouts, pass them explicitly as shown in Branch and commit tracking.

CI/CD Integration

CircleCI orb

Use the TestCo orb to add a run job in one line. Configure the API key as a CircleCI context environment variable.

Config example

version: 2.1
orbs:
  testco: testco/testco@1.2.0
jobs:
  build-test:
    docker:
      - image: cimg/node:20.14
    steps:
      - checkout
      - testco/run:
          suite: unit-tests
          api-key: TESTCO_API_KEY
workflows:
  testco-on-push:
    jobs:
      - build-test:
          context: testco

Orb parameters

CI/CD Integration

Jenkins pipeline

A declarative pipeline that installs the CLI, runs a suite, and archives the report. Inject the API key as a credentials binding.

Declarative pipeline

pipeline {
  agent any
  environment {
    TESTCO_API_KEY = credentials('testco-api-key')
  }
  stages {
    stage('TestCo') {
      steps {
        sh 'curl -fsSL https://cli.testco.com/install.sh | sh'
        sh 'testco run --suite unit-tests'
      }
      post {
        always {
          archiveArtifacts artifacts: 'testco-report.xml', allowEmptyArchive: true
        }
      }
    }
  }
}

CI/CD Integration

Other CI (generic)

Any CI that can run a shell command and set an environment variable can drive TestCo. Install the CLI, set the key, run the suite.

Generic recipe

# 1. Install the CLI
curl -fsSL https://cli.testco.com/install.sh | sh

# 2. Set the API key (from your CI's secret store)
export TESTCO_API_KEY=$TESTCO_API_KEY

# 3. Run the suite
testco run --suite unit-tests

Reporting back to CI

Point TestCo at your report path and consume it in whatever your CI supports (JUnit, JSON, HTML). For programmatic control — trigger runs, poll status — use the REST API.

CLI Reference

TestCo CLI commands

The testco CLI is the primary interface: run suites, check status, rerun failures, and manage configuration.

Common examples

testco run --suite integration-tests --branch main
testco status --run run_xyz789
testco rerun run_xyz789 --test "User API - creates new user"

CLI Reference

CLI configuration

The full .testco.yml spec. Every field, every default, in one place.

Top-level fields

Suite fields

Full example

version: 1
suites:
  - name: unit-tests
    framework: jest
    command: npx jest --ci
    paths: [src/**/*.test.ts]
    timeout: 10m
    retries: 2
    env:
      CI: "true"
    flaky:
      enabled: true
      history_window: 10
      fail_threshold: 3
      retries_on_flaky: 2

CLI Reference

Environment variables

Override configuration and supply secrets without editing .testco.yml.

API Reference

REST API overview

The TestCo REST API exposes runs and suites over HTTPS with bearer-token auth and cursor pagination.

Base URL and auth

• Base URL: https://api.testco.com/v1
• Auth: Authorization: Bearer $TESTCO_API_KEY
• Content type: application/json

Pagination

List endpoints use cursor pagination. Pass limit (default 20, max 100) and use the returned next_cursor to fetch the next page.

Errors

Errors use standard HTTP status codes with a JSON body: {"error": {"code": "not_found", "message": "..."}}. 401 means an invalid or missing key; 429 means you hit the rate limit — back off and retry.

API Reference

List runs

GET /runs — list runs for the current project, with optional filters and cursor pagination.

Request

curl -s "https://api.testco.com/v1/runs?limit=10&status=failed" \
  -H "Authorization: Bearer $TESTCO_API_KEY"

Response

{
  "data": [
    {
      "id": "run_abc123",
      "status": "failed",
      "branch": "main",
      "passed": 44,
      "failed": 3,
      "flaky": 1,
      "created_at": "2026-07-17T09:15:00Z"
    }
  ],
  "pagination": {
    "next_cursor": "cursor_eyJpZCI6InJ1bl9hYmMxMjMifQ==",
    "has_more": true
  }
}

API Reference

Get run details

GET /runs/{id} — fetch a single run, including its test-level results.

Request

curl -s "https://api.testco.com/v1/runs/run_xyz789" \
  -H "Authorization: Bearer $TESTCO_API_KEY"

Response

{
  "id": "run_xyz789",
  "status": "failed",
  "suite": "unit-tests",
  "branch": "main",
  "commit": "a1b2c3d4e5f6",
  "passed": 44,
  "failed": 3,
  "flaky": 1,
  "created_at": "2026-07-17T09:15:00Z",
  "results": [
    { "name": "User API - creates new user", "status": "failed", "duration_ms": 412 }
  ]
}

API Reference

Trigger a run

POST /runs — start a new run for a suite. Returns the run id and a dashboard URL.

Request

curl -X POST https://api.testco.com/v1/runs \
  -H "Authorization: Bearer $TESTCO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "suite": "unit-tests",
    "branch": "main",
    "commit": "a1b2c3d4e5f6"
  }'

Response

{
  "id": "run_xyz789",
  "status": "queued",
  "suite": "unit-tests",
  "branch": "main",
  "created_at": "2026-07-17T10:30:00Z",
  "url": "https://app.testco.com/runs/run_xyz789"
}

API Reference

Rerun failed tests

POST /runs/{id}/rerun — create a new run that re-executes only the tests that failed in the source run.

Request

curl -X POST https://api.testco.com/v1/runs/run_xyz789/rerun \
  -H "Authorization: Bearer $TESTCO_API_KEY" \
  -H "Content-Type: application/json"

Response

{
  "id": "run_xyz987",
  "status": "queued",
  "source_run": "run_xyz789",
  "url": "https://app.testco.com/runs/run_xyz987"
}

API Reference

List suites

GET /suites — list the suites defined in the current project's .testco.yml.

Request

curl -s "https://api.testco.com/v1/suites" \
  -H "Authorization: Bearer $TESTCO_API_KEY"

Response

{
  "data": [
    { "name": "unit-tests", "framework": "jest", "timeout": "10m" },
    { "name": "integration-tests", "framework": "playwright", "timeout": "30m" }
  ]
}

Troubleshooting

Common setup errors

Auth failures, network issues, and rate limits — and the one fix that clears each.

Troubleshooting

Tests not appearing

You ran testco run but the dashboard shows no tests. Almost always a config mismatch or a runner issue.

Checklist

1. Suite name matches. The --suite value must match a name in .testco.yml.
2. Paths resolve. Globs in paths are relative to the repo root. Run testco config validate to check.
3. Framework detected. If auto-detection fails, pass --framework explicitly.
4. Command exits 0 on success. A non-zero exit that isn't a test failure can swallow results. Check the run log.

Troubleshooting

Flaky-test false positives

TestCo flags a test as flaky, but it consistently passes in your local runs. Tune the detection thresholds.

Common causes

• History window too short — a single transient failure flips the flag. Raise history_window.
• Fail threshold too low — fail_threshold: 1 flags on a single fail-then-pass. The default is 3.
• Genuine environmental flakiness — order-dependent tests or shared state. Run with --randomize to confirm.

Adjusting thresholds

See Set up flaky-test detection for the full config reference. Start from the defaults and widen the window before lowering the threshold.

Troubleshooting

Contact support

Three channels, ranked by speed. Include your run id and project slug for the fastest resolution.