Git Pipeline Broken? How to Fix Common CI/CD and Git Problems

TL;DR

CI/CD pipelines break for predictable reasons: dependency failures, Docker build errors, flaky tests, merge conflicts, misconfigured secrets, or permission issues. This article walks through systematic debugging for each scenario, with concrete commands for GitHub Actions and GitLab CI. The key principle: always read the logs first, reproduce locally second, and fix the root cause rather than retrying blindly.

Prerequisites

• Basic familiarity with Git (commits, branches, push/pull)
• Access to a CI/CD platform (GitHub Actions or GitLab CI)
• Command-line Git installed locally (version 2.30+)
• Docker installed locally for reproducing build issues
• Repository permissions sufficient to view pipeline logs and manage secrets

Reading CI/CD Logs

GitHub Actions

GitHub Actions logs are accessible from the Actions tab in your repository. Each workflow run shows individual jobs, and each job expands into steps. Failed steps are marked with a red icon and auto-expanded.

To download logs via the CLI:

# List recent workflow runs
gh run list --limit 10

# View a specific failed run
gh run view 123456789

# Download full logs for offline analysis
gh run view 123456789 --log

# Watch a running workflow in real time
gh run watch 123456789

Key things to look for in the output:

• Exit codes — a non-zero exit code indicates the exact command that failed.
• Timestamps — sudden jumps in time often indicate network timeouts or hanging processes.
• The first error — scroll up past cascading failures to find the root cause. Later errors are often consequences of the first one.

GitLab CI

GitLab CI logs appear under CI/CD > Pipelines in the sidebar. Click into a pipeline, then into the specific job. GitLab offers a useful "search in log" feature directly in the browser.

# Using the GitLab CLI (glab)
glab ci status

# View the log of a specific job
glab ci view <job-id>

# Retry a failed job
glab ci retry <job-id>

In both platforms, enable debug logging when standard logs are insufficient:

# GitHub Actions — add to the env section of your workflow
env:
  ACTIONS_RUNNER_DEBUG: true
  ACTIONS_STEP_DEBUG: true

# GitLab CI — add a variable in the pipeline UI or .gitlab-ci.yml
variables:
  CI_DEBUG_TRACE: "true"

Warning: Debug logging may expose secrets in the log output. Only enable it temporarily and on private repositories.

Common Pipeline Failures

Dependency Install Failures

The most frequent CI failure is a broken dependency installation. This typically manifests as a package manager error during the install step.

Symptoms:

• npm ERR! 404 Not Found — a package was unpublished or the registry is unreachable.
• pip install fails with version conflicts.
• Could not resolve dependencies in Maven or Gradle.

Diagnosis and fix:

# Reproduce locally first — clear caches to match CI
npm cache clean --force
rm -rf node_modules package-lock.json
npm install

# For Python projects
pip install --no-cache-dir -r requirements.txt

# Check if a specific package version still exists
npm view some-package@1.2.3

# Pin your lock file and commit it
git add package-lock.json
git commit -m "fix: pin dependency lock file"

Common root causes:

• Missing lock file — always commit package-lock.json, yarn.lock, Pipfile.lock, or poetry.lock.
• Private registry authentication — CI does not share your local .npmrc or pip.conf. Configure registry tokens as CI secrets.
• Node/Python version mismatch — specify exact versions in your CI config to avoid surprises.

# GitHub Actions — pin Node.js version
steps:
  - uses: actions/setup-node@v4
    with:
      node-version: '20.11.0'
      cache: 'npm'

Docker Build Failures

Docker builds in CI fail for different reasons than locally, primarily due to caching differences and resource limits.

# Reproduce the CI build locally without cache
docker build --no-cache -t myapp:test .

# Check for multi-stage build issues
docker build --target builder -t myapp:builder .

# Inspect layers when debugging size issues
docker history myapp:test

Frequent Docker build errors:

• COPY failed: file not found — the build context does not include the file. Check your .dockerignore.
• Disk space exhaustion — CI runners have limited disk. Use multi-stage builds and clean up in the same layer.
• Platform mismatch — building on ARM locally but CI runs AMD64, or vice versa.

# GitLab CI — Docker build with proper layer caching
build:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_BUILDKIT: "1"
  script:
    - docker build
        --cache-from $CI_REGISTRY_IMAGE:latest
        --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE:latest
        .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

Test Failures

Tests that pass locally but fail in CI — colloquially known as "flaky tests" — are among the most frustrating issues.

Common causes:

• Timing-dependent tests — tests that rely on sleep() or wall-clock time.
• Missing environment variables — test code that reads from .env files not present in CI.
• Database/service dependencies — tests expecting a running database or external API.
• Order-dependent tests — tests that pass in sequence but fail when run in parallel or random order.

# Run tests locally in the same order as CI
npx jest --runInBand --forceExit

# For Python, randomize test order to catch dependencies
pip install pytest-randomly
pytest --randomly-seed=12345

# Run only the failing test with verbose output
npx jest --verbose --testPathPattern="failing-test"
pytest -xvs tests/test_failing.py

# GitHub Actions — add service containers for integration tests
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: testpass
          POSTGRES_DB: testdb
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - run: npm test
        env:
          DATABASE_URL: postgres://postgres:testpass@localhost:5432/testdb

Resolving Merge Conflicts

Merge conflicts are not a CI failure per se, but they block merges and can cause pipeline failures when automatic merges are attempted.

# Update your branch with the latest target branch
git fetch origin
git checkout feature-branch
git rebase origin/main

# When conflicts appear, Git tells you which files are affected
git status

# Open each conflicted file, resolve the markers:
#   <<<<<<< HEAD
#   (your changes)
#   =======
#   (incoming changes)
#   >>>>>>> origin/main

# After resolving each file
git add resolved-file.txt

# Continue the rebase
git rebase --continue

# If things go badly wrong, abort and start over
git rebase --abort

For complex conflicts, use a three-way merge tool:

# Configure a merge tool (e.g., VS Code)
git config --global merge.tool vscode
git config --global mergetool.vscode.cmd 'code --wait --merge $REMOTE $LOCAL $BASE $MERGED'

# Launch the merge tool during conflict resolution
git mergetool

Merge vs. Rebase: Rebasing produces a cleaner history but rewrites commits. Use git merge origin/main instead of rebase when working on shared branches with other developers, to avoid rewriting commits others have already pulled.

Recovering from Force Push

A force push (git push --force) overwrites the remote history. If someone accidentally force-pushes over your work, recovery is possible using git reflog.

# The reflog records every position HEAD has pointed to
git reflog

# Output looks like:
# a1b2c3d HEAD@{0}: pull: Fast-forward
# e4f5g6h HEAD@{1}: commit: add user auth module
# i7j8k9l HEAD@{2}: commit: update API endpoint

# Find the commit before the force push and reset to it
git reset --hard HEAD@{1}

# Or create a new branch from the lost commit
git checkout -b recovered-work e4f5g6h

# Push the recovered branch
git push origin recovered-work

Important: The reflog only exists on machines that had the commits locally. If you never pulled those commits, you cannot recover them from your local machine. Check with teammates or look at CI build artifacts that may have checked out the lost commits.

To prevent force push disasters, use --force-with-lease instead:

# Safe force push — fails if remote has commits you haven't seen
git push --force-with-lease origin feature-branch

# Set it as default behavior
git config --global alias.fpush 'push --force-with-lease'

Docker Builds in CI

Optimizing Docker builds for CI reduces build times dramatically. The key levers are layer caching, multi-stage builds, and BuildKit.

# GitHub Actions — Docker build with GitHub Container Registry and cache
jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/setup-buildx-action@v3

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Structure your Dockerfile to maximize cache hits:

# GOOD: Copy dependency files first, install, then copy source
FROM node:20-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --production=false
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package.json ./
EXPOSE 3000
CMD ["node", "dist/index.js"]

Secrets and Environment Variables in Pipelines

Misconfigured secrets are a silent pipeline killer — the pipeline runs but the application fails at runtime or deploys with missing configuration.

GitHub Actions Secrets

# Referencing secrets in a workflow
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        env:
          API_KEY: ${{ secrets.API_KEY }}
          DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
        run: |
          echo "Deploying with configured secrets..."
          ./deploy.sh

# Set secrets via CLI
gh secret set API_KEY --body "sk-abc123"

# Set from a file (useful for multiline secrets like SSH keys)
gh secret set DEPLOY_KEY < ~/.ssh/deploy_key

# List configured secrets (values are hidden)
gh secret list

# Set environment-specific secrets
gh secret set API_KEY --env production --body "sk-prod-xyz"

GitLab CI Variables

# .gitlab-ci.yml — use variables
deploy:
  stage: deploy
  script:
    - echo "Deploying to $DEPLOY_HOST"
    - scp -i $SSH_PRIVATE_KEY ./build/* user@$DEPLOY_HOST:/app/
  variables:
    DEPLOY_HOST: "production.example.com"
  only:
    - main

Debugging missing secrets:

# In your CI script, check if a secret is set (without printing it)
if [ -z "$API_KEY" ]; then
  echo "ERROR: API_KEY is not set"
  exit 1
fi
echo "API_KEY is configured (length: ${#API_KEY})"

Common pitfalls:

• Secrets are not available to pull request builds from forks (security feature).
• Secrets are not available in if: conditionals in GitHub Actions.
• GitLab variables marked as "protected" are only available on protected branches.
• Multiline secrets (SSH keys, certificates) need careful handling — use base64 encoding if your platform truncates at newlines.

Branch Protection and Deploy Keys

Branch Protection Rules

Branch protection prevents direct pushes and enforces quality gates, but it can also block automated workflows if not configured correctly.

# GitHub — configure branch protection via CLI
gh api repos/{owner}/{repo}/branches/main/protection \
  --method PUT \
  --field required_status_checks='{"strict":true,"contexts":["ci/test","ci/build"]}' \
  --field enforce_admins=true \
  --field required_pull_request_reviews='{"required_approving_review_count":1}' \
  --field restrictions=null

When your pipeline fails because of branch protection:

• "Required status check is failing" — the check name in your CI config must exactly match the name configured in branch protection.
• "Push rejected" — your bot or CI user does not have bypass permissions. Use a deploy key or app token with the right permissions.

Deploy Keys

Deploy keys provide repository-specific SSH access without using a personal account.

# Generate a deploy key pair
ssh-keygen -t ed25519 -C "deploy-key-myrepo" -f deploy_key -N ""

# Add the public key to the repository (read-only by default)
gh repo deploy-key add deploy_key.pub --title "CI Deploy Key"

# For write access (pushes, tags)
gh repo deploy-key add deploy_key.pub --title "CI Deploy Key" --allow-write

# Add the private key as a CI secret
gh secret set DEPLOY_KEY < deploy_key

# Clean up the local key pair
rm deploy_key deploy_key.pub

Using the deploy key in a GitHub Actions workflow:

steps:
  - uses: actions/checkout@v4
    with:
      ssh-key: ${{ secrets.DEPLOY_KEY }}

Troubleshooting Quick Reference

Symptom	Likely Cause	Fix
npm ci fails with ERESOLVE	Lock file out of sync	Run npm install locally, commit package-lock.json
Docker build fails on COPY	File excluded by .dockerignore	Check .dockerignore, adjust paths
Tests pass locally, fail in CI	Missing env vars or services	Add service containers, check secret config
Pipeline never triggers	Wrong branch filter or path filter	Check on: / only: / rules: in CI config
Permission denied on push	Branch protection or missing token	Use deploy key or PAT with proper scopes
Secrets appear as empty strings	Forked PR or protected variable	Run on base repo branch, check variable protection
Force push lost commits	No --force-with-lease	Recover via git reflog, set alias
Cache not working in Docker	BuildKit not enabled	Set DOCKER_BUILDKIT=1, use cache-from

Prevention and Best Practices

• Always commit lock files. This ensures CI installs exactly the same dependency versions as your local environment.
• Pin CI tool versions. Specify exact versions for Node.js, Python, Docker, and any CLI tools in your workflows.
• Use --force-with-lease instead of --force. Make it a global Git alias so you never accidentally overwrite remote history.
• Enable branch protection on main/production branches. Require status checks and code review before merging.
• Validate secrets at the start of pipelines. A simple length check prevents long build times only to fail at the deploy step.
• Reproduce failures locally. Use act for GitHub Actions or run GitLab CI jobs in Docker to reproduce the CI environment on your machine.
• Keep Docker images small. Use multi-stage builds, alpine base images, and proper .dockerignore files to reduce build time and attack surface.
• Rotate secrets on a schedule. Treat CI secrets like production credentials — rotate them regularly and audit access.
• Monitor pipeline duration. A suddenly slow pipeline often indicates a caching regression or a dependency that pulls too much data.
• Document your CI/CD setup. Keep a brief description of each pipeline step in your repository README or a dedicated docs/ci.md file.