Security scan passes locally but fails in CI with different findings

TL;DR — When a security scan passes locally but fails in CI with different findings, the most common cause is not "randomness"; it is that you are not scanning the same thing with the same scanner, database, and flags. Start by printing scanner version, rules DB version, commit SHA, checkout depth, and the exact command in both places; in most incidents, pinning the scanner image/version and fetching full git history fixes it. Reading time: ~6 min

The scenario

You push a small dependency bump at 3:40 PM, your local scan is clean, and the pull request pipeline immediately fails on "high severity vulnerabilities" you cannot reproduce. You rerun the same command on your laptop and still get zero findings. The CI logs show a different scanner version than you expected, plus a warning about shallow clone or an updated vulnerability database. Now you need to prove whether the pipeline is scanning a different commit, a different filesystem, or a different rule set before the team starts waiving findings blindly.

Symptoms

• Local run exits 0, CI exits non-zero:

$ trivy fs --exit-code 1 --severity HIGH,CRITICAL .
2026-10-30T15:42:11Z	INFO	Number of language-specific files: 3
2026-10-30T15:42:12Z	INFO	Detected OS: debian
2026-10-30T15:42:13Z	INFO	No vulnerabilities found
$ echo $?
0

[CI] $ trivy fs --exit-code 1 --severity HIGH,CRITICAL .
2026-10-30T15:43:02Z	INFO	Need to update DB
2026-10-30T15:43:14Z	INFO	Detected OS: debian
2026-10-30T15:43:16Z	HIGH	lodash	4.17.20	CVE-2026-12345	fixed in 4.17.21
[CI] Process completed with exit code 1

• Same tool name, different version or DB revision in logs:

Local: semgrep 1.92.0
CI:    semgrep 1.98.0

• CI logs mention checkout depth, missing refs, or inability to compute baseline:

fatal: bad object origin/main
warning: repository is shallow, some history is missing

• Secret scan finds leaked keys only in CI because ignored files differ:

gitleaks: 3 leaks found
Finding: "AKIA..." in path .env.example

• Container/image scan differs because CI builds a different image than local:

Local image: app:dev@sha256:1c2...
CI image:    registry.example.com/app:pr-184@sha256:9af...

• CI scans generated files or vendored directories your local command skipped:

Found 27 findings in dist/, vendor/, coverage/

Likely causes

Step-by-step diagnosis

1. Compare the exact scanner binary and DB/rules revision.

scanner --version
trivy --version && trivy image --download-db-only 2>/dev/null || true
semgrep --version
gitleaks version

If CI and local show different versions, or CI updates a vulnerability DB/ruleset during the job, this is your problem. Jump to Fixes → Scanner version or vulnerability/rules DB differs.

1. Confirm both environments are scanning the same commit and repository history.

git rev-parse HEAD
git rev-parse --is-shallow-repository
git log --oneline --decorate -3
git merge-base HEAD origin/main 2>/dev/null || true

This is your problem if CI is on a merge commit you never scanned locally, true is returned for shallow repo, or origin/main is missing. Jump to Fixes → CI is scanning a different commit, merge commit, or shallow checkout.

1. Print the exact command, current directory, and ignore files in both places.

pwd
printf '%q ' "$0" "$@"; printf '\n' 2>/dev/null || true
env | sort | grep -E 'SCAN|TRIVY|SEMGREP|GITLEAKS'
find . -maxdepth 3 \( -name '.semgrepignore' -o -name '.trivyignore' -o -name '.gitleaks.toml' -o -name '.gitignore' \) -print

This is your problem if CI runs from repo root but you ran from ./services/api, or CI adds --no-git-ignore, --config auto, --severity HIGH,CRITICAL, or scans dist/ and vendor/. Jump to Fixes → Different scan scope, ignore rules, or working directory.

1. Compare the actual dependency set or built image digest.

sha256sum package-lock.json yarn.lock pnpm-lock.yaml poetry.lock go.sum Gemfile.lock 2>/dev/null
npm ci --ignore-scripts && npm ls --all >/tmp/npm-tree.txt 2>/dev/null || true
docker image inspect app:dev --format '{{.Id}}' 2>/dev/null || true

This is your problem if lockfiles differ, CI installs from a regenerated lockfile, or the image digest in CI is not the one you scanned locally. Jump to Fixes → CI has different dependency/image contents than local.

1. Check whether CI environment variables, proxies, or auth tokens change scanner behavior.

env | sort | grep -E 'HTTP_PROXY|HTTPS_PROXY|NO_PROXY|TOKEN|KEY|TRIVY|SEMGREP|GITLEAKS'
curl -I https://registry-1.docker.io/v2/ 2>/dev/null | sed -n '1,5p'

This is your problem if CI can reach update endpoints and local cannot, or CI injects org-wide policy tokens/config. Jump to Fixes → Environment-specific auth/network changes rule or DB updates.

Typical output shape when a proxy or auth path differs:

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: Basic realm="corp-proxy"

1. If the scan runs in baseline/diff mode, verify the target branch ref exists.

git branch -a
git rev-parse origin/main 2>/dev/null

This is your problem if the scanner compares against origin/main in CI but that ref is absent locally or vice versa. Jump to Fixes → Baseline/diff mode depends on git history or target branch refs.

[Local scan] ---- 1. same cmd? ----> [Same scanner?]
                                   |
                                   v
                           [Same DB/rules?]
                                   |
                                   v
                           [Same commit/tree?]
                                   |
                                   v
                           [Same scope/ignores?]
                                   |
                                   v
                         [Same deps/image/env?]
                                   |
                                   v
                               [Root cause]

Fixes

Scanner version or vulnerability/rules DB differs

Pin the scanner version and stop relying on whatever the runner image currently has.

# Example: use a pinned container image locally and in CI
docker run --rm -v "$PWD:/src" -w /src aquasec/trivy:0.56.2 fs --severity HIGH,CRITICAL --exit-code 1 .
docker run --rm -v "$PWD:/src" -w /src returntocorp/semgrep:1.98.0 semgrep scan --config auto .

If the scanner uses an updateable DB, either prefetch the DB in both places or cache it in CI so both runs use the same revision.

trivy image --download-db-only
trivy clean --scan-cache

Verify it worked:

scanner --version && trivy --version

CI is scanning a different commit, merge commit, or shallow checkout

Fetch full history and print the SHA being scanned in the job log.

git fetch --prune --unshallow || git fetch --prune --depth=0
git fetch origin main:refs/remotes/origin/main
git rev-parse HEAD
git rev-parse origin/main

If your CI scans the synthetic merge result for PRs, reproduce that locally instead of scanning only your feature branch tip.

git fetch origin pull/184/head:pr-184 2>/dev/null || true
# Or locally merge main into your branch before scanning
git checkout my-branch && git merge --no-ff origin/main

Verify it worked:

git rev-parse --is-shallow-repository

Expected output:

false

Different scan scope, ignore rules, or working directory

Normalize the command into a checked-in script and call that script everywhere.

cat > scripts/security-scan.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
trivy fs --severity HIGH,CRITICAL --exit-code 1 --skip-dirs dist,vendor,coverage .
EOF
chmod +x scripts/security-scan.sh
./scripts/security-scan.sh

For secret or SAST tools, commit explicit ignore config instead of depending on local shell aliases or ad hoc excludes.

# .gitleaks.toml
[allowlist]
paths = [
  '''^dist/''',
  '''^coverage/'''
]

# .semgrepignore
dist/
vendor/
coverage/

Verify it worked:

./scripts/security-scan.sh

CI has different dependency/image contents than local

Do not scan an unpinned dependency graph. Commit lockfiles and install from them in CI.

npm ci
pip install --require-hashes -r requirements.txt
bundle config set deployment true && bundle install
go mod verify

For container scans, scan the exact built artifact by digest, not a mutable tag.

docker build -t app:ci .
docker image inspect app:ci --format '{{index .RepoDigests 0}}'
trivy image --exit-code 1 "$(docker image inspect app:ci --format '{{.Id}}')"

Verify it worked:

sha256sum package-lock.json 2>/dev/null && docker image inspect app:ci --format '{{.Id}}'

Environment-specific auth/network changes rule or DB updates

If CI has internet access and local does not, local may be using stale rules. Either allow local updates or freeze updates in CI for reproducibility.

# Example: explicit proxy config
export HTTPS_PROXY=http://proxy.internal:8080
export NO_PROXY=localhost,127.0.0.1,.internal

# Example: run with a pre-downloaded DB/cache path
trivy image --download-db-only --cache-dir .cache/trivy
trivy fs --cache-dir .cache/trivy .

If your org injects policy tokens in CI, document that local reproduction requires the same token.

export SEMGREP_APP_TOKEN=redacted
semgrep scan --config auto .

Verify it worked:

env | grep -E 'HTTPS_PROXY|NO_PROXY|SEMGREP_APP_TOKEN|TRIVY'

Baseline/diff mode depends on git history or target branch refs

Disable diff mode while debugging, or fetch the baseline ref explicitly.

git fetch origin main:refs/remotes/origin/main
semgrep scan --baseline-commit "$(git merge-base HEAD origin/main)" .
# or run a full scan to compare behavior
semgrep scan .

For secret scanners using git history, scan the same range in both places.

gitleaks git --log-opts="origin/main..HEAD"

Verify it worked:

git merge-base HEAD origin/main

Prevention

• Pin scanner versions in CI and local dev wrappers.

# scripts/trivy.sh
docker run --rm -v "$PWD:/src" -w /src aquasec/trivy:0.56.2 "$@"

• Print reproducibility metadata at the top of every security job.

echo "SHA=$(git rev-parse HEAD)"
echo "SHALLOW=$(git rev-parse --is-shallow-repository)"
trivy --version || true
semgrep --version || true

• Use full-history checkout for scanners that rely on git context.

git fetch --prune --depth=0

• Check in one repo-local scan script and call only that from CI.

./scripts/security-scan.sh

• Cache and optionally freeze vulnerability DB/rules revisions per pipeline run so reruns are comparable.

trivy image --download-db-only --cache-dir .cache/trivy

• Fail fast when lockfiles or generated artifacts are dirty before scanning.

git diff --exit-code -- package-lock.json yarn.lock pnpm-lock.yaml Dockerfile

This article was written by an AI system and published pending human review. Verify anything you intend to act on.