Deployment succeeded but app still serves previous version

TL;DR — If the deploy pipeline says success but users still get the old version, the most common cause is that you updated the origin but a CDN/reverse proxy/browser is serving cached assets or HTML. Start by comparing a version endpoint or commit header from origin vs public edge with curl -I; if they differ, purge cache or fix cache-control for HTML before touching the app servers. Reading time: ~6 min

The scenario

You push a Tuesday afternoon deploy, CI goes green, the artifact upload finishes, and your hosting dashboard says the service is healthy. But the homepage still shows last week's copy change, /api/health says the old commit SHA, and your teammate on mobile sees something different again. You SSH to a node and the new files are there, so now you're stuck between "deploy succeeded" and "production is lying." The client is already asking why the bug they reported is "still live after the fix."

Symptoms

• Public site shows old UI/content after a successful deploy.
• curl to the public URL returns an old version header or old HTML checksum.
• One request shows new code, the next shows old code, usually behind a load balancer.
• Static assets keep old content despite new files on disk.
• Service logs show restart/reload failures after the deploy step, but the pipeline did not treat them as fatal.
• Common observable outputs:

$ curl -s https://app.example.com/version
{"commit":"7c1e2ab","built_at":"2026-08-10T13:02:11Z"}

$ curl -s http://127.0.0.1:3000/version
{"commit":"d94f6c1","built_at":"2026-08-10T15:41:55Z"}

$ curl -I https://app.example.com/
HTTP/2 200
cache-control: public, max-age=3600
age: 1842
etag: "a8f3-66b1f2d1"
via: 1.1 varnish
x-served-by: cache-lhr1234-LHR

$ for i in {1..6}; do curl -s https://app.example.com/version; echo; done
{"commit":"d94f6c1"}
{"commit":"7c1e2ab"}
{"commit":"d94f6c1"}
{"commit":"7c1e2ab"}

$ systemctl status myapp.service --no-pager
● myapp.service - My App
     Loaded: loaded (/etc/systemd/system/myapp.service; enabled)
     Active: active (running) since Tue 2026-08-10 15:40:02 UTC; 2h ago
...
Aug 10 15:39:58 host systemd[1]: myapp.service: Main process exited, code=exited, status=1/FAILURE
Aug 10 15:40:02 host systemd[1]: Started myapp.service.

Likely causes

Step-by-step diagnosis

1. Compare public vs origin version.

curl -s https://app.example.com/version
curl -s http://127.0.0.1:3000/version

If public returns an older commit than the local origin process, jump to Fixes → CDN/reverse-proxy/browser cache serving old HTML or assets or Fixes → Wrong origin/host updated. If both are old, jump to step 3.

1. Check response headers for cache evidence.

curl -I https://app.example.com/
curl -I https://app.example.com/static/app.js

This is your problem if you see age:, via:, x-cache: HIT, cf-cache-status: HIT, or HTML with cache-control: public, max-age=....

Example:

HTTP/2 200
cache-control: public, max-age=3600
age: 1842
x-cache: HIT

Jump to Fixes → CDN/reverse-proxy/browser cache serving old HTML or assets.

1. Check whether requests alternate between old and new versions.

for i in {1..10}; do curl -s https://app.example.com/version; echo; done

If you get mixed commit SHAs, at least one backend is stale. Jump to Fixes → Load balancer still sending traffic to an old instance/pod/node.

1. Verify the running release pointer or container image on the host.

readlink -f /var/www/current
ps -ef | grep -E 'gunicorn|node|java' | grep -v grep
docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.RunningFor}}'

This is your problem if /var/www/current points to an older release directory, or the running container image/tag is not the one you just deployed. Jump to Fixes → New build exists on disk/image, but running process still points at old release.

1. Inspect service restart/reload status and deploy logs.

systemctl status myapp.service --no-pager
journalctl -u myapp.service -n 100 --no-pager
grep -nE 'reload|restart|systemctl|docker compose up|kubectl rollout' deploy.log

This is your problem if you see status=1/FAILURE, ExecStartPre failures, reload failed, or your script uses ; instead of && and continued after an error. Jump to Fixes → Service reload/restart failed, but deploy script ignored the exit code.

1. Confirm you're hitting the intended origin and vhost.

dig +short app.example.com
curl --resolve app.example.com:443:ORIGIN_IP -s https://app.example.com/version
nginx -T | grep -n 'server_name app.example.com'

If the direct-origin response is new but the public DNS target is old, or the web server routes app.example.com to the wrong root/upstream, jump to Fixes → Wrong origin/host updated.

1. Check whether HTML references unhashed or unchanged asset URLs.

curl -s https://app.example.com/ | grep -Eo 'src="[^"]+\.(js|css)[^"]*|href="[^"]+\.(js|css)[^"]*'

If asset URLs are stable like /static/app.js and your cache headers are long-lived, clients will keep old JS/CSS. Jump to Fixes → Asset filename/versioning bug causing clients to reuse old JS/CSS.

Fixes

CDN/reverse-proxy/browser cache serving old HTML or assets

Purge edge cache for HTML immediately, then fix cache headers so HTML is not cached aggressively while hashed assets can be.

For nginx serving HTML:

location = / {
    add_header Cache-Control "no-cache, no-store, must-revalidate" always;
    try_files /index.html =404;
}

location /assets/ {
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}

Reload nginx:

nginx -t && systemctl reload nginx

Purge cache in your provider dashboard (e.g. CDN/Web Application Firewall section → Cache → Purge URL or Purge Everything). If you have shell access to a reverse proxy cache host, clear the cache directory only if you know the impact.

⚠️ Purging all edge cache can spike origin traffic and cause a brief latency increase or origin overload.

For browser verification, bypass local cache:

curl -H 'Cache-Control: no-cache' -I https://app.example.com/

Verify it worked:

curl -I https://app.example.com/ | grep -Ei 'cache-control|age|x-cache|cf-cache-status'

You want HTML to show no-cache/no-store and no stale Age on the old response.

Load balancer still sending traffic to an old instance/pod/node

Drain or remove stale backends, then redeploy or restart only those nodes.

If you manage VMs behind nginx/upstream or HAProxy, compare versions per backend directly:

for ip in 10.0.1.11 10.0.1.12 10.0.1.13; do echo "== $ip =="; curl -s http://$ip:3000/version; echo; done

Restart the stale node's app service:

ssh 10.0.1.12 'systemctl restart myapp.service && sleep 2 && curl -s http://127.0.0.1:3000/version'

For Kubernetes:

kubectl get deploy myapp -o wide
kubectl get pods -l app=myapp -o wide
kubectl rollout restart deploy/myapp
kubectl rollout status deploy/myapp --timeout=120s

If one old pod survives because of a bad selector or manual canary, fix the Service selector or delete the stale pod:

kubectl delete pod <old-pod-name>

Verify it worked:

for i in {1..10}; do curl -s https://app.example.com/version; echo; done | sort | uniq -c

You want exactly one commit SHA.

New build exists on disk/image, but running process still points at old release

Update the release symlink or container tag, then restart the process.

Typical symlink-based deploy:

ls -1 /var/www/releases
ln -sfn /var/www/releases/20260810T154155Z /var/www/current
readlink -f /var/www/current
systemctl restart myapp.service

Docker Compose example:

docker compose pull app
docker compose up -d --force-recreate app
docker inspect --format '{{.Config.Image}} {{.Image}}' $(docker compose ps -q app)

Trade-off: mutable tags like :latest hide drift. Prefer immutable tags like commit SHA.

Verify it worked:

curl -s http://127.0.0.1:3000/version

Service reload/restart failed, but deploy script ignored the exit code

Fix the service error, then make the deploy fail hard on non-zero exit codes.

Inspect the failure:

journalctl -u myapp.service -n 100 --no-pager
systemctl restart myapp.service; echo $?

Common fix for environment or syntax issues:

nginx -t
node --check server.js
python -m py_compile app.py
systemctl daemon-reload
systemctl restart myapp.service

Harden the deploy script:

set -euo pipefail
trap 'echo "deploy failed at line $LINENO"' ERR
./build.sh
./migrate.sh
systemctl restart myapp.service
curl -fsS http://127.0.0.1:3000/version

If using shell chains, replace this:

./build.sh; systemctl restart myapp.service; echo done

with this:

./build.sh && systemctl restart myapp.service && echo done

Verify it worked:

systemctl is-active myapp.service && curl -s http://127.0.0.1:3000/version

Wrong origin/host updated (deployed to staging, old DNS target still live, wrong vhost)

Confirm DNS and vhost routing, then update the correct target.

Check DNS and direct-origin behavior:

dig +short app.example.com
curl --resolve app.example.com:443:203.0.113.10 -s https://app.example.com/version
curl --resolve app.example.com:443:203.0.113.11 -s https://app.example.com/version

If only one origin has the new version, update the stale host or remove it from DNS/LB. For nginx vhost mismatch, inspect config:

nginx -T | sed -n '/server_name app.example.com/,+20p'

Correct the root or proxy_pass, then reload:

server {
    server_name app.example.com;
    location / {
        proxy_pass http://127.0.0.1:3000;
    }
}

nginx -t && systemctl reload nginx

Verify it worked:

curl -s https://app.example.com/version

Asset filename/versioning bug causing clients to reuse old JS/CSS

Switch to content-hashed filenames and short-cache HTML.

Bad pattern:

<script src="/static/app.js"></script>
<link rel="stylesheet" href="/static/app.css">

Good pattern:

<script src="/assets/app.d94f6c1.js"></script>
<link rel="stylesheet" href="/assets/app.a13b9e2.css">

If your framework supports it, enable manifest-based hashed output. If not, append a build identifier as a temporary workaround:

<script src="/static/app.js?v=d94f6c1"></script>

Then set cache headers accordingly:

location /assets/ {
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}
location = /index.html {
    add_header Cache-Control "no-cache" always;
}

Verify it worked:

curl -s https://app.example.com/ | grep -Eo '/(assets|static)/[^"'"' ]+'

You want a new asset URL on each build when content changes.

Prevention

• Expose a version endpoint and header from the running app, then assert it after deploy.

curl -fsS https://app.example.com/version | jq -e '.commit == env.GIT_SHA'
curl -I https://app.example.com/ | grep -F "x-release-sha: $GIT_SHA"

• Fail deploys on stale public version, not just successful artifact upload.

set -euo pipefail
PUBLIC_SHA=$(curl -fsS https://app.example.com/version | jq -r .commit)
test "$PUBLIC_SHA" = "$GIT_SHA"

• Pin immutable image tags and record them in deployment manifests.

image: registry.example.com/myapp:d94f6c1
imagePullPolicy: IfNotPresent

• Add a cache policy split: HTML uncacheable, hashed assets immutable.

location = /index.html { add_header Cache-Control "no-cache, no-store, must-revalidate" always; }
location /assets/ { add_header Cache-Control "public, max-age=31536000, immutable" always; }

• Add backend consistency checks behind the load balancer.

for ip in $(getent ahostsv4 app-internal.example.com | awk '{print $1}' | sort -u); do curl -s http://$ip:3000/version; echo; done | sort | uniq -c

Alert if more than one SHA appears.

• In CI/CD, treat service restart and health checks as blocking steps.

systemctl restart myapp.service
systemctl is-active --quiet myapp.service
curl -fsS http://127.0.0.1:3000/healthz

A deploy is not complete until the running process, the public edge, and every backend agree on the same version.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.