How Long to Keep Security Logs and What Retention Really Costs

TL;DR — Most organizations should keep searchable security logs for 30–90 days for active investigation, then keep cheaper archived copies for 6–24 months depending on legal, contractual, and incident-response needs. The key mistake is treating all logs the same: keep high-value security events longer, reduce noisy data early, and move older logs from expensive search storage into cheaper archive storage. Reading time: ~7 min

What it is and where it sits

Security log retention means deciding which security-related events you keep, for how long, and in what storage tier. In an Extended Detection & Response (XDR) setup, logs are the raw evidence that lets you answer questions like: "Was this login normal?", "Which systems talked to the attacker?", and "How long were they inside before we noticed?"

A log is a time-stamped record of an event. In security, common sources are:

• identity systems (login success/failure, MFA challenges)
• firewalls and web application firewalls
• operating systems and endpoint tools
• cloud audit trails (who changed what)
• application access logs
• DNS and network flow logs (records of connections)

Retention is not just a storage setting. It sits between:

• log producers: apps, servers, cloud services, identity providers
• collection: agents, syslog, cloud export, webhooks
• analysis: SIEM/XDR (security tools that search and correlate events)
• archive: cheaper long-term storage
• investigators and auditors: the people who need evidence later

What it replaces is ad hoc behavior like "keep everything forever in the SIEM" or "delete after 7 days because disks fill up." Both are expensive in different ways.

In a typical flow, a user action creates an event, the event is collected into a search system for fast investigation, and older data is rolled into archive storage.

[User] ---- 1. login attempt ----> [App / IdP]
[App / IdP] ---- 2. event log ----> [Collector]
[Collector] ---- 3. normalize/filter ----> [SIEM / XDR]
[Analyst] <---- 4. search recent data ---- [SIEM / XDR]
[SIEM / XDR] ---- 5. age out / export ----> [Archive Storage]
[Analyst] <---- 6. restore older logs ---- [Archive Storage]

The important architecture point: recent logs and old logs do not need to live in the same place. Fast search storage is the expensive part. Archive storage is usually much cheaper, but slower to query.

How it actually works

Here is one realistic end-to-end example.

A customer support employee signs in from a new country at 2:13 AM, then downloads a large set of customer records. Three months later, you discover the account was compromised and need to reconstruct what happened.

Step 1: Events are created at the source

The identity provider logs:

• login success
• impossible travel or unusual location alert
• MFA result

Your app logs:

• account used
• endpoint accessed
• number of records exported
• source IP address

Your cloud platform logs:

• whether any permissions were changed
• whether storage objects were copied or deleted

Step 2: Collection sends logs into your security platform

These events are forwarded by an agent, syslog, or cloud export into your SIEM/XDR. On the way in, many teams do two useful things:

• normalize fields (rename different vendor fields into a common shape like user.email or source.ip)
• filter obvious noise (for example, health checks or duplicate allow logs)

This matters because cost usually scales with either ingested volume (how much data arrives) or stored/searchable volume (how much stays in the expensive tier).

Step 3: Recent logs stay hot; older logs move to archive

Suppose your policy is:

• 60 days searchable in the SIEM/XDR
• 12 months archived in object storage (internet-accessible file storage like S3-compatible buckets)

For the first 60 days, analysts can search instantly. After 60 days, the platform deletes or exports the logs from the hot tier and keeps a compressed copy in archive.

Step 4: An incident happens after 90 days

At day 90, the analyst asks:

• Did the user log in from a new country before the export?
• What IP addresses were involved?
• Were admin permissions changed around the same time?

The last 30 days are still searchable. The older 60–90 day window is no longer in the hot tier, so the team restores the archived logs or queries them in the archive system.

Step 5: Why the retention choice mattered

If you had kept only 30 days, the evidence would be gone. If you had kept 12 months fully searchable, the investigation would be easier but much more expensive. If you had archived the right logs, you still have evidence, just with a slower workflow.

That is the core mechanism: shorter retention in fast search, longer retention in cheap archive, with different rules by log type.

When to use it (and when not to)

You do need a deliberate retention policy if you rely on logs for security investigations, customer commitments, insurance questionnaires, or compliance reviews. You probably do not need year-long full-text search on every web request.

A practical default for many teams

If you want a starting point without overthinking it:

• 90 days searchable for authentication, admin actions, cloud audit, endpoint alerts, VPN, and critical application audit logs
• 12 months archived for the same categories
• 30 days searchable for high-volume lower-value logs like generic web access logs
• 90-180 days archived for those lower-value logs, unless a legal requirement says otherwise

You probably do not need this if...

• you are keeping logs only because "storage is cheap" but nobody reviews or uses them
• the logs contain sensitive personal data you do not actually need
• your only reason for long retention is vague fear, not a real incident-response or legal need

In those cases, reduce scope before increasing retention.

Trade-offs

Every retention decision is a trade between evidence and cost.

What retention usually costs in practice

Vendors price differently, but the cost model usually has some combination of:

• ingest cost: price per GB/day sent in
• hot storage cost: price for keeping data searchable
• archive cost: much lower price per GB/month
• restore/query cost: extra charge or operational time when pulling old data back

A simple way to estimate your own cost is:

daily log volume x retention days x compression factor x storage tier price

Example:

• 20 GB/day of security logs
• 90 days searchable
• 365 days archived total
• hot tier stores roughly near raw/indexed size depending on platform
• archive is compressed and much cheaper

The business lesson is simple: the biggest savings usually come from reducing noisy logs before ingestion, not from shaving a few archive days.

In practice

Below are two practical examples you can adapt.

Example 1: S3-compatible lifecycle policy for archived security logs

If your provider supports object storage lifecycle rules, use the provider dashboard first: in your storage bucket settings, look for a path like Storage → Buckets → your-bucket → Lifecycle rules. Create a rule that transitions older logs to a colder storage class and deletes them after your chosen retention period.

{
  "Rules": [
    {
      "ID": "security-logs-retention",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "security/"
      },
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    }
  ]
}

This keeps logs under the security/ prefix in a cheaper class after 30 days, colder archive after 90 days, and deletes them after 365 days. Gotcha: retrieval from colder tiers can be slow and may cost extra, so do not move data there if your analysts need it weekly.

CLI example if you do have access:

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-security-logs \
  --lifecycle-configuration file://lifecycle.json

Example 2: Linux logrotate policy for local security logs

If an application still writes local audit logs before forwarding them centrally, set a rotation policy so the server does not fill its disk. In a hosting panel, look for a path like Server → Logs → Rotation if your provider exposes it; otherwise this is usually a file on the server.

⚠️ If you set the retention too low here before confirming central forwarding works, you can permanently delete evidence. Verify that logs are arriving in your SIEM/XDR first.

/var/log/myapp/security-audit.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    postrotate
        /usr/bin/systemctl reload myapp >/dev/null 2>&1 || true
    endscript
}

This keeps 30 daily rotated files, compresses older ones, and reloads the app after rotation. Gotcha: local rotation is not a substitute for central retention; if the host is compromised, an attacker may delete local logs.

Example 3: A plain-language retention policy you can adopt internally

This is not machine config, but it is often the missing piece that prevents random decisions.

security_log_retention_policy:
  hot_searchable:
    auth_logs: 90d
    admin_audit_logs: 90d
    cloud_audit_logs: 90d
    endpoint_alerts: 90d
    web_access_logs: 30d
  archive:
    auth_logs: 365d
    admin_audit_logs: 365d
    cloud_audit_logs: 365d
    endpoint_alerts: 365d
    web_access_logs: 180d
  exclusions:
    - healthcheck_requests
    - duplicate_allow_firewall_events
  review_every: 6 months

This gives your agency, security provider, or internal team something concrete to implement. Gotcha: review the exclusions with someone who actually investigates incidents; cutting noisy logs is good, cutting useful evidence is not.

Further reading

• NIST SP 800-92, Guide to Computer Security Log Management
• NIST SP 800-61, Computer Security Incident Handling Guide
• CIS Controls v8, Audit Log Management
• The "Logging" documentation for your SIEM or XDR platform
• The "Lifecycle configuration" section of your cloud object storage docs

This article was written by an AI system and published pending human review. Verify anything you intend to act on.