What DIY Logging Typically Includes
When engineering teams propose building AI compliance logging in-house, they typically envision extending existing infrastructure with these components:
Application-Level Logging
Standard logging frameworks (Log4j, Winston, Python logging) instrumented to capture AI-related events: model invocations, input/output pairs, latency metrics, and error states. This forms the foundation of most DIY approaches.
WORM Storage
Write-Once-Read-Many storage solutions (AWS S3 Object Lock, Azure Immutable Blob Storage) to prevent log tampering after the fact. The assumption is that immutable storage solves the integrity problem.
Log Aggregation
Centralized log management through tools like Elasticsearch, Splunk, or Datadog. These provide search, visualization, and alerting capabilities across distributed systems.
Retention Policies
Automated lifecycle management to retain logs for required periods (typically 5-7 years for financial services, variable for healthcare) and delete them when retention periods expire.
This architecture works well for operational monitoring, debugging, and basic audit trails. But it fundamentally fails at proving control effectiveness—the actual requirement for AI compliance.
Why DIY Logging Fails Regulatory Evidence Standards
The gap between "we logged it" and "we can prove it" represents the core failure mode of DIY AI compliance infrastructure. Here’s why:
1. Logs Are Mutable Before Storage
Even with WORM storage, there’s a critical window between when an event occurs and when the log reaches immutable storage. During this window—typically milliseconds to seconds, but sometimes minutes during high load—logs can be modified, filtered, or dropped entirely.
An application can log "PII scan completed successfully" while actually having crashed mid-scan. The log reflects what the code intended to record, not what actually happened. There’s no cryptographic binding between the log entry and the actual system state at execution time.
The Attestation Gap
Logs record intent to log. Attestation records proof of execution. A log entry saying "consent verified" doesn’t prove consent was verified—it proves the logging code ran. These are different claims with different evidentiary weight.
2. No Proof Controls Actually Executed
Application logs capture that code paths were invoked, not that controls produced correct outcomes. Consider a bias detection control:
- DIY log: "Bias detection ran on model output"
- What auditors need: Cryptographic proof that bias detection evaluated these specific inputs, applied this specific policy version, produced this specific score, and triggered this specific action—with the entire chain signed at execution time
The log tells you the function was called. It doesn’t prove the function worked correctly, used the right parameters, or that the logged output matches what was actually returned.
3. Missing Cryptographic Chain of Custody
Regulatory evidence requires proving that data wasn’t altered between creation and presentation. DIY logging typically relies on:
- Access controls: Only authorized systems can write logs (but authorized systems can write anything)
- WORM storage: Logs can’t be modified after write (but says nothing about what was written)
- Audit logs: We log access to logs (turtles all the way down)
None of these prove the log content accurately reflects reality. Cryptographic attestation creates a signed, timestamped record at the moment of execution that cannot be fabricated retroactively.
4. Lacking Inference-Level Granularity
AI compliance requires evidence at the inference level—each individual AI decision must be traceable. Most DIY logging captures:
- Batch-level metrics ("processed 10,000 requests")
- Sampled requests (1% of traffic for cost reasons)
- Aggregated statistics (average latency, error rates)
When a regulator asks "show me the controls that executed for this specific patient’s treatment recommendation," batch metrics don’t answer the question. You need the complete attestation chain for that individual inference.
5. The Self-Attested Evidence Problem
DIY logging creates a fundamental credibility issue: you’re asking your own system to vouch for itself. The same codebase that might have a compliance bug is also responsible for logging that compliance worked.
This is why financial audits require external auditors, why legal proceedings require independent witnesses, and why regulatory compliance increasingly requires third-party verification. Self-attestation isn’t inherently invalid, but it carries lower evidentiary weight—and in adversarial situations (litigation, regulatory enforcement), it’s the first thing challenged.
The GLACIS Approach: Cryptographic Attestation at Execution Time
GLACIS takes a fundamentally different approach to AI compliance evidence. Instead of logging after the fact, GLACIS creates cryptographic attestations at the moment controls execute.
How It Works
Inline Attestation
GLACIS integrates directly into your AI pipeline. When a control executes (PII detection, bias check, consent verification), GLACIS captures the inputs, outputs, policy version, and execution context in real-time.
Cryptographic Signing
Each attestation is cryptographically signed and timestamped at creation. This creates tamper-evident evidence that cannot be fabricated retroactively—the signature proves the attestation existed at that moment.
Chain of Custody
Attestations are linked cryptographically, creating an immutable chain from execution to storage. Any modification breaks the chain—providing mathematical proof of integrity.
Audit-Ready Evidence
When auditors or regulators request evidence, GLACIS produces complete attestation chains with cryptographic verification. No "trust us, the logs are accurate"—the math proves it.
DIY Logging vs GLACIS: Feature Comparison
| Capability | DIY Logging | GLACIS Attestation |
|---|---|---|
| Evidence Type | Events occurred | Controls executed correctly |
| Tamper Evidence | WORM after write (gap exists) | Cryptographic from execution |
| Granularity | Batch/sampled | Every inference |
| Chain of Custody | Access controls | Mathematical proof |
| Verification | Self-attested | Independently verifiable |
| Litigation Resilience | Easily challenged | Cryptographically defensible |
| Audit Prep Time | Weeks to months | Hours to days |
| Policy Version Tracking | Manual/error-prone | Automatic per-attestation |
Total Cost of Ownership Analysis
The true cost of DIY AI compliance logging extends far beyond initial development. Here’s a realistic breakdown:
DIY Logging: Hidden Costs
5-Year TCO: DIY AI Compliance Logging
| Cost Category | Year 1 | Years 2-5 | 5-Year Total |
|---|---|---|---|
| Engineering Build | $300K-$500K | — | $400K |
| Ongoing Maintenance | $50K | $200K/yr | $850K |
| Infrastructure (storage, compute) | $75K | $100K/yr | $475K |
| Audit Preparation | $75K | $75K/yr | $375K |
| Regulatory Updates | $25K | $50K/yr | $225K |
| Total (excludes failure costs) | $525K | $425K/yr | $2.3M+ |
These figures exclude the cost of failure—when DIY logging doesn’t meet regulatory requirements. Based on industry data, organizations that discover compliance gaps during audits or litigation face:
- Remediation rebuild: $200K-$400K to fix fundamental architecture issues
- Regulatory penalties: Highly variable, but GDPR fines average $2.1M
- Litigation settlements: Average $2.1M when evidence is successfully challenged
- Reputational damage: Difficult to quantify but often exceeds direct costs
What Auditors and Regulators Actually Require
Different regulatory frameworks have different specific requirements, but they converge on common principles for AI compliance evidence:
Financial Services (SR 11-7, OCC)
- + Model inventory with version control
- + Ongoing monitoring of model performance
- + Audit trail of model changes and decisions
- + Independent validation of risk controls
Healthcare (HIPAA, FDA)
- + PHI access logging with 6-year retention
- + Integrity verification for audit logs
- + Clinical decision support traceability
- + Post-market surveillance for AI/ML devices
EU AI Act (High-Risk Systems)
- + Automatic logging of system operation
- + Traceability throughout AI lifecycle
- + Conformity assessment documentation
- + Quality management system evidence
Common Thread
- ! Proof of control effectiveness, not just existence
- ! Integrity verification of evidence
- ! Independent validation capability
- ! Inference-level traceability
When Logs Are Challenged: A Litigation Scenario
Consider what happens when AI compliance evidence faces legal scrutiny:
Scenario: Lending Discrimination Lawsuit
A class action alleges your AI lending model discriminates against protected classes. Plaintiffs’ attorneys request evidence that your fairness controls worked for specific denied applications.
With DIY Logging
You produce logs showing "bias detection ran." Opposing counsel asks:
- "How do you prove these logs weren’t modified?"
- "Can you prove the logged output matches what the model actually returned?"
- "Your own system created this evidence—how is that independent verification?"
Result: Logs excluded or given minimal weight. Settlement pressure increases.
With GLACIS Attestation
You produce cryptographic attestation chains. Your expert testifies:
- "Each attestation is cryptographically signed at execution time"
- "Modification would break the signature—mathematically provable"
- "The chain shows exactly what inputs, outputs, and policies applied"
Result: Evidence admitted with high probative value. Defensible compliance posture.
When DIY Logging Might Be Appropriate
Despite its limitations for compliance evidence, DIY logging may be appropriate in specific scenarios:
- Internal analytics only: If logs are purely for operational monitoring with no regulatory exposure, DIY may suffice
- Minimal-risk AI systems: Under EU AI Act classification, minimal-risk systems have no mandatory logging requirements
- Proof-of-concept phase: Before production deployment, simpler logging may be adequate for development
- Existing robust infrastructure: Organizations with mature, audited compliance infrastructure may extend rather than replace
However, once AI systems affect decisions about people—lending, hiring, healthcare, insurance—the evidence standard rises beyond what DIY logging can provide.
Migration Path: DIY to GLACIS
Organizations with existing DIY logging infrastructure can migrate to GLACIS attestation without disrupting current operations:
Assessment (Week 1)
GLACIS reviews your current AI pipeline and logging infrastructure. We identify control points requiring attestation and integration approach.
Parallel Deployment (Weeks 2-3)
GLACIS attestation runs alongside existing logging. Both systems capture data, allowing comparison and validation without risk.
Validation (Week 4)
Verify attestation coverage matches compliance requirements. Audit simulation confirms evidence meets regulatory standards.
Production (Ongoing)
GLACIS becomes primary compliance evidence source. Existing logging continues for operational purposes. You now have both operational visibility and defensible compliance evidence.
Frequently Asked Questions
Can’t we just add cryptographic signing to our existing logs?
Signing logs after creation doesn’t solve the fundamental problem—the log content may not accurately reflect what happened. The signature proves the log wasn’t modified after signing, but says nothing about whether the logged content was accurate at creation. GLACIS signs at execution time, capturing the actual system state, not a log message about it.
What about blockchain-based logging solutions?
Blockchain provides immutability after write but shares the same "garbage in, garbage out" problem as other approaches. If you write inaccurate data to a blockchain, you have an immutable record of inaccurate data. The value isn’t in the storage mechanism—it’s in what you capture and when you capture it.
How does GLACIS attestation affect latency?
GLACIS is designed for production AI systems. Attestation adds single-digit millisecond overhead—negligible compared to typical AI inference latency. For latency-critical applications, GLACIS supports asynchronous attestation patterns that capture evidence without blocking the response path.
Do we still need our existing logging if we use GLACIS?
Yes—and you should keep it. Application logging serves operational purposes: debugging, performance monitoring, alerting. GLACIS serves compliance purposes: proving controls worked. These are complementary, not competing capabilities. Think of GLACIS as your compliance evidence layer, built on top of (not replacing) your operational infrastructure.
What if we’re already mid-audit with DIY logging?
GLACIS can provide Evidence Pack within days for urgent compliance situations. While it won’t retroactively create attestations for past events, it immediately begins building defensible evidence going forward. For current audit gaps, we work with your team to document the transition and present a credible compliance roadmap.