Comments URL: https://news.ycombinator.com/item?id=47948513

Points: 1

# Comments: 0

The practical answer I've seen hold up: push column-level redaction before the agent layer, not after. A logical replica with PII columns replaced by null or a stable hash gives you the same query surface, plus one audit row per session at the connection pooler, not the app. The AI team gets its data, you get a hard boundary that doesn't rely on prompt engineering.

The harder question is ownership. In a startup where the ML lead, the infra person and the security person are often the same tired CTO at 10pm, the right answer depends on who gets paged when a hallucinated query wakes up the primary. Usually the answer is nobody, which is the real problem behind the technical one.

we went through something similar building a fintech tool — spent months on features nobody asked for because we assumed we understood the problem. what actually changed it: stopped building for what we assumed and started building for one specific thing one customer said they'd pay for that day. just one. then the next.

what was the specific use case you landed on?

we went through something similar building a fintech tool — spent months on features nobody asked for because we assumed we understood the problem. what actually changed it: stopped building for what we assumed and started building for one specific thing one customer said they'd pay for that day. just one. then the next.

what was the specific use case you landed on?

IAM Identity Center (was just called SSO before the rename) with a short session duration -- we do 8-hour max with MFA at login -- handles the traveling employee case cleanly. They re-authenticate, no ticket. Overhead compared to running a VPN server you have to patch is basically zero.

The one thing that bit us: we kept a bastion sitting around "just in case" for way too long before we cleaned it up. It was live for almost a year after we didn't need it anymore. What's the main access pattern you're trying to solve -- DB access, SSH to EC2, or something different?

The "isolate the risk" part is where most early CTOs underestimate the blast radius. A shortcut in the auth layer feels isolated until six months later when three other systems are built on top of assumptions it makes. By then it's not a shortcut anymore, it's load-bearing technical debt.

What's the recovery actually looked like in practice when you've caught it early enough — do you get clean rewrites or is it mostly containment?

One thing worth thinking about beyond detection: even a perfect scanner at install time doesn't protect against skills that start clean and phone home post-install. The runtime layer is a different problem -- restricting what a skill process can actually touch (outbound network, credential paths, filesystem writes outside its own dir) probably matters as much as the intake scan. Seccomp or at minimum per-skill network namespacing would close that gap.

Did any of the 824 original malicious skills survive all 6 passes, or were they each caught by at least one detector?

What I'd add to the defense stack: structured output validation that flags known internal naming patterns before they reach the client, plus anomaly detection on response metadata -- token budget shifts, refusal rates, response shape changes under semantic pressure. At 75% convergence you're past "interesting research" into "reliable extraction technique," which means you need a detection layer, not just prevention.

Have you tested this against non-OpenAI models like Claude or Gemini to see if the naming bleed is GPT-4o-specific or a broader training corpus problem?

SOC 2 and ISO 27001 auditors are now explicitly asking for evidence of data flow controls around AI integrations. "Policy says don't paste customer data into ChatGPT" gets zero credit as a control. What they want to see is technical enforcement with logging — something you can point to and say "here is the record of every outbound prompt from the last 90 days, here is what was flagged." If you can't produce that, it's a finding.

The other gap that's easy to miss: most teams don't know what's sensitive until it's already in a prompt. The classification problem comes before the proxy. What are your teams actually pulling into their prompts right now — internal docs, support tickets, code with credentials, database outputs? That's usually where the real exposure surfaces.

What's worth paying attention to here is that Telus Digital is a BPO/outsourcing company, which means the blast radius almost certainly extends to their clients. If your company has API integrations with Telus Digital, or gave them any kind of federated access to internal systems, now is the time to audit what they held and rotate anything they could have touched. Downstream credential exposure in third-party breaches is consistently underreacted to.

The employee data angle is also interesting that usually means developer workstations and internal tooling were in scope, not just the customer-facing layer. Makes the "how did detection miss it" question even harder. Does anyone know if Telus Digital ran a shared SOC across their outsourcing clients, or was each client siloed?

Vibe coding is genuinely useful for prototyping — I use it. But there's a difference between using AI to move faster and outsourcing your understanding to it entirely. The second one catches up with you the moment something breaks and you have no instinct about where to look.

Curious what the actual product was — did it make it to users before things fell apart, or did it not get that far?

I've seen it go wrong both ways — CTOs who gold-plate systems nobody uses yet, and founders who promise customers features that are a month away from being architecturally possible. Both happen because the two sides aren't seeing the same thing.

The best early-stage CTOs I've watched work well are the ones who treat "no we can't do that" as a last resort. What feature did you promise that you genuinely couldn't deliver, and what did you actually ship instead?

For where you are now (building on top of an existing AI platform, adding domain intelligence and decision workflows), you don't need a co-founder who can invent a new transformer. You need someone who can move fast, understand the product deeply, and ship without a lot of hand-holding. That profile is often a strong contractor or early hire, not someone who needs equal equity to stay motivated.

The co-founder model makes sense when there's genuine tech risk that only a senior technical person can see around — when you're building the infrastructure itself. You're not. The risk you described is clearly on the product and GTM side, which you already own. What's your current thinking on timeline to first deployed version with a real supply chain customer?

What I've seen work is treating LLM API calls like you'd treat external logging — strip or redact anything that looks like a credential before it leaves your process. A simple regex on the request payload costs almost nothing and catches the lazy-paste case.

Are you seeing this as a widespread pattern in your testing, or did this surface from one specific integration?

The CI/CD gap matters a lot too. AI-generated code tends to skip the boring scaffolding: no branch protection, no secret scanning in the pipeline, no way to roll back safely if something goes wrong in prod. That stuff is invisible until it isn't.

What's the actual blocker for the projects you're seeing stall — is it the infrastructure setup itself, or is it more that the founders aren't confident the code is production-safe?

The CI integration is the piece I'd actually lean on first. Most teams I've seen think about agent access controls after they've already deployed, at which point you're doing cleanup instead of prevention. Gating it in the pipeline means the access question gets answered before the agent is running against your Terraform state and live kube contexts.

Are you seeing any patterns in severity distribution — mostly cloud creds coming up critical, or are the kube context exposures landing higher than expected?

The closest parallel I've seen in practice is OAuth scope creep from a few years back — teams installing third-party integrations with broad permissions and never reviewing them. At least those had a permission dialog and an audit log. Agent skills install with one command and the full attack surface is whatever the agent can do in your shell, including your cloud credentials and prod contexts.

What's your signal on whether the malicious installs are actively being exploited or mostly sitting dormant? Wondering if there's any telemetry on runtime execution vs. just install counts.

What makes vendor breaches particularly painful to respond to is that your incident response playbook doesn't really apply. You can't isolate the affected system, you can't pull logs from their infra, and your customers are asking you questions you literally cannot answer for 48-72 hours. The only real leverage you have is contractual — SLAs around breach notification, security attestations, right-to-audit clauses — and most orgs don't negotiate those until after something like this happens.

If you're a startup that processes data through third-party SaaS tools, what's your current process for assessing vendor security posture before integration? Questionnaire-based, SOC 2 report review, something else?

The GPU compute diversion is also underreported as a cost signal. If you have any agentic workloads, you probably want anomaly detection on GPU utilization per job, not just billing alerts — by the time your bill spikes, the damage is already days old.

What runtime isolation are you seeing orgs actually deploy for agent workloads? gVisor, Firecracker, something else? Curious whether this is pushing people toward stronger VM-level boundaries or if network egress controls are the more practical mitigation.