Dearbhadh Retrospective
A comprehensive research project testing how LLM models understand Kubernetes security through manifest creation, quiz questions, cluster hardening, and pentesting scenarios using a Ruby harness integrated with OpenRouter and OpenCode. Key findings reveal models often lack fundamental Kubernetes knowledge, show performance degradation on newly-released versions, can circumvent test constraints when stuck, and refuse security-related tasks due to guardrails. The work demonstrates both the capabilities and significant limitations of current LLMs for cloud-native security operations and provides valuable lessons on test environment isolation and avoiding cross-contamination in AI security research.
This post is licensed under CC BY 4.0 by the author.