Governance-in-the-Loop: Runtime Policy Enforcement for Autonomous and Distributed AI Systems
DOI:
https://doi.org/10.63282/3050-9416.IJAIBDCMS-V7I2P114Keywords:
AI Governance, Runtime Enforcement, Reference Monitor, Distributed AI, Policy Enforcement, Complete Mediation, Autonomous Agents, Delegation Safety, Action Modification, Modify OutcomeAbstract
AI governance mechanisms today are predominantly procedural. Documentation standards, audits, and risk assessments improve transparency but do not constrain runtime behavior. As AI systems evolve into autonomous, distributed platforms that invoke tools, spawn sub-agents, and operate across services, governance violations manifest as execution events rather than documentation failures. This structural mismatch prevents existing approaches from providing enforceable guarantees. We introduce Governance-in-the-Loop (GiL), a runtime architecture that embeds non-bypassable policy enforcement directly into AI execution primitives. GiL integrates Governance Enforcement Points (GEPs) into schedulers, model invocation paths, and inter-service communication layers. Policies support three outcomes: permit, deny, and modify – unlike deny, which cascades into workflow failure, modify preserves system availability by transforming the action into a policy-compliant alternative before execution. Each decision is bound to a verifiable audit artifact. We formalize governance as a complete mediation problem over distributed execution traces, define enforcement invariants, formally argue two core safety properties, and demonstrate differentiation from existing policy enforcement systems. The central argument is that enforceable AI governance requires architectural embedding, not procedural overlay.
References
1. D. Amodei et al., "Concrete Problems in AI Safety," arXiv:1606.06565, 2016.
2. J. P. Anderson, Computer Security Technology Planning Study, U.S. Air Force ESD-TR-73-51, 1972.
3. R. Bommasani et al., "On the Opportunities and Risks of Foundation Models," arXiv:2108.07258, Stanford CRFM, 2021.
4. L. Floridi et al., "AI4People An Ethical Framework for a Good AI Society," Minds and Machines, vol. 28, pp. 689–707, 2018.
5. A. Jobin, M. Ienca, and E. Vayena, "The global landscape of AI ethics guidelines," Nature Machine Intelligence, vol. 1, pp. 389–399, 2019.
6. B. Lampson, "Protection," Proc. Princeton Conf. on Information Sciences and Systems, pp. 437–443, 1971.
7. B. D. Mittelstadt et al., "The ethics of algorithms: Mapping the debate," Big Data & Society, vol. 3, no. 2, 2016.
8. I. D. Raji et al., "Closing the AI accountability gap," in Proc. ACM FAccT, pp. 33–44, 2020.
9. J. H. Saltzer and M. D. Schroeder, "The Protection of Information in Computer Systems," Proc. IEEE, vol. 63, no. 9, pp. 1278–1308, 1975.
10. F. B. Schneider, "Enforceable security policies," ACM TISSEC, vol. 3, no. 1, pp. 30–50, 2000.
11. M. Leucker and C. Schallhart, "A brief account of runtime verification," J. Logic and Algebraic Programming, vol. 78, no. 5, pp. 293–303, 2009.
12. Y. Falcone et al., "A taxonomy for classifying runtime verification tools," Int. J. Software Tools Tech. Transfer, vol. 23, pp. 255–284, 2021.
13. NVIDIA, "NeMo Guardrails: A toolkit for controllable and safe LLM applications," arXiv:2310.10501, 2023.
14. Guardrails AI, "Guardrails: Adding guardrails to large language models,"[Online].Available:https://github.com/guardrails-ai/guardrails, 2023.
15. Open Policy Agent, "OPA: An open source, general-purpose policy engine," [Online]. Available: https://www.openpolicyagent.org, 2023.
16. Y. Bai et al., "Constitutional AI: Harmlessness from AI Feedback," arXiv:2212.08073, Anthropic, 2022.
17. Istio Authors, "Istio: Connect, secure, control, and observe services," [Online]. Available: https://istio.io, 2023.
18. K. Greshake et al., "Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection," in Proc. ACM AISec, 2023.
19. C. Baier and J.-P. Katoen, Principles of Model Checking, MIT Press, 2008.
20. V. Costan and S. Devadas, "Intel SGX Explained," IACR Cryptol. ePrint Arch., Report 2016/086, 2016.
21. European Parliament, "Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act)," Official Journal of the European Union, 2024.
22. European Parliament, "Regulation (EU) 2016/679 (General Data Protection Regulation)," Official Journal of the European Union, 2016.
23. NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), National Institute of Standards and Technology, Gaithersburg, MD, 2023.
