HomeExpertiseAI Governance Framework

    Technical guide from CloudNSite engineering

    AI Governance Framework Built to Pass Audit, Not Just Sit on a Shelf

    CloudNSite implements AI governance programs grounded in NIST AI RMF, ISO/IEC 42001, and the applicable parts of the EU AI Act. Policy, model registry, use-case registry, risk tiering, technical controls, monitoring, and audit evidence. We build the framework, wire it into the systems that produce the evidence, and operate it alongside the engineering team.

    Plan an AI Governance Framework BuildSee how we work

    System diagram

    AI governance control matrix engineering plate showing four risk tiers (low risk, limited, high risk, prohibited) with use case, data classification, human oversight requirement, audit cadence, and approval authority, plus controls (model card, eval suite, incident log) and the policy, implementation, audit cycle.
    AI Governance Control Matrix

    Direct answer

    An AI governance framework is the documented set of policies, registries, risk tiers, controls, and audit evidence that an organization uses to manage AI systems. Production-grade frameworks align to NIST AI RMF, ISO/IEC 42001, and the EU AI Act, and they are wired into the systems that actually produce the evidence rather than maintained as standalone documents.

    Key definitions

    NIST AI Risk Management Framework
    U.S. National Institute of Standards and Technology framework with four functions (Govern, Map, Measure, Manage) and a Generative AI Profile (NIST AI 600-1) that extends the core for foundation-model use cases.
    ISO/IEC 42001
    International standard for AI management systems published in 2023. Defines requirements for establishing, implementing, maintaining, and continually improving an AI management system. Certifiable by accredited bodies.
    EU AI Act
    European Union regulation that classifies AI systems by risk (unacceptable, high, limited, minimal) and assigns obligations per tier. General-purpose AI model obligations apply since August 2025; high-risk system obligations phase in through August 2026 and 2027.
    Model card
    Structured document describing a model's intended use, training data, evaluation results, limitations, and known failure modes. Required artifact in most governance frameworks before a model can be approved for a use case.
    Risk tier
    Classification (typically low, medium, high, prohibited) assigned to each AI use case based on impact on rights, safety, finances, and operations. Drives the depth of controls, review cadence, and approval gates required.

    Six layers of a working AI governance framework

    An AI governance framework is six interlocking layers, not a single policy document. CloudNSite builds each layer against the systems that produce the actual evidence (model registry, ticketing, logs, evaluation harness) so audit responses can be assembled in hours instead of weeks.

    • Policy layer

      Acceptable use policy, data classification rules, vendor due diligence checklist, and model approval policy. Versioned in the same system as the rest of the corporate policy stack and reviewed on a published cadence.

    • Use-case and model registries

      Two linked registries. The use-case registry holds every AI workflow with owner, risk tier, data classes touched, and approval status. The model registry holds every foundation model and fine-tune with provenance, version, evaluation results, and approval scope.

    • Risk tiering and impact assessment

      Each new use case is scored against NIST AI RMF Map function and EU AI Act risk categories. Tier drives the required controls, the review path, and the monitoring depth.

    • Technical controls

      Prompt injection defenses, output validation, PII redaction, retrieval source allow-lists, structured-output enforcement, rate limits, and a documented kill switch. Each control maps to a NIST AI RMF Measure or Manage subcategory.

    • Monitoring and incident response

      Structured logging of prompts, responses, tool calls, and human overrides. Drift detection on eval scores. Incident runbook with severity tiers, on-call rotation, and stakeholder communication template.

    • Audit evidence pipeline

      Automated extracts from the registries, evaluation harness, log store, and ticketing system into the format auditors actually request. SOC 2, ISO 42001, and EU AI Act technical documentation are generated on demand rather than assembled by hand.

    When to use this

    • You operate in a regulated industry (healthcare, financial services, legal, government) where AI workflows touch protected data or high-impact decisions.
    • You have more than two production AI use cases and need a consistent approval and review process across teams.
    • You are pursuing SOC 2, ISO 42001, or HITRUST and the auditor has asked for AI-specific controls.
    • You operate in or sell into the EU and need to demonstrate EU AI Act compliance posture by tier.
    • Your board, legal team, or insurer has asked for a documented AI risk program.

    When not to use this

    • You have one internal chat assistant with no access to protected data and no customer-facing surface. Lightweight acceptable-use guidance is enough.
    • You only consume off-the-shelf SaaS AI features with no model training, retrieval, or tool calling. Use the vendor's documentation and standard third-party risk review.
    • The team is asking for a one-page policy as a checkbox. A framework is operational infrastructure and only pays off if it actually runs.

    How CloudNSite implements it

    1. 1

      Inventory and tier existing AI use cases

      Pull every AI workflow in the organization (sanctioned and shadow), map each to data classes touched and decisions affected, and assign initial risk tiers against NIST AI RMF Map criteria and EU AI Act categories. Output is the seeded use-case registry.

    2. 2

      Build the policy layer and registries

      Author the acceptable use policy, data classification rules, vendor due diligence checklist, and model approval policy. Stand up the use-case and model registries in the system the rest of the organization already uses (Confluence, Notion, a registry tool, or a custom internal app).

    3. 3

      Implement technical controls per tier

      For each risk tier, implement and document the required controls (prompt injection defenses, output validation, PII redaction, source allow-lists, kill switch). Map each control to NIST AI RMF Manage subcategories and ISO 42001 clauses for cross-referencing.

    4. 4

      Wire monitoring and audit pipeline

      Structured logging across LLM calls, tool invocations, and human overrides. Drift detection on the evaluation harness. Automated extracts that produce SOC 2 evidence, ISO 42001 records, and EU AI Act technical documentation from the live systems.

    5. 5

      Run the framework and tune cadence

      Quarterly use-case review, monthly model approval committee, weekly drift report, on-demand audit response. CloudNSite operates the framework alongside your team and tunes the cadence and depth as the AI portfolio matures.

    Tools and standards we use

    Core framework

    NIST AI Risk Management Framework 1.0 + Generative AI Profile (NIST AI 600-1)

    Primary U.S. framework. Govern, Map, Measure, Manage functions with subcategories that we map controls against.

    Management system standard

    ISO/IEC 42001:2023

    International standard for AI management systems. Certifiable. We structure the registries and policy layer to align with its clause structure.

    Regulatory

    EU AI Act (Regulation 2024/1689)

    Risk-tiered obligations for AI systems and general-purpose AI models. We map use cases to the four risk categories and document the obligations per tier.

    Compliance overlap

    SOC 2, HIPAA, HITRUST

    AI-specific controls slot into existing programs. We coordinate with the security team rather than spin up parallel evidence pipelines.

    Evaluation

    Open-source eval harness plus custom fixtures

    We use tooling like Inspect, OpenAI Evals, or a custom harness depending on stack. The output of the harness is direct input to the monitoring layer.

    Observability

    OpenTelemetry plus structured prompt and response logging

    Required for drift detection, incident response, and audit reconstruction. Secrets and protected data are redacted before storage.

    From the field

    SOC 2 evidence collection automation

    Our SOC 2 evidence automation work documents how a CloudNSite-built pipeline extracts control evidence from live systems on demand. The same architecture extends to AI-specific evidence: model registry exports, evaluation history, drift reports, and use-case approval records.

    Read the full case study

    Frequently asked questions

    Is an AI governance framework just a policy document?

    No. A policy document is one layer of six. A working framework includes use-case and model registries, risk tiering, technical controls, monitoring, and an audit evidence pipeline. If it cannot generate an audit response from live systems, it is not yet a framework.

    Do we need both NIST AI RMF and ISO 42001?

    Most U.S. organizations start with NIST AI RMF because it is a risk framework rather than a certification standard. ISO/IEC 42001 is useful when you want a certifiable management system or when customers ask for it. We map controls against both so the framework is one body of evidence, not two.

    How does the EU AI Act apply to a U.S. company?

    If you place AI systems on the EU market, serve EU users, or your output is used in the EU, EU AI Act obligations can apply. We classify your use cases against the four risk tiers and document the obligations and timelines so the legal team can decide where to draw the scoping line.

    Who runs the framework after CloudNSite builds it?

    CloudNSite operates the framework alongside your security, legal, and engineering teams as part of the ongoing engagement. We monitor drift, run the review committees, respond to audits, and ship updates as standards and regulations evolve. We do not hand over a binder and walk away.

    How long does an AI governance framework implementation take?

    A focused implementation covering existing AI workflows usually runs 8 to 14 weeks. ISO 42001 certification readiness or EU AI Act high-risk system documentation extends the timeline. The first audit-ready evidence pipeline typically lands in the first 4 to 6 weeks.

    Can we use this framework with off-the-shelf AI products like ChatGPT Enterprise?

    Yes. Vendor SaaS sits in the use-case registry with the appropriate risk tier. Vendor due diligence, data classification, and acceptable use rules apply. Technical controls focus on the integration points (data flowing in, outputs being used) rather than the model internals.

    How does this interact with existing SOC 2 or HIPAA programs?

    It extends them rather than replacing them. We map AI-specific controls into the existing control catalogue and reuse the security team's evidence pipeline. The audit response is one program with AI controls included, not two parallel programs.

    What is the difference between AI governance and model risk management?

    Model risk management (MRM) is a discipline mature in financial services, focused on quantitative model validation, monitoring, and governance under regulatory expectations like SR 11-7. AI governance is broader and covers non-quantitative use cases, foundation models, and general-purpose AI. The two overlap and a financial services framework typically integrates both.

    Related CloudNSite work