Data use, training, and retention
- Is customer content (prompts, files, outputs) used to train or improve models? Is that the default for our plan, and is it stated in the contract?
- How long are prompts, outputs, and files retained? Can we configure retention, including zero retention?
- Are prompts or outputs reviewed by humans (for abuse monitoring or quality)? Under what conditions?
- Where is data processed and stored? Can we choose a region?
- Can we delete our data on request, and how is deletion confirmed?
- Is data used to build profiles or for advertising?
Model providers and sub-processors
Many AI products are built on models from other companies. Your data may flow to the vendor and to one or more model providers.
- Which model providers process our data?
- Do the vendor's terms with those providers match what the vendor promises us (training, retention)?
- Is there a published sub-processor list and a change-notification process?
Security posture
- Independent attestations (e.g., SOC 2 Type II report, ISO/IEC 27001 certificate) — request the report, not just the logo.
- SSO (SAML/OIDC) and SCIM provisioning on our plan.
- MFA enforcement.
- Admin audit logs, and whether they can be exported to our SIEM.
- Tenant isolation approach.
- Vulnerability disclosure program and breach notification commitment (timeline).
Contracts
- Data processing agreement (DPA) for personal data.
- Business Associate Agreement (BAA) if PHI will be processed — confirm it covers the specific product and plan.
- Confidentiality terms appropriate for privileged or trade-secret information.
- Intellectual property: who owns outputs; indemnities for IP claims where offered.
- Liability caps and security obligations.
- Termination assistance and data return.
Capabilities in your environment
The review should cover what the product can do once connected — not just what it stores.
- Which OAuth scopes or API permissions does it request? Are they the minimum?
- Can it send, write, delete, execute, or pay? Can those actions require approval?
- Does it browse the web or process external content (prompt-injection exposure)?
- Can admins restrict features, connectors, and plugins?
Tier the review to the risk
| Tier | Criteria | Review depth |
|---|---|---|
| Tier 1 | Regulated data (PHI, card data, privileged) or high-impact actions | Full questionnaire, attestation report, contract review, security sign-off |
| Tier 2 | Internal or personal data, read-only | Standard questionnaire, DPA, admin controls check |
| Tier 3 | Public data only, no integrations | Lightweight checklist and policy acknowledgement |
Record the outcome — approved, approved with conditions, or rejected — with the date and reviewer. FlashTower's vendor fields (DPA, BAA, no-training, retention, security review) feed directly into the risk rules for every AI system that uses that vendor.
Red flags
- The vendor cannot say which model providers process your data.
- Training on customer data is on by default and cannot be disabled contractually.
- No way to configure or confirm deletion of prompts and files.
- No SSO on any plan, or SSO only at a price tier far above your need, with no alternative controls.
- No audit logs available to customers.
- Marketing claims of compliance ("HIPAA compliant", "GDPR certified") with no supporting agreement or report.
- OAuth requests for broad write scopes when the product only needs to read.
Ongoing monitoring
Vendor review is not one-time. AI vendors ship features quickly — new connectors, agent modes, memory, and browsing — and each can change the risk profile. Subscribe to vendor changelogs and sub-processor notifications, re-review Tier 1 vendors annually, and re-review any vendor when it introduces a capability that lets it act in your environment.
A short AI vendor questionnaire
For Tier 2 vendors, these ten questions are usually enough. Ask for answers in writing and keep them with the review record:
- Is our content used to train or improve any model? Where is that stated contractually?
- What is the default retention for prompts, outputs, and files on our plan, and what is the minimum configurable?
- Which third-party model providers and sub-processors process our data, and in which regions?
- Do humans at your company or your providers ever review our content? When?
- Which independent security attestations can you share (report, not logo)?
- Do you support SSO and enforce MFA on our plan?
- Are admin and user activity logs available to us, and can they be exported?
- What permissions does your product request in our systems, and can they be reduced?
- Can your product take actions (send, write, delete, execute)? Can those require approval?
- What is your breach-notification commitment, and who is our security contact?
Frequently asked questions
Is a SOC 2 report enough to approve an AI vendor?
No. A SOC 2 report addresses security controls, but not whether the vendor trains on your data, how long it retains prompts, or which model providers see your data. Ask those questions separately.
Do we need a BAA for every AI tool?
Only for tools that will create, receive, maintain, or transmit PHI on your behalf. The practical rule: if PHI may go in, a BAA covering that specific product and plan must be in place first.
How long should an AI vendor review take?
With a tiered approach, low-risk tools can be reviewed in a day or two, standard tools within one to two weeks, and high-risk tools handling regulated data in a few weeks. Speed matters: slow reviews push employees toward unapproved alternatives.
What if the vendor won't answer our questions?
Treat unanswered questions as unfavorable answers. For regulated data or high-impact capabilities, an unwilling vendor is usually a reason to choose another product or restrict the use to public data.