For decision-makers
Before you deploy: the questions that actually matter
Most agent pitches focus on capability. The questions that actually decide whether a small company can run one in production are less exciting, and rarely get asked up front. Here they are.
Which model, and what does it actually cost?
Model choice is a real trade-off, not a footnote. A model hosted in your own jurisdiction may cost many times more per request than a US or Chinese alternative — but for most project-management workloads the absolute volume is small enough that the difference is cents per query, not a line item that changes the business case. Ask for the real per-request number before you decide it matters.
Who pays for usage, and how is it metered?
Someone needs an account with a spending cap from day one — not after the first surprising bill. Decide up front whether you or your vendor holds that account, and set a small hard limit while you're still validating the pilot.
Where does it actually run?
"In the cloud" isn't an answer. A well-scoped deployment runs in an isolated container on infrastructure you already control, with a small, auditable codebase — not a black-box service with access to everything. Ask how many lines of code you'd actually need to read to audit it.
Does your data train anyone else's model?
Get an explicit answer, in writing, not an assumption. The safer default: logs and queries stay on your own infrastructure, and anything a vendor wants to reuse — even anonymized — comes back to you for approval first.
Can this be talked about publicly?
Decide separately whether you're comfortable being named in public case studies versus mentioned only in private conversation with peers. These are genuinely different levels of exposure, and a vendor should ask which one you're agreeing to rather than assume.
Is the existing contract actually enough?
An NDA and standard terms of service may already cover a pilot. But it's worth explicitly checking that assumption rather than discovering a gap later — especially once the agent starts proposing or taking actions, not just reporting.
What happens when it's wrong, and who finds out?
This is the one most pitches skip. An agent that can't tell you how confident it is will eventually state a wrong answer as a fact. Ask specifically what the agent does when it's uncertain — does it guess, hedge, or stop and ask a person — before you ask what it can do when it's right. See why enterprise agents still fail for the underlying reason this matters more than raw capability.
A practical way to start
Start read-only. A reporting-only phase — the agent answers questions about data that already exists, with permissions respected, and changes nothing — lets you validate accuracy, latency and trust with no downside risk before it's allowed to propose or take any action. That's the sequence we're following on our own pilot; see the roadmap.