Promptfoo

Promptfoo is a CLI/library that runs declarative evaluation configs against LLM prompts, comparing outputs across providers (OpenAI, Anthropic, Azure, Bedrock, Ollama, and others), and a separate red-teaming mode that generates adversarial probes to find prompt injection, jailbreak, data-leak, and other vulnerabilities in a live LLM application. Evaluations run 100% locally — prompts and results don't leave the machine unless you choose to share them — and there's no agent or SDK to install in the target app; it tests black-box or gray-box against anything reachable over HTTP. It plugs into CI/CD for regression testing on every prompt or model change. Source: GitHub README (https://github.com/promptfoo/promptfoo) and Promptfoo docs (https://www.promptfoo.dev/docs/intro/).

25,234 GitHub stars and 2,331 forks as of 2026-09-18 (https://api.github.com/repos/promptfoo/promptfoo), with 346+ recorded contributors (https://www.promptfoo.dev/about). The npm package took 2,633,102 downloads in the 30 days ending 2026-09-16 (https://api.npmjs.org/downloads/point/last-month/promptfoo), and the repo shipped a release (0.123.0) as recently as 2026-09-10 with a further code push on 2026-09-17, so this is an actively maintained project, not a dormant one. Promptfoo itself claims (unverified by us) more than 350,000 developers have used it, 130,000 monthly active, and adoption at over 25% of the Fortune 500 — company claim, stated on its own blog (https://www.promptfoo.dev/blog/promptfoo-joining-openai/) and repeated in OpenAI's acquisition announcement.

On 2026-03-09 OpenAI announced it had agreed to acquire Promptfoo, to fold its testing/red-teaming technology into OpenAI Frontier, OpenAI's platform for building AI "coworkers" (https://openai.com/index/openai-to-acquire-promptfoo/). Promptfoo's own announcement confirms the open-source project will remain open source and MIT-licensed, and that the team — co-founders Ian Webster (CEO) and Michael D'Angelo (CTO), plus a roughly 23-person company — will continue supporting existing users (https://www.promptfoo.dev/blog/promptfoo-joining-openai/). Both announcements describe the deal as subject to "customary closing conditions"; we found no independent source confirming the acquisition has since closed, and the GitHub organization, npm package, and release cadence are all still running under the standalone promptfoo name as of 2026-09-18. For a buyer, the practical question isn't whether the code still works — it does, and is still shipping — but whether you're comfortable with an AI-security testing tool being owned by one of the model vendors (OpenAI) it's used to independently evaluate. That's a trade-off to weigh, not a defect.

Community is free and open source (MIT), with all evaluation features, all model-provider integrations, and red teaming up to 10,000 probes per month included at no charge; it can run locally or be self-hosted (https://www.promptfoo.dev/pricing). Enterprise (fully managed SaaS) and Enterprise On-Prem (self-hosted with a dedicated runner, for complete data isolation) add team/SSO/RBAC, continuous monitoring, a compliance dashboard, and priority support with an SLA — but neither publishes a price; both require contacting sales for a custom quote (https://www.promptfoo.dev/pricing). That means a buyer can start free and stay free for a meaningful amount of red-teaming before hitting the probe cap, but can't get a real number for anything beyond Community without a sales call.

The core package is MIT-licensed, confirmed against the repo's own LICENSE file (https://raw.githubusercontent.com/promptfoo/promptfoo/main/LICENSE) — no source-available or BSL clause. No open vulnerabilities were found for the promptfoo npm package in OSV.dev or in GitHub's security-advisories feed for the repo, checked 2026-09-18 (https://api.osv.dev/v1/query, https://api.github.com/repos/promptfoo/promptfoo/security-advisories — both empty). Promptfoo publishes a Trust Center (Vanta-hosted, at https://trust.promptfoo.dev) and a public status page (https://status.promptfoo.dev), though we could not verify current uptime figures from the status page — it returned an internal data-fetch error at time of check rather than a working history.