YourAITools

LLM Evaluation Bundle: RAG + Prompt Testing

Know whether a prompt edit, model switch or chunking change made your AI feature better or worse.

LLM Evaluation Bundle: RAG + Prompt Testing product illustration
Download$29

Prompt Regression Testing Kit

Prompt testing tool: YAML test cases, 17 checks incl. JSON Schema and LLM grader, flaky detection, baseline regressions, JUnit. OpenAI, Anthropic, Ollama. $29.

View Prompt Regression Testing Kit
Download$29

RAG Evaluation Kit

RAG evaluation kit: hit@k, recall, MRR, nDCG, faithfulness, context recall and answer F1 on your own questions. BM25 baseline, chunk sweep, CI gates. $29.

View RAG Evaluation Kit

What it is

Two dependency-free Python tools for teams shipping LLM features: test your prompts like code, and measure your RAG pipeline on your own questions.

FolderWhat it does
plugins/prompt-regression-testing-kit/YAML test suites for prompts, 17 assertion types (incl. JSON Schema and an LLM grader), flaky detection, baselines, JUnit/Markdown reports for CI
plugins/rag-evaluation-kit/retrieval hit@k, recall, MRR, nDCG; faithfulness that flags invented numbers; context recall, answer F1; BM25 baseline and chunk-size sweep; run comparison
./run_tests.sh   # runs both tools' tests
./demo.sh        # runs both demos offline (no API keys)

Bought separately: $29 + $29 = $58. Both work with OpenAI, Anthropic, OpenRouter, Ollama/any OpenAI-compatible server, or your own program.

What you get

The screenshot is real output from the kit’s own commands. Tested from a fresh download of this exact zip: ./run_tests.sh passed on Python 3.13 and 3.10.

Real terminal output from LLM Evaluation Bundle: RAG + Prompt Testing: demo and passing tests

Use with Claude Code

/plugin marketplace add ./llm-evaluation-bundle
/plugin install rag-evaluation-kit@youraitools-llm-eval

Honest limits

Scores are only as good as your test cases; LLM-graded checks depend on the grader model. The limits in each tool's README apply.

Format and license

Zip, 63 KB, v1.0, 52 files. You may use and modify this kit for your own work and for clients you serve, including paid client work. Not affiliated with or endorsed by OpenAI, Anthropic or Ollama.

FAQ

Are these the same as the single kits?
Yes, the same two kits.
Do I need API keys?
Not for the demos and tests; real runs use your OpenAI, Anthropic, OpenRouter key or a local Ollama server.
Do they replace promptfoo or Ragas?
They cover the core checks without dependencies; those projects have far more features and integrations.
How do I get the files?
Pay by card right here on youraitools.co: your download links appear on this page the moment the payment goes through (claim window: 72 hours, up to 5 downloads per 7 days). Need the files re-sent later? Email hello@youraitools.co.
Refunds?
Email hello@youraitools.co within 14 days if it doesn’t work and we can’t fix it.
Can I pay with crypto?
Yes. Pay the same price in USDC or USDT on Base, Ethereum, Arbitrum, Optimism, Polygon or Solana from the "Pay with crypto" button, with no account or KYC. We check the payment on-chain and show your download links as soon as it confirms.

Get LLM Evaluation Bundle: RAG + Prompt Testing

LLM evaluation tools: prompt regression tests with baselines and CI, plus RAG metrics (hit@k, MRR, nDCG, faithfulness). Python, no deps. $45 ($58 separately).

$45 one-time ($58 separately)

Buy and downloadPay with crypto

Pairs well with

Download$29

Vibe Coding Security Audit

Security audit for AI-built apps: exposed API keys, NEXT_PUBLIC secrets, Supabase service_role, open API routes, Stripe webhooks, CORS. Grade + fixes. $29.

View Vibe Coding Security Audit
Download$19

Supabase RLS Checker

Supabase RLS checker for migrations: tables without RLS, using (true) policies, user_metadata, slow auth.uid(), definer views. Fix SQL included. $19.

View Supabase RLS Checker
Download$19

Dockerfile Linter & Best Practices Checker

Dockerfile linter with fixes: non-root user, pinned slim images, secrets in ENV/ARG, apt cache, COPY order, .dockerignore. SARIF + Claude Code skill. $19.

View Dockerfile Linter & Best Practices Checker

Related

1 email a week: new x402 + MCP tools and free kits.

Short weekly digest of new x402 + MCP tools and free kits, including the MCP OAuth Checker. No spam, unsubscribe any time.