Skip to main content
Get up and running with the Vals Platform in minutes. This guide walks you through creating an account, setting up a test suite, and running your first evaluation.

​
1. Create an Account

Sign up at platform.vals.ai and confirm your email.

​
2. Create a Test Suite

Once logged in, navigate to the Suites page and click + New Suite. Quick start: Click the dropdown on the right of the button and select New Suite from Library to import a pre-built suite (e.g. Basic Examples, LegalBench, CUAD). To create a suite from scratch:
  1. Give your suite a title and description.
  2. Click Add Test to create your first test.
  3. Enter a Test Input — the prompt you want to send to the model.
  4. Add one or more Checks — each check verifies a specific aspect of the model’s output (e.g. includes, is_concise, grammar).
For a full walkthrough, see Creating a Test Suite.

​
3. Run Your Suite

Click the Start Run button in the upper-right corner of the test suite page. Choose a model (e.g. GPT-4o), configure any parameters, and press Run. Once the run completes, you can view results including pass rates, confidence scores, and per-check feedback on the Results page.

​
4. (Optional) Use the SDK

You can also create and run test suites programmatically with the Python SDK.

​
Install

pip install valsai

​
Authenticate

Create an API key from the Admin page of the platform, then set it in your environment: 
export VALS_API_KEY=<YOUR_API_KEY>

​
Create and Run a Suite

from vals import Suite, Test, Check

async def main():
    suite = Suite(
        title="My First Suite",
        tests=[
            Test(
                input_under_test="What is QSBS?",
                checks=[
                    Check(operator="includes", criteria="C Corporation"),
                    Check(operator="grammar"),
                ]
            )
        ],
    )
    await suite.create()

    run = await suite.run(model="openai/gpt-4o-mini", wait_for_completion=True)
    print(f"Run URL: {run.url}")
    print(f"Pass rate: {run.pass_rate}")
For more details, see the full SDK documentation.

​
Next Steps