nagyist · pull · May 9, 2025 · May 11, 2025 · May 11, 2025 · May 11, 2025
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -29,11 +29,18 @@
 			"extensions": [
 				"ms-python.python",
 				"ms-python.vscode-pylance",
+				"ms-python.vscode-python-envs",
 				"charliermarsh.ruff",
 				"mtxr.sqltools",
 				"mtxr.sqltools-driver-pg",
+				"esbenp.prettier-vscode",
+				"mechatroner.rainbow-csv",
 				"ms-vscode.vscode-node-azure-pack",
-                "esbenp.prettier-vscode"
+				"esbenp.prettier-vscode",
+				"twixes.pypi-assistant",
+				"ms-python.vscode-python-envs",
+				"teamsdevapp.vscode-ai-foundry",
+				"ms-windows-ai-studio.windows-ai-studio"
 			],
 			// Set *default* container specific settings.json values on container create.
 			"settings": {

diff --git a/.env.sample b/.env.sample
@@ -37,8 +37,7 @@ OLLAMA_EMBED_MODEL=nomic-embed-text
 OLLAMA_EMBEDDING_COLUMN=embedding_nomic
 # Needed for GitHub Models:
 GITHUB_TOKEN=YOUR-GITHUB-TOKEN
-GITHUB_BASE_URL=https://models.inference.ai.azure.com
-GITHUB_MODEL=gpt-4o
-GITHUB_EMBED_MODEL=text-embedding-3-large
+GITHUB_MODEL=openai/gpt-4o
+GITHUB_EMBED_MODEL=openai/text-embedding-3-large
 GITHUB_EMBED_DIMENSIONS=1024
 GITHUB_EMBEDDING_COLUMN=embedding_3l
diff --git a/.github/workflows/app-tests.yaml b/.github/workflows/app-tests.yaml
@@ -28,10 +28,8 @@ jobs:
       fail-fast: false
       matrix:
         os: ["ubuntu-latest", "macos-latest-xlarge", "macos-13", "windows-latest"]
-        python_version: ["3.9", "3.10", "3.11", "3.12"]
+        python_version: ["3.10", "3.11", "3.12"]
         exclude:
-          - os: macos-latest-xlarge
-            python_version: "3.9"
           - os: macos-latest-xlarge
             python_version: "3.10"
     env:
@@ -123,7 +121,7 @@ jobs:
             key: mypy${{ matrix.os }}-${{ matrix.python_version }}-${{ hashFiles('requirements-dev.txt', 'src/backend/requirements.txt', 'src/backend/pyproject.toml') }}
 
         - name: Run MyPy
-          run: python3 -m mypy .
+          run: python3 -m mypy . --python-version ${{ matrix.python_version }}
 
         - name: Run Pytest
           run: python3 -m pytest -s -vv --cov --cov-fail-under=85

diff --git a/.github/workflows/auto_approve_copilot.yaml b/.github/workflows/auto_approve_copilot.yaml
@@ -0,0 +1,15 @@
+name: Auto-approve Copilot PRs
+
+on:
+  pull_request:
+    types: [opened, synchronize]
+
+jobs:
+  auto-approve:
+    runs-on: ubuntu-latest
+    if: github.actor == 'copilot-swe-agent'
+    steps:
+      - name: Auto-approve PR from Copilot agent
+        uses: hmarr/auto-approve-action@v3
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.gitignore b/.gitignore
@@ -149,3 +149,6 @@ static/
 
 # Playwright test trace
 test-results/
+
+# Test virtual environments
+test_venv*/
diff --git a/.vscode/launch.json b/.vscode/launch.json
@@ -21,6 +21,14 @@
         "module": "uvicorn",
         "args": ["fastapi_app:create_app", "--factory", "--reload"],
         "justMyCode": false
+    },
+    {
+        "name": "Python: Current File",
+        "type": "debugpy",
+        "request": "launch",
+        "program": "${file}",
+        "console": "integratedTerminal",
+        "justMyCode": false
     }
   ],
   "compounds": [

diff --git a/README.md b/README.md
@@ -69,7 +69,7 @@ A related option is VS Code Dev Containers, which will open the project in your
 
     * [Azure Developer CLI (azd)](https://aka.ms/install-azd)
     * [Node.js 18+](https://nodejs.org/download/)
-    * [Python 3.9+](https://www.python.org/downloads/)
+    * [Python 3.10+](https://www.python.org/downloads/)
     * [PostgreSQL 14+](https://www.postgresql.org/download/)
     * [pgvector](https://github.com/pgvector/pgvector)
     * [Docker Desktop](https://www.docker.com/products/docker-desktop/)

diff --git a/docs/images/redteam_dashboard.png b/docs/images/redteam_dashboard.png
diff --git a/docs/images/redteam_logs.png b/docs/images/redteam_logs.png
diff --git a/docs/safety_evaluation.md b/docs/safety_evaluation.md
@@ -1,17 +1,17 @@
 # Evaluating RAG answer safety
 
-When deploying a RAG app to production, you should evaluate the safety of the answers generated by the RAG flow. This is important to ensure that the answers are appropriate and do not contain any harmful or sensitive content. This project includes scripts that use  Azure AI services to simulate an adversarial user and evaluate the safety of the answers generated in response to those adversarial queries.
+When deploying a RAG app to production, you should evaluate the safety of the answers generated by the RAG flow. This is important to ensure that the answers are appropriate and do not contain any harmful or sensitive content. This project includes scripts that use the [azure-ai-evaluation SDK](https://pypi.org/project/azure-ai-evaluation/#history) to perform an [automated safety scan with an AI Red Teaming agent](https://learn.microsoft.com/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent).
 
 * [Deploy an Azure AI project](#deploy-an-azure-ai-project)
 * [Setup the evaluation environment](#setup-the-evaluation-environment)
-* [Simulate and evaluate adversarial users](#simulate-and-evaluate-adversarial-users)
-* [Review the safety evaluation results](#review-the-safety-evaluation-results)
+* [Run red teaming agent](#run-red-teaming-agent)
+* [Review the red teaming results](#review-the-red-teaming-results)
 
 ## Deploy an Azure AI project
 
-In order to use the adversarial simulator and safety evaluators, you need an Azure AI project inside an Azure AI Hub.
+In order to use the Red Teaming agent, you need an Azure AI project inside Azure AI Foundry.
 
-1. Run this command to tell `azd` to provision an Azure AI project and hub:
+1. Run this command to tell `azd` to provision an Azure AI project:
 
     ```shell
     azd env set USE_AI_PROJECT true
@@ -45,63 +45,62 @@ In order to use the adversarial simulator and safety evaluators, you need an Azu
     .evalenv\Scripts\activate
     ```
 
-1. Install the dependencies for the safety evaluation script:
+3. Install the dependencies for the safety evaluation script:
 
   ```bash
   pip install uv
   uv pip install -r evals/requirements.txt
   ```
 
-## Simulate and evaluate adversarial users
+## Run the red teaming agent
 
-Run the following command to simulate adversarial queries and evaluate the safety of the answers generated in response to those queries:
+To run the red teaming agent, you need to have the RAG app running at a deployed URL or local URL. Consult the main README for deployment instructions local server instructions.
+
+In the same terminal where you activated `.evalenv`, run this command to perform a red teaming scan:
 
 ```shell
-python evals/safety_evaluation.py --target_url <TARGET_URL> --max_simulations <MAX_RESULTS>
+python evals/safety_evaluation.py --target_url <TARGET_URL> --questions_per_category <NUM_QUESTIONS>
 ```
 
-* `--target_url`: The target URL for the callback. Default is `http://127.0.0.1:8000/chat`, so make sure that the RAG app is running locally. If you are running the RAG app in a different environment, set this to the correct URL.
-* `--max_simulations`: The maximum number of simulated user queries. Default is `200`. The higher the number, the longer the evaluation will take. The default of `200` simulations will take about 25 minutes to run, which includes both the time to generate the simulated data and the time to evaluate it.
-
-## Review the safety evaluation results
-
-That command will generate a report in `safety_results.json`.
-Results will look like this:
-
-```json
-{
-  "hate_unfairness": {
-    "low_count": 20,
-    "score_total": 8,
-    "mean_score": 0.4,
-    "low_rate": 1.0
-  },
-  "sexual": {
-    "low_count": 20,
-    "score_total": 9,
-    "mean_score": 0.45,
-    "low_rate": 1.0
-  },
-  "violence": {
-    "low_count": 20,
-    "score_total": 9,
-    "mean_score": 0.45,
-    "low_rate": 1.0
-  },
-  "self_harm": {
-    "low_count": 20,
-    "score_total": 10,
-    "mean_score": 0.5,
-    "low_rate": 1.0
-  }
-}
-```
+* `--target_url`: The target URL for the callback. Default is `http://127.0.0.1:8000/chat`, which assumes the RAG app is running locally. If you want to scan a deployed app instead, set this to `https://DEPLOYEDURL/chat`.
+* `--questions_per_category`: The number of questions per risk category to ask during the scan. There are four risk categories (`hate_unfairness`, `sexual`, `violence`, and `self_harm`). The default is 5 questions per category, but you can decrease this number to 1 for test runs, or increase it for a more thorough evaluation. Note that the total questions asked will be higher, as the red teaming agent is configured to try [multiple attack strategies](https://learn.microsoft.com/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent#supported-attack-strategies), where it transforms the original question in different ways.
+
+## Review the red teaming results
+
+That command will save results both in Azure AI Foundry and in the local `evals/redteams` directory.
+
+### Review results in Azure AI Foundry
+
+Navigate from the URL in the terminal output to the results in Azure AI Foundry.
+First you'll see overall attack metrics:
+
+![Red Teaming Agent Results](images/redteam_dashboard.png)
+
+Then you can dig into the data and see the questions and answers received from your app:
+
+![Red Teaming Agent Results](images/redteam_logs.png)
+
+Note that the questions are intentionally adversarial, so you should mentally prepare for that before looking at the results. The goal is to see if your app can handle these adversarial queries and provide safe answers. Even if your scan results in a 0% attack success rate, you should still review the questions and answers to ensure that you're happy with the way your app responds to these adversarial queries.
+
+Learn more in the [Red Teaming Agent documentation](https://learn.microsoft.com/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent#viewing-your-results-in-azure-ai-foundry-project).
+
+## Review results locally
+
+In addition to the results in Azure AI Foundry, you can also review the results locally in the `evals/redteams` directory. The results are saved in JSON format.
+
+Each file in this directory corresponds to a single red teaming scan, and contains the following fields:
+
+* `scorecard`: A summary of the scan results, including the attack success rate and the number of questions asked.
+* `parameters`: The parameters used for the scan, including the risk categories and attack strategies.
+* `attack_details`: A list of the questions asked during the scan, along with the answers received from your app.
+* `studio_url`: A link to the Azure AI Foundry studio where you can view the results in more detail.
 
-The ideal score is `low_rate` of 1.0 and `mean_score` of 0.0. The `low_rate` indicates the fraction of answers that were reported as "Low" or "Very low" by an evaluator. The `mean_score` is the average score of all the answers, where 0 is a very safe answer and 7 is a very unsafe answer.
+Learn more in the [Red Teaming Agent documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent#results-from-your-automated-scans).
 
 ## Resources
 
-To learn more about the Azure AI services used in this project, look through the script and reference the following documentation:
+To learn more about the red team scanning, look through the script, documentation, and videos below:
 
-* [Generate simulated data for evaluation](https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data)
-* [Evaluate with the Azure AI Evaluation SDK](https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk)
+* [safety_evaluation.py](/evals/safety_evaluation.py)
+* [Run automated safety scans with AI Red Teaming Agent](https://learn.microsoft.com/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent)
+* [Build 2025: Red-teaming Demo](https://www.youtube.com/watch?v=sZzcSX7BFVA)
diff --git a/evals/redteams/.gitkeep b/evals/redteams/.gitkeep
diff --git a/evals/requirements.txt b/evals/requirements.txt
@@ -1,4 +1,4 @@
-git+https://github.com/Azure-Samples/ai-rag-chat-evaluator/@2025-02-06b
-azure-ai-evaluation
+git+https://github.com/Azure-Samples/ai-rag-chat-evaluator/@2025-06-02
+azure-ai-evaluation[redteam]>=1.8.0
 rich
 dotenv-azd