Portfolio / Projects / LLM Red Team
AI SECURITY · CASE STUDY

ACADEMIC ENGINEERING PROJECT DEVELOPMENT IN PROGRESS

Red Team Agent for LLM Application Auditing.

An experimental toolchain for evaluating security behavior in LLM-powered applications through controlled adversarial scenarios, explicit verdict rules and reproducible evidence.

PythonFastAPIPydanticOllamapytestLLM Security

Publication boundary. This is a public technical overview, not a public code release. The implementation repository remains private, and work-in-progress features are identified separately from validated observations.

01 / PROBLEM

Why test the application, not just the model?

LLM applications combine model responses with application instructions, retrieved content, access controls and external tools. A model's behavior in a conversation is not sufficient evidence that the complete system respects its security boundaries.

This project explores a practical audit question: Can a controlled testing process reveal instruction-following failures and sensitive-information disclosure without mistaking an ambiguous answer for a confirmed vulnerability?

01Authorized scope

Local or explicitly permitted targets only; the prototype is not an Internet scanner.

02Evidence-driven verdicts

Scenario-specific checks distinguish proven failures from uncertainty and technical errors.

03Synthetic data

Test content and canary markers are fictitious; no production secrets are needed.

02 / SYSTEM DESIGN

A separate audit engine and local target.

The implementation distinguishes UniAssist, a fictional university chatbot used as a test target, from the independent audit engine. UniAssist is not the name of the overall Red Team project.

Target application

A FastAPI endpoint orchestrates requests to a local Ollama model. Its context uses invented university data and controlled markers to probe confidentiality behavior.

Audit engine

Scenario validation, HTTP execution and per-scenario evaluators separate attack delivery from evidence assessment. Requests are tracked rather than silently retried.

Evidence outputs

Development branches implement structured JSON and JSONL results and a standalone HTML report. A technically failed request is classified as an error, not a vulnerability.

Experimental extensions

Adaptive variants, an installable CLI, optional attacker-LLM follow-ups and a local dashboard exist on development branches. They are not presented as a finished public release.

03 / OBSERVED EVIDENCE

A small test set, reported without inflated claims.

An initial experiment with the local model recorded three manual security checks. These are exploratory observations, not a statistical security benchmark.

CHECKOBSERVATIONINTERPRETATION
Command requestRefusedExpected behavior for this specific probe.
Context canaryDisclosedA canary deliberately included in model context was reproduced.
Restricted requestRefusedExpected behavior for this specific probe.
What the disclosure does—and does not—show.

Reproduction of a canary placed inside the model's context demonstrates a disclosure weakness in that test configuration. It does not establish access to an unrelated private file, exfiltration from a production system, or the effectiveness of the entire future audit engine.

Verdict discipline

PASS and FAIL require scenario-specific evidence; INCONCLUSIVE is reserved for ambiguous responses, and ERROR for failed requests. An absent canary does not prove universal safety.

Reproducibility

Testing uses synthetic context, explicit scenario identifiers, HTTP requests and response traces. Broader conclusions will require manually reviewed examples and repeatable runs.

04 / ENGINEERING LOG

Implementation status.

The status below refers to the private development repository as inspected in November 2026. A feature existing in a branch is different from a feature merged, validated against a real model, or released.

01

Local test target Implemented

FastAPI, a mocked model client for unit tests, synthetic public/private fixtures and Ollama integration. Initial real-model behavior recorded.

02

Baseline audit core Implemented in development

Scenario validation, sequential requests, deterministic verdicts, structured logs and HTML reporting.

03

Adaptive evaluation Development branch

Controlled follow-up variants with request budgets and annotation preparation. Empirical benchmark results are not yet established.

04

CLI and local interfaces Draft PR / awaiting review

Installable command-line workflow, optional attacker-LLM support, local web views and offline checks. Real integration and security review remain pending.

05 / NEXT MILESTONES

What still needs to be demonstrated?

01

Real integration

Exercise the complete CLI and local interfaces with Ollama, and test optional providers without committing API secrets.

02

Human-reviewed results

Collect a small set of actual target responses, annotate verdicts and calculate only metrics supported by the reviewed data.

03

Safe release

Review packaging, local-only UI restrictions, documentation and sensitive content before considering public source publication.

PRIVATE SOURCE · PUBLIC METHODOLOGY

Evidence without premature disclosure.

The source repository is intentionally private while integration and verification are completed. This page will evolve only when new work can be demonstrated accurately.

Explore other projects