← PROJECT INDEXCASE STUDY / 01

ThinkTrace

Specialist AI agents check claims, identify logical fallacies, and evaluate reasoning.

View source ↗Open project ↗

DISCIPLINE

AI Agents

TIMELINE

Mar to Apr 2026

BUILT WITH

AI agents, PostgreSQL, Pinecone, Neo4j

System architectureFIG. 01

Input & parsing
The submitted argument is converted into a claim tree, with context specific to the organization.
Parallel checks
Logic mapping, fallacy detection, and fact checking run in parallel on the extracted claims.
Evidence & scoring
Fact checking retrieves supporting sources. The scorer combines evidence, fallacies, and the argument graph.
Report
Structured results are assembled, stored, and made available for review.

AI AgentsConceptual overview

INSIDE THE SYSTEM / SELECT A STAGE

The submitted argument is converted into a claim tree, with context specific to the organization.

Problem

Evaluating an argument involves several different tasks: checking its claims, identifying logical fallacies, and assessing the reasoning. ThinkTrace brings these perspectives together in a single analysis workflow.

Implementation

I built a platform that extracts claims, runs logic mapping, fallacy detection, and fact checking in parallel, then combines their findings with an epistemic scorer. PostgreSQL stores records, Pinecone retrieves relevant evidence, and Neo4j stores relationships. Each organization has separate context and access controls.

Technical decisions

Used PostgreSQL, Pinecone, and Neo4j for distinct storage and retrieval needs.
Separated organizations using isolated namespaces, tenant identity determined by the server, and access controls.
Applied caching by content hash to avoid repeating the same analysis and unnecessary LLM calls.
Added JWT and OTP authentication, organization and IP rate limits, tracing, and calibrated evaluation cases.

Results

In my project tests, cached analyses returned in about 460 ms. Caching lets the application reuse an existing analysis instead of making another set of model calls.

NEXT / KEEP EXPLORING

AI Voice Agent

Inside ThinkTrace

Follow a request.

Step through a real test case and see how the analysis fits together.

A test argument from my evaluation suite

Case eval_003 tests whether the analysis catches unsupported marketing claims.

You should buy this supplement because celebrities use it. It has no side effects. Big pharma doesn't want you to know about it. Thousands of five-star reviews prove it works.

Turn the argument into claims

The ingestion agent extracts a claim tree. It distinguishes premises, conclusions, and supporting claims before analysis begins.

In this example, the recommendation to buy the supplement depends on celebrity use, reviews, and unsupported safety claims.

Three checks, working in parallel

Logic mapper

Maps how the claims support or contradict one another.

Fallacy detector

Looks for named reasoning errors in the argument.

Fact checker

Checks verifiable claims against external sources.

The analysis graph waits for these branches before scoring the argument.

Combine findings, then check the test targets

The scorer receives the claim tree, argument graph, detected fallacies, and fact checks.

This evaluation case expects an overall score between 0 and 35 and checks for these fallacies:

These are the evaluation targets defined in the repository. The walkthrough is a guided example, not a live model run.

Step 1 of 4