We Dogfooded SpecFact CLI on Itself — Here's What Happened

TL;DR: We ran SpecFact CLI on its own codebase in two ways: (1) Brownfield analysis discovered 19 features and 49 stories in seconds, found 24 deviations, and blocked the merge. (2) Contract enhancement added contracts to our core telemetry module with 7-step validation (all tests passed). Total time: seconds for analysis (varies by codebase size), ~3 minutes for contract enhancement. 🚀

The Challenge

We built SpecFact CLI and wanted to validate that it actually works in the real world. So we did what every good developer does: we dogfooded it.

"Dogfooding" is a well-known tech term meaning "eating your own dog food" — using your own product. It's a common practice to validate that tools work in real-world scenarios.

Our Goal: Analyze the SpecFact CLI codebase itself and demonstrate:

1. How fast brownfield analysis is
2. How enforcement actually blocks bad code
3. How the complete workflow works end-to-end
4. How contract enhancement works on real production code

---

Part 1: Brownfield Analysis ⚡

First, we analyzed the existing codebase to see what features it discovered:

specfact import from-code specfact-cli --repo . --confidence 0.5

Note: Analysis time varies by codebase size. For ~19 Python files, this typically completes in a few seconds. Larger codebases may take longer as the CLI performs AST analysis, Semgrep pattern detection, and dependency graph building.

Output:

🔍 Analyzing Python files...
✓ Found 19 features
✓ Detected themes: CLI, Validation
✓ Total stories: 49

✓ Analysis complete!
Project bundle written to: .specfact/projects/specfact-cli/

What It Discovered

The brownfield analysis extracted 19 features from our codebase:

Feature	Stories	Confidence	What It Does
Enforcement Config	3	0.9	Configuration for contract enforcement and quality gates
Code Analyzer	2	0.7	Analyzes Python code to auto-derive plan bundles
Plan Comparator	1	0.7	Compares two plan bundles to detect deviations
Report Generator	3	0.9	Generator for validation and deviation reports
Protocol Generator	3	0.9	Generator for protocol YAML files
Plan Generator	3	0.9	Generator for plan bundle YAML files
FSM Validator	3	1.0	FSM validator for protocol validation
Schema Validator	2	0.7	Schema validator for plan bundles and protocols
Git Operations	5	1.0	Helper class for Git operations
Logger Setup	3	1.0	Utility class for standardized logging setup

Total: 49 user stories auto-generated with Fibonacci story points.

Time taken: A few seconds for 19 Python files (varies by machine and codebase complexity) 🏎️

---

Setting Enforcement Rules 🎯

Next, we configured quality gates to block HIGH severity violations:

specfact enforce stage --preset balanced

Output:

Setting enforcement mode: balanced
  Enforcement Mode:  
      BALANCED       
┏━━━━━━━━━━┳━━━━━━━━┓
┃ Severity ┃ Action ┃
┡━━━━━━━━━━╇━━━━━━━━┩
│ HIGH     │ BLOCK  │
│ MEDIUM   │ WARN   │
│ LOW      │ LOG    │
└──────────┴────────┘

✓ Enforcement mode set to balanced

What this means:

• 🚫 HIGH severity deviations → BLOCK the merge (exit code 1)
• ⚠️ MEDIUM severity deviations → WARN but allow (exit code 0)
• 📝 LOW severity deviations → LOG silently (exit code 0)

---

Comparing Plans — The Magic Moment 🔍

We already had a manual plan in our repo (created during initial development with specfact plan init). Now we compare it against the auto-derived plan from the brownfield analysis:

# In AI IDE (recommended): /specfact.compare --code-vs-plan
# Or via CLI (basic):
specfact plan compare --code-vs-plan

Plan loading time depends on the number of features and stories detected.

Results

Deviations Found: 24 total

• 🔴 HIGH: 2 (Missing features from manual plan)
• 🟡 MEDIUM: 19 (Extra implementations found in code)
• 🔵 LOW: 3 (Metadata mismatches)

🔴 HIGH Severity — BLOCKED!

The comparison found that our manual plan calls it FEATURE-ENFORCEMENT, but the code has FEATURE-ENFORCEMENTCONFIG. This is a real deviation — our naming doesn't match!

🟡 MEDIUM Severity — Warned

We have 19 utility features (YAML utils, Git operations, validators, etc.) that exist in code but aren't documented in our minimal manual plan.

This is exactly what we want! It shows us undocumented features that should either be added to the plan or removed if not needed.

---

Enforcement In Action 🚫

With balanced enforcement enabled:

❌ Enforcement BLOCKED: 2 deviation(s) violate quality gates
Fix the blocking deviations or adjust enforcement config

Exit Code: 1 (BLOCKED) ❌

In CI/CD: This would fail the PR and prevent the merge until we fix the deviations.

Switching to Minimal Enforcement

Let's try again with minimal enforcement (never blocks):

specfact enforce stage --preset minimal
specfact plan compare

Result:

✅ Enforcement PASSED: No blocking deviations

Exit Code: 0 (PASSED) ✅

Same deviations, different outcome: With minimal enforcement, even HIGH severity issues are downgraded to warnings. Perfect for exploration phase!

---

Part 2: Contract Enhancement (Production Use Case) 🎯

After validating the brownfield analysis, we took it a step further: we used SpecFact CLI to enhance one of our own core modules with contracts.

Goal: Add beartype, icontract, and CrossHair contracts to src/specfact_cli/telemetry.py — a core module that handles privacy-first telemetry.

Generate Contract Enhancement Prompt

# In AI IDE (recommended): /specfact.07-contracts --apply all-contracts

# Or via CLI (basic):
specfact generate contracts-prompt src/specfact_cli/telemetry.py --bundle specfact-cli-test --apply all-contracts

What happened:

• CLI analyzed the telemetry module (543 lines)
• Generated a structured prompt for AI IDEs
• Included instructions for beartype, icontract, and CrossHair

AI IDE Enhancement + 7-Step Validation

The AI IDE (Cursor) enhanced the code, then ran comprehensive validation:

Step	Check	Result
1/7	File Size Check	✅ 678 lines (was 543)
2/7	Syntax Validation	✅ Python compilation passed
3/7	AST Structure Comparison	✅ All 23 definitions preserved
4/7	Contract Imports Verification	✅ beartype, icontract imports verified
5/7	Code Quality Checks	✅ Ruff linting passed
6/7	Test Execution	✅ 10/10 tests passed
7/7	Diff Preview	✅ Changes reviewed

---

What We Achieved

Contracts Applied

1. beartype decorators: Runtime type checking for all public APIs
2. icontract decorators: Preconditions and postconditions where appropriate
3. CrossHair tests: Property-based test functions for edge case discovery

Production Value

This demonstrates real production use:

• Enhanced a core module (telemetry) used throughout the CLI
• Applied all three contract types
• All tests passed (10/10) — no regressions
• Fast validation (< 10 seconds for comprehensive 7-step process)

---

Key Takeaways

1. Speed ⚡

Task	Typical Time*
Analyze 19 Python files	~3–5 seconds
Set enforcement	< 1 second
Compare plans	~2–5 seconds
Total Analysis	~10–15 seconds

*Times vary by machine, codebase size, and number of features/stories detected.

2. Accuracy 🎯

• Discovered 19 features we actually built
• Generated 49 user stories with meaningful titles
• Detected real naming inconsistencies

3. Enforcement Works 🚫

• Balanced mode: Blocked execution due to 2 HIGH deviations (exit 1)
• Minimal mode: Passed with warnings (exit 0)
• CI/CD ready: Exit codes work perfectly with any CI system

4. Real Value 💎

The tool found real issues:

1. Naming inconsistency between manual plan and code
2. Undocumented features that should be documented
3. Documentation gaps that need addressing

These are actual questions that need answers, not false positives!

---

Try It Yourself

Option A: AI IDE Mode (Recommended)

For best results, use the slash commands in your AI IDE (Cursor, VS Code + Copilot, etc.). The LLM enriches the CLI output with semantic understanding, business context, and "why" reasoning.

# Step 1: Install slash commands in your AI IDE
specfact init --ide cursor  # or --ide vscode

# Step 2: In your AI IDE, use the slash command:
/specfact.01-import --repo .

# The slash command runs CLI + LLM enrichment for better feature detection

Option B: CLI-Only Mode (Quick Start)

For quick validation or CI/CD, you can run the CLI directly (no LLM enrichment):

# Clone SpecFact CLI
git clone https://github.com/nold-ai/specfact-cli.git
cd specfact-cli

# Step 1: Analyze the codebase (CLI-only, no LLM enrichment)
specfact import from-code specfact-cli --repo .

# Step 2: Set enforcement rules
specfact enforce stage --preset balanced

# Step 3: Compare plans (requires existing manual plan)
specfact plan compare --code-vs-plan

# Step 4: Run full validation suite
specfact repro --verbose

Note: CLI-only mode is fast and deterministic but provides basic feature detection. AI IDE mode (slash commands) provides significantly better results through LLM-enriched analysis.

---

Conclusion

SpecFact CLI works. We proved it by running it on itself.

• ⚡ Fast: Analyzed 19 files in seconds
• 🎯 Accurate: Found 24 real deviations
• 🚫 Blocks bad code: Enforcement prevented merge with HIGH violations
• 🛡️ Comprehensive: 7-step validation for contract enhancement
• ✅ Production-ready: All tests passed, no regressions

Built by dogfooding — This example is real, not fabricated. We ran SpecFact CLI on itself and documented the actual outcomes.