AI-Assisted Backlog Refinement: Empowering DevOps and Agile Teams

TL;DR: SpecFact CLI v0.26+ introduces AI-assisted backlog refinement that transforms unstructured backlog items into standardized, template-compliant work items. This guide walks you through authentication, searching, refinement workflows, custom template creation, Definition of Ready validation, and cross-backlog synchronization for complete DevOps workflow integration.

New in v0.26+: Backlog refinement is available in SpecFact CLI v0.26.0 and later. Install via pip install specfact-cli or uvx specfact-cli@latest.

Introduction: The Backlog Refinement Challenge

DevOps and Agile teams face a constant challenge: backlog items come in all shapes and sizes. Some are well-structured user stories with clear acceptance criteria. Others are informal notes, bug reports, or feature requests that need refinement before they're ready for development.

Traditional approaches require manual effort to:

Standardize work item formats across teams
Enforce corporate templates (user stories, defects, spikes, enablers)
Validate Definition of Ready (DoR) criteria
Maintain consistency when syncing between different backlog tools
Migrate backlogs between platforms without losing structure

SpecFact CLI's backlog refinement feature solves these challenges by combining AI assistance with template-driven workflows. It helps teams:

✅ Automatically detect which template matches each backlog item
✅ Refine unstructured input into template-compliant format using AI
✅ Validate Definition of Ready before items enter sprints
✅ Create custom templates for your organization's specific needs
✅ Sync seamlessly between GitHub Issues and Azure DevOps
✅ Migrate backlogs between tools with full content preservation

Getting Started: Authentication and Setup

Step 1: Install SpecFact CLI

First, ensure you have SpecFact CLI installed:

# Install via pip
pip install specfact-cli

# Or use uvx (recommended for latest version)
uvx specfact-cli@latest backlog refine --help

Step 2: Authenticate with GitHub

SpecFact CLI uses GitHub CLI authentication. If you already have gh authenticated, you're ready to go:

# Verify GitHub authentication
gh auth status

# If not authenticated, authenticate GitHub CLI
gh auth login

# Then verify SpecFact can access GitHub
specfact auth github

Note: SpecFact CLI automatically uses your existing GitHub CLI credentials, so no additional token management is required.

Step 3: Authenticate with Azure DevOps

Azure DevOps supports two authentication methods:

Option A: OAuth (Interactive Browser or Device Code)

For the best user experience, use interactive browser authentication:

# Interactive browser (default - tries first, falls back to device code)
specfact auth azure-devops

# Force device code flow (for SSH/headless environments)
specfact auth azure-devops --use-device-code

Token Refresh: OAuth tokens expire after ~1 hour, but SpecFact automatically refreshes them using a persistent token cache (similar to Azure CLI). Tokens refresh automatically for ~90 days without user interaction.

Option B: Personal Access Token (PAT)

For longer-lived tokens (up to 1 year), use a Personal Access Token:

# Store PAT directly
specfact auth azure-devops --pat your_pat_token

# Or set environment variable
export AZURE_DEVOPS_TOKEN=your_pat_token

Creating a PAT: Go to https://dev.azure.com/{org}/_usersSettings/tokens and create a token with "Work Items (Read & Write)" permissions.

Step 4: Verify Backlog Accessibility

Before refining, verify you can access your backlogs:

GitHub Verification

# Test GitHub access by listing issues
specfact backlog refine github \
  --repo-owner "your-org" \
  --repo-name "your-repo" \
  --preview \
  --state open

If this succeeds, you can access the repository's issues.

Azure DevOps Verification

# Test ADO access by listing work items
specfact backlog refine ado \
  --ado-org "your-org" \
  --ado-project "your-project" \
  --preview \
  --state Active

If this succeeds, you can access the project's work items.

Debug Mode: If you encounter authentication issues, use --debug to see detailed diagnostic information:

specfact --debug backlog refine github --repo-owner "your-org" --repo-name "your-repo"

Searching and Querying Backlog Items

SpecFact CLI provides flexible options for finding and filtering backlog items before refinement.

GitHub Issues: Search and Filter

Using GitHub Search Syntax

Leverage GitHub's powerful search syntax for precise queries:

# Search for open issues with specific labels
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --search "is:open label:feature label:enhancement"

# Search for issues assigned to a user
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --search "is:open assignee:username"

# Search for issues in a milestone
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --search "is:open milestone:\"Sprint 1\""

Using Filter Options

Alternatively, use dedicated filter options for common queries:

# Filter by labels
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature,enhancement \
  --state open

# Filter by assignee
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --assignee "username" \
  --state open

# Filter by sprint (if using GitHub Projects)
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --sprint "Sprint 1" \
  --state open

Azure DevOps: Query Work Items

Azure DevOps uses state-based filtering:

# Filter by state
specfact backlog refine ado \
  --ado-org "myorg" \
  --ado-project "myproject" \
  --state Active

# Filter by iteration/sprint
specfact backlog refine ado \
  --ado-org "myorg" \
  --ado-project "myproject" \
  --iteration "Project\\Sprint 1" \
  --state Active

# Filter by assignee
specfact backlog refine ado \
  --ado-org "myorg" \
  --ado-project "myproject" \
  --assignee "[email protected]" \
  --state Active

Combining Filters

You can combine multiple filters for precise targeting:

# GitHub: Multiple filters
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature,enhancement \
  --state open \
  --assignee "dev1" \
  --sprint "Sprint 1"

# ADO: Multiple filters
specfact backlog refine ado \
  --ado-org "myorg" \
  --ado-project "myproject" \
  --state Active \
  --iteration "Project\\Sprint 1" \
  --assignee "[email protected]"

The Refinement Process

Backlog refinement transforms unstructured backlog items into standardized, template-compliant work items using AI assistance.

How It Works

SpecFact CLI follows a CLI-first architecture:

SpecFact CLI generates prompts/instructions for your IDE AI copilot (Cursor, Claude Code, etc.)
IDE AI copilot executes those instructions using its native LLM
IDE AI copilot feeds results back to SpecFact CLI
SpecFact CLI validates and processes the results

This means: You get AI-powered refinement without SpecFact CLI directly invoking LLM APIs. Your IDE AI copilot handles the AI processing, while SpecFact CLI orchestrates the workflow.

Template Detection

SpecFact automatically detects which template matches each backlog item:

User Story - Features and enhancements
Defect - Bug reports and fixes
Spike - Research and investigation tasks
Enabler - Technical debt and infrastructure work

Templates can be customized by:

Framework - Scrum, Kanban, SAFe
Persona - Product Owner, Developer, QA
Provider - GitHub, Azure DevOps

Refinement Workflow

Step 1: Preview Refinements (Safe Mode)

By default, refinement runs in preview mode - you can review all changes before applying them:

# Preview refinements without writing
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --preview

This shows you:

Which template was detected for each item
Confidence scores for template matching
Proposed refined content
Items that need manual review (low confidence)

Step 2: Interactive Refinement

For items with low confidence scores, SpecFact prompts you to:

Accept the AI-refined content
Skip the item (leave unchanged)
Edit manually before accepting

# Interactive refinement (default behavior)
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --state open

Step 3: Write Refinements to Backlog

When you're satisfied with the refinements, write them to the backlog:

# Write refinements to backlog (explicit opt-in)
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --write

Safety Note: The --write flag is required to prevent accidental overwrites. Preview mode is the default.

Step 4: Auto-Accept High-Confidence Refinements

For items with high confidence scores (≥0.85), you can auto-accept without manual review:

# Auto-accept high-confidence refinements
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --write \
  --auto-accept-high-confidence

Framework and Persona Context

Provide context to improve template matching and refinement quality:

# Scrum framework with Product Owner persona
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --framework scrum \
  --persona product-owner \
  --labels feature

# Kanban framework
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --framework kanban \
  --labels feature

Creating Custom Templates

SpecFact CLI comes with default templates, but you can create custom templates tailored to your organization's needs.

Template Structure

Templates are YAML files that define the structure and fields for work items:

# resources/templates/backlog/my_custom_template.yaml
name: "My Custom User Story"
description: "Custom user story template for our organization"
type: "user_story"
framework: "scrum"  # Optional: scrum, kanban, safe
persona: "product-owner"  # Optional: product-owner, developer, qa
provider: "github"  # Optional: github, ado

fields:
  - name: "title"
    required: true
    description: "User story title"
  
  - name: "description"
    required: true
    description: "User story description"
  
  - name: "acceptance_criteria"
    required: true
    description: "List of acceptance criteria"
    format: "list"
  
  - name: "business_value"
    required: false
    description: "Business value statement"
  
  - name: "technical_notes"
    required: false
    description: "Technical implementation notes"

validation:
  min_description_length: 50
  required_fields: ["title", "description", "acceptance_criteria"]

Template Location

Place custom templates in your project's template directory:

# Initialize SpecFact in your project (if not already done)
specfact init

# This creates: .specfact/templates/backlog/
# Add your custom templates here:
.specfact/templates/backlog/my_custom_template.yaml

Using Custom Templates

Reference your custom template by name:

# Use custom template
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --template my_custom_template \
  --labels feature \
  --write

Template Matching Priority

SpecFact matches templates in order of specificity:

Provider + Framework + Persona (most specific)
Framework + Persona
Framework
Default (least specific)

Example: If you specify --framework scrum --persona product-owner, SpecFact will look for:

github_scrum_product-owner_user_story.yaml
scrum_product-owner_user_story.yaml
scrum_user_story.yaml
user_story.yaml (default)

Definition of Ready (DoR) Validation

Definition of Ready ensures backlog items meet quality criteria before entering sprints. SpecFact CLI can validate DoR criteria during refinement.

What is Definition of Ready?

Definition of Ready typically includes:

Clear title and description
Acceptance criteria defined
Dependencies identified
Size/effort estimated
Business value articulated
Technical approach documented (if needed)

Checking Definition of Ready

Use the --check-dor flag to validate items against DoR criteria:

# Check Definition of Ready before refinement
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --check-dor

This will:

Identify items missing required fields
Flag items with incomplete acceptance criteria
Highlight items that need additional information
Provide recommendations for making items "ready"

Custom DoR Rules

Define custom DoR rules in your template:

# resources/templates/backlog/user_story_with_dor.yaml
name: "User Story with DoR"
type: "user_story"

fields:
  # ... field definitions ...

definition_of_ready:
  rules:
    - field: "acceptance_criteria"
      condition: "count >= 3"
      message: "At least 3 acceptance criteria required"
    
    - field: "description"
      condition: "length >= 100"
      message: "Description must be at least 100 characters"
    
    - field: "business_value"
      condition: "present"
      message: "Business value must be specified"
    
    - field: "dependencies"
      condition: "identified"
      message: "All dependencies must be identified"

DoR Validation Workflow

Integrate DoR validation into your refinement workflow:

# Step 1: Check DoR for all items
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --check-dor

# Step 2: Refine items that pass DoR
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --write \
  --auto-accept-high-confidence

Cross-Backlog Synchronization

SpecFact CLI enables seamless synchronization between different backlog tools, supporting both full and partial sync, as well as complete backlog migration.

Understanding Sync Modes

SpecFact supports three sync modes:

export-only - Export from OpenSpec/bundle to backlog tool
import-only - Import from backlog tool to OpenSpec/bundle

bidirectional

Full Synchronization

Sync all items between backlogs:

GitHub to Azure DevOps

# Step 1: Import GitHub issues into bundle
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle migration \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli"

# Step 2: Export bundle to Azure DevOps
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle migration \
  --ado-org "myorg" \
  --ado-project "myproject"

Azure DevOps to GitHub

# Step 1: Import ADO work items into bundle
specfact sync bridge \
  --adapter ado \
  --mode import-only \
  --bundle migration \
  --ado-org "myorg" \
  --ado-project "myproject"

# Step 2: Export bundle to GitHub
specfact sync bridge \
  --adapter github \
  --mode export-only \
  --bundle migration \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli"

Partial Synchronization

Sync specific items by ID:

# Import specific GitHub issues
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle partial-sync \
  --backlog-ids 110,112,115 \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli"

# Export specific items to ADO
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle partial-sync \
  --change-ids add-feature-x,add-feature-y \
  --ado-org "myorg" \
  --ado-project "myproject"

Bidirectional Synchronization

Keep backlogs in sync automatically:

# Bidirectional sync between GitHub and OpenSpec
specfact sync bridge \
  --adapter github \
  --mode bidirectional \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli"

# Bidirectional sync between ADO and OpenSpec
specfact sync bridge \
  --adapter ado \
  --mode bidirectional \
  --ado-org "myorg" \
  --ado-project "myproject"

How it works: Bidirectional sync detects changes in both systems and updates the other accordingly. Status changes, content updates, and new items are synchronized automatically.

Backlog Migration

Migrate entire backlogs between tools with full content preservation:

Complete Migration Workflow

# Step 1: Import all items from source backlog
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle full-migration \
  --repo-owner "old-org" \
  --repo-name "old-repo"

# Step 2: Refine items (optional but recommended)
specfact backlog refine github \
  --repo-owner "old-org" \
  --repo-name "old-repo" \
  --write \
  --auto-accept-high-confidence

# Step 3: Export to target backlog
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle full-migration \
  --ado-org "new-org" \
  --ado-project "new-project"

State Preservation During Migration

SpecFact preserves work item states during migration:

Source (GitHub)	Intermediate (OpenSpec)	Target (ADO)
`open`	`proposed`	`New`
`in-progress`	`in-progress`	`Active`
`closed`	`applied`	`Closed`

Lossless Migration: All original content, formatting, and metadata are preserved during migration. You can migrate back and forth without losing information.

Combining Refinement with Sync

Refine items before syncing to ensure consistency:

# Step 1: Refine GitHub issues
specfact backlog refine github \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli" \
  --labels feature \
  --write \
  --auto-accept-high-confidence

# Step 2: Import refined items into bundle
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle refined-items \
  --repo-owner "nold-ai" \
  --repo-name "specfact-cli"

# Step 3: Export to ADO with refined content
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle refined-items \
  --ado-org "myorg" \
  --ado-project "myproject"

Real-World Workflow Examples

Example 1: Sprint Planning Preparation

Scenario: Product Owner creates informal GitHub issues. DevOps team needs to refine them into Scrum-compliant user stories before sprint planning.

# Step 1: Refine all feature issues
specfact backlog refine github \
  --repo-owner "my-org" \
  --repo-name "my-repo" \
  --framework scrum \
  --persona product-owner \
  --labels feature \
  --check-dor \
  --write \
  --auto-accept-high-confidence

# Step 2: Import refined items to OpenSpec for tracking
specfact backlog refine github \
  --repo-owner "my-org" \
  --repo-name "my-repo" \
  --labels feature \
  --bundle sprint-1 \
  --auto-bundle

# Step 3: Sync to Azure DevOps for development team
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle sprint-1 \
  --ado-org "myorg" \
  --ado-project "myproject"

Example 2: Cross-Team Collaboration

Scenario: Public-facing features tracked in GitHub, internal implementation tracked in Azure DevOps. Keep both in sync.

# Step 1: Refine GitHub issues (public)
specfact backlog refine github \
  --repo-owner "public-org" \
  --repo-name "public-repo" \
  --labels feature \
  --write

# Step 2: Sync to ADO (internal)
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle cross-team \
  --repo-owner "public-org" \
  --repo-name "public-repo"

specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle cross-team \
  --ado-org "internal-org" \
  --ado-project "internal-project"

# Step 3: Bidirectional sync for status updates
specfact sync bridge \
  --adapter ado \
  --mode bidirectional \
  --bundle cross-team \
  --ado-org "internal-org" \
  --ado-project "internal-project"

Example 3: Tool Migration

Scenario: Migrating from GitHub Issues to Azure DevOps while preserving all content and history.

# Step 1: Import all GitHub issues
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle migration \
  --repo-owner "old-org" \
  --repo-name "old-repo"

# Step 2: Refine items to match ADO templates
specfact backlog refine github \
  --repo-owner "old-org" \
  --repo-name "old-repo" \
  --framework scrum \
  --write \
  --auto-accept-high-confidence

# Step 3: Export to Azure DevOps
specfact sync bridge \
  --adapter ado \
  --mode export-only \
  --bundle migration \
  --ado-org "new-org" \
  --ado-project "new-project"

# Step 4: Verify migration (import back to verify)
specfact sync bridge \
  --adapter ado \
  --mode import-only \
  --bundle verification \
  --ado-org "new-org" \
  --ado-project "new-project"

Best Practices

1. Always Preview Before Writing

Use preview mode to review refinements before applying them:

# Preview first
specfact backlog refine github \
  --repo-owner "my-org" \
  --repo-name "my-repo" \
  --labels feature \
  --preview

# Then write when satisfied
specfact backlog refine github \
  --repo-owner "my-org" \
  --repo-name "my-repo" \
  --labels feature \
  --write

2. Use Framework and Persona Context

Provide context to improve refinement quality:

# Better refinement with context
specfact backlog refine github \
  --framework scrum \
  --persona product-owner \
  --labels feature

3. Validate Definition of Ready

Check DoR before items enter sprints:

# Check DoR before refinement
specfact backlog refine github \
  --check-dor \
  --labels feature

4. Use Bundles for Organization

Organize related items into bundles:

# Group items by sprint
specfact backlog refine github \
  --bundle sprint-1 \
  --sprint "Sprint 1"

5. Test Sync Before Full Migration

Test with a small subset before migrating entire backlogs:

# Test with a few items first
specfact sync bridge \
  --adapter github \
  --mode import-only \
  --bundle test-migration \
  --backlog-ids 110,112,115 \
  --repo-owner "my-org" \
  --repo-name "my-repo"

Conclusion

SpecFact CLI's backlog refinement feature empowers DevOps and Agile teams to:

✅ Standardize work items automatically using AI-assisted refinement
✅ Enforce corporate templates with custom template support
✅ Validate Definition of Ready before items enter sprints
✅ Sync seamlessly between GitHub and Azure DevOps
✅ Migrate backlogs between tools with full content preservation
✅ Maintain consistency across teams and tools

Whether you're preparing for sprint planning, collaborating across teams, or migrating between tools, SpecFact CLI provides the automation and standardization you need to keep your backlog management efficient and consistent.

Ready to transform your backlog management? Start with specfact backlog refine --help and explore the possibilities!

Additional Resources

📖 Backlog Refinement Guide - Complete documentation
📚 Command Reference - All available options
🔗 DevOps Adapter Integration - Cross-tool synchronization
💬 GitHub Discussions - Community support

Introduction: The Backlog Refinement Challenge

Getting Started: Authentication and Setup

Step 1: Install SpecFact CLI

Step 2: Authenticate with GitHub

Step 3: Authenticate with Azure DevOps

Option A: OAuth (Interactive Browser or Device Code)

Option B: Personal Access Token (PAT)

Step 4: Verify Backlog Accessibility

GitHub Verification

Azure DevOps Verification

Searching and Querying Backlog Items

GitHub Issues: Search and Filter

Using GitHub Search Syntax

Using Filter Options

Azure DevOps: Query Work Items

Combining Filters

The Refinement Process

How It Works

Template Detection

Refinement Workflow

Step 1: Preview Refinements (Safe Mode)

Step 2: Interactive Refinement

Step 3: Write Refinements to Backlog

Step 4: Auto-Accept High-Confidence Refinements

Framework and Persona Context

Creating Custom Templates

Template Structure

Template Location

Using Custom Templates

Template Matching Priority

Definition of Ready (DoR) Validation

What is Definition of Ready?

Checking Definition of Ready

Custom DoR Rules

DoR Validation Workflow

Cross-Backlog Synchronization

Understanding Sync Modes

Full Synchronization

GitHub to Azure DevOps

Azure DevOps to GitHub

Partial Synchronization

Bidirectional Synchronization

Backlog Migration

Complete Migration Workflow

State Preservation During Migration

Combining Refinement with Sync

Real-World Workflow Examples

Example 1: Sprint Planning Preparation

Example 2: Cross-Team Collaboration

Example 3: Tool Migration

Best Practices

1. Always Preview Before Writing

2. Use Framework and Persona Context

3. Validate Definition of Ready

4. Use Bundles for Organization

5. Test Sync Before Full Migration

Conclusion

Additional Resources

Ready to try SpecFact?