Hiddenden/openrabbit
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity

Merge pull request 'feat: Add @codebot explain-diff command for plain-language PR explanations' (#26) from feature/code-diff-explainer into dev

Browse Source 
Reviewed-on: #26
This commit was merged in pull request #26.
This commit is contained in:
Latte
2025-12-29 12:53:32 +00:00
parent 1d468e360e 37f3eb45d0
commit c712fe670b
8 changed files with 680 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitea/workflows/ai-chat.yml
View File

@@ -26,6 +26,7 @@ jobs:
!contains(github.event.comment.body, '@codebot security') &&
!contains(github.event.comment.body, '@codebot summarize') &&
!contains(github.event.comment.body, '@codebot changelog') &&
!contains(github.event.comment.body, '@codebot explain-diff') &&
!contains(github.event.comment.body, '@codebot review-again') &&
!contains(github.event.comment.body, '@codebot setup-labels')
runs-on: ubuntu-latest

3
.gitea/workflows/ai-comment-reply.yml
View File

@@ -1,7 +1,7 @@
name: AI Comment Reply
# WORKFLOW ROUTING:
# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, changelog, review-again, setup-labels
# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, changelog, explain-diff, review-again, setup-labels
# Other workflows: ai-issue-triage.yml (@codebot triage), ai-chat.yml (free-form questions)
on:
@@ -24,6 +24,7 @@ jobs:
contains(github.event.comment.body, '@codebot security') ||
contains(github.event.comment.body, '@codebot summarize') ||
contains(github.event.comment.body, '@codebot changelog') ||
contains(github.event.comment.body, '@codebot explain-diff') ||
contains(github.event.comment.body, '@codebot review-again') ||
contains(github.event.comment.body, '@codebot setup-labels'))
steps:

75
CLAUDE.md
View File

@@ -217,6 +217,7 @@ Prompts are stored in `tools/ai-review/prompts/` as Markdown files:
- `base.md` - Base instructions for all reviews
- `pr_summary.md` - PR summary generation template
- `changelog.md` - Keep a Changelog format generation template
- `explain_diff.md` - Plain-language diff explanation template
- `issue_triage.md` - Issue classification template
- `issue_response.md` - Issue response template
@@ -533,6 +534,79 @@ The `@codebot changelog` command generates Keep a Changelog format entries from
- Excluded from ai-chat.yml to prevent duplicate runs
- No automatic triggering - manual command only
### Code Diff Explainer
The `@codebot explain-diff` command translates technical code changes into plain language for non-technical stakeholders.
**Key Features:**
- Plain-language explanations without jargon
- File-by-file breakdown with "what" and "why" context
- Architecture impact analysis
- Breaking change detection
- Perfect for PMs, designers, and new team members
**Implementation Details:**
1. **Command Handler** - `PRAgent._handle_explain_diff_command()`:
- Triggered by `@codebot explain-diff` in PR comments
- Fetches PR title, description, and full diff
- Loads `prompts/explain_diff.md` template
- Formats prompt with PR context
2. **LLM Analysis** - Generates plain-language JSON:
```json
{
"overview": "High-level summary in everyday language",
"key_changes": [
{
"file": "path/to/file.py",
"status": "new|modified|deleted",
"explanation": "What changed (no jargon)",
"why_it_matters": "Business/user impact"
}
],
"architecture_impact": {
"description": "System-level effects explained simply",
"new_dependencies": ["External libraries added"],
"affected_components": ["System parts impacted"]
},
"breaking_changes": ["User-facing breaking changes"],
"technical_details": { /* Stats for reference */ }
}
```
3. **Formatting** - `_format_diff_explanation()`:
- Converts JSON to readable markdown
- Uses emojis for visual categorization (➕ new, 📝 modified, 🗑️ deleted)
- Highlights breaking changes prominently
- Includes technical summary for developers
- Omits empty sections for clean output
4. **Prompt Engineering** - `prompts/explain_diff.md`:
- **Avoids jargon**: "API" → "connection point between systems"
- **Explains why**: Not just what changed, but why it matters
- **Uses analogies**: "Caching" → "memory system for faster loading"
- **Focus on impact**: Who is affected and how
- **Groups changes**: Combines related files into themes
- **Translates concepts**: Technical terms → everyday language
**Plain Language Rules:**
- ❌ "Refactored authentication middleware" → ✅ "Updated login system for better security"
- ❌ "Implemented Redis caching" → ✅ "Added memory to make pages load 10x faster"
- ❌ "Database migration" → ✅ "Updated how data is stored"
**Common Use Cases:**
- New team members understanding large PRs
- Non-technical reviewers (PMs, designers) reviewing features
- Documenting architectural decisions
- Learning from other developers' code
**Workflow Safety:**
- Only triggers on PR comments (not issue comments)
- Included in ai-comment-reply.yml workflow conditions
- Excluded from ai-chat.yml to prevent duplicate runs
- No automatic triggering - manual command only
### Review-Again Command Implementation
The `@codebot review-again` command allows manual re-triggering of PR reviews without new commits.
@@ -591,6 +665,7 @@ Example commands:
- `@codebot suggest` - Suggest solutions
- `@codebot summarize` - Generate PR summary or issue summary (works on both)
- `@codebot changelog` - Generate Keep a Changelog format entries (PR comments only)
- `@codebot explain-diff` - Explain code changes in plain language (PR comments only)
- `@codebot setup-labels` - Automatic label setup (built-in, not in config)
- `@codebot review-again` - Re-run PR review without new commits (PR comments only)

62
README.md
View File

@@ -12,7 +12,7 @@ Enterprise-grade AI code review system for **Gitea** with automated PR review, i
| **PR Summaries** | Auto-generate comprehensive PR summaries with change analysis and impact assessment |
| **Issue Triage** | On-demand classification, labeling, priority assignment via `@codebot triage` |
| **Chat** | Interactive AI chat with codebase search and web search tools |
| **@codebot Commands** | `@codebot summarize`, `changelog`, `explain`, `suggest`, `triage`, `review-again` in comments |
| **@codebot Commands** | `@codebot summarize`, `changelog`, `explain-diff`, `explain`, `suggest`, `triage`, `review-again` in comments |
| **Codebase Analysis** | Health scores, tech debt tracking, weekly reports |
| **Security Scanner** | 17 OWASP-aligned rules for vulnerability detection |
| **Enterprise Ready** | Audit logging, metrics, Prometheus export |
@@ -192,6 +192,7 @@ In any PR comment:
|---------|-------------|
| `@codebot summarize` | Generate a comprehensive PR summary with changes, files affected, and impact |
| `@codebot changelog` | Generate Keep a Changelog format entries ready for CHANGELOG.md |
| `@codebot explain-diff` | Explain code changes in plain language for non-technical stakeholders |
| `@codebot review-again` | Re-run AI code review on current PR state without new commits |
#### PR Summary (`@codebot summarize`)
@@ -281,6 +282,65 @@ Adds new feature without affecting existing functionality
- **Main components:** auth/, api/users/, database/
```
#### Diff Explainer (`@codebot explain-diff`)
**Features:**
- 📖 Translates technical changes into plain language
- 🎯 Perfect for non-technical stakeholders (PMs, designers)
- 🔍 File-by-file breakdown with "what" and "why"
- 🏗️ Architecture impact analysis
- ⚠️ Breaking change detection
- 📊 Technical summary for reference
**When to use:**
- New team members reviewing complex PRs
- Non-technical reviewers need to understand changes
- Documenting architectural decisions
- Learning from others' code
**Example output:**
```markdown
## 📖 Code Changes Explained (PR #123)
### 🎯 Overview
This PR adds user authentication using secure tokens that expire after 24 hours, enabling users to log in securely without storing passwords in the application.
### 🔍 What Changed
#### ➕ `auth/jwt.py` (new)
**What changed:** Creates secure tokens for logged-in users
**Why it matters:** Enables the app to remember who you are without constantly asking for your password
#### 📝 `api/users.py` (modified)
**What changed:** Added a login page where users can sign in
**Why it matters:** Users can now create accounts and access their personal data
---
### 🏗️ Architecture Impact
Introduces a security layer across the entire application, ensuring only authenticated users can access protected features.
**New dependencies:**
- PyJWT (for creating secure tokens)
- bcrypt (for password encryption)
**Affected components:**
- API (all endpoints now check authentication)
- Database (added user credentials storage)
---
### ⚠️ Breaking Changes
- **All API endpoints now require authentication - existing scripts need to be updated**
---
### 📊 Technical Summary
- **Files changed:** 5
- **Lines:** +200 / -10
- **Components:** auth/, api/
```
#### Review Again (`@codebot review-again`)
**Features:**

240
tests/test_ai_review.py
View File

@@ -1052,5 +1052,245 @@ class TestChangelogGeneration:
assert "changelog" in config["interaction"]["commands"]
class TestDiffExplanation:
"""Test diff explanation functionality."""
def test_explain_diff_prompt_exists(self):
"""Verify explain_diff.md prompt file exists."""
prompt_path = os.path.join(
os.path.dirname(__file__),
"..",
"tools",
"ai-review",
"prompts",
"explain_diff.md",
)
assert os.path.exists(prompt_path), "explain_diff.md prompt file not found"
def test_explain_diff_prompt_formatting(self):
"""Test that explain_diff.md can be formatted with placeholders."""
prompt_path = os.path.join(
os.path.dirname(__file__),
"..",
"tools",
"ai-review",
"prompts",
"explain_diff.md",
)
with open(prompt_path) as f:
prompt = f.read()
# Check for key elements
assert "plain language" in prompt.lower() or "plain-language" in prompt.lower()
assert "overview" in prompt.lower()
assert "key changes" in prompt.lower() or "key_changes" in prompt.lower()
assert "architecture" in prompt.lower()
assert "JSON" in prompt
# Should be able to format with pr_title and pr_description
try:
formatted = prompt.format(
pr_title="Test PR Title", pr_description="Test PR Description"
)
assert "Test PR Title" in formatted
assert "Test PR Description" in formatted
except KeyError as e:
pytest.fail(f"Prompt has unescaped placeholders: {e}")
def test_pr_agent_can_handle_explain_diff_command(self):
"""Test that PRAgent can handle @codebot explain-diff in PR comments."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Test explain-diff command in PR comment
event_data = {
"action": "created",
"issue": {
"number": 123,
"pull_request": {}, # Indicates this is a PR
},
"comment": {
"body": "@codebot explain-diff please",
},
}
assert agent.can_handle("issue_comment", event_data) is True
def test_pr_agent_can_handle_explain_diff_case_insensitive(self):
"""Test that explain-diff command is case-insensitive."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Test various casings
for body in [
"@codebot EXPLAIN-DIFF",
"@codebot Explain-Diff",
"@codebot ExPlAiN-dIfF",
]:
event_data = {
"action": "created",
"issue": {
"number": 123,
"pull_request": {},
},
"comment": {"body": body},
}
assert agent.can_handle("issue_comment", event_data) is True
def test_pr_agent_ignores_explain_diff_on_non_pr(self):
"""Test that explain-diff command is ignored on regular issues."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Regular issue (no pull_request field)
event_data = {
"action": "created",
"issue": {
"number": 123,
# No pull_request field
},
"comment": {
"body": "@codebot explain-diff",
},
}
assert agent.can_handle("issue_comment", event_data) is False
def test_format_diff_explanation_structure(self):
"""Test _format_diff_explanation generates correct structure."""
from agents.pr_agent import PRAgent
agent = PRAgent(config={})
explanation_data = {
"overview": "This PR adds user authentication using JWT tokens",
"key_changes": [
{
"file": "auth/jwt.py",
"status": "new",
"explanation": "Creates JSON Web Tokens for authenticated users",
"why_it_matters": "Enables secure stateless authentication",
},
{
"file": "api/users.py",
"status": "modified",
"explanation": "Added login endpoint",
"why_it_matters": "Users can now authenticate",
},
],
"architecture_impact": {
"description": "Introduces authentication layer across all API endpoints",
"new_dependencies": ["PyJWT", "bcrypt"],
"affected_components": ["API", "Database"],
},
"breaking_changes": ["All API endpoints now require authentication"],
"technical_details": {
"files_changed": 5,
"insertions": 200,
"deletions": 10,
"main_components": ["auth/", "api/"],
},
}
result = agent._format_diff_explanation(explanation_data, 123)
# Verify structure
assert "## 📖 Code Changes Explained (PR #123)" in result
assert "### 🎯 Overview" in result
assert "user authentication using JWT tokens" in result
assert "### 🔍 What Changed" in result
assert "➕ `auth/jwt.py` (new)" in result
assert "📝 `api/users.py` (modified)" in result
assert "**What changed:**" in result
assert "**Why it matters:**" in result
assert "### 🏗️ Architecture Impact" in result
assert "PyJWT" in result
assert "### ⚠️ Breaking Changes" in result
assert "All API endpoints now require authentication" in result
assert "### 📊 Technical Summary" in result
assert "Files changed:** 5" in result
assert "+200 / -10" in result
def test_format_diff_explanation_empty_sections(self):
"""Test that empty sections are not included in output."""
from agents.pr_agent import PRAgent
agent = PRAgent(config={})
explanation_data = {
"overview": "Small bugfix",
"key_changes": [
{
"file": "app.py",
"status": "modified",
"explanation": "Fixed typo",
"why_it_matters": "",
}
],
"architecture_impact": {},
"breaking_changes": [],
"technical_details": {},
}
result = agent._format_diff_explanation(explanation_data, 123)
# Only overview and key changes should be present
assert "### 🎯 Overview" in result
assert "### 🔍 What Changed" in result
assert "### 🏗️ Architecture Impact" not in result
assert "### ⚠️ Breaking Changes" not in result
def test_config_has_explain_diff_command(self):
"""Verify config.yml has explain-diff command."""
config_path = os.path.join(
os.path.dirname(__file__), "..", "tools", "ai-review", "config.yml"
)
import yaml
with open(config_path) as f:
config = yaml.safe_load(f)
assert "interaction" in config
assert "commands" in config["interaction"]
assert "explain-diff" in config["interaction"]["commands"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])

202
tools/ai-review/agents/pr_agent.py
View File

@@ -100,7 +100,15 @@ class PRAgent(BaseAgent):
)
has_summarize = f"{mention_prefix} summarize" in comment_body.lower()
has_changelog = f"{mention_prefix} changelog" in comment_body.lower()
return is_pr and (has_review_again or has_summarize or has_changelog)
has_explain_diff = (
f"{mention_prefix} explain-diff" in comment_body.lower()
)
return is_pr and (
has_review_again
or has_summarize
or has_changelog
or has_explain_diff
)
return False
@@ -116,6 +124,8 @@ class PRAgent(BaseAgent):
return self._handle_summarize_command(context)
elif f"{mention_prefix} changelog" in comment_body.lower():
return self._handle_changelog_command(context)
elif f"{mention_prefix} explain-diff" in comment_body.lower():
return self._handle_explain_diff_command(context)
elif f"{mention_prefix} review-again" in comment_body.lower():
return self._handle_review_again(context)
@@ -1211,3 +1221,193 @@ class PRAgent(BaseAgent):
lines.append(f"- **Main components:** {', '.join(components)}")
return "\n".join(lines)
def _handle_explain_diff_command(self, context: AgentContext) -> AgentResult:
"""Handle @codebot explain-diff command from PR comments.
Generates plain-language explanation of code changes for non-technical stakeholders.
Args:
context: Agent context with event data
Returns:
AgentResult with success status and actions taken
"""
issue = context.event_data.get("issue", {})
pr_number = issue.get("number")
comment_author = (
context.event_data.get("comment", {}).get("user", {}).get("login", "user")
)
self.logger.info(
f"Generating diff explanation for PR #{pr_number} at user request"
)
try:
# Get PR data
pr = self.gitea.get_pull_request(context.owner, context.repo, pr_number)
pr_title = pr.get("title", "")
pr_description = pr.get("body", "")
# Get PR diff
diff = self._get_diff(context.owner, context.repo, pr_number)
if not diff.strip():
error_msg = (
f"@{comment_author}\n\n"
f"{self.AI_DISCLAIMER}\n\n"
"**⚠️ Diff Explanation Failed**\n\n"
"No changes found in this PR to explain."
)
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, error_msg
)
return AgentResult(
success=False,
message=f"No diff to explain for PR #{pr_number}",
)
# Load explain_diff prompt
prompt_template = self.load_prompt("explain_diff")
prompt = prompt_template.format(
pr_title=pr_title,
pr_description=pr_description or "(No description provided)",
)
prompt = f"{prompt}\n{diff}"
# Call LLM to generate explanation
result = self.call_llm_json(prompt)
# Format the explanation comment
explanation_comment = self._format_diff_explanation(result, pr_number)
# Post explanation comment
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, explanation_comment
)
return AgentResult(
success=True,
message=f"Generated diff explanation for PR #{pr_number}",
actions_taken=["Posted diff explanation comment"],
)
except Exception as e:
self.logger.error(f"Failed to generate diff explanation: {e}")
# Post error message
error_msg = (
f"@{comment_author}\n\n"
f"{self.AI_DISCLAIMER}\n\n"
"**⚠️ Diff Explanation Failed**\n\n"
f"I encountered an error while generating the explanation: {str(e)}\n\n"
"This could be due to:\n"
"- The PR is too large to analyze\n"
"- The LLM service is temporarily unavailable\n"
"- An unexpected error occurred"
)
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, error_msg
)
return AgentResult(
success=False,
message=f"Failed to generate diff explanation for PR #{pr_number}",
error=str(e),
)
def _format_diff_explanation(self, explanation_data: dict, pr_number: int) -> str:
"""Format diff explanation data into readable markdown.
Args:
explanation_data: JSON data from LLM containing explanation
pr_number: PR number for reference
Returns:
Formatted markdown explanation
"""
lines = [
self.AI_DISCLAIMER,
"",
f"## 📖 Code Changes Explained (PR #{pr_number})",
"",
]
# Overview
overview = explanation_data.get("overview", "")
if overview:
lines.append("### 🎯 Overview")
lines.append(overview)
lines.append("")
# Key changes
key_changes = explanation_data.get("key_changes", [])
if key_changes:
lines.append("### 🔍 What Changed")
lines.append("")
for change in key_changes:
file_path = change.get("file", "unknown")
status = change.get("status", "modified")
explanation = change.get("explanation", "")
why_it_matters = change.get("why_it_matters", "")
# Status emoji
status_emoji = {"new": "➕", "modified": "📝", "deleted": "🗑️"}
emoji = status_emoji.get(status, "📝")
lines.append(f"#### {emoji} `{file_path}` ({status})")
lines.append(f"**What changed:** {explanation}")
if why_it_matters:
lines.append(f"**Why it matters:** {why_it_matters}")
lines.append("")
# Architecture impact
arch_impact = explanation_data.get("architecture_impact", {})
if arch_impact and arch_impact.get("description"):
lines.append("---")
lines.append("")
lines.append("### 🏗️ Architecture Impact")
lines.append(arch_impact.get("description", ""))
lines.append("")
new_deps = arch_impact.get("new_dependencies", [])
if new_deps:
lines.append("**New dependencies:**")
for dep in new_deps:
lines.append(f"- {dep}")
lines.append("")
affected = arch_impact.get("affected_components", [])
if affected:
lines.append("**Affected components:**")
for comp in affected:
lines.append(f"- {comp}")
lines.append("")
# Breaking changes
breaking = explanation_data.get("breaking_changes", [])
if breaking:
lines.append("---")
lines.append("")
lines.append("### ⚠️ Breaking Changes")
for change in breaking:
lines.append(f"- **{change}**")
lines.append("")
# Technical details
tech = explanation_data.get("technical_details", {})
if tech:
lines.append("---")
lines.append("")
lines.append("### 📊 Technical Summary")
files = tech.get("files_changed", 0)
additions = tech.get("insertions", 0)
deletions = tech.get("deletions", 0)
lines.append(f"- **Files changed:** {files}")
lines.append(f"- **Lines:** +{additions} / -{deletions}")
components = tech.get("main_components", [])
if components:
lines.append(f"- **Components:** {', '.join(components)}")
return "\n".join(lines)

1
tools/ai-review/config.yml
View File

@@ -68,6 +68,7 @@ interaction:
- security
- summarize # Generate PR summary (works on both issues and PRs)
- changelog # Generate Keep a Changelog format entries (PR comments only)
- explain-diff # Explain code changes in plain language (PR comments only)
- triage
- review-again

99
tools/ai-review/prompts/explain_diff.md Normal file
View File

@@ -0,0 +1,99 @@
You are an experienced technical writer explaining code changes to **non-technical stakeholders** (product managers, designers, business analysts).
Your goal is to translate complex code diffs into **clear, plain-language explanations** that anyone can understand, regardless of their technical background.
---
## Requirements
Analyze the PR diff and generate a structured explanation with:
1. **Overview** - High-level summary in 1-2 sentences (what changed and why)
2. **Key Changes** - File-by-file breakdown in plain language
3. **Architecture Impact** - How this affects the overall system
4. **Breaking Changes** - Any changes that affect existing functionality (if applicable)
5. **Technical Details** - Summary of files, lines, and components (for reference)
---
## Output Format
Return a JSON object with this structure:
```json
{{{{
"overview": "One or two sentence summary of what this PR accomplishes",
"key_changes": [
{{{{
"file": "path/to/file.py",
"status": "new" | "modified" | "deleted",
"explanation": "Plain language explanation of what changed in this file",
"why_it_matters": "Why this change is important or what problem it solves"
}}}}
],
"architecture_impact": {{{{
"description": "How this affects the overall system architecture",
"new_dependencies": ["List of new libraries or services added"],
"affected_components": ["List of system components that are impacted"]
}}}},
"breaking_changes": [
"List of changes that break backward compatibility or affect existing features"
],
"technical_details": {{{{
"files_changed": 15,
"insertions": 450,
"deletions": 120,
"main_components": ["List of main directories/components affected"]
}}}}
}}}}
```
---
## Rules for Plain Language Explanations
1. **Avoid jargon**: Use everyday language, not technical terms
- ❌ Bad: "Refactored the authentication middleware to use JWT tokens"
- ✅ Good: "Updated the login system to use secure tokens that expire after 24 hours"
2. **Explain the "why", not just the "what"**
- ❌ Bad: "Added new function `calculate_total()`"
- ✅ Good: "Added calculation logic to automatically sum up order totals, preventing manual errors"
3. **Use analogies and real-world examples**
- ❌ Bad: "Implemented caching layer using Redis"
- ✅ Good: "Added a memory system that remembers frequently accessed data, making the app load 10x faster"
4. **Focus on user impact**
- ❌ Bad: "Optimized database queries"
- ✅ Good: "Made the search feature faster by improving how we retrieve data"
5. **Group related changes together**
- Instead of listing 10 small files, say "Updated 10 files across the payment system to fix checkout bugs"
6. **Be specific about impact**
- "This change affects all users on the mobile app"
- "This only impacts admin users"
- "This is internal cleanup with no user-visible changes"
7. **Translate technical concepts**
- API → "connection point between systems"
- Database migration → "updating how data is stored"
- Refactoring → "cleaning up code without changing behavior"
- Dependency → "external library or tool we use"
8. **Highlight risks clearly**
- "This requires a system restart"
- "Users will need to log in again"
- "This changes how existing features work"
---
## PR Information
**Title:** {pr_title}
**Description:** {pr_description}
**Diff:**