Hiddenden/openrabbit
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity

Merge pull request 'feat: Add @codebot changelog command for Keep a Changelog format generation' (#25) from feature/pr-changelog-generator into dev

Browse Source 
Reviewed-on: #25
This commit was merged in pull request #25.
This commit is contained in:
Latte
2025-12-29 11:53:57 +00:00
parent c6dbb0acf6 15beb0fb5b
commit 1d468e360e
8 changed files with 628 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitea/workflows/ai-chat.yml
View File

@@ -25,6 +25,7 @@ jobs:
!contains(github.event.comment.body, '@codebot suggest') &&
!contains(github.event.comment.body, '@codebot security') &&
!contains(github.event.comment.body, '@codebot summarize') &&
!contains(github.event.comment.body, '@codebot changelog') &&
!contains(github.event.comment.body, '@codebot review-again') &&
!contains(github.event.comment.body, '@codebot setup-labels')
runs-on: ubuntu-latest

3
.gitea/workflows/ai-comment-reply.yml
View File

@@ -1,7 +1,7 @@
name: AI Comment Reply
# WORKFLOW ROUTING:
# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, review-again, setup-labels
# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, changelog, review-again, setup-labels
# Other workflows: ai-issue-triage.yml (@codebot triage), ai-chat.yml (free-form questions)
on:
@@ -23,6 +23,7 @@ jobs:
contains(github.event.comment.body, '@codebot suggest') ||
contains(github.event.comment.body, '@codebot security') ||
contains(github.event.comment.body, '@codebot summarize') ||
contains(github.event.comment.body, '@codebot changelog') ||
contains(github.event.comment.body, '@codebot review-again') ||
contains(github.event.comment.body, '@codebot setup-labels'))
steps:

66
CLAUDE.md
View File

@@ -216,6 +216,7 @@ Prompts are stored in `tools/ai-review/prompts/` as Markdown files:
- `base.md` - Base instructions for all reviews
- `pr_summary.md` - PR summary generation template
- `changelog.md` - Keep a Changelog format generation template
- `issue_triage.md` - Issue classification template
- `issue_response.md` - Issue response template
@@ -468,6 +469,70 @@ The PR summary feature automatically generates comprehensive summaries for pull
- Standardized documentation format
- Pre-review context for reviewers
### PR Changelog Generation
The `@codebot changelog` command generates Keep a Changelog format entries from PR diffs.
**Key Features:**
- Generates structured changelog entries following Keep a Changelog format
- Categorizes changes: Added/Changed/Deprecated/Removed/Fixed/Security
- Automatically detects breaking changes
- Includes technical details (files changed, LOC, components)
- Output is ready to copy-paste into CHANGELOG.md
**Implementation Details:**
1. **Command Handler** - `PRAgent._handle_changelog_command()`:
- Triggered by `@codebot changelog` in PR comments
- Fetches PR title, description, and diff
- Loads `prompts/changelog.md` template
- Formats prompt with PR context
2. **LLM Analysis** - Generates structured JSON:
```json
{
"changelog": {
"added": ["New features"],
"changed": ["Changes to existing functionality"],
"fixed": ["Bug fixes"],
"security": ["Security fixes"]
},
"breaking_changes": ["Breaking changes"],
"technical_details": {
"files_changed": 15,
"insertions": 450,
"deletions": 120,
"main_components": ["auth/", "api/"]
}
}
```
3. **Formatting** - `_format_changelog()`:
- Converts JSON to Keep a Changelog markdown format
- Uses emojis for visual categorization (✨ Added, 🔄 Changed, 🐛 Fixed)
- Highlights breaking changes prominently
- Includes technical summary at the end
- Omits empty sections for clean output
4. **Prompt Engineering** - `prompts/changelog.md`:
- User-focused language (not developer jargon)
- Filters noise (formatting, typos, minor refactoring)
- Groups related changes
- Active voice, concise entries
- Maximum 100 characters per entry
**Common Use Cases:**
- Preparing release notes
- Maintaining CHANGELOG.md
- Customer-facing announcements
- Version documentation
**Workflow Safety:**
- Only triggers on PR comments (not issue comments)
- Included in ai-comment-reply.yml workflow conditions
- Excluded from ai-chat.yml to prevent duplicate runs
- No automatic triggering - manual command only
### Review-Again Command Implementation
The `@codebot review-again` command allows manual re-triggering of PR reviews without new commits.
@@ -525,6 +590,7 @@ Example commands:
- `@codebot explain` - Explain the issue
- `@codebot suggest` - Suggest solutions
- `@codebot summarize` - Generate PR summary or issue summary (works on both)
- `@codebot changelog` - Generate Keep a Changelog format entries (PR comments only)
- `@codebot setup-labels` - Automatic label setup (built-in, not in config)
- `@codebot review-again` - Re-run PR review without new commits (PR comments only)

50
README.md
View File

@@ -12,7 +12,7 @@ Enterprise-grade AI code review system for **Gitea** with automated PR review, i
| **PR Summaries** | Auto-generate comprehensive PR summaries with change analysis and impact assessment |
| **Issue Triage** | On-demand classification, labeling, priority assignment via `@codebot triage` |
| **Chat** | Interactive AI chat with codebase search and web search tools |
| **@codebot Commands** | `@codebot summarize`, `explain`, `suggest`, `triage`, `review-again` in comments |
| **@codebot Commands** | `@codebot summarize`, `changelog`, `explain`, `suggest`, `triage`, `review-again` in comments |
| **Codebase Analysis** | Health scores, tech debt tracking, weekly reports |
| **Security Scanner** | 17 OWASP-aligned rules for vulnerability detection |
| **Enterprise Ready** | Audit logging, metrics, Prometheus export |
@@ -191,6 +191,7 @@ In any PR comment:
| Command | Description |
|---------|-------------|
| `@codebot summarize` | Generate a comprehensive PR summary with changes, files affected, and impact |
| `@codebot changelog` | Generate Keep a Changelog format entries ready for CHANGELOG.md |
| `@codebot review-again` | Re-run AI code review on current PR state without new commits |
#### PR Summary (`@codebot summarize`)
@@ -233,6 +234,53 @@ This PR implements automatic PR summary generation...
Adds new feature without affecting existing functionality
```
#### Changelog Generator (`@codebot changelog`)
**Features:**
- 📋 Generates Keep a Changelog format entries
- 🏷️ Categorizes changes (Added/Changed/Fixed/Removed/Security)
- ⚠️ Detects breaking changes automatically
- 📊 Includes technical details (files, LOC, components)
- 📝 Ready to copy-paste into CHANGELOG.md
**When to use:**
- Preparing release notes
- Maintaining CHANGELOG.md
- Customer-facing announcements
- Version documentation
**Example output:**
```markdown
## 📋 Changelog for PR #123
### ✨ Added
- User authentication system with JWT tokens
- Password reset functionality via email
### 🔄 Changed
- Updated database schema for user table
- Refactored login endpoint for better error handling
### 🐛 Fixed
- Session timeout bug causing premature logouts
- Security vulnerability in password validation
### 🔒 Security
- Fixed XSS vulnerability in user input validation
---
### ⚠️ BREAKING CHANGES
- **Removed legacy API endpoint /api/v1/old - migrate to /api/v2**
---
### 📊 Technical Details
- **Files changed:** 15
- **Lines:** +450 / -120
- **Main components:** auth/, api/users/, database/
```
#### Review Again (`@codebot review-again`)
**Features:**

227
tests/test_ai_review.py
View File

@@ -825,5 +825,232 @@ class TestPRSummaryGeneration:
assert "post_as_comment" in config["agents"]["pr"]["auto_summary"]
class TestChangelogGeneration:
"""Test changelog generation functionality."""
def test_changelog_prompt_exists(self):
"""Verify changelog.md prompt file exists."""
prompt_path = os.path.join(
os.path.dirname(__file__),
"..",
"tools",
"ai-review",
"prompts",
"changelog.md",
)
assert os.path.exists(prompt_path), "changelog.md prompt file not found"
def test_changelog_prompt_formatting(self):
"""Test that changelog.md can be formatted with placeholders."""
prompt_path = os.path.join(
os.path.dirname(__file__),
"..",
"tools",
"ai-review",
"prompts",
"changelog.md",
)
with open(prompt_path) as f:
prompt = f.read()
# Check for key elements
assert "changelog" in prompt.lower()
assert "added" in prompt.lower()
assert "changed" in prompt.lower()
assert "fixed" in prompt.lower()
assert "breaking" in prompt.lower()
assert "JSON" in prompt
# Should be able to format with pr_title and pr_description
try:
formatted = prompt.format(
pr_title="Test PR Title", pr_description="Test PR Description"
)
assert "Test PR Title" in formatted
assert "Test PR Description" in formatted
except KeyError as e:
pytest.fail(f"Prompt has unescaped placeholders: {e}")
def test_pr_agent_can_handle_changelog_command(self):
"""Test that PRAgent can handle @codebot changelog in PR comments."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Test changelog command in PR comment
event_data = {
"action": "created",
"issue": {
"number": 123,
"pull_request": {}, # Indicates this is a PR
},
"comment": {
"body": "@codebot changelog please",
},
}
assert agent.can_handle("issue_comment", event_data) is True
def test_pr_agent_can_handle_changelog_case_insensitive(self):
"""Test that changelog command is case-insensitive."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Test various casings
for body in [
"@codebot CHANGELOG",
"@codebot Changelog",
"@codebot ChAnGeLoG",
]:
event_data = {
"action": "created",
"issue": {
"number": 123,
"pull_request": {},
},
"comment": {"body": body},
}
assert agent.can_handle("issue_comment", event_data) is True
def test_pr_agent_ignores_changelog_on_non_pr(self):
"""Test that changelog command is ignored on regular issues."""
from agents.pr_agent import PRAgent
config = {
"agents": {
"pr": {
"enabled": True,
}
},
"interaction": {
"mention_prefix": "@codebot",
},
}
agent = PRAgent(config=config)
# Regular issue (no pull_request field)
event_data = {
"action": "created",
"issue": {
"number": 123,
# No pull_request field
},
"comment": {
"body": "@codebot changelog",
},
}
assert agent.can_handle("issue_comment", event_data) is False
def test_format_changelog_structure(self):
"""Test _format_changelog generates correct Keep a Changelog format."""
from agents.pr_agent import PRAgent
agent = PRAgent(config={})
changelog_data = {
"changelog": {
"added": ["User authentication system", "Password reset feature"],
"changed": ["Updated database schema", "Refactored login endpoint"],
"fixed": ["Session timeout bug", "Security vulnerability"],
"security": ["Fixed XSS vulnerability in user input"],
},
"breaking_changes": ["Removed legacy API endpoint /api/v1/old"],
"technical_details": {
"files_changed": 15,
"insertions": 450,
"deletions": 120,
"main_components": ["auth/", "api/users/", "database/"],
},
}
result = agent._format_changelog(changelog_data, 123)
# Verify structure
assert "## 📋 Changelog for PR #123" in result
assert "### ✨ Added" in result
assert "User authentication system" in result
assert "### 🔄 Changed" in result
assert "Updated database schema" in result
assert "### 🐛 Fixed" in result
assert "Session timeout bug" in result
assert "### 🔒 Security" in result
assert "Fixed XSS vulnerability" in result
assert "### ⚠️ BREAKING CHANGES" in result
assert "Removed legacy API endpoint" in result
assert "### 📊 Technical Details" in result
assert "Files changed:** 15" in result
assert "+450 / -120" in result
assert "auth/" in result
def test_format_changelog_empty_sections(self):
"""Test that empty sections are not included in output."""
from agents.pr_agent import PRAgent
agent = PRAgent(config={})
changelog_data = {
"changelog": {
"added": ["New feature"],
"changed": [],
"deprecated": [],
"removed": [],
"fixed": [],
"security": [],
},
"breaking_changes": [],
"technical_details": {},
}
result = agent._format_changelog(changelog_data, 123)
# Only Added section should be present
assert "### ✨ Added" in result
assert "New feature" in result
assert "### 🔄 Changed" not in result
assert "### 🗑️ Removed" not in result
assert "### 🐛 Fixed" not in result
assert "### ⚠️ BREAKING CHANGES" not in result
def test_config_has_changelog_command(self):
"""Verify config.yml has changelog command."""
config_path = os.path.join(
os.path.dirname(__file__), "..", "tools", "ai-review", "config.yml"
)
import yaml
with open(config_path) as f:
config = yaml.safe_load(f)
assert "interaction" in config
assert "commands" in config["interaction"]
assert "changelog" in config["interaction"]["commands"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])

192
tools/ai-review/agents/pr_agent.py
View File

@@ -99,7 +99,8 @@ class PRAgent(BaseAgent):
f"{mention_prefix} review-again" in comment_body.lower()
)
has_summarize = f"{mention_prefix} summarize" in comment_body.lower()
return is_pr and (has_review_again or has_summarize)
has_changelog = f"{mention_prefix} changelog" in comment_body.lower()
return is_pr and (has_review_again or has_summarize or has_changelog)
return False
@@ -113,6 +114,8 @@ class PRAgent(BaseAgent):
)
if f"{mention_prefix} summarize" in comment_body.lower():
return self._handle_summarize_command(context)
elif f"{mention_prefix} changelog" in comment_body.lower():
return self._handle_changelog_command(context)
elif f"{mention_prefix} review-again" in comment_body.lower():
return self._handle_review_again(context)
@@ -1021,3 +1024,190 @@ class PRAgent(BaseAgent):
message=f"Failed to generate PR summary for PR #{pr_number}",
error="Summary generation failed",
)
def _handle_changelog_command(self, context: AgentContext) -> AgentResult:
"""Handle @codebot changelog command from PR comments.
Generates Keep a Changelog format entries for the PR.
Args:
context: Agent context with event data
Returns:
AgentResult with success status and actions taken
"""
issue = context.event_data.get("issue", {})
pr_number = issue.get("number")
comment_author = (
context.event_data.get("comment", {}).get("user", {}).get("login", "user")
)
self.logger.info(f"Generating changelog for PR #{pr_number} at user request")
try:
# Get PR data
pr = self.gitea.get_pull_request(context.owner, context.repo, pr_number)
pr_title = pr.get("title", "")
pr_description = pr.get("body", "")
# Get PR diff
diff = self._get_diff(context.owner, context.repo, pr_number)
if not diff.strip():
error_msg = (
f"@{comment_author}\n\n"
f"{self.AI_DISCLAIMER}\n\n"
"**⚠️ Changelog Generation Failed**\n\n"
"No changes found in this PR to analyze."
)
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, error_msg
)
return AgentResult(
success=False,
message=f"No diff to generate changelog for PR #{pr_number}",
)
# Load changelog prompt
prompt_template = self.load_prompt("changelog")
prompt = prompt_template.format(
pr_title=pr_title,
pr_description=pr_description or "(No description provided)",
)
prompt = f"{prompt}\n{diff}"
# Call LLM to generate changelog
result = self.call_llm_json(prompt)
# Format the changelog comment
changelog_comment = self._format_changelog(result, pr_number)
# Post changelog comment
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, changelog_comment
)
return AgentResult(
success=True,
message=f"Generated changelog for PR #{pr_number}",
actions_taken=["Posted changelog comment"],
)
except Exception as e:
self.logger.error(f"Failed to generate changelog: {e}")
# Post error message
error_msg = (
f"@{comment_author}\n\n"
f"{self.AI_DISCLAIMER}\n\n"
"**⚠️ Changelog Generation Failed**\n\n"
f"I encountered an error while generating the changelog: {str(e)}\n\n"
"This could be due to:\n"
"- The PR is too large to analyze\n"
"- The LLM service is temporarily unavailable\n"
"- An unexpected error occurred"
)
self.gitea.create_issue_comment(
context.owner, context.repo, pr_number, error_msg
)
return AgentResult(
success=False,
message=f"Failed to generate changelog for PR #{pr_number}",
error=str(e),
)
def _format_changelog(self, changelog_data: dict, pr_number: int) -> str:
"""Format changelog data into Keep a Changelog format.
Args:
changelog_data: JSON data from LLM containing changelog entries
pr_number: PR number for reference
Returns:
Formatted markdown changelog
"""
lines = [
self.AI_DISCLAIMER,
"",
f"## 📋 Changelog for PR #{pr_number}",
"",
]
changelog = changelog_data.get("changelog", {})
# Added
added = changelog.get("added", [])
if added:
lines.append("### ✨ Added")
for item in added:
lines.append(f"- {item}")
lines.append("")
# Changed
changed = changelog.get("changed", [])
if changed:
lines.append("### 🔄 Changed")
for item in changed:
lines.append(f"- {item}")
lines.append("")
# Deprecated
deprecated = changelog.get("deprecated", [])
if deprecated:
lines.append("### ⚠️ Deprecated")
for item in deprecated:
lines.append(f"- {item}")
lines.append("")
# Removed
removed = changelog.get("removed", [])
if removed:
lines.append("### 🗑️ Removed")
for item in removed:
lines.append(f"- {item}")
lines.append("")
# Fixed
fixed = changelog.get("fixed", [])
if fixed:
lines.append("### 🐛 Fixed")
for item in fixed:
lines.append(f"- {item}")
lines.append("")
# Security
security = changelog.get("security", [])
if security:
lines.append("### 🔒 Security")
for item in security:
lines.append(f"- {item}")
lines.append("")
# Breaking changes
breaking = changelog_data.get("breaking_changes", [])
if breaking:
lines.append("---")
lines.append("")
lines.append("### ⚠️ BREAKING CHANGES")
for item in breaking:
lines.append(f"- **{item}**")
lines.append("")
# Technical details
tech = changelog_data.get("technical_details", {})
if tech:
lines.append("---")
lines.append("")
lines.append("### 📊 Technical Details")
files = tech.get("files_changed", 0)
additions = tech.get("insertions", 0)
deletions = tech.get("deletions", 0)
lines.append(f"- **Files changed:** {files}")
lines.append(f"- **Lines:** +{additions} / -{deletions}")
components = tech.get("main_components", [])
if components:
lines.append(f"- **Main components:** {', '.join(components)}")
return "\n".join(lines)

1
tools/ai-review/config.yml
View File

@@ -67,6 +67,7 @@ interaction:
- suggest
- security
- summarize # Generate PR summary (works on both issues and PRs)
- changelog # Generate Keep a Changelog format entries (PR comments only)
- triage
- review-again

91
tools/ai-review/prompts/changelog.md Normal file
View File

@@ -0,0 +1,91 @@
You are an experienced software developer creating changelog entries for release notes following the **Keep a Changelog** format (https://keepachangelog.com/).
Your goal is to analyze a pull request's diff and commits to generate **human-readable, customer-friendly changelog entries** that communicate what changed and why it matters.
---
## Requirements
Analyze the PR and generate changelog entries categorized by:
1. **Added** - New features or functionality
2. **Changed** - Changes to existing functionality
3. **Deprecated** - Features that will be removed in future versions
4. **Removed** - Features that have been removed
5. **Fixed** - Bug fixes
6. **Security** - Security vulnerability fixes
Additional analysis:
- **Breaking Changes** - Changes that break backward compatibility
- **Technical Details** - Files changed, lines of code, main components affected
---
## Output Format
Return a JSON object with this structure:
```json
{{{{
"changelog": {{{{
"added": ["List of new features or functionality"],
"changed": ["List of changes to existing functionality"],
"deprecated": ["List of deprecated features"],
"removed": ["List of removed features"],
"fixed": ["List of bug fixes"],
"security": ["List of security fixes"]
}}}},
"breaking_changes": ["List of breaking changes, if any"],
"technical_details": {{{{
"files_changed": 15,
"insertions": 450,
"deletions": 120,
"main_components": ["List of main components/directories affected"]
}}}}
}}}}
```
---
## Rules
1. **Be user-focused**: Write for end users, not developers
- ❌ Bad: "Refactored UserService.authenticate() method"
- ✅ Good: "Improved login performance and reliability"
2. **Be specific and actionable**: Include what changed and the benefit
- ❌ Bad: "Updated authentication"
- ✅ Good: "Added JWT token authentication for improved security"
3. **Filter noise**: Ignore formatting changes, typos, minor refactoring unless user-visible
- Skip: "Fixed linting issues", "Updated whitespace", "Renamed internal variable"
- Include: "Fixed crash on invalid input", "Improved error messages"
4. **Detect breaking changes**: Look for:
- API endpoint changes (removed/renamed endpoints)
- Configuration changes (removed/renamed config keys)
- Dependency version upgrades with breaking changes
- Database schema changes requiring migrations
- Removed features or deprecated functionality
5. **Group related changes**: Combine similar changes into one entry
- ❌ "Added user login", "Added user logout", "Added password reset"
- ✅ "Added complete user authentication system with login, logout, and password reset"
6. **Use active voice**: Start with a verb
- ✅ "Added", "Fixed", "Improved", "Updated", "Removed"
7. **Keep entries concise**: One line per change, maximum 100 characters
8. **Output only JSON**: No additional text before or after the JSON object
---
## PR Information
**Title:** {pr_title}
**Description:** {pr_description}
**Diff:**