diff --git a/.gitea/workflows/ai-chat.yml b/.gitea/workflows/ai-chat.yml index 0b620e9..8c3f336 100644 --- a/.gitea/workflows/ai-chat.yml +++ b/.gitea/workflows/ai-chat.yml @@ -25,6 +25,7 @@ jobs: !contains(github.event.comment.body, '@codebot suggest') && !contains(github.event.comment.body, '@codebot security') && !contains(github.event.comment.body, '@codebot summarize') && + !contains(github.event.comment.body, '@codebot changelog') && !contains(github.event.comment.body, '@codebot review-again') && !contains(github.event.comment.body, '@codebot setup-labels') runs-on: ubuntu-latest diff --git a/.gitea/workflows/ai-comment-reply.yml b/.gitea/workflows/ai-comment-reply.yml index a845400..4804a2d 100644 --- a/.gitea/workflows/ai-comment-reply.yml +++ b/.gitea/workflows/ai-comment-reply.yml @@ -1,7 +1,7 @@ name: AI Comment Reply # WORKFLOW ROUTING: -# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, review-again, setup-labels +# This workflow handles SPECIFIC commands: help, explain, suggest, security, summarize, changelog, review-again, setup-labels # Other workflows: ai-issue-triage.yml (@codebot triage), ai-chat.yml (free-form questions) on: @@ -23,6 +23,7 @@ jobs: contains(github.event.comment.body, '@codebot suggest') || contains(github.event.comment.body, '@codebot security') || contains(github.event.comment.body, '@codebot summarize') || + contains(github.event.comment.body, '@codebot changelog') || contains(github.event.comment.body, '@codebot review-again') || contains(github.event.comment.body, '@codebot setup-labels')) steps: diff --git a/CLAUDE.md b/CLAUDE.md index 749662b..fbca82f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -216,6 +216,7 @@ Prompts are stored in `tools/ai-review/prompts/` as Markdown files: - `base.md` - Base instructions for all reviews - `pr_summary.md` - PR summary generation template +- `changelog.md` - Keep a Changelog format generation template - `issue_triage.md` - Issue classification template - `issue_response.md` - Issue response template @@ -468,6 +469,70 @@ The PR summary feature automatically generates comprehensive summaries for pull - Standardized documentation format - Pre-review context for reviewers +### PR Changelog Generation + +The `@codebot changelog` command generates Keep a Changelog format entries from PR diffs. + +**Key Features:** +- Generates structured changelog entries following Keep a Changelog format +- Categorizes changes: Added/Changed/Deprecated/Removed/Fixed/Security +- Automatically detects breaking changes +- Includes technical details (files changed, LOC, components) +- Output is ready to copy-paste into CHANGELOG.md + +**Implementation Details:** + +1. **Command Handler** - `PRAgent._handle_changelog_command()`: + - Triggered by `@codebot changelog` in PR comments + - Fetches PR title, description, and diff + - Loads `prompts/changelog.md` template + - Formats prompt with PR context + +2. **LLM Analysis** - Generates structured JSON: + ```json + { + "changelog": { + "added": ["New features"], + "changed": ["Changes to existing functionality"], + "fixed": ["Bug fixes"], + "security": ["Security fixes"] + }, + "breaking_changes": ["Breaking changes"], + "technical_details": { + "files_changed": 15, + "insertions": 450, + "deletions": 120, + "main_components": ["auth/", "api/"] + } + } + ``` + +3. **Formatting** - `_format_changelog()`: + - Converts JSON to Keep a Changelog markdown format + - Uses emojis for visual categorization (✨ Added, 🔄 Changed, 🐛 Fixed) + - Highlights breaking changes prominently + - Includes technical summary at the end + - Omits empty sections for clean output + +4. **Prompt Engineering** - `prompts/changelog.md`: + - User-focused language (not developer jargon) + - Filters noise (formatting, typos, minor refactoring) + - Groups related changes + - Active voice, concise entries + - Maximum 100 characters per entry + +**Common Use Cases:** +- Preparing release notes +- Maintaining CHANGELOG.md +- Customer-facing announcements +- Version documentation + +**Workflow Safety:** +- Only triggers on PR comments (not issue comments) +- Included in ai-comment-reply.yml workflow conditions +- Excluded from ai-chat.yml to prevent duplicate runs +- No automatic triggering - manual command only + ### Review-Again Command Implementation The `@codebot review-again` command allows manual re-triggering of PR reviews without new commits. @@ -525,6 +590,7 @@ Example commands: - `@codebot explain` - Explain the issue - `@codebot suggest` - Suggest solutions - `@codebot summarize` - Generate PR summary or issue summary (works on both) +- `@codebot changelog` - Generate Keep a Changelog format entries (PR comments only) - `@codebot setup-labels` - Automatic label setup (built-in, not in config) - `@codebot review-again` - Re-run PR review without new commits (PR comments only) diff --git a/README.md b/README.md index dc8b0be..c1cdc4b 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ Enterprise-grade AI code review system for **Gitea** with automated PR review, i | **PR Summaries** | Auto-generate comprehensive PR summaries with change analysis and impact assessment | | **Issue Triage** | On-demand classification, labeling, priority assignment via `@codebot triage` | | **Chat** | Interactive AI chat with codebase search and web search tools | -| **@codebot Commands** | `@codebot summarize`, `explain`, `suggest`, `triage`, `review-again` in comments | +| **@codebot Commands** | `@codebot summarize`, `changelog`, `explain`, `suggest`, `triage`, `review-again` in comments | | **Codebase Analysis** | Health scores, tech debt tracking, weekly reports | | **Security Scanner** | 17 OWASP-aligned rules for vulnerability detection | | **Enterprise Ready** | Audit logging, metrics, Prometheus export | @@ -191,6 +191,7 @@ In any PR comment: | Command | Description | |---------|-------------| | `@codebot summarize` | Generate a comprehensive PR summary with changes, files affected, and impact | +| `@codebot changelog` | Generate Keep a Changelog format entries ready for CHANGELOG.md | | `@codebot review-again` | Re-run AI code review on current PR state without new commits | #### PR Summary (`@codebot summarize`) @@ -233,6 +234,53 @@ This PR implements automatic PR summary generation... Adds new feature without affecting existing functionality ``` +#### Changelog Generator (`@codebot changelog`) + +**Features:** +- 📋 Generates Keep a Changelog format entries +- 🏷️ Categorizes changes (Added/Changed/Fixed/Removed/Security) +- ⚠️ Detects breaking changes automatically +- 📊 Includes technical details (files, LOC, components) +- 📝 Ready to copy-paste into CHANGELOG.md + +**When to use:** +- Preparing release notes +- Maintaining CHANGELOG.md +- Customer-facing announcements +- Version documentation + +**Example output:** +```markdown +## 📋 Changelog for PR #123 + +### ✨ Added +- User authentication system with JWT tokens +- Password reset functionality via email + +### 🔄 Changed +- Updated database schema for user table +- Refactored login endpoint for better error handling + +### 🐛 Fixed +- Session timeout bug causing premature logouts +- Security vulnerability in password validation + +### 🔒 Security +- Fixed XSS vulnerability in user input validation + +--- + +### ⚠️ BREAKING CHANGES +- **Removed legacy API endpoint /api/v1/old - migrate to /api/v2** + +--- + +### 📊 Technical Details +- **Files changed:** 15 +- **Lines:** +450 / -120 +- **Main components:** auth/, api/users/, database/ +``` + #### Review Again (`@codebot review-again`) **Features:** diff --git a/tests/test_ai_review.py b/tests/test_ai_review.py index c9d780d..e5d8f98 100644 --- a/tests/test_ai_review.py +++ b/tests/test_ai_review.py @@ -825,5 +825,232 @@ class TestPRSummaryGeneration: assert "post_as_comment" in config["agents"]["pr"]["auto_summary"] +class TestChangelogGeneration: + """Test changelog generation functionality.""" + + def test_changelog_prompt_exists(self): + """Verify changelog.md prompt file exists.""" + prompt_path = os.path.join( + os.path.dirname(__file__), + "..", + "tools", + "ai-review", + "prompts", + "changelog.md", + ) + assert os.path.exists(prompt_path), "changelog.md prompt file not found" + + def test_changelog_prompt_formatting(self): + """Test that changelog.md can be formatted with placeholders.""" + prompt_path = os.path.join( + os.path.dirname(__file__), + "..", + "tools", + "ai-review", + "prompts", + "changelog.md", + ) + with open(prompt_path) as f: + prompt = f.read() + + # Check for key elements + assert "changelog" in prompt.lower() + assert "added" in prompt.lower() + assert "changed" in prompt.lower() + assert "fixed" in prompt.lower() + assert "breaking" in prompt.lower() + assert "JSON" in prompt + + # Should be able to format with pr_title and pr_description + try: + formatted = prompt.format( + pr_title="Test PR Title", pr_description="Test PR Description" + ) + assert "Test PR Title" in formatted + assert "Test PR Description" in formatted + except KeyError as e: + pytest.fail(f"Prompt has unescaped placeholders: {e}") + + def test_pr_agent_can_handle_changelog_command(self): + """Test that PRAgent can handle @codebot changelog in PR comments.""" + from agents.pr_agent import PRAgent + + config = { + "agents": { + "pr": { + "enabled": True, + } + }, + "interaction": { + "mention_prefix": "@codebot", + }, + } + + agent = PRAgent(config=config) + + # Test changelog command in PR comment + event_data = { + "action": "created", + "issue": { + "number": 123, + "pull_request": {}, # Indicates this is a PR + }, + "comment": { + "body": "@codebot changelog please", + }, + } + + assert agent.can_handle("issue_comment", event_data) is True + + def test_pr_agent_can_handle_changelog_case_insensitive(self): + """Test that changelog command is case-insensitive.""" + from agents.pr_agent import PRAgent + + config = { + "agents": { + "pr": { + "enabled": True, + } + }, + "interaction": { + "mention_prefix": "@codebot", + }, + } + + agent = PRAgent(config=config) + + # Test various casings + for body in [ + "@codebot CHANGELOG", + "@codebot Changelog", + "@codebot ChAnGeLoG", + ]: + event_data = { + "action": "created", + "issue": { + "number": 123, + "pull_request": {}, + }, + "comment": {"body": body}, + } + assert agent.can_handle("issue_comment", event_data) is True + + def test_pr_agent_ignores_changelog_on_non_pr(self): + """Test that changelog command is ignored on regular issues.""" + from agents.pr_agent import PRAgent + + config = { + "agents": { + "pr": { + "enabled": True, + } + }, + "interaction": { + "mention_prefix": "@codebot", + }, + } + + agent = PRAgent(config=config) + + # Regular issue (no pull_request field) + event_data = { + "action": "created", + "issue": { + "number": 123, + # No pull_request field + }, + "comment": { + "body": "@codebot changelog", + }, + } + + assert agent.can_handle("issue_comment", event_data) is False + + def test_format_changelog_structure(self): + """Test _format_changelog generates correct Keep a Changelog format.""" + from agents.pr_agent import PRAgent + + agent = PRAgent(config={}) + + changelog_data = { + "changelog": { + "added": ["User authentication system", "Password reset feature"], + "changed": ["Updated database schema", "Refactored login endpoint"], + "fixed": ["Session timeout bug", "Security vulnerability"], + "security": ["Fixed XSS vulnerability in user input"], + }, + "breaking_changes": ["Removed legacy API endpoint /api/v1/old"], + "technical_details": { + "files_changed": 15, + "insertions": 450, + "deletions": 120, + "main_components": ["auth/", "api/users/", "database/"], + }, + } + + result = agent._format_changelog(changelog_data, 123) + + # Verify structure + assert "## 📋 Changelog for PR #123" in result + assert "### ✨ Added" in result + assert "User authentication system" in result + assert "### 🔄 Changed" in result + assert "Updated database schema" in result + assert "### 🐛 Fixed" in result + assert "Session timeout bug" in result + assert "### 🔒 Security" in result + assert "Fixed XSS vulnerability" in result + assert "### ⚠️ BREAKING CHANGES" in result + assert "Removed legacy API endpoint" in result + assert "### 📊 Technical Details" in result + assert "Files changed:** 15" in result + assert "+450 / -120" in result + assert "auth/" in result + + def test_format_changelog_empty_sections(self): + """Test that empty sections are not included in output.""" + from agents.pr_agent import PRAgent + + agent = PRAgent(config={}) + + changelog_data = { + "changelog": { + "added": ["New feature"], + "changed": [], + "deprecated": [], + "removed": [], + "fixed": [], + "security": [], + }, + "breaking_changes": [], + "technical_details": {}, + } + + result = agent._format_changelog(changelog_data, 123) + + # Only Added section should be present + assert "### ✨ Added" in result + assert "New feature" in result + assert "### 🔄 Changed" not in result + assert "### 🗑️ Removed" not in result + assert "### 🐛 Fixed" not in result + assert "### ⚠️ BREAKING CHANGES" not in result + + def test_config_has_changelog_command(self): + """Verify config.yml has changelog command.""" + config_path = os.path.join( + os.path.dirname(__file__), "..", "tools", "ai-review", "config.yml" + ) + + import yaml + + with open(config_path) as f: + config = yaml.safe_load(f) + + assert "interaction" in config + assert "commands" in config["interaction"] + assert "changelog" in config["interaction"]["commands"] + + if __name__ == "__main__": pytest.main([__file__, "-v"]) diff --git a/tools/ai-review/agents/pr_agent.py b/tools/ai-review/agents/pr_agent.py index 70f00a9..ce00d50 100644 --- a/tools/ai-review/agents/pr_agent.py +++ b/tools/ai-review/agents/pr_agent.py @@ -99,7 +99,8 @@ class PRAgent(BaseAgent): f"{mention_prefix} review-again" in comment_body.lower() ) has_summarize = f"{mention_prefix} summarize" in comment_body.lower() - return is_pr and (has_review_again or has_summarize) + has_changelog = f"{mention_prefix} changelog" in comment_body.lower() + return is_pr and (has_review_again or has_summarize or has_changelog) return False @@ -113,6 +114,8 @@ class PRAgent(BaseAgent): ) if f"{mention_prefix} summarize" in comment_body.lower(): return self._handle_summarize_command(context) + elif f"{mention_prefix} changelog" in comment_body.lower(): + return self._handle_changelog_command(context) elif f"{mention_prefix} review-again" in comment_body.lower(): return self._handle_review_again(context) @@ -1021,3 +1024,190 @@ class PRAgent(BaseAgent): message=f"Failed to generate PR summary for PR #{pr_number}", error="Summary generation failed", ) + + def _handle_changelog_command(self, context: AgentContext) -> AgentResult: + """Handle @codebot changelog command from PR comments. + + Generates Keep a Changelog format entries for the PR. + + Args: + context: Agent context with event data + + Returns: + AgentResult with success status and actions taken + """ + issue = context.event_data.get("issue", {}) + pr_number = issue.get("number") + comment_author = ( + context.event_data.get("comment", {}).get("user", {}).get("login", "user") + ) + + self.logger.info(f"Generating changelog for PR #{pr_number} at user request") + + try: + # Get PR data + pr = self.gitea.get_pull_request(context.owner, context.repo, pr_number) + pr_title = pr.get("title", "") + pr_description = pr.get("body", "") + + # Get PR diff + diff = self._get_diff(context.owner, context.repo, pr_number) + if not diff.strip(): + error_msg = ( + f"@{comment_author}\n\n" + f"{self.AI_DISCLAIMER}\n\n" + "**⚠️ Changelog Generation Failed**\n\n" + "No changes found in this PR to analyze." + ) + self.gitea.create_issue_comment( + context.owner, context.repo, pr_number, error_msg + ) + return AgentResult( + success=False, + message=f"No diff to generate changelog for PR #{pr_number}", + ) + + # Load changelog prompt + prompt_template = self.load_prompt("changelog") + prompt = prompt_template.format( + pr_title=pr_title, + pr_description=pr_description or "(No description provided)", + ) + prompt = f"{prompt}\n{diff}" + + # Call LLM to generate changelog + result = self.call_llm_json(prompt) + + # Format the changelog comment + changelog_comment = self._format_changelog(result, pr_number) + + # Post changelog comment + self.gitea.create_issue_comment( + context.owner, context.repo, pr_number, changelog_comment + ) + + return AgentResult( + success=True, + message=f"Generated changelog for PR #{pr_number}", + actions_taken=["Posted changelog comment"], + ) + + except Exception as e: + self.logger.error(f"Failed to generate changelog: {e}") + + # Post error message + error_msg = ( + f"@{comment_author}\n\n" + f"{self.AI_DISCLAIMER}\n\n" + "**⚠️ Changelog Generation Failed**\n\n" + f"I encountered an error while generating the changelog: {str(e)}\n\n" + "This could be due to:\n" + "- The PR is too large to analyze\n" + "- The LLM service is temporarily unavailable\n" + "- An unexpected error occurred" + ) + self.gitea.create_issue_comment( + context.owner, context.repo, pr_number, error_msg + ) + + return AgentResult( + success=False, + message=f"Failed to generate changelog for PR #{pr_number}", + error=str(e), + ) + + def _format_changelog(self, changelog_data: dict, pr_number: int) -> str: + """Format changelog data into Keep a Changelog format. + + Args: + changelog_data: JSON data from LLM containing changelog entries + pr_number: PR number for reference + + Returns: + Formatted markdown changelog + """ + lines = [ + self.AI_DISCLAIMER, + "", + f"## 📋 Changelog for PR #{pr_number}", + "", + ] + + changelog = changelog_data.get("changelog", {}) + + # Added + added = changelog.get("added", []) + if added: + lines.append("### ✨ Added") + for item in added: + lines.append(f"- {item}") + lines.append("") + + # Changed + changed = changelog.get("changed", []) + if changed: + lines.append("### 🔄 Changed") + for item in changed: + lines.append(f"- {item}") + lines.append("") + + # Deprecated + deprecated = changelog.get("deprecated", []) + if deprecated: + lines.append("### ⚠️ Deprecated") + for item in deprecated: + lines.append(f"- {item}") + lines.append("") + + # Removed + removed = changelog.get("removed", []) + if removed: + lines.append("### 🗑️ Removed") + for item in removed: + lines.append(f"- {item}") + lines.append("") + + # Fixed + fixed = changelog.get("fixed", []) + if fixed: + lines.append("### 🐛 Fixed") + for item in fixed: + lines.append(f"- {item}") + lines.append("") + + # Security + security = changelog.get("security", []) + if security: + lines.append("### 🔒 Security") + for item in security: + lines.append(f"- {item}") + lines.append("") + + # Breaking changes + breaking = changelog_data.get("breaking_changes", []) + if breaking: + lines.append("---") + lines.append("") + lines.append("### ⚠️ BREAKING CHANGES") + for item in breaking: + lines.append(f"- **{item}**") + lines.append("") + + # Technical details + tech = changelog_data.get("technical_details", {}) + if tech: + lines.append("---") + lines.append("") + lines.append("### 📊 Technical Details") + + files = tech.get("files_changed", 0) + additions = tech.get("insertions", 0) + deletions = tech.get("deletions", 0) + lines.append(f"- **Files changed:** {files}") + lines.append(f"- **Lines:** +{additions} / -{deletions}") + + components = tech.get("main_components", []) + if components: + lines.append(f"- **Main components:** {', '.join(components)}") + + return "\n".join(lines) diff --git a/tools/ai-review/config.yml b/tools/ai-review/config.yml index 706e359..6fcfaf5 100644 --- a/tools/ai-review/config.yml +++ b/tools/ai-review/config.yml @@ -67,6 +67,7 @@ interaction: - suggest - security - summarize # Generate PR summary (works on both issues and PRs) + - changelog # Generate Keep a Changelog format entries (PR comments only) - triage - review-again diff --git a/tools/ai-review/prompts/changelog.md b/tools/ai-review/prompts/changelog.md new file mode 100644 index 0000000..3b5b1b2 --- /dev/null +++ b/tools/ai-review/prompts/changelog.md @@ -0,0 +1,91 @@ +You are an experienced software developer creating changelog entries for release notes following the **Keep a Changelog** format (https://keepachangelog.com/). + +Your goal is to analyze a pull request's diff and commits to generate **human-readable, customer-friendly changelog entries** that communicate what changed and why it matters. + +--- + +## Requirements + +Analyze the PR and generate changelog entries categorized by: + +1. **Added** - New features or functionality +2. **Changed** - Changes to existing functionality +3. **Deprecated** - Features that will be removed in future versions +4. **Removed** - Features that have been removed +5. **Fixed** - Bug fixes +6. **Security** - Security vulnerability fixes + +Additional analysis: +- **Breaking Changes** - Changes that break backward compatibility +- **Technical Details** - Files changed, lines of code, main components affected + +--- + +## Output Format + +Return a JSON object with this structure: + +```json +{{{{ + "changelog": {{{{ + "added": ["List of new features or functionality"], + "changed": ["List of changes to existing functionality"], + "deprecated": ["List of deprecated features"], + "removed": ["List of removed features"], + "fixed": ["List of bug fixes"], + "security": ["List of security fixes"] + }}}}, + "breaking_changes": ["List of breaking changes, if any"], + "technical_details": {{{{ + "files_changed": 15, + "insertions": 450, + "deletions": 120, + "main_components": ["List of main components/directories affected"] + }}}} +}}}} +``` + +--- + +## Rules + +1. **Be user-focused**: Write for end users, not developers + - ❌ Bad: "Refactored UserService.authenticate() method" + - ✅ Good: "Improved login performance and reliability" + +2. **Be specific and actionable**: Include what changed and the benefit + - ❌ Bad: "Updated authentication" + - ✅ Good: "Added JWT token authentication for improved security" + +3. **Filter noise**: Ignore formatting changes, typos, minor refactoring unless user-visible + - Skip: "Fixed linting issues", "Updated whitespace", "Renamed internal variable" + - Include: "Fixed crash on invalid input", "Improved error messages" + +4. **Detect breaking changes**: Look for: + - API endpoint changes (removed/renamed endpoints) + - Configuration changes (removed/renamed config keys) + - Dependency version upgrades with breaking changes + - Database schema changes requiring migrations + - Removed features or deprecated functionality + +5. **Group related changes**: Combine similar changes into one entry + - ❌ "Added user login", "Added user logout", "Added password reset" + - ✅ "Added complete user authentication system with login, logout, and password reset" + +6. **Use active voice**: Start with a verb + - ✅ "Added", "Fixed", "Improved", "Updated", "Removed" + +7. **Keep entries concise**: One line per change, maximum 100 characters + +8. **Output only JSON**: No additional text before or after the JSON object + +--- + +## PR Information + +**Title:** {pr_title} + +**Description:** {pr_description} + +**Diff:** +