PR Intent Checker GitHub Action (MCP Server Packaged)

This GitHub Action uses an AI model (via Azure OpenAI Service) to analyze the code changes in a Pull Request (PR) and compare them against the requirements specified in a linked GitHub Issue. It helps identify potential "intent drift" early in the development cycle.

Key Difference: This version packages the GitHub MCP Server inside the action's Docker container, allowing it to interact with the GitHub API via the Model Context Protocol without relying on external libraries like PyGithub or externally running MCP servers.

How it Works (MCP Packaged Version)

1. Trigger: The action runs automatically when a Pull Request is opened or updated in a repository where it's configured.
2. Container Start: The action runs inside a Docker container. The entrypoint.sh script within the container first starts the GitHub MCP server (npx @modelcontextprotocol/server-github) in the background, passing the necessary GITHUB_TOKEN.
3. Python Script Execution: The entrypoint.sh script then executes the main Python script (src/main.py).
4. Link Issue: The Python script reads the event payload to find the PR number.
5. Fetch Data (via MCP): The Python script (src/github_api.py) communicates with the background MCP server via stdin/stdout using JSON-RPC messages. It sends requests to the MCP server to:
Fetch the PR code diff.
Find the linked issue number (checking timeline events first, then PR body).
Fetch the body of the linked issue.
6. AI Analysis: It sends the issue requirements and the code diff to your configured Azure OpenAI model (e.g., GPT-4o) using a predefined prompt (prompts/intent_check.prompty).
7. Evaluation: The AI model evaluates whether the code changes satisfy the requirements.
8. Report Result: The action parses the AI's response to determine a PASS or FAIL result.
9. Status Check & Comment (via MCP): It sets the action outputs (result, explanation). It then uses the MCP server again (via stdin/stdout) to post the AI's explanation as a comment on the PR. It exits with a success (0) or failure (1) code to set the status check.
10. Cleanup: When the Python script finishes, the entrypoint.sh script ensures the background MCP server process is terminated.

    name: PR Intent Check
on:

      pull_request:

        types: [opened, synchronize, reopened]
permissions:

      contents: read

      pull-requests: write

      issues: read
jobs:

      intent-check:

        runs-on: ubuntu-latest

        steps:

          - name: Checkout code

            uses: actions/checkout@v4

            with:

              fetch-depth: 0 # Required to get diff
- name: Run PR Intent Checker

            # Replace with the path to this action repository

            uses: your-username/mcp-server-pkg-pr-intent-checker@main

            id: intent_checker

            with:

              github_token: ${{ secrets.GITHUB_TOKEN }}

              # Pass Azure credentials from secrets

              azure_openai_endpoint: ${{ secrets.AZURE_OPENAI_ENDPOINT }}

              azure_openai_key: ${{ secrets.AZURE_OPENAI_KEY }}

              azure_openai_deployment: ${{ secrets.AZURE_OPENAI_DEPLOYMENT }}

# Optional: Explicitly fail job if checker fails

          - name: Check result from intent checker

            if: steps.intent_checker.outputs.result == 'FAIL'

            run: |

              echo "Intent Check Failed based on LLM evaluation."

              exit 1

    

2. Configure Secrets: In your repository's Settings -> Secrets and variables -> Actions, add the following secrets:
AZURE_OPENAI_ENDPOINT: Your Azure OpenAI resource endpoint URL.
AZURE_OPENAI_KEY: Your Azure OpenAI API key.
AZURE_OPENAI_DEPLOYMENT: The deployment name of your model (e.g., gpt-4o-wukev).
(Note: Also add these secrets to the pr-intent-checker action repository itself if you haven't already).

Closes #<number>
Closes: #<number>
Closed #<number>
Fixes #<number>
Fixes: #<number>
Fixed #<number>
Resolves #<number>
Resolves: #<number>
Resolved #<number>

(Case is ignored, e.g., closes #123 works too).

Example PR Description:

    This PR implements the factorial function.
Closes #4

    

Inputs

github_token: (Required) The GitHub token. Usually ${{ secrets.GITHUB_TOKEN }}.
azure_openai_endpoint: (Required) Your Azure OpenAI endpoint URL.
azure_openai_key: (Required) Your Azure OpenAI API key.
azure_openai_deployment: (Required) Your Azure OpenAI model deployment name.

Outputs

result: The result of the evaluation (PASS or FAIL).
explanation: The explanation provided by the AI model. If the result is FAIL, this explanation may include specific code snippets (formatted using Markdown diff syntax) highlighting the areas of concern identified by the AI.

Contributing

Feel free to submit issues or pull requests to the your-username/mcp-server-pkg-pr-intent-checker repository (replace with the actual repository path).