Intelligent GitHub Issue Refinement & Duplicate Detection System
1. Project Objective
What are we building?
We are building an Agentic AI system that helps a developer before submitting a new issue to a large GitHub repository.
The system will:
- Accept a GitHub issue title and description.
- Detect spelling and grammar problems.
- Correct the language.
- Refine the issue into a professional technical report.
- Search historical GitHub issues.
- Retrieve semantically similar issues using RAG.
- Ask an LLM-based agent to determine whether the issue is actually a duplicate.
- Produce one final recommendation:
DUPLICATE
POSSIBLE_DUPLICATE
READY_TO_PUBLISH
The application does not actually create the GitHub issue. Its responsibility ends with the recommendation
Understand the Agentic Nature
This is not simply an LLM chatbot.
The project contains:
LLM
+
Tools
+
RAG
+
Persistent Knowledge
+
State
+
Conditional Routing
+
Agentic Workflow
Pasted markdown
LLM
Provides language understanding and reasoning.
Embedding Model
Converts text into vectors.
Vector Database
Stores and searches historical issue embeddings.
RAG
Provides relevant historical issues to the LLM.
Tools
Interact with external systems such as GitHub and the vector database.
Agents
Perform specialized responsibilities.
Agentic Workflow
Coordinates:
Agents
Tools
State
Transitions
Conditions
Decisions
Below is a practical reference solution for the uploaded problem statement, using:
- Python
- LangGraph for agentic workflow
- LangChain
- Ollama
- Qwen for LLM reasoning
- nomic-embed-text for embeddings
- Chroma for vector storage
- GitHub REST API for issue collection
The solution follows the original architecture: GitHub ingestion → normalization → embeddings → vector search → language correction → refinement → duplicate analysis → final decision. Pasted markdown
1. Solution Architecture
We will implement the project in two pipelines.
Pipeline 1 – Offline Knowledge Base
GitHub Repository
|
v
GitHub API
|
v
Issue Collector
|
v
issues.json
|
v
Normalize Issues
|
v
Embedding Model
|
v
Chroma Vector DB
Pipeline 2 – Online Agentic Workflow
User Issue
|
v
Issue Intake
|
v
Language Agent
|
v
Correction
|
v
Refinement Agent
|
v
Embedding
|
v
Vector Search
|
v
Top-K Candidates
|
v
Duplicate Analysis Agent
|
v
Decision
|
+------ DUPLICATE
|
+------ POSSIBLE_DUPLICATE
|
+------ READY_TO_PUBLISH
This follows the required architecture where vector similarity retrieves candidates, while the LLM performs the final duplicate reasoning. Pasted markdown
2. Project Structure
Create the following project:
github-issue-agent/
│
├── .env
├── requirements.txt
├── README.md
│
├── data/
│ ├── issues.json
│ └── sync_state.json
│
├── config/
│ └── settings.py
│
├── models/
│ └── schemas.py
│
├── ingestion/
│ ├── github_client.py
│ ├── issue_collector.py
│ └── synchronizer.py
│
├── rag/
│ ├── embeddings.py
│ ├── indexer.py
│ └── retriever.py
│
├── agents/
│ ├── language_agent.py
│ ├── refinement_agent.py
│ └── duplicate_agent.py
│
├── workflow/
│ ├── state.py
│ ├── nodes.py
│ └── graph.py
│
└── app/
└── main.py
This also keeps ingestion, RAG, agents, workflow and models separate as required by the problem statement. Pasted markdown
3. Step 1 – Create Virtual Environment
What?
Create an isolated Python environment.
Why?
The project has several Python dependencies. A virtual environment prevents dependency conflicts with other projects.
How?
Windows:
Linux/macOS:
source .venv/bin/activate
4. Step 2 – Install Packages
Create requirements.txt.
langchain
langchain-core
langchain-community
langchain-ollama
langchain-chroma
langgraph
chromadb
pydantic
python-dotenv
requests
Install:
pip install -r requirements.txt
5. Step 3 – Install Ollama Models
What?
We use Ollama for local LLM and embedding generation.
Why?
The problem statement recommends Ollama with models such as Qwen/Llama and nomic-embed-text. Pasted markdown
Install/pull models:
ollama pull nomic-embed-text
Verify:
You should see something similar to:
NAME
qwen3
nomic-embed-text
6. Step 4 – Configuration
Create:
import osfrom dotenv import load_dotenv# Load values from .envload_dotenv()# GitHub repository to analyzeGITHUB_REPOSITORY = os.getenv( "GITHUB_REPOSITORY", "kubernetes/kubernetes")# Number of issues to retrieve during the demoISSUE_LIMIT = int( os.getenv("ISSUE_LIMIT", "100"))# Local Ollama modelsLLM_MODEL = os.getenv( "LLM_MODEL", "qwen3")EMBEDDING_MODEL = os.getenv( "EMBEDDING_MODEL", "nomic-embed-text")# Number of candidates retrieved from ChromaTOP_K = int( os.getenv("TOP_K", "10"))# Vector similarity thresholdSIMILARITY_THRESHOLD = float( os.getenv("SIMILARITY_THRESHOLD", "0.75")
7. Step 5 – Environment File
Create .env.
GITHUB_REPOSITORY=kubernetes/kubernetes
ISSUE_LIMIT=100
LLM_MODEL=qwen3
EMBEDDING_MODEL=nomic-embed-text
TOP_K=10
SIMILARITY_THRESHOLD=0.75
CHROMA_DIRECTORY=./data/chroma
If you use a GitHub token:
GITHUB_TOKEN=your_token_here
Do not commit .env to Git.
8. Step 6 – Define Issue Schema
What?
Define the structure of an issue.
Why?
GitHub API data should be converted into a predictable application model.
Create:
from pydantic import BaseModelfrom typing import List, Optionalclass GitHubIssue(BaseModel): """ Represents a normalized GitHub issue. """ id: int number: int title: str body: str = "" state: str labels: List[str] = [] created_at: str updated_at: str url: strclass DuplicateResult(BaseModel): """ Structured result returned by the Duplicate Analysis Agent. """ status: str issue_number: Optional[int] = None
Structured output is important because the original problem statement recommends structured JSON/Pydantic-style responses. Pasted markdown
9. Step 7 – GitHub Client
Create:
ingestion/github_client.py
import requestsfrom config.settings import GITHUB_TOKENclass GitHubClient: """ Responsible only for communicating with GitHub. """ BASE_URL = "https://api.github.com" def __init__(self): self.session = requests.Session() # Add authentication if a GitHub token is available. if GITHUB_TOKEN: self.session.headers.update({ "Authorization": f"Bearer {GITHUB_TOKEN}" }) self.session.headers.update({ "Accept": "application/vnd.github+json" }) def get_issues( self, repository: str, limit: int = 100 ): """ Retrieve issues from a public GitHub repository. We request both open and closed issues because historical issues are required for duplicate detection. """
10. Step 8 – Collect GitHub Issues
Create:
ingestion/issue_collector.py
import jsonfrom pathlib import Pathfrom ingestion.github_client import GitHubClientfrom config.settings import ( GITHUB_REPOSITORY, ISSUE_LIMIT)def collect_issues(): client = GitHubClient() print( f"Collecting issues from " f"{GITHUB_REPOSITORY}..." ) raw_issues = client.get_issues( GITHUB_REPOSITORY, ISSUE_LIMIT ) normalized = [] for issue in raw_issues: # GitHub's /issues endpoint can also return pull requests. # We only want actual issues. if "pull_request" in issue: continue normalized.append({ "id": issue["id"], "number": issue["number"],
Run:
python ingestion/issue_collector.py
Expected:
Collecting issues from kubernetes/kubernetes...
Issues saved: 95
The original specification requires the collected dataset to become the source dataset for RAG. Pasted markdown
11. Step 9 – Understand Normalization
A raw GitHub issue may look like:
Title:
Pod fails to restart after node failure
Body:
After the worker node recovers...
We convert it into a searchable document:
Issue ID: 12345
Title:
Pod fails to restart after node failure
Description:
After the worker node recovers...
Labels:
kind/bug, sig/node
Why?
The embedding model needs meaningful text representing the issue.
12. Step 10 – Create Embedding Service
Create:
from langchain_ollama import OllamaEmbeddingsfrom config.settings import EMBEDDING_MODELdef create_embedding_model(): """ Create the embedding model. Ollama runs the model locally, so the application does not need to send issue text to an external embedding service. """ return OllamaEmbeddings( model=EMBEDDING_MODEL )
13. Step 11 – Create RAG Documents
Create:
import jsonfrom langchain_core.documents import Documentfrom langchain_chroma import Chromafrom rag.embeddings import create_embedding_modelfrom config.settings import CHROMA_DIRECTORYdef load_issues(): with open( "data/issues.json", "r", encoding="utf-8" ) as file: return json.load(file)def create_documents(): issues = load_issues() documents = [] for issue in issues: # Combine the important parts of the issue # into one searchable text document. content = f"""Issue ID: {issue['number']}Title:{issue['title']}
14. Step 12 – Build the Vector Database
Run:
Conceptually:
Issue
|
v
Text
|
v
Embedding Model
|
v
Vector
|
v
Chroma
For example:
"Pod fails to restart"
|
v
[0.021, -0.153, 0.481, ...]
The embedding model converts issue content into vectors, while the vector database stores and searches those vectors. Pasted markdown
15. Step 13 – Create the Retriever
Create:
from langchain_chroma import Chromafrom rag.embeddings import create_embedding_modelfrom config.settings import ( CHROMA_DIRECTORY, TOP_K)def get_vector_store(): embeddings = create_embedding_model() return Chroma( persist_directory=CHROMA_DIRECTORY, embedding_function=embeddings, collection_name="github_issues" )def search_similar_issues(query: str): vector_store = get_vector_store() # Retrieve the Top-K semantically similar issues. results = vector_store.similarity_search_with_score( query, k=TOP_K ) return results
16. Important RAG Principle
Suppose we have:
We should not do this:
Instead:
50,000 Issues
|
v
Vector Database
|
v
New Issue Embedding
|
v
Vector Search
|
v
Top 10 Candidates
|
v
LLM
This is one of the most important design principles in the assignment. Pasted markdown
17. Step 14 – Create Agent State
Create:
from typing import TypedDict, List, Dictclass IssueState(TypedDict, total=False): # Original user input original_title: str original_description: str # Corrected version corrected_title: str corrected_description: str # Professionally refined version refined_title: str refined_description: str # Language errors detected spelling_errors: List[str] # Issues retrieved from vector database candidate_issues: List[Dict] # Duplicate analysis duplicate_status: str duplicate_issue_id: int | None similarity_score: float | None # Final application decision final_decision: str # Explanation reason: str
This corresponds directly to the shared IssueState described in the problem statement. Pasted markdown
18. Step 15 – Create the Language Agent
Create:
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef create_llm(): return ChatOllama( model=LLM_MODEL, temperature=0 )def analyze_language( title: str, description: str): llm = create_llm() prompt = f"""You are a technical language correction agent.Analyze the following GitHub issue.Title:{title}Description:{description}Tasks:1. Detect spelling errors.2. Detect grammar problems.3. Correct the language.
Why?
The first agent is responsible only for language quality.
It should not make the duplicate decision.
This separation of responsibilities is what makes the workflow agentic.
19. Step 16 – Create Refinement Agent
Create:
agents/refinement_agent.py
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef refine_issue( title: str, description: str): llm = ChatOllama( model=LLM_MODEL, temperature=0 ) prompt = f"""You are a GitHub technical issue refinement agent.Rewrite the following issue as a professionaltechnical issue report.Title:{title}Description:{description}Create:1. Clear technical title2. Description3. Expected Behavior4. Actual BehaviorRules:
The problem statement specifically requires that refinement must not invent technical facts. Pasted markdown
20. Step 17 – Create Duplicate Analysis Agent
This is the most important reasoning agent.
Create:
agents/duplicate_agent.py
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef analyze_duplicates( refined_issue: str, candidates: list): llm = ChatOllama( model=LLM_MODEL, temperature=0 ) candidate_text = "" for candidate, score in candidates: candidate_text += f"""-----------------------------------Issue Number:{candidate.metadata.get("issue_number")}Similarity Score:{score}Issue:{candidate.page_content}URL:{candidate.metadata.get("url")}-----------------------------------""" prompt = f"""
The important distinction is:
Embedding Model
|
v
Candidate Retrieval
LLM
|
v
Semantic Duplicate Reasoning
The source explicitly states that vector similarity alone should not determine duplicate status. Pasted markdown
21. Step 18 – Create LangGraph Nodes
Create:
from workflow.state import IssueStatefrom agents.language_agent import analyze_languagefrom agents.refinement_agent import refine_issuefrom rag.retriever import search_similar_issuesdef issue_input_node( state: IssueState): # Preserve the original user input. print("\n[Issue Intake]") return statedef language_check_node( state: IssueState): print("\n[Language Agent]") result = analyze_language( state["original_title"], state["original_description"] ) # For a production implementation, parse the # structured LLM response instead of storing raw text. state["corrected_title"] = state["original_title"] state["corrected_description"] = result return state
22. Step 19 – Create Duplicate Node
Add to workflow/nodes.py:
from agents.duplicate_agent import analyze_duplicatesdef duplicate_analysis_node( state: IssueState): print("\n[Duplicate Analysis Agent]") # Convert our simplified candidate list back into # information that can be supplied to the LLM. candidates = [] for item in state["candidate_issues"]: candidates.append( ( item["title"], item["score"] ) ) refined_issue = ( state["refined_title"] + "\n" + state["refined_description"] ) result = analyze_duplicates( refined_issue, candidates ) # For demonstration, retain the LLM response. state["reason"] = result
23. Step 20 – Create Decision Node
Add:
def decision_node( state: IssueState): print("\n[Decision Node]") status = state.get( "duplicate_status", "NOT_DUPLICATE" ) if status == "DUPLICATE": state["final_decision"] = "DUPLICATE" elif status == "POSSIBLE_DUPLICATE": state["final_decision"] = "POSSIBLE_DUPLICATE" else: state["final_decision"] = "READY_TO_PUBLISH" return state
The required final application decisions are:
DUPLICATE
POSSIBLE_DUPLICATE
READY_TO_PUBLISH
Pasted markdown
24. Step 21 – Build the LangGraph
Create:
from langgraph.graph import StateGraph, START, ENDfrom workflow.state import IssueStatefrom workflow.nodes import ( issue_input_node, language_check_node, refinement_node, rag_search_node, duplicate_analysis_node, decision_node)def build_graph(): # Create graph using our shared state. graph = StateGraph(IssueState) # Register nodes. graph.add_node( "issue_input", issue_input_node ) graph.add_node( "language_check", language_check_node ) graph.add_node( "refinement", refinement_node ) graph.add_node(
25. Step 22 – Run the Application
Create:
from workflow.graph import build_graphdef main(): print("=" * 60) print("GitHub Intelligent Issue Refinement Agent") print("=" * 60) # Get user input. title = input( "\nEnter issue title: " ) description = input( "Enter issue description: " ) # Create LangGraph application. app = build_graph() # Initial workflow state. initial_state = { "original_title": title, "original_description": description, "corrected_title": "", "corrected_description": "", "refined_title": "", "refined_description": "",
Run:
26. Example Input
Enter:
Title:
pod is not restrting after crash
Description:
when pod crash sometime its not coming back
i tried restart but same problem happening
The workflow should conceptually execute:
[Issue Intake]
↓
[Language Agent]
↓
[Correction]
↓
[Refinement Agent]
↓
[RAG Search]
↓
[Top 10 Candidates]
↓
[Duplicate Analysis Agent]
↓
[Decision]
27. Expected Refined Result
The LLM should produce something conceptually similar to:
Title:
Pod does not automatically restart after an unexpected crash
Description:
In certain situations, a pod that terminates unexpectedly
does not restart automatically.
The pod remains unavailable until it is manually restarted.
Expected Behavior:
The pod should automatically recover according to the
configured restart policy.
Actual Behavior:
The pod occasionally remains unavailable after the crash
and requires manual intervention.
This follows the refinement example from the problem statement. Pasted markdown
28. Example RAG Result
The vector database may return:
Top 10 Candidates
#9321 0.91
#15102 0.87
#20452 0.79
#41002 0.65
...
This means:
0.91 → very semantically similar
0.87 → highly similar
0.79 → reasonably similar
0.65 → less similar
But remember:
The LLM must examine the actual issue content.
29. Final Duplicate Analysis
The Duplicate Analysis Agent should compare:
New Issue
|
+-- Problem
+-- Component
+-- Symptoms
+-- Trigger
+-- Expected Behavior
+-- Actual Behavior
+-- Technical Context
against:
Then produce:
or:
or:
The source specifically recommends these classifications for the analysis stage. Pasted markdown
30. Final Decision Logic
The application converts the analysis into:
Duplicate Analysis
|
+------------+------------+
| | |
v v v
DUPLICATE POSSIBLE NOT DUPLICATE
DUPLICATE |
| | |
v v v
DUPLICATE POSSIBLE READY TO
DUPLICATE PUBLISH
31. Important Improvement – Use Structured Output
The above code demonstrates the architecture, but for a proper production-quality trainee solution, do not depend on parsing free-form LLM text.
Instead, define:
class DuplicateResult(BaseModel): status: str issue_number: int | None confidence: float reason: str
Then ask the LLM to return structured output.
Conceptually:
{
"status": "DUPLICATE",
"issue_number": 9321,
"confidence": 0.91,
"reason": "Both issues describe pods remaining unavailable after unexpected termination."
}
This directly follows the structured-output requirement in the problem statement. Pasted markdown
32. Important Improvement – Incremental Synchronization
The first version above demonstrates ingestion, but the complete project also requires incremental synchronization.
Create:
Conceptually:
{
"last_sync_time": "2026-10-06T08:00:00Z",
"issues": {
"9321": {
"updated_at": "2026-10-05T10:20:00Z",
"content_hash": "abc123"
}
}
}
During the next synchronization:
GitHub Issue
|
v
Existing?
/ \
No Yes
| |
Insert updated_at/hash changed?
/ \
No Yes
| |
Ignore Re-index
This avoids unnecessary embedding generation and duplicate vector insertion. Pasted markdown
33. Complete End-to-End Flow
After all components are implemented, the complete system becomes:
GITHUB
|
v
GitHub API / CLI
|
v
Issue Collector
|
v
issues.json
|
v
Normalize
|
v
Embeddings
|
v
CHROMA DB
|
|
|
USER ----------------+
|
v
Issue Intake
|
v
Language Agent
|
+---- Errors ----> Correction
|
v
Refinement Agent
|
v
Query Embedding
|
v
Vector Search
|
v
Top 10 Candidates
|
v
Duplicate Analysis Agent
|
v
Decision Node
|
+--------------------+--------------------+
| | |
v v v
DUPLICATE POSSIBLE_DUPLICATE READY_TO_PUBLISH
This represents the offline and online data flow defined in the source problem statement. Pasted markdown
34. What, Why and How – Project Summary
| Component | What? | Why? | How? |
|---|
| GitHub API | Gets historical issues | Need knowledge base | REST API |
issues.json | Stores normalized issues | Local source dataset | JSON |
| Synchronizer | Finds changes | Avoid reprocessing | ID + timestamp + hash |
| Embedding Model | Text → vector | Semantic search | Ollama |
| Chroma | Stores vectors | Fast similarity retrieval | Vector DB |
| Language Agent | Checks language | Improve input quality | Qwen |
| Refinement Agent | Creates technical report | Standardize issue | Qwen |
| Retriever | Finds candidates | Reduce search space | Chroma |
| Duplicate Agent | Reasons about duplicates | Similarity alone isn't enough | Qwen |
| LangGraph | Controls workflow | State + routing + agents | Graph |
| Decision Node | Produces final recommendation | Business outcome | Conditional logic |
35. Most Important Viva Explanation
If asked:
"Why is this an Agentic AI project and not just an RAG chatbot?"
A good answer is:
This system combines an LLM, tools, RAG, persistent knowledge, shared state, multiple specialized agents and conditional workflow routing. The Language Agent handles language correction, the Refinement Agent creates a technical report, the RAG tool retrieves historical GitHub issues, and the Duplicate Analysis Agent reasons over those candidates before the Decision Node produces the final recommendation.
The source explicitly describes the project as demonstrating LLM + Tools + RAG + Persistent Knowledge + State + Conditional Routing + Agentic Workflow. Pasted markdown
36. One Critical Correction to Keep in the Final Implementation
The similarity score should not directly produce:
Instead:
New Issue
|
v
Embedding
|
v
Vector Search
|
v
Top 5–10 Candidates
|
v
Duplicate Analysis Agent
|
+---------+---------+
| | |
v v v
DUPLICATE POSSIBLE NOT DUPLICATE
|
v
Final Decision
That distinction is the core RAG + Agentic AI design principle of this assignment. Pasted markdown
Recommended implementation sequence for trainees
Do not try to build the whole project at once. Implement and demonstrate it in these stages:
Stage 1 → GitHub API + issues.json
Stage 2 → Incremental synchronization
Stage 3 → Embedding generation
Stage 4 → Chroma vector database
Stage 5 → Semantic search
Stage 6 → Language Agent
Stage 7 → Refinement Agent
Stage 8 → Duplicate Analysis Agent
Stage 9 → LangGraph state/workflow
Stage 10 → Final DUPLICATE / POSSIBLE_DUPLICATE / READY_TO_PUBLISH
Stage 11 → Automated tests
Stage 12 → Optional FastAPI/Streamlit UI
This staged approach makes it much easier for trainees to understand what each component does, why it is required, and how it contributes to the final Agentic AI system.
