Saturday, October 17, 2026

AGENTIC AI PROJECT

Intelligent GitHub Issue Refinement & Duplicate Detection System

1. Project Objective

What are we building?

We are building an Agentic AI system that helps a developer before submitting a new issue to a large GitHub repository.

The system will:

  1. Accept a GitHub issue title and description.
  2. Detect spelling and grammar problems.
  3. Correct the language.
  4. Refine the issue into a professional technical report.
  5. Search historical GitHub issues.
  6. Retrieve semantically similar issues using RAG.
  7. Ask an LLM-based agent to determine whether the issue is actually a duplicate.
  8. Produce one final recommendation:
DUPLICATE
POSSIBLE_DUPLICATE
READY_TO_PUBLISH

The application does not actually create the GitHub issue. Its responsibility ends with the recommendation

Understand the Agentic Nature

This is not simply an LLM chatbot.

The project contains:

LLM
 +
Tools
 +
RAG
 +
Persistent Knowledge
 +
State
 +
Conditional Routing
 +
Agentic Workflow

Pasted markdown

LLM

Provides language understanding and reasoning.

Embedding Model

Converts text into vectors.

Vector Database

Stores and searches historical issue embeddings.

RAG

Provides relevant historical issues to the LLM.

Tools

Interact with external systems such as GitHub and the vector database.

Agents

Perform specialized responsibilities.

Agentic Workflow

Coordinates:

Agents
Tools
State
Transitions
Conditions 

Decisions 


Below is a practical reference solution for the uploaded problem statement, using:

  • Python
  • LangGraph for agentic workflow
  • LangChain
  • Ollama
  • Qwen for LLM reasoning
  • nomic-embed-text for embeddings
  • Chroma for vector storage
  • GitHub REST API for issue collection

The solution follows the original architecture: GitHub ingestion → normalization → embeddings → vector search → language correction → refinement → duplicate analysis → final decision. Pasted markdown


1. Solution Architecture

We will implement the project in two pipelines.

Pipeline 1 – Offline Knowledge Base

GitHub Repository
       |
       v
GitHub API
       |
       v
Issue Collector
       |
       v
issues.json
       |
       v
Normalize Issues
       |
       v
Embedding Model
       |
       v
Chroma Vector DB

Pipeline 2 – Online Agentic Workflow

User Issue
    |
    v
Issue Intake
    |
    v
Language Agent
    |
    v
Correction
    |
    v
Refinement Agent
    |
    v
Embedding
    |
    v
Vector Search
    |
    v
Top-K Candidates
    |
    v
Duplicate Analysis Agent
    |
    v
Decision
    |
    +------ DUPLICATE
    |
    +------ POSSIBLE_DUPLICATE
    |
    +------ READY_TO_PUBLISH

This follows the required architecture where vector similarity retrieves candidates, while the LLM performs the final duplicate reasoning. Pasted markdown


2. Project Structure

Create the following project:

github-issue-agent/
│
├── .env
├── requirements.txt
├── README.md
│
├── data/
│   ├── issues.json
│   └── sync_state.json
│
├── config/
│   └── settings.py
│
├── models/
│   └── schemas.py
│
├── ingestion/
│   ├── github_client.py
│   ├── issue_collector.py
│   └── synchronizer.py
│
├── rag/
│   ├── embeddings.py
│   ├── indexer.py
│   └── retriever.py
│
├── agents/
│   ├── language_agent.py
│   ├── refinement_agent.py
│   └── duplicate_agent.py
│
├── workflow/
│   ├── state.py
│   ├── nodes.py
│   └── graph.py
│
└── app/
    └── main.py

This also keeps ingestion, RAG, agents, workflow and models separate as required by the problem statement. Pasted markdown


3. Step 1 – Create Virtual Environment

What?

Create an isolated Python environment.

Why?

The project has several Python dependencies. A virtual environment prevents dependency conflicts with other projects.

How?

python -m venv .venv

Windows:

.venv\Scripts\activate

Linux/macOS:

source .venv/bin/activate

4. Step 2 – Install Packages

Create requirements.txt.

langchain
langchain-core
langchain-community
langchain-ollama
langchain-chroma
langgraph
chromadb
pydantic
python-dotenv
requests

Install:

pip install -r requirements.txt

5. Step 3 – Install Ollama Models

What?

We use Ollama for local LLM and embedding generation.

Why?

The problem statement recommends Ollama with models such as Qwen/Llama and nomic-embed-text. Pasted markdown

Install/pull models:

ollama pull qwen3
ollama pull nomic-embed-text

Verify:

ollama list

You should see something similar to:

NAME
qwen3
nomic-embed-text

6. Step 4 – Configuration

Create:

config/settings.py
import osfrom dotenv import load_dotenv# Load values from .envload_dotenv()# GitHub repository to analyzeGITHUB_REPOSITORY = os.getenv(    "GITHUB_REPOSITORY",    "kubernetes/kubernetes")# Number of issues to retrieve during the demoISSUE_LIMIT = int(    os.getenv("ISSUE_LIMIT", "100"))# Local Ollama modelsLLM_MODEL = os.getenv(    "LLM_MODEL",    "qwen3")EMBEDDING_MODEL = os.getenv(    "EMBEDDING_MODEL",    "nomic-embed-text")# Number of candidates retrieved from ChromaTOP_K = int(    os.getenv("TOP_K", "10"))# Vector similarity thresholdSIMILARITY_THRESHOLD = float(    os.getenv("SIMILARITY_THRESHOLD", "0.75")

7. Step 5 – Environment File

Create .env.

GITHUB_REPOSITORY=kubernetes/kubernetes

ISSUE_LIMIT=100

LLM_MODEL=qwen3

EMBEDDING_MODEL=nomic-embed-text

TOP_K=10

SIMILARITY_THRESHOLD=0.75

CHROMA_DIRECTORY=./data/chroma

If you use a GitHub token:

GITHUB_TOKEN=your_token_here

Do not commit .env to Git.


8. Step 6 – Define Issue Schema

What?

Define the structure of an issue.

Why?

GitHub API data should be converted into a predictable application model.

Create:

models/schemas.py
from pydantic import BaseModelfrom typing import List, Optionalclass GitHubIssue(BaseModel):    """    Represents a normalized GitHub issue.    """    id: int    number: int    title: str    body: str = ""    state: str    labels: List[str] = []    created_at: str    updated_at: str    url: strclass DuplicateResult(BaseModel):    """    Structured result returned by the Duplicate Analysis Agent.    """    status: str    issue_number: Optional[int] = None

Structured output is important because the original problem statement recommends structured JSON/Pydantic-style responses. Pasted markdown


9. Step 7 – GitHub Client

Create:

ingestion/github_client.py
import requestsfrom config.settings import GITHUB_TOKENclass GitHubClient:    """    Responsible only for communicating with GitHub.    """    BASE_URL = "https://api.github.com"    def __init__(self):        self.session = requests.Session()        # Add authentication if a GitHub token is available.        if GITHUB_TOKEN:            self.session.headers.update({                "Authorization": f"Bearer {GITHUB_TOKEN}"            })        self.session.headers.update({            "Accept": "application/vnd.github+json"        })    def get_issues(        self,        repository: str,        limit: int = 100    ):        """        Retrieve issues from a public GitHub repository.        We request both open and closed issues because        historical issues are required for duplicate detection.        """

10. Step 8 – Collect GitHub Issues

Create:

ingestion/issue_collector.py
import jsonfrom pathlib import Pathfrom ingestion.github_client import GitHubClientfrom config.settings import (    GITHUB_REPOSITORY,    ISSUE_LIMIT)def collect_issues():    client = GitHubClient()    print(        f"Collecting issues from "        f"{GITHUB_REPOSITORY}..."    )    raw_issues = client.get_issues(        GITHUB_REPOSITORY,        ISSUE_LIMIT    )    normalized = []    for issue in raw_issues:        # GitHub's /issues endpoint can also return pull requests.        # We only want actual issues.        if "pull_request" in issue:            continue        normalized.append({            "id": issue["id"],            "number": issue["number"],

Run:

python ingestion/issue_collector.py

Expected:

Collecting issues from kubernetes/kubernetes...
Issues saved: 95

The original specification requires the collected dataset to become the source dataset for RAG. Pasted markdown


11. Step 9 – Understand Normalization

A raw GitHub issue may look like:

Title:
Pod fails to restart after node failure

Body:
After the worker node recovers...

We convert it into a searchable document:

Issue ID: 12345

Title:
Pod fails to restart after node failure

Description:
After the worker node recovers...

Labels:
kind/bug, sig/node

Why?

The embedding model needs meaningful text representing the issue.


12. Step 10 – Create Embedding Service

Create:

rag/embeddings.py
from langchain_ollama import OllamaEmbeddingsfrom config.settings import EMBEDDING_MODELdef create_embedding_model():    """    Create the embedding model.    Ollama runs the model locally, so the application    does not need to send issue text to an external    embedding service.    """    return OllamaEmbeddings(        model=EMBEDDING_MODEL    )

13. Step 11 – Create RAG Documents

Create:

rag/indexer.py
import jsonfrom langchain_core.documents import Documentfrom langchain_chroma import Chromafrom rag.embeddings import create_embedding_modelfrom config.settings import CHROMA_DIRECTORYdef load_issues():    with open(        "data/issues.json",        "r",        encoding="utf-8"    ) as file:        return json.load(file)def create_documents():    issues = load_issues()    documents = []    for issue in issues:        # Combine the important parts of the issue        # into one searchable text document.        content = f"""Issue ID: {issue['number']}Title:{issue['title']}

14. Step 12 – Build the Vector Database

Run:

python rag/indexer.py

Conceptually:

Issue
  |
  v
Text
  |
  v
Embedding Model
  |
  v
Vector
  |
  v
Chroma

For example:

"Pod fails to restart"
          |
          v
[0.021, -0.153, 0.481, ...]

The embedding model converts issue content into vectors, while the vector database stores and searches those vectors. Pasted markdown


15. Step 13 – Create the Retriever

Create:

rag/retriever.py
from langchain_chroma import Chromafrom rag.embeddings import create_embedding_modelfrom config.settings import (    CHROMA_DIRECTORY,    TOP_K)def get_vector_store():    embeddings = create_embedding_model()    return Chroma(        persist_directory=CHROMA_DIRECTORY,        embedding_function=embeddings,        collection_name="github_issues"    )def search_similar_issues(query: str):    vector_store = get_vector_store()    # Retrieve the Top-K semantically similar issues.    results = vector_store.similarity_search_with_score(        query,        k=TOP_K    )    return results

16. Important RAG Principle

Suppose we have:

50,000 historical issues

We should not do this:

50,000 Issues
      |
      v
     LLM

Instead:

50,000 Issues
      |
      v
Vector Database
      |
      v
New Issue Embedding
      |
      v
Vector Search
      |
      v
Top 10 Candidates
      |
      v
LLM

This is one of the most important design principles in the assignment. Pasted markdown


17. Step 14 – Create Agent State

Create:

workflow/state.py
from typing import TypedDict, List, Dictclass IssueState(TypedDict, total=False):    # Original user input    original_title: str    original_description: str    # Corrected version    corrected_title: str    corrected_description: str    # Professionally refined version    refined_title: str    refined_description: str    # Language errors detected    spelling_errors: List[str]    # Issues retrieved from vector database    candidate_issues: List[Dict]    # Duplicate analysis    duplicate_status: str    duplicate_issue_id: int | None    similarity_score: float | None    # Final application decision    final_decision: str    # Explanation    reason: str

This corresponds directly to the shared IssueState described in the problem statement. Pasted markdown


18. Step 15 – Create the Language Agent

Create:

agents/language_agent.py
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef create_llm():    return ChatOllama(        model=LLM_MODEL,        temperature=0    )def analyze_language(    title: str,    description: str):    llm = create_llm()    prompt = f"""You are a technical language correction agent.Analyze the following GitHub issue.Title:{title}Description:{description}Tasks:1. Detect spelling errors.2. Detect grammar problems.3. Correct the language.

Why?

The first agent is responsible only for language quality.

It should not make the duplicate decision.

This separation of responsibilities is what makes the workflow agentic.


19. Step 16 – Create Refinement Agent

Create:

agents/refinement_agent.py
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef refine_issue(    title: str,    description: str):    llm = ChatOllama(        model=LLM_MODEL,        temperature=0    )    prompt = f"""You are a GitHub technical issue refinement agent.Rewrite the following issue as a professionaltechnical issue report.Title:{title}Description:{description}Create:1. Clear technical title2. Description3. Expected Behavior4. Actual BehaviorRules:

The problem statement specifically requires that refinement must not invent technical facts. Pasted markdown


20. Step 17 – Create Duplicate Analysis Agent

This is the most important reasoning agent.

Create:

agents/duplicate_agent.py
from langchain_ollama import ChatOllamafrom config.settings import LLM_MODELdef analyze_duplicates(    refined_issue: str,    candidates: list):    llm = ChatOllama(        model=LLM_MODEL,        temperature=0    )    candidate_text = ""    for candidate, score in candidates:        candidate_text += f"""-----------------------------------Issue Number:{candidate.metadata.get("issue_number")}Similarity Score:{score}Issue:{candidate.page_content}URL:{candidate.metadata.get("url")}-----------------------------------"""    prompt = f"""

The important distinction is:

Embedding Model
      |
      v
Candidate Retrieval

LLM
      |
      v
Semantic Duplicate Reasoning

The source explicitly states that vector similarity alone should not determine duplicate status. Pasted markdown


21. Step 18 – Create LangGraph Nodes

Create:

workflow/nodes.py
from workflow.state import IssueStatefrom agents.language_agent import analyze_languagefrom agents.refinement_agent import refine_issuefrom rag.retriever import search_similar_issuesdef issue_input_node(    state: IssueState):    # Preserve the original user input.    print("\n[Issue Intake]")    return statedef language_check_node(    state: IssueState):    print("\n[Language Agent]")    result = analyze_language(        state["original_title"],        state["original_description"]    )    # For a production implementation, parse the    # structured LLM response instead of storing raw text.    state["corrected_title"] = state["original_title"]    state["corrected_description"] = result    return state

22. Step 19 – Create Duplicate Node

Add to workflow/nodes.py:

from agents.duplicate_agent import analyze_duplicatesdef duplicate_analysis_node(    state: IssueState):    print("\n[Duplicate Analysis Agent]")    # Convert our simplified candidate list back into    # information that can be supplied to the LLM.    candidates = []    for item in state["candidate_issues"]:        candidates.append(            (                item["title"],                item["score"]            )        )    refined_issue = (        state["refined_title"]        + "\n"        + state["refined_description"]    )    result = analyze_duplicates(        refined_issue,        candidates    )    # For demonstration, retain the LLM response.    state["reason"] = result

23. Step 20 – Create Decision Node

Add:

def decision_node(    state: IssueState):    print("\n[Decision Node]")    status = state.get(        "duplicate_status",        "NOT_DUPLICATE"    )    if status == "DUPLICATE":        state["final_decision"] = "DUPLICATE"    elif status == "POSSIBLE_DUPLICATE":        state["final_decision"] = "POSSIBLE_DUPLICATE"    else:        state["final_decision"] = "READY_TO_PUBLISH"    return state

The required final application decisions are:

DUPLICATE
POSSIBLE_DUPLICATE
READY_TO_PUBLISH

Pasted markdown


24. Step 21 – Build the LangGraph

Create:

workflow/graph.py
from langgraph.graph import StateGraph, START, ENDfrom workflow.state import IssueStatefrom workflow.nodes import (    issue_input_node,    language_check_node,    refinement_node,    rag_search_node,    duplicate_analysis_node,    decision_node)def build_graph():    # Create graph using our shared state.    graph = StateGraph(IssueState)    # Register nodes.    graph.add_node(        "issue_input",        issue_input_node    )    graph.add_node(        "language_check",        language_check_node    )    graph.add_node(        "refinement",        refinement_node    )    graph.add_node(

25. Step 22 – Run the Application

Create:

app/main.py
from workflow.graph import build_graphdef main():    print("=" * 60)    print("GitHub Intelligent Issue Refinement Agent")    print("=" * 60)    # Get user input.    title = input(        "\nEnter issue title: "    )    description = input(        "Enter issue description: "    )    # Create LangGraph application.    app = build_graph()    # Initial workflow state.    initial_state = {        "original_title": title,        "original_description": description,        "corrected_title": "",        "corrected_description": "",        "refined_title": "",        "refined_description": "",

Run:

python app/main.py

26. Example Input

Enter:

Title:
pod is not restrting after crash

Description:

when pod crash sometime its not coming back
i tried restart but same problem happening

The workflow should conceptually execute:

[Issue Intake]
      ↓
[Language Agent]
      ↓
[Correction]
      ↓
[Refinement Agent]
      ↓
[RAG Search]
      ↓
[Top 10 Candidates]
      ↓
[Duplicate Analysis Agent]
      ↓
[Decision]

27. Expected Refined Result

The LLM should produce something conceptually similar to:

Title:
Pod does not automatically restart after an unexpected crash

Description:

In certain situations, a pod that terminates unexpectedly
does not restart automatically.

The pod remains unavailable until it is manually restarted.

Expected Behavior:
The pod should automatically recover according to the
configured restart policy.

Actual Behavior:
The pod occasionally remains unavailable after the crash
and requires manual intervention.

This follows the refinement example from the problem statement. Pasted markdown


28. Example RAG Result

The vector database may return:

Top 10 Candidates

#9321     0.91
#15102    0.87
#20452    0.79
#41002    0.65
...

This means:

0.91 → very semantically similar
0.87 → highly similar
0.79 → reasonably similar
0.65 → less similar

But remember:

Similarity ≠ Duplicate

The LLM must examine the actual issue content.


29. Final Duplicate Analysis

The Duplicate Analysis Agent should compare:

New Issue
    |
    +-- Problem
    +-- Component
    +-- Symptoms
    +-- Trigger
    +-- Expected Behavior
    +-- Actual Behavior
    +-- Technical Context

against:

Candidate Issue #9321

Then produce:

DUPLICATE

or:

POSSIBLE_DUPLICATE

or:

NOT_DUPLICATE

The source specifically recommends these classifications for the analysis stage. Pasted markdown


30. Final Decision Logic

The application converts the analysis into:

                   Duplicate Analysis
                          |
             +------------+------------+
             |            |            |
             v            v            v
         DUPLICATE   POSSIBLE       NOT DUPLICATE
                     DUPLICATE           |
             |            |              |
             v            v              v
         DUPLICATE   POSSIBLE       READY TO
                     DUPLICATE       PUBLISH

31. Important Improvement – Use Structured Output

The above code demonstrates the architecture, but for a proper production-quality trainee solution, do not depend on parsing free-form LLM text.

Instead, define:

class DuplicateResult(BaseModel):    status: str    issue_number: int | None    confidence: float    reason: str

Then ask the LLM to return structured output.

Conceptually:

{
    "status": "DUPLICATE",
    "issue_number": 9321,
    "confidence": 0.91,
    "reason": "Both issues describe pods remaining unavailable after unexpected termination."
}

This directly follows the structured-output requirement in the problem statement. Pasted markdown


32. Important Improvement – Incremental Synchronization

The first version above demonstrates ingestion, but the complete project also requires incremental synchronization.

Create:

data/sync_state.json

Conceptually:

{
    "last_sync_time": "2026-10-06T08:00:00Z",

    "issues": {
        "9321": {
            "updated_at": "2026-10-05T10:20:00Z",
            "content_hash": "abc123"
        }
    }
}

During the next synchronization:

GitHub Issue
     |
     v
Existing?
   /   \
 No     Yes
 |       |
Insert   updated_at/hash changed?
         /        \
       No          Yes
       |            |
     Ignore       Re-index

This avoids unnecessary embedding generation and duplicate vector insertion. Pasted markdown


33. Complete End-to-End Flow

After all components are implemented, the complete system becomes:

                    GITHUB
                      |
                      v
              GitHub API / CLI
                      |
                      v
               Issue Collector
                      |
                      v
                 issues.json
                      |
                      v
                 Normalize
                      |
                      v
                Embeddings
                      |
                      v
                CHROMA DB
                      |
                      |
                      |
USER ----------------+
 |
 v
Issue Intake
 |
 v
Language Agent
 |
 +---- Errors ----> Correction
 |
 v
Refinement Agent
 |
 v
Query Embedding
 |
 v
Vector Search
 |
 v
Top 10 Candidates
 |
 v
Duplicate Analysis Agent
 |
 v
Decision Node
 |
 +--------------------+--------------------+
 |                    |                    |
 v                    v                    v
DUPLICATE       POSSIBLE_DUPLICATE    READY_TO_PUBLISH

This represents the offline and online data flow defined in the source problem statement. Pasted markdown


34. What, Why and How – Project Summary

ComponentWhat?Why?How?
GitHub APIGets historical issuesNeed knowledge baseREST API
issues.jsonStores normalized issuesLocal source datasetJSON
SynchronizerFinds changesAvoid reprocessingID + timestamp + hash
Embedding ModelText → vectorSemantic searchOllama
ChromaStores vectorsFast similarity retrievalVector DB
Language AgentChecks languageImprove input qualityQwen
Refinement AgentCreates technical reportStandardize issueQwen
RetrieverFinds candidatesReduce search spaceChroma
Duplicate AgentReasons about duplicatesSimilarity alone isn't enoughQwen
LangGraphControls workflowState + routing + agentsGraph
Decision NodeProduces final recommendationBusiness outcomeConditional logic

35. Most Important Viva Explanation

If asked:

"Why is this an Agentic AI project and not just an RAG chatbot?"

A good answer is:

This system combines an LLM, tools, RAG, persistent knowledge, shared state, multiple specialized agents and conditional workflow routing. The Language Agent handles language correction, the Refinement Agent creates a technical report, the RAG tool retrieves historical GitHub issues, and the Duplicate Analysis Agent reasons over those candidates before the Decision Node produces the final recommendation.

The source explicitly describes the project as demonstrating LLM + Tools + RAG + Persistent Knowledge + State + Conditional Routing + Agentic Workflow. Pasted markdown


36. One Critical Correction to Keep in the Final Implementation

The similarity score should not directly produce:

0.91 → DUPLICATE

Instead:

                    New Issue
                       |
                       v
                 Embedding
                       |
                       v
                 Vector Search
                       |
                       v
              Top 5–10 Candidates
                       |
                       v
            Duplicate Analysis Agent
                       |
             +---------+---------+
             |         |         |
             v         v         v
         DUPLICATE  POSSIBLE   NOT DUPLICATE
                       |
                       v
                Final Decision

That distinction is the core RAG + Agentic AI design principle of this assignment. Pasted markdown


Recommended implementation sequence for trainees

Do not try to build the whole project at once. Implement and demonstrate it in these stages:

Stage 1  → GitHub API + issues.json
Stage 2  → Incremental synchronization
Stage 3  → Embedding generation
Stage 4  → Chroma vector database
Stage 5  → Semantic search
Stage 6  → Language Agent
Stage 7  → Refinement Agent
Stage 8  → Duplicate Analysis Agent
Stage 9  → LangGraph state/workflow
Stage 10 → Final DUPLICATE / POSSIBLE_DUPLICATE / READY_TO_PUBLISH
Stage 11 → Automated tests
Stage 12 → Optional FastAPI/Streamlit UI

This staged approach makes it much easier for trainees to understand what each component does, why it is required, and how it contributes to the final Agentic AI system.

No comments:

Post a Comment