Ep 06: Building an End-to-End LLM Resume Evaluator & Job Matcher
A comprehensive, production-oriented guide to building an automated AI Job Description & Resume Matcher. This system extracts structured JSON from arbitrary Job Descriptions (JDs), parses heterogeneous resume files (.pdf and .docx), runs candidate evaluations against strict requirements via an LLM, and produces ranked diagnostic fit scores.
1. End-to-End System Architecture
Manual resume screening by HR departments is slow, inconsistent, and often inefficient. Traditional Applicant Tracking Systems (ATS) rely on simple keyword searching, which fails to evaluate contextual competence, hands-on project complexity, or non-standard formatting.
By combining Pydantic schema enforcement, document ingestion tools, and Large Language Models (LLMs), we can convert unformatted candidate resumes and raw job postings into structured JSON objects, run semantic evaluation prompts, and generate actionable fit scores () alongside detailed diagnostic breakdowns.
High-Level System Workflow
2. Infrastructure Setup & Network Resilience
The Cost of Quality vs. Rapid Production
Engineering high-fidelity AI systems requires rigorous preparation. Rapidly pumping out code or content without local validation leads to severe degradation in output quality.
Production Reality: A single hour of reliable output typically requires around four hours of end-to-end development:
- Research & Documentation Review: 1 Hour
- Prototyping & Error Debugging: 1 Hour
- Pedagogical / Instructional Structuring: 30 Minutes
- Execution & Generation: 1 Hour
- Deployment & Publishing: 30 Minutes
API Rate Limiting & Denial of Service (DoS) Mechanics
When processing batches of files against remote LLM endpoints (e.g., Groq, OpenAI), system developers frequently encounter Rate Limit Errors (HTTP 429 Too Many Requests).
Real-Life Analogy: The 1-Crore Highway Congestion
Imagine a city's main highway designed to accommodate normal commuting traffic. If a single individual suddenly deploys cars onto that same highway simultaneously, the entire road network paralyzes. Regular commuters (scooters, buses) are completely blocked.
In computer networking, this traffic overload is called a Denial of Service (DoS) condition. To protect server infrastructure, cloud provider APIs implement Rate Limiters. If an application fires requests back-to-back without delays, the server temporarily blocks the client IP/token.
To make batch processing pipelines resilient against rate limits:
- Use explicit delays (e.g.,
time.sleep(5)) between sequential LLM calls. - Implement exponential backoff fallback logic for rate-limit exception handling.
3. Parsing Unstructured Job Descriptions into Pydantic Schemas
Job descriptions (JDs) posted on corporate career portals contain unstructured text describing responsibilities, minimum requirements, and nice-to-have qualifications. Operating on raw text strings leads to inconsistent prompt evaluations. We must enforce strict output typing using pydantic.
Defining the JobD Pydantic Schema
from pydantic import BaseModel, Field
from typing import List, Optional
class JobD(BaseModel):
role: str = Field(description="The exact job title or role, e.g., SDE-1")
required_skills: List[str] = Field(description="Mandatory technical skills and qualifications")
preferred_skills: List[str] = Field(description="Optional/preferred skills, e.g., Docker, AWS")
min_experience: Optional[str] = Field(default=None, description="Minimum required years of experience")
educational_requirements: Optional[str] = Field(default=None, description="Degree requirements, e.g., B.Tech/BS in CS")
responsibilities: List[str] = Field(description="Key duties and core expectations")
Prompt Engineering for Schema Extraction
When requesting structured output from an LLM, prompts must enforce compliance and eliminate common failure modes:
- Schema Echoing: Weak models often copy the JSON schema structure instead of populating it with extracted file data.
- Hallucination: Models invent experience numbers or degrees not listed in the input.
JOB_SYSTEM_PROMPT = """
You are an expert HR Assistant. Your task is to analyze the provided Job Description text and extract structured information.
CRITICAL RULES:
1. Return ONLY valid JSON matching the provided JobD schema.
2. Fill the schema with ACTUAL information extracted from the text.
3. Do NOT return the schema definition itself.
4. Do NOT include meta-properties like 'title', 'type', or '$defs'.
5. If minimum experience is not mentioned, set 'min_experience' to null.
6. Absolute Rule: Do NOT invent or hallucinate information. If it is missing, leave it null or empty.
"""
4. Designing Resilient Resume Schemas
Candidate resumes exhibit high structural ambiguity. Unlike JDs, resumes vary drastically in organization, naming conventions, and extraneous details (e.g., hobbies, marital status, irrelevant certificates).
Handling Structural Ambiguity
If a schema mandates fields like email or phone without defaults, the extraction engine will throw a validation error whenever a candidate omits them. We use nested models combined with optional fields (Optional[...] = None) to ensure system resilience.
from pydantic import BaseModel, Field
from typing import List, Optional
class Experience(BaseModel):
company_name: Optional[str] = Field(default=None, description="Name of the organization")
role: Optional[str] = Field(default=None, description="Designation or job title held")
duration: Optional[str] = Field(default=None, description="Tenure or employment period")
description: Optional[str] = Field(default=None, description="Summary of accomplishments and work done")
skills_used: List[str] = Field(default_factory=list, description="Technologies applied during this tenure")
class Resume(BaseModel):
name: Optional[str] = Field(default=None, description="Full candidate name")
email: Optional[str] = Field(default=None, description="Contact email address")
phone: Optional[str] = Field(default=None, description="Contact phone number")
total_experience_years: Optional[float] = Field(default=None, description="Calculated total professional experience in years")
skills: List[str] = Field(default_factory=list, description="List of technical and core skills")
experiences: List[Experience] = Field(default_factory=list, description="List of past work or internship experiences")
projects: List[str] = Field(default_factory=list, description="Notable academic or independent technical projects")
certifications: List[str] = Field(default_factory=list, description="Professional certifications or qualifications")
5. File Ingestion Engine (PDF & DOCX Extraction)
Candidate submissions generally arrive as either Adobe PDF (.pdf) or Microsoft Word (.docx) documents.
Comparison of Extraction Mechanics
The docx Table Extraction Trap
A common oversight when parsing .docx resumes is extracting only doc.paragraphs. Candidates frequently place their educational qualifications (e.g., 10th/12th marks, CGPA, Degree info) inside explicit Word tables. Skipping table iteration strips out these academic credentials before the LLM ever sees them.
from pathlib import Path
import pypdf
import docx
def read_pdf(file_path: Path) -> str:
"""Extracts raw text from all pages of a PDF file."""
reader = pydantic_pdf = pypdf.PdfReader(file_path)
text_content = []
for page in reader.pages:
extracted = page.extract_text()
if extracted:
text_content.append(extracted)
return "\n".join(text_content)
def read_docx(file_path: Path) -> str:
"""Extracts raw text from paragraphs AND tables of a DOCX file."""
doc = docx.Document(file_path)
full_text = []
# 1. Extract standard paragraphs
for para in doc.paragraphs:
if para.text.strip():
full_text.append(para.text.strip())
# 2. Extract content buried inside document tables
for table in doc.tables:
for row in table.rows:
row_data = [cell.text.strip() for cell in row.cells if cell.text.strip()]
if row_data:
full_text.append(" | ".join(row_data))
return "\n".join(full_text)
def read_resume(file_path: Path) -> Optional[str]:
"""Routes document to correct parser based on file extension."""
suffix = file_path.suffix.lower()
if suffix == ".pdf":
return read_pdf(file_path)
elif suffix in [".docx", ".doc"]:
return read_docx(file_path)
else:
# Reject non-standard document extensions (e.g., images/screenshots)
return None
6. Batch Resume Ingestion & Parsing Pipeline
With document reading functions established, we can build a directory scanner loop to process every resume, extract its text, call the LLM to map it onto our Resume schema, and insert rate-limit delays.
7. Matcher Engine & Scoring Metrics
The final step is evaluating candidate fit by comparing the structured JobD JSON against the structured Resume JSON.
Defining the Match Result Model
class MatchResult(BaseModel):
score: float = Field(description="Numerical fit score from 0.0 to 100.0 based on JD compliance")
details: dict = Field(description="Diagnostic details containing candidate_name, strengths, missing_skills, and final_verdict")
The Evaluation Prompt Strategy
Instead of asking a general question ("Is this candidate good?"), we instruct the LLM to act as an objective Senior HR Technical Recruiter. The model must analyze requirement overlaps, flag key missing qualifications, and assign a balanced composite score.
MATCH_SYSTEM_PROMPT = """
You are an expert HR Technical Recruiter conducting rigorous candidate screening.
Given a Job Description JSON and a Candidate Resume JSON:
1. Compare required skills vs. candidate skills.
2. Check if minimum experience and educational criteria are satisfied.
3. Assess nice-to-have/preferred skills (e.g., DevOps, Cloud, AI tools).
4. Provide a numerical match score from 0.0 to 100.0.
5. Provide a short, direct final verdict explaining WHY this score was given.
Output MUST strictly match the MatchResult schema.
"""
8. Empirical Case Study: Evaluating Candidate Fit
To observe the system under realistic conditions, we evaluate four diverse resumes against an Amazon SDE-1 Job Posting.
Job Description Summary (Amazon SDE-1)
- Role: Software Development Engineer I
- Required: Strong fundamentals in C++, Java, or Python; Data Structures & Algorithms (DSA); Computer Science degree.
- Preferred: Cloud platforms (AWS), CI/CD pipelines, Docker, Kubernetes, Database systems (SQL/NoSQL).
Candidate Performance & Evaluation Breakdown
Technical Takeaway: Fluff vs. Substance
The LLM accurately distinguished real engineering capability from superficial resume filler:
- Fluff Penalization: Candidates who listed non-technical fluff (e.g., "Green Campus Team Lead", "Sports Coordinator") or superficial copy-paste scripts (e.g., standard boilerplate Twitter Sentiment Analysis) were heavily penalized.
- Substance Reward: Candidates demonstrating actual systems work (e.g., CI/CD deployment pipelines, Ansible playbooks, Dockerization, microservices) received high match scores.
9. Complete End-to-End Implementation Code
The complete script below integrates document parsing, Pydantic schema generation, rate-limit safeguards, and candidate ranking.
import os
import time
import json
from pathlib import Path
from typing import List, Optional
import pypdf
import docx
from pydantic import BaseModel, Field
from openai import OpenAI
# Initialize Client (Using Groq or OpenAI-compatible endpoint)
client = OpenAI(
base_url="https://api.groq.com/openai/v1",
api_key=os.environ.get("GROQ_API_KEY", "your_api_key_here")
)
MODEL_NAME = "llama-3.3-70b-versatile"
# ==========================================
# 1. PYDANTIC SCHEMAS
# ==========================================
class JobD(BaseModel):
role: str = Field(description="Job title")
required_skills: List[str] = Field(description="Mandatory skills")
preferred_skills: List[str] = Field(description="Preferred skills")
min_experience: Optional[str] = Field(default=None)
educational_requirements: Optional[str] = Field(default=None)
responsibilities: List[str] = Field(description="Key responsibilities")
class Experience(BaseModel):
company_name: Optional[str] = Field(default=None)
role: Optional[str] = Field(default=None)
duration: Optional[str] = Field(default=None)
description: Optional[str] = Field(default=None)
skills_used: List[str] = Field(default_factory=list)
class Resume(BaseModel):
name: Optional[str] = Field(default=None)
email: Optional[str] = Field(default=None)
phone: Optional[str] = Field(default=None)
total_experience_years: Optional[float] = Field(default=None)
skills: List[str] = Field(default_factory=list)
experiences: List[Experience] = Field(default_factory=list)
projects: List[str] = Field(default_factory=list)
certifications: List[str] = Field(default_factory=list)
class MatchResult(BaseModel):
score: float
details: dict
# ==========================================
# 2. DOCUMENT READERS
# ==========================================
def read_pdf(file_path: Path) -> str:
reader = pypdf.PdfReader(file_path)
return "\n".join([page.extract_text() for page in reader.pages if page.extract_text()])
def read_docx(file_path: Path) -> str:
doc = docx.Document(file_path)
lines = [p.text.strip() for p in doc.paragraphs if p.text.strip()]
for table in doc.tables:
for row in table.rows:
row_cells = [cell.text.strip() for cell in row.cells if cell.text.strip()]
if row_cells:
lines.append(" | ".join(row_cells))
return "\n".join(lines)
def read_resume(file_path: Path) -> Optional[str]:
suffix = file_path.suffix.lower()
if suffix == ".pdf":
return read_pdf(file_path)
elif suffix in [".docx", ".doc"]:
return read_docx(file_path)
return None
# ==========================================
# 3. LLM EXTRACTION & MATCHING FUNCTIONS
# ==========================================
def parse_job_description(jd_text: str) -> JobD:
system_prompt = (
"You are an HR Assistant. Extract structured information from the Job Description into valid JSON.\n"
"Strictly follow the JobD schema properties. Do not invent details."
)
response = client.chat.completions.create(
model=MODEL_NAME,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"Extract JD data:\n{jd_text}"}
],
response_format={"type": "json_object"}
)
raw_json = json.loads(response.choices[0].message.content)
return JobD(**raw_json)
def parse_resume(resume_text: str) -> Resume:
system_prompt = (
"You are an expert Resume Parser. Parse candidate details into JSON matching the Resume schema.\n"
"Map work history, internships, and projects accurately. Leave missing items null or empty lists."
)
response = client.chat.completions.create(
model=MODEL_NAME,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"Parse the following resume content:\n{resume_text}"}
],
response_format={"type": "json_object"}
)
raw_json = json.loads(response.choices[0].message.content)
return Resume(**raw_json)
def compute_final_score(job: JobD, parsed_resume: Resume) -> MatchResult:
system_prompt = (
"You are a Senior HR Recruiter. Compare the Job Description JSON with the Candidate Resume JSON.\n"
"Provide a match score (0.0 to 100.0) and diagnostic details containing candidate_name, matching_skills, "
"missing_skills, and short_final_verdict."
)
user_prompt = f"JOB DESCRIPTION:\n{job.model_dump_json()}\n\nCANDIDATE RESUME:\n{parsed_resume.model_dump_json()}"
response = client.chat.completions.create(
model=MODEL_NAME,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
],
response_format={"type": "json_object"}
)
raw_json = json.loads(response.choices[0].message.content)
return MatchResult(
score=float(raw_json.get("score", 0.0)),
details=raw_json.get("details", raw_json)
)
# ==========================================
# 4. MAIN BATCH EVALUATION PIPELINE
# ==========================================
def run_pipeline(jd_raw_text: str, resume_dir_path: str):
print("[1/4] Parsing Job Description...")
job = parse_job_description(jd_raw_text)
print(f"-> Parsed Role: {job.role}")
resume_folder = Path(resume_dir_path)
all_results = []
print("\n[2/4] Ingesting & Scoring Resumes...")
for file_path in resume_folder.iterdir():
if file_path.suffix.lower() not in [".pdf", ".docx", ".doc"]:
continue
print(f"\nProcessing File: {file_path.name}")
resume_text = read_resume(file_path)
if not resume_text:
continue
# Extract structured resume JSON
parsed_res = parse_resume(resume_text)
time.sleep(5) # Rate-limiting safeguard
# Compare JD vs Resume
match = compute_final_score(job, parsed_res)
time.sleep(5) # Rate-limiting safeguard
candidate_name = parsed_res.name or file_path.stem
print(f"-> Candidate: {candidate_name} | Score: {match.score}")
all_results.append({
"name": candidate_name,
"score": match.score,
"details": match.details
})
# Sort candidates by score descending
sorted_results = sorted(all_results, key=lambda x: x["score"], reverse=True)
print("\n==========================================")
print(" FINAL RANKING RESULTS ")
print("==========================================")
print("\nTOP CANDIDATES:")
for cand in sorted_results[:2]:
print(f"• {cand['name']} - Score: {cand['score']}/100")
print(f" Verdict: {cand['details'].get('short_final_verdict', 'N/A')}\n")
if len(sorted_results) > 2:
print("BOTTOM CANDIDATES:")
for cand in sorted_results[-2:]:
print(f"• {cand['name']} - Score: {cand['score']}/100")
print(f" Verdict: {cand['details'].get('short_final_verdict', 'N/A')}\n")
# Execution Entry Point
if __name__ == "__main__":
sample_jd = """
Amazon SDE-1 Job Requirement:
Looking for a Software Development Engineer I.
Required: B.Tech/BS in Computer Science. Strong knowledge of C++, Java, or Python and DSA.
Preferred: Cloud experience with AWS, Docker, Kubernetes, CI/CD pipelines, and Database knowledge.
Responsibilities: Write clean code, maintain microservices, and collaborate on system design.
"""
# Assuming 'resumes/' contains candidate PDF/DOCX files
# run_pipeline(sample_jd, "./resumes")
Summary and Key Takeaways
- Structured Schemas over Unstructured Text: Operating directly on raw text strings causes prompt ambiguity. Converting both Job Descriptions and Resumes into Pydantic Schemas enables deterministic downstream comparison.
- Resilience to Resume Formatting Ambiguity: Resumes vary wildly. Use optional fields (
Optional[...] = None) across Pydantic models to prevent extraction crashes when candidates omit details like phone numbers or emails. - DOCX Ingestion Must Parse Tables: Simple paragraph extraction in
python-docxmisses text stored in tables. Ingesting bothdoc.paragraphsanddoc.tablesis required to capture academic marks and grades. - Rate Limiting & DoS Safeguards: Sequential, unthrottled API calls to LLM endpoints trigger
HTTP 429rate-limit errors. Inserting explicit delays (time.sleep(5)) maintains pipeline stability during batch runs. - Anti-Hallucination Prompting: System prompts must explicitly instruct the LLM not to echo the raw JSON schema definition and not to invent non-existent candidate details.
- Substance Over Resume Fluff: LLM matchers effectively penalize non-technical fluff (e.g., campus leadership titles) and generic copy-paste projects, prioritizing verifiable systems engineering skills (CI/CD, Docker, cloud platforms).
- End-to-End Evaluation Workflow: A production-grade AI resume evaluator parses files -> extracts structured schema -> executes an HR-persona evaluation prompt -> outputs diagnostic fit scores () -> ranks candidates automatically.