AI Engineering: Building a Google Careers Agent for Role Recommendation and Resume Gap Analysis
Automating job search and application evaluation requires bridging structured web data extraction with deep unstructured content synthesis. This chapter covers the end-to-end implementation of an agentic workflow using LangChain and LLMs that scrapes open positions from Google Careers, filters roles against a candidate's resume, performs granular gap analysis across qualifications and responsibilities, and compiles a role-by-role preparation report.
1. Context & Motivation: Automating Job Alignment
Navigating corporate job portals like Google Careers involves significant manual overhead. A typical candidate must:
- Search & Filter: Select parameters such as location (e.g., India), employment type (Full-time), experience level (Early, Mid, Senior), and functional area (Software Engineering).
- Inspect Listings: Individually open dozens of positions to review three core sections:
- Minimum Qualifications: Mandatory baseline technical skills and years of experience.
- Preferred Qualifications: Ideal candidate characteristics and advanced skill sets.
- Responsibilities: Daily engineering tasks and product scope.
- Cross-Evaluate: Compare listed requirements against personal qualifications listed on their resume to identify gaps or strong fits.
By building an agentic workflow, this process is automated through a multi-pass pipeline: coarse web-scraping and filtering are executed first, followed by fine-grained semantic evaluation using Large Language Models (LLMs).
2. Agentic System Architecture & System Design
The architecture uses a two-pass evaluation pipeline. Pass 1 performs rapid broad matching across open job listings using job titles and high-level candidate profiles. Pass 2 executes deep semantic gap analysis on the top-matching subset of positions.
End-to-End Execution Flow
Component Breakdown
3. Data Acquisition & Scraping: Web Content Pipeline
Scraping modern single-page or dynamically populated career sites requires URL normalization and strict pattern matching to filter out non-job navigation links (e.g., "About Us", "Application Status").
Base URL & Query Parameter Construction
The scraper constructs targeted HTTP requests with parametric query strings:
BASE_URL = "https://www.google.com/about/careers/applications/jobs/results"
JOB_LINK_PREFIX = "jobs/results/"
FULL_URL_PREFIX = "https://www.google.com/about/careers/applications/"
# Default filters applied to Google Careers search
SEARCH_FILTERS = {
"location": "India",
"employment_type": "FULL_TIME",
"company": "Google",
"target_level": "MID" # Options: EARLY, MID, ADVANCED, EXECUTIVE
}
Filtering Logic & Pagination Termination
Pagination is handled by incrementing a page counter (page_id) in an infinite loop. The process terminates when a page yields zero valid job listing links.
The string pattern matching algorithm ensures only canonical job endpoints are retrieved:
import urllib.parse
from typing import List
def get_google_job_urls(raw_urls: List[str], target_prefix: str, base_prefix: str) -> List[str]:
"""
Filters raw extracted page links to keep only valid Google job listing endpoints,
prepending the canonical web domain prefix.
"""
valid_job_urls = []
for url in raw_urls:
if target_prefix in url:
# Construct absolute URL path
full_url = urllib.parse.urljoin(base_prefix, url)
if full_url not in valid_job_urls:
valid_job_urls.append(full_url)
return valid_job_urls
Note: Modern career portals often wrap job listings under internal paths like
jobs/results/123456789. Filtering by prefix prevents the LLM from processing irrelevant boilerplate text from general career guidance pages.
4. Parallel Content Extraction & Document Ingestion
Parsing job listings sequentially via single-threaded HTTP requests creates bottleneck delays. To maintain high throughput, page contents are fetched concurrently.
Parallel Web Loading Strategy
Given job listing URLs, sequential extraction requires total network latency time. By batching requests in parallel (e.g., limit ), network round-trip delays are significantly reduced.
import asyncio
from langchain_community.document_loaders import WebBaseLoader, PyPDFLoader
from langchain_core.documents import Document
def get_url_content(urls: List[str], batch_size: int = 10) -> List[Document]:
"""
Fetches URL content in parallel batches to optimize network throughput.
"""
all_documents = []
# Process requests in concurrent chunks of size `batch_size`
for i in range(0, len(urls), batch_size):
chunk_urls = urls[i:i + batch_size]
loader = WebBaseLoader(web_paths=chunk_urls)
loader.requests_per_second = batch_size
docs = loader.load()
all_documents.extend(docs)
return all_documents
Candidate Resume Ingestion
Candidate profiles stored in PDF format are processed using PyPDFLoader. Multi-page documents are merged into a single plain-text string for down-pipeline prompt contexts.
def get_resume_content(pdf_path: str) -> str:
"""
Loads a PDF resume and merges page content into a unified text document.
"""
loader = PyPDFLoader(pdf_path)
pages = loader.load()
resume_text = "\n".join([page.page_content for page in pages])
return resume_text
5. Candidate Resume Processing & Top-N Role Filtering
Passing the complete page content of 100+ scraped job listings into an LLM context window causes excessive latency and token consumption costs. The agent mitigates this by executing a First-Pass LLM Filter using job titles alone.
Dynamic Model Provider Selection
The codebase includes an abstraction layer to route prompts between proprietary APIs (e.g., OpenAI gpt-4) and open-weights models (e.g., GPT-OSS 120B hosted on HuggingFace).
from langchain_openai import ChatOpenAI
from langchain_community.llms import HuggingFaceEndpoint
def get_model(use_huggingface: bool = False):
"""
Factory function providing model abstraction across commercial and open-source models.
"""
if use_huggingface:
return HuggingFaceEndpoint(
repo_id="openai/gpt-oss-120b",
temperature=0.1,
max_new_tokens=2048
)
return ChatOpenAI(model="gpt-4", temperature=0.0)
First-Pass Matching Prompt & Extraction Logic
RELATED_JOB_ROLES_PROMPT = """
You are an expert career advisor specializing in technical recruitment at Google.
You are provided with a list of comma-separated open job titles and a candidate's resume content.
Job Titles:
{job_titles}
Candidate Resume Content:
{resume_content}
Tasks:
Select the top {top_n} most relevant job roles from the provided list that best align with the candidate's core background, skill set, and experience level.
Return ONLY the exact, unmodified job titles, separated by newlines. Do not alter titles or append bullet points.
"""
def get_related_job_roles(
open_jobs: List[dict],
resume_text: str,
top_n: int = 5
) -> List[dict]:
"""
Screens candidate resume against job titles to return full objects for top-N matches.
"""
# Create comma-separated listing of titles
all_titles = ", ".join([job["title"] for job in open_jobs])
prompt = RELATED_JOB_ROLES_PROMPT.format(
job_titles=all_titles,
resume_content=resume_text,
top_n=top_n
)
model = get_model(use_huggingface=False)
response = model.invoke(prompt)
# Parse new-line delimited output titles
selected_titles = [line.strip().lower() for line in response.content.split("\n") if line.strip()]
# Filter original job dictionary array to keep matched job objects
recommended_job_profiles = []
for job in open_jobs:
if job["title"].lower().strip() in selected_titles:
recommended_job_profiles.append(job)
return recommended_job_profiles
6. Deep Qualification Extraction & RAG Micro-Chains
Once top candidate roles are selected, the agent executes targeted extraction micro-chains on each role's raw HTML page content. This isolates specific requirements into three modular components:
- Minimum Qualifications
- Preferred Qualifications
- Job Responsibilities
Section Extraction Execution Micro-Chains
RAG_EXTRACTION_PROMPT = """
You are a precise document extraction system.
Extract the requested information from the job posting below. Do not infer or extrapolate.
If the information is not present, respond with "Not Specified".
Job Posting Context:
{context}
Extraction Question:
{question}
"""
def extract_job_section(job_content: str, question: str) -> str:
"""
Executes a micro-chain call to isolate specific sections of a job posting.
"""
model = get_model()
prompt = RAG_EXTRACTION_PROMPT.format(context=job_content, question=question)
response = model.invoke(prompt)
return response.content.strip()
# Specialized Question Constants
MIN_QUAL_QUESTION = "What are the exact Minimum Qualifications listed for this role?"
PREF_QUAL_QUESTION = "What are the exact Preferred Qualifications listed for this role?"
RESPONSIBILITIES_QUESTION = "What are the core Responsibilities listed for this role?"
7. Role-Level Gap Analysis & Markdown Synthesis
With qualifications extracted into structured strings, the agent invokes an evaluation prompt to compare the candidate's resume against the requirements for each selected role.
Evaluation Prompt Architecture
RESUME_ANALYSIS_PROMPT = """
You are an expert Google technical talent recruiter and career coach.
Analyze the provided candidate resume against the explicit criteria for the target role.
Target Role Criteria:
- Minimum Qualifications: {min_quals}
- Preferred Qualifications: {pref_quals}
- Job Responsibilities: {responsibilities}
Candidate Resume:
{resume_text}
Generate a comprehensive Markdown analysis report covering:
1. **Qualification Assessment Table**:
| Requirement | Requirement Type (Min/Pref) | Meets Requirement (Yes/No) | Gap / Evidence from Resume |
2. **Key Skill & Experience Gaps**: Outline specific missing experience (e.g., years of experience, specialized tools, degree mismatches).
3. **Actionable Recommendations**: Clear technical updates, project improvements, or certs needed to qualify.
4. **Preparation Strategy**: Focus areas for technical interviews for this specific position.
Format your entire response cleanly using strictly structured Markdown.
"""
Structure of Generated Markdown Gap Analysis Report
For each analyzed role, the agent outputs a structured markdown report section containing criteria compliance metrics and direct links back to Google Careers:
# Google Job Role Analysis Report (India)
## 1. Role: [Software Engineer, AI Infrastructure, Cloud ML](https://www.google.com/about/careers/applications/jobs/results/12345678)
### Qualification Assessment Table
| Requirement | Requirement Type | Meets Requirement | Gap / Evidence from Resume |
| :--- | :--- | :--- | :--- |
| 5+ years experience in C++ or Python | Minimum | **No** | Resume shows 4 years experience. |
| Bachelor's degree in CS or STEM | Minimum | **Yes** | Holds B.Tech in Electronics & Telecom. |
| Experience with ML infrastructure tools (KubeFlow, Ray) | Preferred | **No** | Only general cloud experience logged. |
### Key Skill & Experience Gaps
* **Years of Experience Gap**: Candidate falls short of the mandatory 5-year requirement by 1 year.
* **ML Infrastructure Orchestration**: Missing hands-on experience with production ML frameworks like Ray or Kubernetes-based ML pipelines.
### Actionable Action Items
* Highlight distributed computing projects on resume to bridge the experience gap.
* Build and document an end-to-end ML serving pipeline using Ray/KubeFlow in the knowledge portfolio.
8. Complete Codebase Implementation
The script below integrates URL extraction, PDF parsing, micro-chain section parsing, LLM-based role matching, gap evaluation, and file output into a unified application.
import os
import json
import urllib.parse
from typing import List, Dict, Any
from langchain_community.document_loaders import WebBaseLoader, PyPDFLoader
from langchain_openai import ChatOpenAI
# ------------------------------------------------------------------------------
# Configuration & Constants
# ------------------------------------------------------------------------------
BASE_CAREERS_URL = "https://www.google.com/about/careers/applications/jobs/results"
JOB_PREFIX = "jobs/results"
DOMAIN_PREFIX = "https://www.google.com/about/careers/applications/"
RESUME_PATH = "knowledge_base/resume.pdf"
OUTPUT_REPORT_PATH = "report/report.md"
# ------------------------------------------------------------------------------
# Core Helper Functions
# ------------------------------------------------------------------------------
def get_model():
"""Initializes standard LLM chat engine."""
return ChatOpenAI(model="gpt-4", temperature=0.0)
def scrape_google_job_links(max_pages: int = 5) -> List[str]:
"""
Simulates pagination across Google Careers using query params
and collects unique filtered job URLs.
"""
extracted_job_urls = []
for page in range(1, max_pages + 1):
# Construct paginated filtering URL
query_params = {
"q": "Software Engineer",
"location": "India",
"page": str(page)
}
target_url = f"{BASE_CAREERS_URL}?{urllib.parse.urlencode(query_params)}"
try:
loader = WebBaseLoader(target_url)
docs = loader.load()
# Simple link parsing logic based on text scraping
page_content = docs[0].page_content if docs else ""
# Filter URLs containing target prefix
# In production, use BeautifulSoup over raw page_content
raw_links = [line.strip() for line in page_content.split() if JOB_PREFIX in line]
page_job_urls = []
for link in raw_links:
full_url = urllib.parse.urljoin(DOMAIN_PREFIX, link)
if full_url not in extracted_job_urls:
page_job_urls.append(full_url)
if not page_job_urls:
print(f"[Info] Zero job links extracted on page {page}. Halting scraping.")
break
extracted_job_urls.extend(page_job_urls)
except Exception as e:
print(f"[Error] Failed to fetch page {page}: {e}")
break
return extracted_job_urls
def parse_job_pages(urls: List[str]) -> List[Dict[str, Any]]:
"""Fetches job descriptions in bulk and returns structured objects."""
print(f"[Info] Extracting content for {len(urls)} job pages...")
loader = WebBaseLoader(urls)
docs = loader.load()
jobs_data = []
for doc in docs:
source_url = doc.metadata.get("source", "")
# Extract title line assuming title format from metadata or content header
title = doc.metadata.get("title", "Google Software Engineering Role")
jobs_data.append({
"title": title,
"url": source_url,
"content": doc.page_content
})
return jobs_data
def load_resume_text(pdf_path: str) -> str:
"""Extracts plain text content from PDF resume."""
if not os.path.exists(pdf_path):
raise FileNotFoundError(f"Resume file not found at path: {pdf_path}")
loader = PyPDFLoader(pdf_path)
pages = loader.load()
return "\n".join([page.page_content for page in pages])
# ------------------------------------------------------------------------------
# Agent Matching & Synthesis Workflow
# ------------------------------------------------------------------------------
def run_agentic_job_matching():
print("=== Phase 1: Ingesting Job Listings & Resume ===")
resume_text = load_resume_text(RESUME_PATH)
job_urls = scrape_google_job_links(max_pages=3)
if not job_urls:
print("[Error] No job links retrieved. Process terminated.")
return
scraped_jobs = parse_job_pages(job_urls)
print("\n=== Phase 2: First-Pass LLM Candidate Screening ===")
titles_list = "\n".join([f"- {job['title']}" for job in scraped_jobs])
selection_prompt = f"""
Compare the candidate's resume below with these Google Job Titles.
Candidate Resume:
{resume_text[:2000]} # Truncated for initial screening call
Open Google Positions:
{titles_list}
Identify up to 3 position titles that best match this candidate.
Return ONLY the exact position titles, separated by newlines.
"""
model = get_model()
selection_response = model.invoke(selection_prompt)
matched_titles = [t.strip().lower() for t in selection_response.content.split("\n") if t.strip()]
matched_jobs = [
job for job in scraped_jobs
if any(m_title in job["title"].lower() for m_title in matched_titles)
]
if not matched_jobs:
print("[Warning] No direct title matches returned by LLM screening. Defaulting to first 2 jobs.")
matched_jobs = scraped_jobs[:2]
print(f"[Success] Screened down to {len(matched_jobs)} target positions.")
print("\n=== Phase 3: Deep Qualification Extraction & Analysis ===")
final_report_sections = []
for job in matched_jobs:
print(f"[Analyzing Role] {job['title']}")
# Micro-Chain 1: Min Qualifications
min_q = model.invoke(
f"Extract Minimum Qualifications from text:\n\n{job['content'][:4000]}"
).content
# Micro-Chain 2: Preferred Qualifications
pref_q = model.invoke(
f"Extract Preferred Qualifications from text:\n\n{job['content'][:4000]}"
).content
# Micro-Chain 3: Responsibilities
resp = model.invoke(
f"Extract Job Responsibilities from text:\n\n{job['content'][:4000]}"
).content
# Micro-Chain 4: Final Gap Synthesis
synthesis_prompt = f"""
Role Title: {job['title']}
Role Link: {job['url']}
Criteria:
- Minimum Qualifications: {min_q}
- Preferred Qualifications: {pref_q}
- Responsibilities: {resp}
Candidate Resume:
{resume_text}
Provide a structured evaluation in clean Markdown format:
1. Role Title (as Markdown Header with link)
2. Qualification Evaluation Table (Columns: Requirement, Type, Meets [Yes/No], Gap Notes)
3. Resume Gaps and Missing Skills
4. Actionable Steps to Improve Resume Match
"""
role_analysis = model.invoke(synthesis_prompt).content
final_report_sections.append(role_analysis)
# --------------------------------------------------------------------------
# Output Synthesis
# --------------------------------------------------------------------------
print("\n=== Phase 4: Compiling Final Markdown Report ===")
os.makedirs(os.path.dirname(OUTPUT_REPORT_PATH), exist_ok=True)
report_header = "# Google Career Roles Analysis Report\n\n"
report_header += "This automated report analyzes candidate resume fit against extracted Google open roles.\n\n---\n\n"
full_markdown = report_header + "\n\n---\n\n".join(final_report_sections)
with open(OUTPUT_REPORT_PATH, "w", encoding="utf-8") as f:
f.write(full_markdown)
print(f"[Complete] Detailed gap analysis saved to '{OUTPUT_REPORT_PATH}'.")
if __name__ == "__main__":
run_agentic_job_matching()
9. Real-World Execution, Edge Cases & Debugging Mindset
Building production-grade web agents requires handling non-deterministic network conditions, dynamic web layouts, and unstructured LLM responses.
Common Production Failure Modes & Mitigation Strategies
1. Junk URL Leakage during Web Extraction
- Issue: The web scraper extracts links pointing to generic career information pages (e.g.,
/about/careers/applications/faq) instead of valid job postings. - Solution: Enforce explicit prefix validation (
jobs/results/) and verify that extracted job documents contain explicit headers for minimum qualifications before passing them to the LLM.
2. Title Substring Mismatches
- Issue: The first-pass screening prompt returns slightly altered job titles (e.g., returning "Software Engineer III, Infrastructure" when the listing was titled "Software Engineer 3, Infrastructure, Cloud ML"), breaking exact string matching logic.
- Solution: Normalize title strings by stripping punctuation, converting to lowercase, and performing fuzzy matching or checking for substring inclusions rather than relying on exact string equality (
==).
3. LLM Rate Limiting Under Concurrency
- Issue: Sending parallel requests via
WebBaseLoaderwhile running multi-pass LLM prompts can trigger429 Too Many RequestsAPI errors. - Solution: Throttle operations using concurrency primitives (
asyncio.Semaphore(10)) and wrap API calls in retry handlers configured with exponential backoff.
4. Non-Standard Resume Content
- Issue: PDF resumes containing embedded image tables, non-standard text encodings, or scanned image text yield empty strings from standard text extractors like
PyPDFLoader. - Solution: Add validation checks verifying that the extracted resume text length exceeds a minimal threshold (). If validation fails, route the document through an optical character recognition (OCR) fallback parser (such as
pdf2imagepaired withpytesseract).
Summary and Key Takeaways
- Two-Pass Screening Architecture: Avoid passing large collections of scraped documents directly into an LLM context. Use a two-pass pipeline that screens job titles first before performing deep semantic analysis on a selected subset.
- Prefix-Based URL Filtering: Robust scraping relies on strict URL path matching (e.g., checking for
jobs/results/) to filter out irrelevant platform and navigation links. - Parallel Content Fetching: Use parallel batching strategies () when fetching job page contents to minimize overall execution latency.
- Modular Micro-Chains: Break complex extraction tasks down into dedicated micro-chains (e.g., separate prompts for minimum qualifications, preferred qualifications, and responsibilities) to prevent contextual hallucination.
- Explicit Role Alignment: Map extraction outputs directly back to candidate resume details using structured evaluation tables that explicitly evaluate requirements as met or unmet.
- Actionable Gap Analysis: Direct the synthesis prompt to return practical, concrete advice—such as missing tools, experience gaps, or interview preparation focus areas—rather than generic evaluation summaries.
- Traceable Documentation: Include direct job listing links alongside generated markdown summaries so candidates can jump straight from evaluation insights to application pages.