Data Analytics🏢 Equitabl.io Singapore
Indeed Job Data Scraper with AI Enhancement
Advanced web scraper extracting job data from Indeed.com with GPT-powered extraction of experience requirements and skills.
Project Overview
🎯
Objective
Build comprehensive job data scraper with AI-powered extraction of structured information from job descriptions
💼
My Role
Data Engineer & Web Scraping Specialist
⏱️
Timeline
2 weeks
🛠️
Tech Stack
Python, Scrapy, OpenAI GPT
📈
Key Results
- ✓Implemented Scrapy-based scraper supporting 10+ countries (US, UK, Canada, Singapore, Australia, etc.)
- ✓Integrated GPT API to extract years of experience and required skills from job descriptions
- ✓Built multi-stage ETL pipeline with staging tables and normalized database schema
- ✓Automated currency conversion with daily exchange rate updates
Impact
Multi-country job data extraction
Value
AI-enhanced job description parsing
🔗Project Links
GitHub RepositoryPrivate
Tools & Technologies
PythonScrapyOpenAI GPTPostgreSQLREST API