Pipeline that cut extraction from 3 hours to 5 minutes
Built internal tooling for automated web scraping at HIPTIFY Solutions, with real-time storage, export flows, and a React interface that turned a slow manual workflow into a much faster operational pipeline.
Automation stack 2024 Pipeline that cut extraction from 3 hours to 5 minutes
Overview
The core problem was workflow speed. Manual extraction and processing were consuming hours, limiting throughput and adding unnecessary operational effort.
I built an internal pipeline that automated scraping, persisted data in MongoDB, exported it through Google Sheets, and exposed the workflow through a React-based interface so the process was faster and easier to run.
Stack
- Puppeteer
- MongoDB
- React
- Google Sheets API
Outcome
- Reduced processing time from roughly 3 hours to 5 minutes
- Connected scraping, storage, and export into one usable system
- Turned a backend workflow into a clearer internal tool