Data Engineer

Weekday

Pune

On-site

INR 1,000,000 - 1,500,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Weekday is seeking a skilled Data Engineer to build automated data extraction solutions for healthcare vendor portals. You will transform, validate, and migrate sensitive patient data using Python/Node.js, SQL, and browser automation tools.

You will interact with US-based customers, manage multiple migrations under tight timelines, and ensure HIPAA-compliant data handling. Location includes Pune/Bangalore with full-time engagement and a strong focus on reusable data connectors.

Qualifications

  • 4–7 years of experience in data engineering, system integrations, or data migration.
  • Strong programming skills in Python or Node.js/TypeScript with production scripts and pipelines.
  • Hands-on experience with browser automation tools such as Playwright, Puppeteer, or Selenium.

Responsibilities

  • Design, develop, and maintain automated crawlers to access healthcare vendor portals.
  • Extract and retrieve structured information including patient records, insurance details, and appointments.
  • Transform, validate, and normalise extracted data using SQL and Pandas.
  • Load data into target systems via REST APIs with proper sequencing and retries.
  • Develop reusable connectors to onboard new vendors and reduce one-off scripting.
  • Coordinate with US-based customers to manage migrations and ensure data security.

Skills

Python
Web Scraping
Data Engineering
Data Migration
Browser Automation
REST API Integration
SQL
Data Processing

Tools

Playwright
Puppeteer
Selenium
REST APIs

Job description

This role is for one of Weekday’s clients Salary range: Rs 1000000 - Rs 1500000 (ie INR 10 - 15 LPA) Min Experience: 4+ years Location: Pune, Bangalore JobType: full-time

We are looking for a skilled Data Engineer with hands‑on experience in data engineering, web scraping, system integrations, and data migration. The role involves building automated data extraction solutions to retrieve information from healthcare vendor portals, transforming and validating the extracted data, and migrating it into target systems through REST APIs. The ideal candidate should have strong Python or Node.js/TypeScript programming skills, practical experience with browser automation tools, and the ability to build reusable, scalable data connectors. The candidate must also be comfortable interacting with US‑based customers, managing multiple migrations under tight deadlines, and handling sensitive patient information in compliance with HIPAA requirements.

Key Responsibilities
  • 1. Data Extraction and Web Scraping Design, develop, and maintain automated crawlers to access healthcare vendor portals such as WebPT, Prompt, Raintree, Clinicient, and Jane. Extract and retrieve structured information, including patient records, insurance details, appointments, provider information, documents, and account balances. Handle portal‑specific challenges such as multi‑factor authentication (MFA), session timeouts, pagination, rate limits, and frequent UI or layout changes. Build reliable browser automation workflows using Playwright, Puppeteer, Selenium, or equivalent tools.
  • 2. Data Transformation and Validation Map vendor‑specific fields to the target data model and ensure accurate data transformation. Clean, deduplicate, validate, and normalise extracted datasets. Identify unmapped fields, inconsistencies, and missing information, and flag them for resolution. Use SQL and data‑processing libraries such as Pandas to perform data wrangling and validation.
  • 3. Data Migration and API Integration Load extracted and transformed data into target systems through REST APIs. Ensure data is migrated in the correct sequence with appropriate authentication, batching, pagination, and retry mechanisms. Implement structured logging, error handling, reconciliation, and rollback processes to maintain data integrity. Perform test migrations, production go‑live cutovers, and post‑migration validation.
  • 4. Reusable Connector Development Convert one‑time scripts into reusable, configurable, and scalable data connectors. Develop standardised frameworks that reduce the time and effort required to onboard new healthcare vendors. Improve automation, reliability, and maintainability across data extraction and migration workflows.
  • 5. Customer Coordination and Migration Execution Work closely with customers and Onboarding Managers to obtain portal credentials, understand migration requirements, and finalise data scope. Coordinate customer sign‑offs before production go‑live. Manage test data loads, migration cutovers, and reconciliation activities. Troubleshoot and resolve post‑go‑live data issues promptly. Manage multiple migration projects simultaneously while meeting fixed delivery timelines. Communicate effectively with non‑technical US‑based customers in English.
  • 6. Data Security and Compliance Handle sensitive patient information securely throughout extraction, processing, storage, and migration. Follow HIPAA requirements and applicable data protection practices. Ensure appropriate access controls and secure handling of customer credentials and healthcare data.
Must-Have Skills
  • Python Agentic AI Web Scraping Data Engineering and Data Migration Browser Automation REST API Integration SQL and Data Processing
Good-to-Have Skills
  • Pandas Playwright Puppeteer Selenium Node.js/TypeScript Workflow Orchestration: Airflow, Prefect, or Temporal AWS and Docker Large Language Models (LLMs) for automated field mapping and document parsing
Qualifications and Experience

4–7 years of experience in data engineering, system integrations, or data migration. Strong programming skills in Python or Node.js/TypeScript, with experience developing production‑ready scripts and pipelines. Hands‑on experience with browser automation tools such as Playwright, Puppeteer, or Selenium, including login workflows, MFA, and dynamic web pages. Strong understanding of SQL, data transformation, and data‑wrangling techniques. Experience integrating with REST APIs at scale, including authentication, pagination, batching, and retry handling. Proven ability to develop reusable automation frameworks and configurable connectors rather than relying solely on one‑off scripts. Strong problem‑solving skills and the ability to manage multiple data migration projects under tight deadlines. Excellent English communication skills and confidence interacting with non‑technical customers in the US. Preferred Experience Exposure to US healthcare data and Electronic Health Record (EHR) or Practice Management (PM) systems, preferably in Physical Therapy (PT), Occupational Therapy (OT), or Speech‑Language Pathology (SLP). Familiarity with healthcare data standards and terminology, including HL7, FHIR, X12 (837/835), CPT, and ICD‑10 codes.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

TigerHire • Pune District

On-site
INR 1,000,000 - 1,500,000
Senior Consultant - Data Integration Developer
Senior Consultant - Data Integration Developer

enGen Global • Chennai

On-site
INR 1,200,000 - 2,200,000
Data Engineer
Data Engineer

Mango Sciences India • Mumbai, Delhi, Bengaluru

Remote
INR 2,400,000 - 4,200,000
Data and Information Architect
Data and Information Architect

Santan Intellect • Chennai

On-site
INR 3,200,000 - 5,200,000
Manager- Data Ops
Manager- Data Ops

Innovaccer Inc. • Dadri

On-site
INR 4,500,000 - 7,500,000
Consultant - Data Modeler
Consultant - Data Modeler

HCA Healthcare UK • Hyderabad

On-site
INR 900,000 - 1,500,000
LEAD FULL STACK -Remote
LEAD FULL STACK -Remote

Innova Solutions • India

Remote
INR 3,000,000 - 6,000,000
Sr. Data Engineer
Sr. Data Engineer

Raapid • Ahmedabad District

On-site
INR 2,600,000 - 3,800,000
Senior Data Engineer
Senior Data Engineer

Athenahealth, Inc. • Chennai

On-site
INR 1,500,000 - 1,900,000
Tester III - Software Testing (ETL testing, Big Query, SQL, AWS/Azure)
Tester III - Software Testing (ETL testing, Big Query, SQL, AWS/Azure)

UST • Ernakulam

On-site
INR 1,200,000 - 2,100,000