Azure Data Engineer

Healthworksai; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Healthworksai in the United States is seeking an experienced Data Engineer to build reliable, scalable data pipelines for CPG clients. The role blends strong technical foundations with analytics value, including Python ETL automation and collaboration with client teams.

You'll develop and maintain ETL/ELT workflows using Azure Data Factory and Databricks, ingest data from APIs and web sources, and enable dashboards via Power BI or Tableau in partnership with analysts.

Qualifications

• 4+ years of experience in Data Engineering or Data Analytics.
• Strong SQL and Python skills, with data manipulation libraries.
• Experience with API-based data ingestion and data transformation.
• Experience with cloud data platforms (Azure) and ETL/ELT workflows.

Responsibilities

• Collaborate with client teams and stakeholders to gather data requirements.
• Design and maintain ETL/ELT pipelines across Azure Data Factory and Databricks.
• Ingest data from external APIs and perform data cleaning and backfill handling.
• Automate reporting workflows to improve efficiency.
• Explore AI-enabled approaches to optimize data processes and analytics.
• Build reusable Python utilities and modules to promote code reuse.
• Partner with analytics to structure raw data for reporting in BI tools.

Skills

Client interface

Data pipeline design

Cloud platforms

Python

SQL

Communication

Adaptability

Education

Graduate/Postgraduate from Tier 1/Tier 2 institutions

Tools

Azure Data Factory

Databricks

Key Vault

Blob/ADLS

Git

Python (pandas, requests, PySpark)

Job description

Job Summary

Were seeking an experienced and skilled Data Engineer to work with our Consumer-Packaged Goods (CPG) clients and help build reliable, scalable data pipelines that drive strategic insights. This role is suited for someone who combines strong technical foundations with a solid understanding of how data enables analytics and business value. This is a dynamic, client-facing role that may involve working across adjacent areas such as data analytics, AI or even data science use cases based on project needs. This person will wear many hats in the role, but much of the focus will be on building out our Python ETL processes and automation.

What You'll Do:

• Client Interface: Collaborate directly with client teams, often interfacing with senior stakeholders from global CPG firms to understand data requirements and deliver engineering solutions.


• Data Pipeline Development: Design and maintain robust ETL/ELT workflows across Azure Data Factory, Databricks, and related cloud platforms.


• API Integrations: Ingest data from external APIs and web-sources including web scraping, and ensure error handling, backfilling, and retry mechanisms are in place for resilience.


• Automation: Streamline client reporting through automation for enhanced efficiency.


• AI-Enabled Solutions: Explore and leverage AI capabilities to develop intelligent and automated solutions that simplify data processes, accelerate analysis, improve operational efficiency, and address evolving client needs.


• Utilities & Reusability: Build and maintain utility functions, helper scripts, and custom Python libraries to promote code reuse, modularity, and consistency across projects.


• Collaboration with Analysts: Partner with analytics teams to structure raw data for reporting (Power BI, Tableau) and downstream insights.


• Scalability & Governance: Ensure data quality, security, and reusability, while optimizing performance and adhering to best practices in data governance.


• Cross-functional Flexibility: Opportunity to contribute in analytical or developmental tasks such as building dashboards, web-based tools, or supporting ML workflows when required.

Experience You'll Need:

• Education: Graduate/Postgraduate from Tier 1/Tier 2 institutions.


• Professional Track: 4+ years of experience in Data Engineering/Data Analytics.


• Technical Expertise:
  
  
  • o Hands-on experience with Azure - ADF, Databricks, Key Vault, Blob/ADLS
  
  
  • o Strong proficiency in SQL, Python (with libraries like pandas, requests, pySpark).
  
  
  • o Ability to design reusable utility functions and develop custom Python modules/libraries to streamline repetitive tasks and improve code maintainability.
  
  
  • o Experience with API-based data ingestion, data cleaning, and transformation.
  
  
  • o Familiarity with version control (Git), modular code structuring, and collaborative development practices.
  
  


• Communication Prowess: Good communication skills, with the ability to explain complex technical concepts clearly to both technical and non-technical stakeholders.


• Adaptability: Willingness to wear multiple hats and contribute to broader data and analytics.