Senior Data Engineer - Azure Databricks & PySpark Mastery

Applaudo Studios

United States

Remote

USD 140,000 - 190,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Applaudo Studios is seeking aSenior Data Engineer to design, build, and maintain scalable data pipelines using Azure Databricks and Python. The role involves integrating data from APIs, databases, cloud storage, and SFTP, and delivering business-ready datasets for enterprise use.

You will own development of data transformations with PySpark/Spark SQL, manage data warehouse structures, and ensure data quality and performance.

Qualifications

  • 5+ years of professional experience in Data Engineering.
  • Strong hands-on experience with Azure Databricks.
  • Advanced Python skills for data processing and pipeline development.
  • Solid experience with Apache Spark, including PySpark and/or Spark SQL.
  • Advanced SQL proficiency and data warehousing knowledge.

Responsibilities

  • Design, build, and maintain reliable, production-grade data pipelines using Azure Databricks and Python.
  • Develop scalable data transformations using PySpark or Spark SQL to integrate data from multiple business systems.
  • Ingest and process data from APIs, relational databases, cloud storage, and SFTP sources.
  • Integrate data from ticketing, CRM, marketing, retail, and other enterprise platforms into a centralized Data Hub.
  • Develop and maintain data warehouse structures, dimensional models, and data marts aligned with business requirements.
  • Write, optimize, and maintain SQL transformations to ensure data accuracy and processing efficiency.
  • Implement data quality validations and consistency checks between source systems and target models.
  • Build and maintain outbound data feeds for third-party systems according to defined specifications.
  • Investigate data discrepancies, troubleshoot pipeline issues, and implement appropriate corrective actions.
  • Document data pipelines, transformation logic, dependencies, and technical processes.
  • Collaborate with team members and stakeholders to clarify requirements, identify risks early, and deliver reliable data solutions.
  • Communicate progress, blockers, and potential delays proactively, proposing clear next steps to keep deliverables on track.

Skills

Azure Databricks
Python
Apache Spark
PySpark
Spark SQL
SQL
Data modeling
Data warehousing
Data quality
REST APIs
SFTP
Azure data services
Troubleshooting

Job description

Applaudo Studios is seeking aSenior Data Engineer to design, build, and maintain scalable data pipelines using Azure Databricks and Python. The role involves integrating data from APIs, databases, cloud storage, and SFTP, and delivering business-ready datasets for enterprise use.

You will own development of data transformations with PySpark/Spark SQL, manage data warehouse structures, and ensure data quality and performance.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

TechDigital Group • Fort Worth (TX)

On-site
USD 100,000 - 130,000
Senior Data Engineer: Databricks, Spark & Azure
Senior Data Engineer: Databricks, Spark & Azure

Ciandt • United States

Remote
BRL 110,000 - 190,000
Health insurance
Meal allowance
Childcare support
+7
Senior Data Engineer — Azure & Databricks Pipelines
Senior Data Engineer — Azure & Databricks Pipelines

Sansar Tec, LLC • Haddon Heights (NJ)

On-site
USD 110,000 - 140,000
Senior Data Engineer - Hybrid (Azure Databricks)
Senior Data Engineer - Hybrid (Azure Databricks)

Publicis Sapient • Arlington (VA)

Hybrid
USD 109,000 - 130,000
Inclusive workplace
Learning & development opportunities
Competitive compensation and benefits
+4
Senior Data Engineer on-site)
Senior Data Engineer on-site)

Ziosk • Dallas (TX)

On-site
USD 140,000 - 190,000
Senior Data Engineer – Databricks
Senior Data Engineer – Databricks

Quantum Technologies LLC • Madison (WI)

On-site
USD 110,000 - 140,000
Senior Data Engineer & Client Liaison, Azure Databricks
Senior Data Engineer & Client Liaison, Azure Databricks

ExlService Holdings, Inc. • New York (NY)

Remote
USD 167,000 - 175,000
Senior Data Engineer: Azure & Databricks
Senior Data Engineer: Azure & Databricks

Publicis Sapient • Houston (TX)

Hybrid
USD 104,000 - 120,000
Competitive compensation and benefits
Senior Data Engineer
Senior Data Engineer

Topcoder • California

On-site
USD 120,000 - 170,000
Senior Data Engineer - Databricks/PySpark/SQL | Remote
Senior Data Engineer - Databricks/PySpark/SQL | Remote

HumanIT Digital Consulting • United States

Remote
USD 28,000 - 37,000
15th month salary
Health insurance for family
Birthday off
+3