Senior Data Engineer (Azure Databricks)

Applaudo Studios

United States

À distance

USD 140 000 - 190 000

Plein temps

14 jours+

Une candidature complète en une minute

Un CV et une lettre de motivation personnalisés, prêts à envoyer.

Passez les filtres ATS

Résumé du poste

Applaudo Studios is seeking aSenior Data Engineer to design, build, and maintain scalable data pipelines using Azure Databricks and Python. The role involves integrating data from APIs, databases, cloud storage, and SFTP, and delivering business-ready datasets for enterprise use.

You will own development of data transformations with PySpark/Spark SQL, manage data warehouse structures, and ensure data quality and performance.

Qualifications

  • 5+ years of professional experience in Data Engineering.
  • Strong hands-on experience with Azure Databricks.
  • Advanced Python skills for data processing and pipeline development.
  • Solid experience with Apache Spark, including PySpark and/or Spark SQL.
  • Advanced SQL proficiency and data warehousing knowledge.

Responsabilités

  • Design, build, and maintain reliable, production-grade data pipelines using Azure Databricks and Python.
  • Develop scalable data transformations using PySpark or Spark SQL to integrate data from multiple business systems.
  • Ingest and process data from APIs, relational databases, cloud storage, and SFTP sources.
  • Integrate data from ticketing, CRM, marketing, retail, and other enterprise platforms into a centralized Data Hub.
  • Develop and maintain data warehouse structures, dimensional models, and data marts aligned with business requirements.
  • Write, optimize, and maintain SQL transformations to ensure data accuracy and processing efficiency.
  • Implement data quality validations and consistency checks between source systems and target models.
  • Build and maintain outbound data feeds for third-party systems according to defined specifications.
  • Investigate data discrepancies, troubleshoot pipeline issues, and implement appropriate corrective actions.
  • Document data pipelines, transformation logic, dependencies, and technical processes.
  • Collaborate with team members and stakeholders to clarify requirements, identify risks early, and deliver reliable data solutions.
  • Communicate progress, blockers, and potential delays proactively, proposing clear next steps to keep deliverables on track.

Connaissances

Azure Databricks
Python
Apache Spark
PySpark
Spark SQL
SQL
Data modeling
Data warehousing
Data quality
REST APIs
SFTP
Azure data services
Troubleshooting

Description du poste

About You

You are a Senior Data Engineer passionate about building scalable, production-grade data pipelines and transforming complex data from multiple sources into reliable, business-ready datasets. You thrive when working with Azure Databricks, Python, and Spark to solve data integration challenges and support enterprise data ecosystems. You enjoy connecting data across business platforms, optimizing data transformations, and ensuring accuracy and consistency throughout the data lifecycle. You take ownership of your work, ask the right questions early, and proactively communicate risks and blockers. You value clear documentation, technical discipline, and delivering high-quality solutions that support business needs. Strong English communication skills, with the ability to collaborate effectively with a US-based team.

You Bring to Applaudo the Following Competencies
  • 5+ years of professional experience in Data Engineering, designing, building, and maintaining production-grade data pipelines.
  • Strong hands-on experience with Azure Databricks.
  • Advanced Python skills for data processing, transformation, and pipeline development.
  • Solid experience with Apache Spark, including PySpark and/or Spark SQL, for processing data at scale.
  • Advanced SQL proficiency, including complex transformations, data validation, and query performance optimization.
  • Strong knowledge of data warehousing and dimensional modeling, including facts, dimensions, and data marts.
  • Experience integrating data from multiple sources, including REST APIs, relational databases, cloud storage, and SFTP.
  • Hands-on experience with Microsoft Azure data services for data ingestion, storage, processing, and integration.
  • Experience implementing data quality checks and validating consistency between source systems and target data models.
  • Ability to troubleshoot pipeline failures, investigate data discrepancies, and implement reliable solutions.
  • Strong problem-solving, analytical, and organizational skills.
  • Ability to work autonomously, take ownership, and follow established engineering practices and documentation standards.
  • Strong English communication skills, with the ability to collaborate effectively with a US-based team.
  • Nice to have: Experience migrating SQL-based data workloads to Azure Databricks.
You Will Be Accountable for the Following Responsibilities
  • Design, build, and maintain reliable, production-grade data pipelines using Azure Databricks and Python.
  • Develop scalable data transformations using PySpark or Spark SQL to integrate data from multiple business systems.
  • Ingest and process data from APIs, relational databases, cloud storage, and SFTP sources.
  • Integrate data from ticketing, CRM, marketing, retail, and other enterprise platforms into a centralized Data Hub.
  • Develop and maintain data warehouse structures, dimensional models, and data marts aligned with business requirements.
  • Write, optimize, and maintain SQL transformations to ensure data accuracy and processing efficiency.
  • Implement data quality validations and consistency checks between source systems and target models.
  • Build and maintain outbound data feeds for third-party systems according to defined specifications.
  • Investigate data discrepancies, troubleshoot pipeline issues, and implement appropriate corrective actions.
  • Document data pipelines, transformation logic, dependencies, and technical processes.
  • Collaborate with team members and stakeholders to clarify requirements, identify risks early, and deliver reliable data solutions.
  • Communicate progress, blockers, and potential delays proactively, proposing clear next steps to keep deliverables on track.
Qualifications
  • Strong proficiency in Azure Databricks, Python, and Apache Spark.
  • Advanced SQL skills and solid knowledge of data warehousing and dimensional modeling.
  • Proven experience developing and maintaining production data pipelines.
  • Hands-on experience with Microsoft Azure data services and integrating data from multiple enterprise systems.
  • Strong understanding of data quality, validation, and consistency across source and target systems.
  • Strong ownership, autonomy, documentation, and communication skills.
  • Ability to communicate effectively in English and collaborate with a US-based enterprise team.
About Us

We Are Engineered Different. At Applaudo, talented people design, build, and scale meaningful, AI-powered solutions that create real business impact. As an AI-native organization, we collaborate across design, development, cloud, data, and artificial intelligence to turn ideas into scalable products that transform how companies operate, make decisions, and grow. We are building a high-performance culture grounded in five values: Empowering Excellence, Collaborative Teamwork, Unsolicited Respect, Consistent Transparency, and Efficient Communication. These define how we work, how we support one another, and how we hold ourselves accountable. Applaudo is a place for people who want to learn fast, take ownership, and work alongside strong teams they are proud to belong to. Joining us means being part of an organization that is evolving intentionally, investing in modern ways of working, and leading AI-native transformation at scale.

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Data Engineer (temporary)
Data Engineer (temporary)

Applaudo Studios • États-Unis

À distance
USD 120 000 - 180 000
Backend Developer (Python)
Backend Developer (Python)

Applaudo Studios • États-Unis

À distance
USD 90 000 - 130 000
Senior Site Reliability Engineer
Senior Site Reliability Engineer

Applaudo Studios • Kentucky

Hybride
USD 150 000 - 190 000
Hybrid work schedule
On-site Tuesdays and Thursdays in Lous
On-call coverage
Senior Site Reliability Engineer
Senior Site Reliability Engineer

Applaudo Studios • Louisville (KY)

Hybride
USD 140 000 - 190 000
Senior Offensive Security Engineer (Web & API)
Senior Offensive Security Engineer (Web & API)

Applaudo Studios • États-Unis

À distance
USD 83 000 - 138 000
AI Engineer (temporary contract)
AI Engineer (temporary contract)

Applaudo Studios • États-Unis

À distance
USD 83 000 - 152 000
Senior Offensive Security Engineer (Azure & Entra ID)
Senior Offensive Security Engineer (Azure & Entra ID)

Applaudo Studios • États-Unis

À distance
USD 83 000 - 124 000
Restaurant d'entreprise
Data Engineer
Data Engineer

Addepto • États-Unis

À distance
USD 110 000 - 170 000
Flexible remote work options
Professional training and conferences
Equipment provided (laptop and devices
+2
Data Engineering Leader (AI-First) — Remote
Data Engineering Leader (AI-First) — Remote

IVANS Insurance Solutions • États-Unis

Hybride
USD 135 000 - 185 000
Medical, Dental, and Vision Coverage
Holiday and Vacation Time
Health & Wellness Days
+1
Sr. Manager, Data Engineering
Sr. Manager, Data Engineering

Applied Systems • États-Unis

À distance
USD 135 000 - 185 000
Medical, Dental, Vision Coverage
Holiday and Vacation Time
Health & Wellness Days
+1