Senior Data Engineer

Mphasis; California; Hybride; USD 63 000 - 135 000; Plein temps

Résumé du poste

Mphasis in Ontario, CA is seeking an experienced Lead Data Platform Engineer to drive the design, build, and optimization of our enterprise data platform. You will lead technical strategy, guide engineering best practices, and collaborate with cross-functional teams to deliver scalable, secure data solutions across Hadoop, Hive, Spark, and Azure—including Databricks.

You will oversee architecture, develop pipelines with Azure Data services, enforce data governance, and mentor engineers.

Qualifications

• 8+ years of experience in data engineering or data platform roles, with 3+ years in a technical lead or architect capacity.

Responsabilités

• Lead the architecture, development, and enhancement of large-scale data platforms supporting analytics, machine learning, and operational workloads.
• Design and optimize big data solutions leveraging Hadoop ecosystem technologies (HDFS, Hive, Spark, Oozie, Ranger, Airflow).
• Develop and manage data pipelines and transformations using Azure Data services (Data Lake Storage, Data Factory, Synapse, Databricks).
• Implement and enforce robust data governance, security, and quality frameworks across all data layers.
• Partner with data engineering, analytics, product, and infrastructure teams to translate business needs into scalable designs.
• Drive performance tuning, capacity planning, and cost optimization across on-premise and cloud data platforms.
• Mentor and guide data engineers; establish engineering standards and best practices.
• Oversee CI/CD practices, deployment, and monitoring for data workflows.
• Evaluate emerging technologies and contribute to long-term platform strategy.

Connaissances

HDFS

Hive

Spark

Oozie

Ranger

Airflow

Python

SQL

Data modeling

DevOps

Outils

Azure Data Lake Storage

Azure Data Factory

Azure Synapse

Azure Functions

Key Vault

Databricks

Delta Lake

Unity Catalog

Description du poste

About The Role

We are looking for an experienced Lead Data Platform Engineer to drive the design, build, and optimization of our enterprise data platform. This role will lead technical strategy, guide engineering best practices, and work closely with cross-functional teams to deliver scalable, secure, and high-performing data solutions across Hadoop, Hive, and Azure—including Databricks.

Job Title: Lead Data Platform Engineer

Location: Ontario, CA, Hybrid

Key Responsibilities

• Lead the architecture, development, and enhancement of large-scale data platforms supporting analytics, machine learning, and operational workloads.
• Design and optimize big data solutions leveraging Hadoop ecosystem technologies such as HDFS, Hive, Spark, YARN, and related components.
• Develop and manage data pipelines and transformations using Azure Data services (Data Lake Storage, Data Factory, Synapse, Databricks).
• Implement and enforce robust data governance, security, and quality frameworks across all data layers.
• Partner with data engineering, analytics, product, and infrastructure teams to translate business needs into scalable technical designs.
• Drive performance tuning, capacity planning, and cost optimization across both on-premise and cloud data platforms.
• Mentor and technically guide data engineers; establish engineering standards, reusable patterns, and best practices.
• Oversee CI/CD practices, deployment, and monitoring for data workflows.
• Evaluate emerging technologies and contribute to long-term platform strategy and modernization initiatives.

Required Qualifications

• 8+ years of experience in data engineering or data platform roles, with 3+ years in a technical lead or architect capacity.
• Strong hands-on experience with Hadoop ecosystem components (HDFS, Hive, Spark, Oozie, Ranger, Airflow etc.).
• Deep expertise with Azure data services (Azure Data Lake Storage, Azure Data Factory, Azure Synapse, Azure Functions, Key Vault).
• Advanced experience building solutions in Databricks, including Spark optimization, Delta Lake, and Unity Catalog.
• Proficiency in Python, SQL, and distributed data processing frameworks.
• Experience with DevOps practices, CI/CD pipelines, and Infrastructure-as-Code (e.g., Terraform or ARM).
• Strong understanding of data modeling, storage formats (Parquet, ORC, Delta), and data governance frameworks.
• Proven ability to lead technical teams, communicate clearly, and influence architecture decisions.

Preferred Qualifications

• Experience migrating on-prem Hadoop workloads to cloud platforms (preferably Azure Databricks).
• Knowledge of real-time data processing (Kafka, Event Hubs, Spark Streaming).

Soft Skills

• Strong communication and collaboration abilities.
• Prior experience mentoring engineers and contributing to team culture.

Other Details

Salary range $63000 to $134500