Lead Data Engineer | PySpark, Databricks & Data Lakes

Infinity Outsourcing; Mount Laurel Township (NJ); On-site; USD 67,502 - 77,145; Full time

Job summary

A leading data engineering firm is seeking a Lead Data Engineer with expertise in PySpark to manage data platforms and architectures. This role demands 12+ years of experience, a strong knowledge of ETL and Data Warehousing, and the ability to lead multiple projects. The candidate should possess hands-on skills in using Pyspark and must be adept at working with both structured and unstructured data. Strong organizational and communication skills are essential for success in this dynamic environment.

Qualifications

• 12+ years of experience in data engineering and architecture.
• Hands-on experience in PySpark is mandatory.
• Must possess strong communication and collaboration skills.

Responsibilities

• Lead the development of data platforms and architectures.
• Coordinate with various technical teams for project management.
• Own the end-to-end development lifecycle including testing and deployment.

Skills

PySpark

Databricks

Data Warehousing

ETL

SQL Server

Data Integration

Cloud Architecture

GitHub

Job description

A leading data engineering firm is seeking a Lead Data Engineer with expertise in PySpark to manage data platforms and architectures. This role demands 12+ years of experience, a strong knowledge of ETL and Data Warehousing, and the ability to lead multiple projects. The candidate should possess hands-on skills in using Pyspark and must be adept at working with both structured and unstructured data. Strong organizational and communication skills are essential for success in this dynamic environment.