Azure Data Engineer - Cloud ETL & Databricks Expert (Onsite)

MassGenics, Inc.; Woodbridge Township (NJ); On-site; USD 110,000 - 117,000; Full time

Benefits offered by this job

Medical & pharmacy coverage

401(k)

HSA and FSA

Job summary

Innova Solutions is seeking a Cloud Data Engineer for a 12+ month contract in Iselin, NJ (onsite). You will design, build and maintain scalable data pipelines and Azure cloud platforms, handling large datasets with Delta Lake and Databricks.

Responsibilities include developing PySpark ETL, optimizing Databricks jobs, and implementing data governance. Requires 5+ years in Python/SQL Server and cloud data warehousing experience. Referral bonus offered.

Qualifications

• 5+ years Python coding experience.
• 5+ years SQL Server based development of large datasets
• 5+ years with ETL pipelines using Databricks PySpark.
• Experience in cloud data warehouses like Synapse, BigQuery, Redshift, Snowflake.
• Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
• Leadership in enterprise-wide Cloud Data Platform migration.
• Cloud based data architecture, messaging, and analytics.
• Cloud certifications.
• Experience with Airflow is a plus.

Responsibilities

• Build and optimize data pipelines for ingestion, transformation and loading with data quality.
• Design, develop, and deploy Spark programs in Databricks to process large data volumes.
• Leverage Delta Lake, DWH, data integration, cloud design and data modeling.
• Proficient in Python and SQL programming.
• Develop data warehouse dimensional models.
• Work with streaming/integration technologies for data ingestion.
• Handle structured, semi-structured and unstructured data.
• Optimize Databricks jobs for performance and scalability.
• Monitor and troubleshoot Databricks jobs and bottlenecks.
• Implement data management, security and governance in Databricks; design enterprise DWH solutions.
• Write SQL queries and stored procedures; reverse engineer processes.
• Perform code reviews to ensure requirements alignment and standards.

Skills

Python

SQL

Databricks PySpark

Cloud data warehouses

Data warehousing modelling

Azure Cloud

ETL pipelines

Airflow

Data governance

SQL + stored procedures

Education

BA degree in engineering or computer science

Tools

Databricks

Spark

SQL Server

Azure Synapse

BigQuery

Redshift

Snowflake

Airflow

Job description

Innova Solutions is seeking a Cloud Data Engineer for a 12+ month contract in Iselin, NJ (onsite). You will design, build and maintain scalable data pipelines and Azure cloud platforms, handling large datasets with Delta Lake and Databricks.

Responsibilities include developing PySpark ETL, optimizing Databricks jobs, and implementing data governance. Requires 5+ years in Python/SQL Server and cloud data warehousing experience. Referral bonus offered.