Senior Data Engineer - Microsoft Fabric & Spark Pipelines

RAPS Consulting Inc; Newark (NJ); Hybrid; USD 140,000 - 180,000; Full time

Job summary

RAPS Consulting Inc. in Newark, NJ is seeking an experienced Data Engineer for a 12–24 month hybrid role.

You will design, develop and orchestrate end-to-end data pipelines using Microsoft Fabric or Azure Synapse Analytics, building scalable data frameworks and config-driven architectures. You will own ETL/ELT processes with SQL, Spark, and Python, while collaborating with architecture and engineering teams to ensure robust, enterprise-grade solutions.

Qualifications

• 10+ years of experience as a Data Engineer.
• Strong experience with Microsoft Fabric OR strong background in Azure Synapse Analytics with Azure data services.
• Hands-on experience with SQL, Data Modeling, ETL / ELT Pipelines.
• Experience with Spark notebooks and pipelines or Python experience for data engineering use cases.
• Proven ability to design and orchestrate scalable data solutions.
• Experience working in environments with CI/CD processes.
• Familiarity with enterprise documentation standards (e.g., Confluence).

Responsibilities

• Design, develop, and orchestrate end-to-end data pipelines using Microsoft Fabric.
• Build scalable and reusable data frameworks with config-driven pipeline architecture.
• Develop and maintain Spark notebooks and pipelines.
• Implement ETL/ELT processes using SQL, Spark, and Python.
• Perform data modeling and transformations for analytics and reporting.
• Collaborate with architectural and engineering teams to design robust solutions.
• Ensure adherence to enterprise standards including code quality and deployment guidelines.
• Own deliverables end-to-end with minimal supervision.

Skills

Data engineering

CI/CD

Python scripting

SQL proficiency

Data modeling

Distributed data processing

Tools

Microsoft Fabric

Azure Synapse Analytics

Spark notebooks

Python

Job description

RAPS Consulting Inc. in Newark, NJ is seeking an experienced Data Engineer for a 12–24 month hybrid role.

You will design, develop and orchestrate end-to-end data pipelines using Microsoft Fabric or Azure Synapse Analytics, building scalable data frameworks and config-driven architectures. You will own ETL/ELT processes with SQL, Spark, and Python, while collaborating with architecture and engineering teams to ensure robust, enterprise-grade solutions.