Senior Data Engineer

NFI; Camden (NJ); On-site; USD 104,626 - 139,501; Full time

Benefits offered by this job

Medical

Dental

Vision

401(k) Plan

Wellness Program

Life Insurance

Paid Time Off

Paid Parental Leave

Job summary

NFI is seeking a Senior Data Engineer in Camden, NJ to lead refactoring of legacy SQL logic into modern PySpark pipelines within a Lakehouse on Azure Synapse. You will partner with a vendor to design CI/CD and governance for the Fabric migration, while mentoring junior engineers and ensuring data quality and security.

You will own end-to-end ETL/ELT pipelines, optimize performance across the Azure stack, and champion best practices, data lineage, and robust production stability in a fast-paced

Qualifications

• 5+ years of dedicated data engineering or related experience in enterprise cloud environments.
• Deep expertise in advanced T-SQL, including complex stored procedures.
• Strong Python and PySpark skills for data processing and pipeline development.
• Hands-on experience with core Azure data services (Synapse, Pipeline, ADLS Gen2).
• Ability to review, debug, and optimize code including AI-generated solutions.
• Proven track record building robust, scalable data pipelines in production.

Responsibilities

• Lead refactoring of legacy logic (SQL Stored Procedures) into PySpark pipelines within a Lakehouse architecture.
• Collaborate with a third-party vendor to design CI/CD pipelines and governance for Fabric migration.
• Design, build, and optimize ETL/ELT pipelines across Azure for high performance and data integrity.
• Own data platform health by refactoring fragile processes and reducing production bottlenecks.
• Mentor junior data engineers and establish best practices for maintenance and quality.
• Implement data quality, security, and lineage tracking across ingestion and migration paths.

Skills

SQL Mastery

PySpark/Python

Azure Expertise

Code Evaluation

ETL/ELT Pipelines

Problem Solving

Tools

Azure Synapse Analytics

Azure Pipeline

ADLS Gen2

Job description

Overview

Are you passionate about transforming legacy data systems and driving enterprise-scale platform modernizations? Do you thrive in modern cloud environments and want to help shape our strategic migration to Microsoft Fabric? Join our innovative Data & Analytics team as a Senior Data Engineer, where you'll play a pivotal role in refactoring, optimizing, and scaling our core enterprise data platform.

You’ll build robust, scalable PySpark pipelines and modern Lakehouse structures within our Azure Synapse environment, while also helping lay the foundation for our upcoming Microsoft Fabric migration in collaboration with external architecture partners and mentoring junior team members. This is your opportunity to make a lasting impact at the intersection of hands‑on data engineering and next‑generation modernization—solving complex pipeline bottlenecks today while architecting the ecosystem of tomorrow.

We embrace modern development tools and encourage the use of AI companions (like GitHub Copilot or Google Gemini) to speed up workflows. However, we believe that true senior‑level engineering requires the foundational depth to question, validate, and ruthlessly scrutinize AI‑generated solutions before they ever hit production. Code accountability ultimately rests with the engineer.

This role will be located onsite in Camden, NJ.

Responsibilities

• Lead the technical execution of refactoring legacy logic (predominantly SQL Stored Procedures) into modern, optimized PySpark pipelines within a Lakehouse architecture.
• Partner alongside a third‑party architecture vendor to help influence, design, and implement foundational frameworks, CI/CD pipelines, and governance standards for our upcoming Microsoft Fabric environment.
• Actively design, build, and optimize robust ETL/ELT pipelines, ensuring high performance, scalability, and data integrity across the Azure ecosystem.
• Take ownership of data platform health by refactoring fragile processes, minimizing technical debt, and building highly stable frameworks that reduce production bottlenecks.
• Act as a technical guide and escalation point for associate data engineers, establishing engineering best practices and setting up the team to successfully manage ongoing platform maintenance.
• Implement frameworks that ensure high data quality, strict security standards, and comprehensive lineage tracking across all data ingestion and migration paths.

Qualifications

• Experience: 5+ years of dedicated data engineering or related experience, with a proven track record of designing, building, and deploying pipelines in enterprise cloud environments.
• SQL Mastery: Deep expertise in advanced T‑SQL, including extensive experience writing, debugging, and reverse‑engineering complex Stored Procedures.
• Advanced PySpark/Python: Strong proficiency in Python and PySpark for data processing, with the capacity to translate complex, legacy SQL‑driven logic into efficient notebook‑based pipelines.
• Azure Expertise: Strong hands‑on experience with core cloud services, specifically:
  • Azure Synapse Analytics
  • Azure Pipeline
  • Azure Data Lake Storage (ADLS Gen2)
• Critical Code Evaluation: Possess the deep, foundational understanding of SQL and PySpark internals required to critically evaluate, debug, and optimize complex code—whether written by peers or generated by AI systems.
• ETL/ELT Pipeline Management: Proven track record of building, running, and troubleshooting robust data pipelines in high‑availability production environments.
• Problem Solver: A passion for performance tuning, troubleshooting bottlenecked processes, and re‑engineering legacy code for maximum resilience.

Preferred Qualifications (Nice‑to‑Have)

• Conceptual understanding of, or early hands‑on exposure to, Microsoft Fabric tools (OneLake, Lakehouses, Warehouses).
• Experience collaborating with external systems integrators, consultants, or architecture vendors to deploy enterprise‑level frameworks.
• Strong foundation in Git‑based workflows, DevOps principles, and CI/CD concepts in a data engineering ecosystem.

Compensation and Benefits

Base salary range: $104,626.08 – 139,501.44 USD per annum. Position also eligible for an annual discretionary bonus, targeted at 3%. Employees are also eligible for a robust benefit program including Medical, Dental, Vision, Prescription Drug Coverage, 401(k) Plan, Wellness Program, Life Insurance, Paid Time Off, and Paid Parental Leave.

Equal Opportunity Employer / Protected Veterans / Individuals With Disabilities

The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60‑1.35(c)

LA County Applicants: The Company will consider qualified applicants, including those with criminal histories, in a manner consistent with applicable state and local laws.