Senior Data Engineer

Rearc; New York (NY); On-site; USD 160,000 - 200,000; Full time

Job summary

Rearc is seeking a Senior Data Engineer in New York, New York. In this role, you'll engage directly with clients to tackle complex data challenges, design and build effective data solutions, and set technical standards for our data platforms. Your expertise in Databricks and strong code-building skills will be critical as you work with client teams, mentor junior engineers, and contribute to meaningful data projects.

Your deep knowledge of data engineering, combined with skills in Apache Spark and modern cloud platforms, will empower you to deliver scalable solutions while promoting best practices in DataOps.

Qualifications

• 6+ years of hands-on data engineering experience with production-grade data platforms.
• Expert-level in Apache Spark, performance tuning, and optimization.
• Strong experience with Python and familiarity with Scala.

Responsibilities

• Contribute to architecture and ensure solutions meet client expectations.
• Design and implement scalable data pipelines and lakehouse architectures.
• Support technical delivery by identifying blockers and maintaining timelines.
• Mentor junior engineers through hands-on pairing and feedback.
• Establish and promote modern data engineering standards and practices.

Skills

Data engineering

Apache Spark

Python

Databricks

Cloud platforms (AWS, Azure, GCP)

DataOps mindset

Job description

As a Senior Data Engineer at Rearc, you'll be the technical anchor on complex, client-facing data engineering engagements, someone who can sit across the table from a client's data leadership team, understand their most difficult data challenges, and then go head-down and build a solution. You'll bring deep, hands‑on expertise with Databricks and the broader modern data stack, and you'll set the technical standard for how we design, build, and deliver data platforms that actually work in production. You'll write code, build pipelines, and architect solutions side‑by‑side with your team and your clients.

What You Bring

• 6+ years of hands‑on data engineering experience, designing and delivering production‑grade data platforms
• Expert‑level in Apache Spark, including runtime internals, performance tuning, and optimisation, you understand what's happening under the hood and use that knowledge to build pipelines that perform at scale.
• You write clean, production‑quality code in Python, with Scala experience a strong plus for deeper Spark and performance‑critical work.
• You've built and productionalized solutions on the platform, including Delta Lake architectures, Unity Catalog governance, and Databricks Workflows. Databricks certification is a strong plus.
• You have real, working experience across at least two major cloud platforms (AWS, Azure, GCP) with genuine depth in at least one, including cloud‑native services such as AWS Redshift/Glue/S3, Azure Synapse/Data Factory/ADLS, or Google BigQuery/Dataflow/GCS.
• You bring a DataOps mindset: CI/CD for data pipelines, automated testing, observability, and infrastructure‑as‑code are standard practice for you, not afterthoughts.
• Your experience spans ETL/ELT design, data warehousing, lakehouse architecture, and data modelling, and you know when to apply each approach.
• Your communication skills allow you to engage technical and non‑technical stakeholders equally well, from a client's CTO to a junior engineer on your team.

What You'll Do

• Contribute to Client Data Engagements : Serve as a senior technical contributor on client projects. Contribute to the architecture, guide the build, and help ensure the solution shipped matches what was promised.
• Build and Productionize Data Solutions : Design and implement scalable, reliable data pipelines and lakehouse architectures on Databricks and cloud platforms. You're hands‑on keyboard, you write code, review code, and set the engineering standard for the engagement.
• Architect for Scale and Reliability : Translate complex client requirements into robust technical designs, reference architectures, and data models built to last in production.
• Support Technical Delivery : Support technical scope and timelines, identify blockers early, and partner with project managers and client stakeholders to keep engagements on track.
• Mentor Data Engineers : Coach junior and mid‑level engineers through hands‑on pairing, code review, and direct feedback, raising the floor for everyone around you.
• Promote Knowledge Sharing : Contribute technical blogs, reference architectures, and internal guides that reflect hard‑won lessons from real client work.
• Champion DataOps Practices : Establish and enforce modern data engineering standards across engagements, automated testing, pipeline observability, version control, CI/CD, and documentation.

The base salary range for this role is $160,000 - $200,000 USD per year. Compensation may vary based on skills, experience, and location.

160,000 - 200,000 USD per year (Hybrid (New York, New York, US))