Seniorn Data Engineer

TechShack

United States

Remote

USD 83,000 - 152,000

Full time

19 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

TechShack, a New York-based payments company, is seeking an experienced Data Engineer for a remote contract role. You will design, build and optimize scalable data pipelines on Databricks to support high-volume transaction processing, reporting and analytics in a regulated environment.

Working closely with analytics, product and engineering teams, you will implement Delta Lake architecture, batch and streaming ETL/ELT, and tune Spark jobs and AWS components. US Eastern Time hours apply.

Qualifications

  • 5+ years of hands-on Databricks experience.
  • Strong Python, PySpark and SQL skills.
  • Experience with Delta Lake, Unity Catalog and Databricks Workflows.
  • Strong AWS experience (S3, IAM, Glue, Lambda).
  • Experience building production-grade, high-volume data pipelines.
  • Based in the US and able to work Eastern Time hours.

Responsibilities

  • Design, build and maintain scalable data pipelines on Databricks.
  • Develop batch and streaming ETL/ELT workflows using PySpark and Spark SQL.
  • Implement and manage Delta Lake architecture with bronze, silver and gold layers.
  • Ensure data reliability, quality and governance across payment and transaction data.
  • Tune Spark jobs and Databricks clusters for performance and cost.
  • Collaborate with analytics, product and engineering stakeholders.

Skills

Databricks
Python
PySpark
SQL
Delta Lake
Unity Catalog
Databricks Workflows
AWS

Job description

Location: Remote (US based, must work Eastern Time hours)

Industry: Payments / FinTech

Start: ASAP

The Opportunity

TechShack is partnering with a New York based payments business that is looking for an experienced Data Engineer to join its data platform team on a contract basis. The platform runs mainly on AWS. You'll build and optimise the data pipelines that support high-volume transaction processing, reporting and analytics in a fast-moving, regulated environment.

The role is fully remote. Because the team is based in New York, you need to be located in, and work to, US Eastern Time.

What you'll be doing
  • Designing, building and maintaining scalable data pipelines on Databricks
  • Developing batch and streaming ETL/ELT workflows using PySpark and Spark SQL
  • Implementing and managing Delta Lake architecture, including medallion (bronze, silver, gold) layers
  • Making sure data is reliable, of high quality and well governed across payment and transaction datasets
  • Tuning Spark jobs and Databricks clusters for performance and cost
  • Working closely with analytics, product and engineering stakeholders to deliver data solutions
What we're looking for
  • 5+ years of hands-on Databricks experience (essential)
  • Strong Python, PySpark and SQL skills
  • Solid experience with Delta Lake, Unity Catalog and Databricks Workflows
  • Strong AWS experience (S3, IAM, Glue, Lambda or similar), as the environment is predominantly AWS
  • A background in building production-grade, high-volume data pipelines
  • Based in the US and able to work Eastern Time hours
Nice to have
  • Experience in payments, FinTech or financial services
  • Exposure to real-time streaming (Kafka, Structured Streaming)
  • CI/CD and infrastructure-as-code experience (Terraform, Databricks Asset Bundles)
  • Databricks certification (Data Engineer Professional + more)
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer - Databricks & Delta Lake (Remote, ET)
Senior Data Engineer - Databricks & Delta Lake (Remote, ET)

TechShack • United States

Remote
USD 83,000 - 152,000
Senior Data Engineer (Databricks) - Contractor
Senior Data Engineer (Databricks) - Contractor

Job Envoy • United States

Remote
USD 140,000 - 190,000
Fully remote in US
6-month+ contract option
Flexible start date
+2
Senior Data Engineer (Databricks) - Contractor
Senior Data Engineer (Databricks) - Contractor

Jobgether • United States

Remote
USD 96,000 - 165,000
Remote work
Contract 6 months
Extension possibility
Data Engineer - Databricks (Remote)
Data Engineer - Databricks (Remote)

Lumenalta • United States

Remote
MXN 1,426,533 - 2,139,800
Data Engineer
Data Engineer

Virtusa Corporation • United States

Remote
USD 120,000 - 160,000
Senior Databricks Developer
Senior Databricks Developer

Robotic Process Automation LLC • Pennsylvania

On-site
USD 95,000 - 128,000
Senior Databricks Developer
Senior Databricks Developer

Valiant Technologies LLC • Pennsylvania

On-site
USD 92,000 - 131,000
Databricks Engineer
Databricks Engineer

Princeton IT Services, Inc • United States

Remote
USD 130,000 - 180,000
Data Engineer - Databricks
Data Engineer - Databricks

Open Systems Technologies • New York (NY)

Remote
USD 137,760 - 179,088
Senior Databricks Developer
Senior Databricks Developer

Robotics Technologies LLC • Pennsylvania

On-site
USD 92,000 - 131,000