Data Engineer Databricks / Python / Database

MACHINE LEARNING TECHNOLOGIES LLC; Jersey City (NJ); On-site; USD 50,000 - 70,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is hiring a Data Engineer in Jersey City, NJ, to design and optimize scalable data pipelines using Databricks and Python. Ideal candidates should have experience with database engineering, Apache Spark, and cloud platforms.

This role involves collaborating with data scientists and ensuring data quality and compliance. Candidates must have a strong understanding of ETL/ELT processes and large-scale datasets.

Qualifications

• Proven experience as a Data Engineer with expertise in Databricks.
• Strong programming skills in Python and experience with SQL.
• Familiarity with cloud platforms and CI/CD methodologies.

Responsibilities

• Design and maintain ETL/ELT workflows on Databricks.
• Develop scalable data pipelines using Python and Spark.
• Implement data quality and monitoring frameworks.

Skills

Databricks

Python

Database Engineering

Apache Spark

Cloud Platforms (AWS/Azure/GCP)

ETL/ELT tools

CI/CD for data pipelines

Job description

Overview

We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, and Database engineering. The ideal candidate will design, build, and optimize scalable data pipelines, ensure data quality, and support advanced analytics and business intelligence initiatives across the organization.

Job Details

• Job ID: J52293
• Job Title: Data Engineer Databricks / Python / Database
• Location: Jersey City, NJ
• Duration: 12 Months + Extension
• Hourly Rate: Depending on Experience (DOE)
• Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, L2-EAD, GC-EAD
• Client: To Be Discussed Later
• Employment Type: W-2, 1099, C2C

Key Responsibilities

• Design, build, and maintain ETL/ELT workflows on Databricks.
• Develop scalable data pipelines using Python and Spark.
• Optimize data transformation logic for performance and cost efficiency.
• Work with structured and unstructured data from various sources.
• Develop and manage data models, schemas, and tables across databases (SQL/NoSQL).
• Implement data quality, validation, and monitoring frameworks.
• Collaborate with Data Scientists, Analysts, and Cloud Engineers to support analytics and ML workloads.
• Ensure data security, governance, and compliance in all solutions.
• Troubleshoot pipeline failures and continuously improve pipeline reliability.
• Integrate data from APIs, streaming sources, and cloud storage.

Required Skills

• Databricks (mandatory) – Delta Lake, Notebooks, Spark SQL, Jobs, Workflows
• Python – Data processing, Spark APIs, automation, optimization
• Database Engineering – Strong SQL, query tuning, schema design, stored procedures
• Apache Spark – RDDs/DataFrames, optimisation, partitioning
• Cloud Platforms (one or more): AWS / Azure / GCP
• ETL/ELT tools: ADF, Glue, Dataflow, or equivalent
• Experience working with large-scale datasets
• CI/CD for data pipelines (Git, DevOps, Jenkins, etc.)

Equal Opportunity Employer

MACHINE LEARNING TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. MACHINE LEARNING TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will MACHINE LEARNING TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

© 2008-2018, Machine Learning Technologies LLC. All Rights Reserved.