Senior Data Engineer - Databricks

DATAECONOMY Inc

New Jersey

On-site

USD 130,000 - 180,000

Full time

18 hours ago

Be an early applicant

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

DATAECONOMY Inc. in New Jersey seeks a Senior/Lead Data Engineer to design and build scalable data pipelines on Databricks, delivering robust Bronze-Silver-Gold architectures and reliable data workflows.

You will collaborate across teams to implement Delta Live Tables, Unity Catalog, and Lakehouse patterns, applying best practices in streaming, data quality, and cloud environments (AWS/Azure/GCP) to empower analytics at scale.

Qualifications

Strong Python (PySpark) and SQL programming
Databricks stack: Spark, Delta Lake, Workflows, Unity Catalog
ETL/ELT pipelines with Medallion Architecture (Bronze/Silver/Gold)
Delta Live Tables, Auto-Loader, Structured Streaming
Data modeling: star/snowflake, normalization/denormalization
CI/CD, Git, job orchestration
Cloud experience: AWS, Azure, or GCP
7–10+ years in data engineering
Knowledge of medallion architecture, DLT and Unity Catalog

Responsibilities

Build scalable, production-grade ETL/ELT pipelines using Databricks.
Ingest structured, semi-structured, and streaming data into Bronze/Silver/Gold layers.
Develop optimized transformations, data quality rules, and reusable components.
Implement best practices for job orchestration, monitoring, and automation.
Hands-on experience with Spark, Delta Lake, Workflows, Unity Catalog.
Strong SQL and performance tuning for large-scale data projects.
Work in cloud environments (AWS/Azure/GCP) and modern lakehouse concepts.

Skills

Python (PySpark)

SQL programming

Databricks

Spark

Delta Lake

Workflows

Unity Catalog

DLT / Delta Live Tables

Auto-Loader

Structured Streaming

Dimensional data modeling

CI/CD

Git

Cloud (AWS/Azure/GCP)

Orchestration (Airflow)

Education

Bachelor's degree in CS or related field

7–10+ years data engineering experience

Tools

Databricks

Spark

Delta Lake

Unity Catalog

DLT

Delta Live Tables

Auto-Loader

Lakehouse concepts

Job description

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.

Senior/Lead Data Engineer — Databricks

Rutherford, NJ/ Jersey City, NJ

Full-time

Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
Develop optimized transformations, data quality rules, and reusable framework components.
Implement best practices for job orchestration, monitoring, alerting, and automation.
Hands‑on experience: Spark, Delta Lake, Workflows, Unity Catalog.
Strong SQL programming and performance tuning skills.
Experience with cloud environments (AWS/Azure/GCP).
Experience with modern data lakehouse concepts and distributed systems.
Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
Should be able to explain these concepts using real project examples and architecture decisions.
Knowledge of medallion architecture, DLT and unity catalog within Databricks.