Data Engineers - Databricks - W2

American Unit, Inc; Philadelphia (Philadelphia County); Hybrid; USD 70,000 - 85,000; Full time

Job summary

A leading telecommunications provider is seeking a Data Engineer to optimize their Databricks environment. The role involves building data transformations and implementing Medallion architecture. Candidates should have over 5 years of data engineering experience with strong skills in Databricks. This contract position offers competitive pay in Philadelphia. Remote candidates in the Eastern Time Zone are also welcome.

Qualifications

• Hands-on experience with Databricks Asset Bundles (DABs) for CI/CD automation.
• Proven experience implementing Medallion architecture patterns in production environments.
• Strong proficiency in Databricks platform capabilities and data processing frameworks.

Responsibilities

• Optimize the current Databricks environment.
• Build and enhance data transformations.
• Support the implementation of Medallion Architecture.

Skills

Databricks Asset Bundling

Code reviews

Architectural discussions

Data transformations

Graph data integration patterns

Medallion Architecture

Education

5+ years of experience in data engineering

3 years on Databricks platform

Tools

Databricks

CI/CD automation tools

Data visualization tools

Job description

Account Manager - Delivery (Recruitment - US) -- Only W2

Location: Candidates located in the Tri-State area (NY, NJ, CT) will be given first preference, as the role could require onsite presence 1–2 days per week in Philadelphia. However, we are also open to remote candidates based in the Eastern Time Zone (EST).

Anticipated Contract Duration: likely until end of year, potential extensions based on project completion.

Project Details: The team of Data Engineers will play a crucial role in a large Data Engineering project beginning on the GIS Software team at a large, enterprise-level telecommunications provider. These data engineers will be focused on optimizing the current Databricks environment through building and developing data transformations, designing and implementing graph data integration patterns and relationship mapping strategies, implementing DABs for streamlined deployment workflows, and Medallion Architecture Implementation. In addition, these data engineers should have experience with Databricks Asset Bundling for CI/CD automation, conducting code reviews for adherence to development standards, and be comfortable participating in architectural decisions and technical solution design for this project.

Candidates should have experience with:

• Databricks Asset Bundling to support CI/CD automation
• Conducting code reviews to ensure adherence to development standards
• Contributing to architectural discussions and technical solution design

The ideal candidate will focus on optimizing the current Databricks environment by:

• Building and enhancing data transformations
• Designing and implementing graph data integration patterns and relationship mapping strategies
• Supporting the implementation of Medallion Architecture

Data Engineer Qualifications:

• 5+ years of experience in data engineering with at least 3 years focused on the Databricks platform
• Hands‑on experience with Databricks Asset Bundles (DABs) for CI/CD automation
• Proven experience implementing Medallion architecture patterns in production environments
• Strong proficiency in Databricks platform capabilities and data processing frameworks
• Experience with data lake management concepts and transactional data operations

Nice to Have Skills:

• Experience with real‑time streaming data processing using Databricks Structured Streaming
• Knowledge of machine learning workflows and MLOps practices within Databricks
• Experience with Infrastructure as Code tools and deployment automation
• Familiarity with containerization technologies and modern DevOps practices
• Previous consulting or managed services experience
• Experience with data visualization tools and business intelligence platforms
• Knowledge of graph algorithms and network analysis techniques
• Design and implement robust data pipelines using Databricks platform capabilities
• Develop and maintain ETL/ELT processes following best practices for performance and reliability
• Build and optimize data transformations within Databricks environments
• Design and implement graph data integration patterns and relationship mapping strategies
• Implement and manage Databricks Asset Bundles (DABs) for streamlined deployment workflows
• Configure and maintain CI/CD pipelines using Asset Bundles for development, staging, and production environments
• Participate in architectural decisions and technical solution design
• Stay current with Databricks platform updates and emerging data engineering technologies

Seniority level

Mid‑Senior level

Employment type

Contract

Job function

Information Technology

Philadelphia, PA $70,000.00‑$85,000.00 2 days ago