Hybrid Data Engineer: Databricks Pipelines & ETL

Infinitive; North Dakota; Hybrid; USD 75,000 - 120,000; Full time

Job summary

Infinitive is seeking a Consultant, Data Engineer to build and maintain data pipelines on Databricks for client projects. You will work on project teams with senior engineers, architects and analysts, writing production Python, PySpark and SQL code.

The role is ideal for engineers with 1–3 years of experience who want client-facing work and exposure to new data platforms. The client location is McLean, VA, with hybrid on-site work.

Qualifications

• Bachelor's degree in CS or related field or equivalent experience.
• 1–3 years of professional data engineering or related experience.
• Hands-on experience with Databricks on professional or substantial academic project.
• Proficiency in Python for data processing and PySpark.
• Strong SQL skills including joins, window functions, and aggregations.
• Experience with Spark DataFrames and Spark SQL.
• Familiarity with at least one cloud platform (AWS/Azure/GCP).
• Clear written and verbal communication; able to explain technical work to non-technical stakeholders.
• Authorized to work in the United States.

Responsibilities

• Build and maintain batch and incremental ETL pipelines in Databricks using Python, PySpark and Spark SQL.
• Ingest data from relational databases, APIs, flat files and cloud storage into Delta Lake tables.
• Implement bronze, silver and gold layers following medallion architecture patterns.
• Write and tune SQL for transformations, data validation and reporting tables.
• Add data quality checks, handle schema changes and troubleshoot failed jobs.
• Schedule and monitor workloads with Databricks Workflows and Unity Catalog permissions.
• Tune Spark jobs for performance and cost via partitioning and cluster config.
• Use Git for version control and participate in code reviews and CI/CD pipelines.
• Document pipelines, mappings and runbooks for client teams.
• Join client meetings to clarify requirements and report progress.

Skills

Python

PySpark

SQL

Databricks

Data Engineering

Education

Bachelor's degree in CS or related

Tools

Databricks

Delta Lake

Git

CI/CD

Cloud platforms (AWS/Azure/GCP)

Job description

Infinitive is seeking a Consultant, Data Engineer to build and maintain data pipelines on Databricks for client projects. You will work on project teams with senior engineers, architects and analysts, writing production Python, PySpark and SQL code.

The role is ideal for engineers with 1–3 years of experience who want client-facing work and exposure to new data platforms. The client location is McLean, VA, with hybrid on-site work.