Databricks Data Engineer - Spark, ETL & Cloud Pipelines

Smart IT Frame LLC; Reston (VA); On-site; USD 90,000 - 120,000; Full time

Job summary

Smart IT Frame LLC is looking for a data engineer to design, build, and maintain data pipelines on the Databricks platform in Reston, Virginia. The role requires strong proficiency in Python and hands-on experience with Apache Spark and AWS services.

The ideal candidate will collaborate with cross-functional teams to optimize performance and ensure data quality, governance, and operational excellence across data platforms. This is an opportunity to work in a dynamic environment with current technologies.

Qualifications

• Hands-on experience with Apache Spark and big data processing frameworks.
• Strong proficiency in Python programming.
• Experience with AWS cloud services including S3, EMR, Lambda, SQS, SNS, and PostgreSQL.

Responsibilities

• Build scalable and reliable data engineering solutions on the Databricks platform.
• Create and maintain efficient ETL/ELT pipelines for data ingestion and transformation.
• Optimize Spark jobs and Databricks workloads for performance and cost efficiency.

Skills

Databricks

Apache Spark

Python programming

AWS services

Data quality assurance

Tools

PostgreSQL

S3

EMR

Lambda

SQS

SNS

Job description

Smart IT Frame LLC is looking for a data engineer to design, build, and maintain data pipelines on the Databricks platform in Reston, Virginia. The role requires strong proficiency in Python and hands-on experience with Apache Spark and AWS services.

The ideal candidate will collaborate with cross-functional teams to optimize performance and ensure data quality, governance, and operational excellence across data platforms. This is an opportunity to work in a dynamic environment with current technologies.