Data Engineer: AWS, Databricks & Delta Lake

Socket.dev; Reston (VA); On-site; USD 90,000 - 130,000; Full time

Job summary

Acuity Inc. is seeking a skilled Data Engineer to design and deliver AWS cloud-scale data platforms for federal clients. The role emphasizes Spark, Delta Lake, and Delta Lake-optimized pipelines in Databricks, with an eye toward enterprise data modernization.

You'll build scalable data pipelines, implement governance, and collaborate with architects to deploy cloud-native solutions on AWS. The position requires 2+ years in data engineering and a BA/BS degree, and offers the opportunity to impact

Qualifications

• 2+ years data engineering and Agile analytics experience.
• 2+ years retrieving, parsing, and processing structured/unstructured data.
• 1–2 years building scalable ETL/ELT workflows for reporting/analytics.
• 1+ years building cloud data solutions with AWS and Databricks.
• Experience with data quality, validation frameworks, and storage optimization.
• BA or BS degree.

Responsibilities

• Build and maintain PySpark-based data pipelines in Databricks for ingestion, transformation, and enrichment of data.
• Design Delta Lake tables optimized for ACID, partition pruning, schema enforcement, and query performance.
• Develop ETL/ELT workflows integrating multiple source systems into a centralized data warehouse.
• Leverage Spark SQL and DataFrame APIs for business rules and warehouse modeling.
• Collaborate with data architects to implement cloud-native data solutions on AWS (S3, Glue, RDS, IAM).
• Optimize pipeline performance with partitioning, caching, and cost-performance tuning.
• Deploy and version data assets using Git-integrated workflows and CI/CD tools (GitLab/Jenkins).
• Monitor pipeline health and cluster utilization via Databricks and AWS CloudWatch.

Skills

Spark

Delta Lake

Databricks

Data Pipelines

AWS

ACID compliance

Git & CI/CD

Data Quality

SQL

Schema enforcement

Education

BA or BS

Tools

Databricks notebooks

Job description

Acuity Inc. is seeking a skilled Data Engineer to design and deliver AWS cloud-scale data platforms for federal clients. The role emphasizes Spark, Delta Lake, and Delta Lake-optimized pipelines in Databricks, with an eye toward enterprise data modernization.

You'll build scalable data pipelines, implement governance, and collaborate with architects to deploy cloud-native solutions on AWS. The position requires 2+ years in data engineering and a BA/BS degree, and offers the opportunity to impact