Junior AWS Databricks Data Engineer: Pipelines & Analytics

SAIC; College Park (MD); On-site; USD 40,000 - 80,000; Full time

Job summary

SAIC is seeking a Junior AWS Databricks Data Engineer to design and maintain scalable data pipelines on AWS for analytics and reporting. You will turn data from databases, APIs, and files into reliable datasets for business insights.

The role involves building ETL/ELT pipelines with Databricks, PySpark, Spark SQL, and Delta Lake, creating Bronze-Silver-Gold data layers, and ingesting data from Oracle, SQL Server, REST APIs, and S3.

Qualifications

• 5+ years of technology experience.
• 3+ years of AWS engineering experience.
• Strong Python and SQL skills, with hands-on experience in PySpark, Databricks, and Delta Lake.
• Experience building data pipelines on AWS, particularly with Amazon S3, AWS Glue, and Amazon Redshift.
• Knowledge of incremental loading, dimensional data modeling, data quality practices, and Spark performance tuning.
• Experience with Git, CI/CD workflows, and Agile delivery.

Responsibilities

• Develop batch and incremental ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Delta Lake.
• Build and maintain Bronze, Silver, and Gold data layers using Medallion Architecture.
• Ingest and transform structured and semi-structured data from sources such as Oracle, SQL Server, REST APIs, and Amazon S3.
• Implement data quality checks, including schema validation, deduplication, reconciliation, and business rule enforcement.
• Optimize Spark jobs and Delta tables to improve processing time and control cloud resource use.
• Publish curated datasets to Amazon Redshift and support reporting in Power BI, Tableau, or Amazon QuickSight.
• Orchestrate, monitor, and troubleshoot production pipelines using Databricks Workflows, Apache Airflow, and AWS monitoring services.
• Collaborate with analysts, architects, QA, and DevOps teams on requirements, releases, and production support.

Skills

Python

SQL

PySpark

Databricks

Delta Lake

AWS

ETL/ELT

Git

CI/CD

Agile

Spark tuning

Tools

Databricks

AWS Glue

Amazon Redshift

Amazon S3

Airflow

Git

CI/CD

Job description

SAIC is seeking a Junior AWS Databricks Data Engineer to design and maintain scalable data pipelines on AWS for analytics and reporting. You will turn data from databases, APIs, and files into reliable datasets for business insights.

The role involves building ETL/ELT pipelines with Databricks, PySpark, Spark SQL, and Delta Lake, creating Bronze-Silver-Gold data layers, and ingesting data from Oracle, SQL Server, REST APIs, and S3.