Senior Data Engineer — AWS Cloud Pipelines (Remote)

Attain Talent; United States; Hybrid; USD 110,000 - 140,000; Full time

Benefits offered by this job

Remote Work (Hybrid)

Medical, Dental, Vision

401(k) with matching

Generous PTO

Professional development budget

Job summary

Attain Talent is seeking a Data Engineer to design, build, and maintain scalable data pipelines and platforms for a GovCon client. You will work with Databricks, PySpark, Python, and SQL (PostgreSQL) to model and transform large data sets in a secure, AWS-based environment.

Responsibilities include evolving ETL/ELT workflows, integrating multiple data sources, and contributing to AI-enabled data solutions with Bedrock and vector search.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related technical field
• 4+ years of professional experience in data engineering or related domains
• Strong hands-on experience with Databricks, Apache Spark (PySpark), Python, SQL (PostgreSQL), and dbt for large-scale data engineering, ETL/ELT development, data transformation, and data modeling.
• Designing, building, and maintaining AWS-native data platforms using AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, AWS Step Functions, Amazon S3, Amazon Redshift, Amazon RDS, AWS DMS, and Amazon CloudWatch.
• Developing scalable data pipelines, workflow orchestration, and data integration solutions across enterprise environments.
• Working with modern data lake technologies including Apache Iceberg and data formats such as Parquet, ORC, and Avro.
• Designing and optimizing solutions using relational and NoSQL databases including PostgreSQL, Redshift, Oracle, GraphDB, and other NoSQL platforms.
• Building reliable, high-performance data platforms through performance tuning, system optimization, and enterprise-scale ETL/ELT architectures.
• Java development and modern CI/CD practices using Harness.
• Strong analytical and problem-solving skills
• Experience working in agile, iterative software development environments
• Ability to quickly learn and apply new technologies and domain knowledge
• Excellent written and verbal communication skills, with the ability to explain complex topics to diverse audiences

Responsibilities

• Design, build, and maintain scalable data pipelines and data models.
• Develop and optimize AWS-native data platforms using Glue, EMR, MWAA, Lambda, and Redshift.
• Create high-performance ingestion, transformation, and orchestration workflows for structured and semi-structured data.
• Design and optimize analytical data platforms using Athena, Trino, Hive, OpenSearch, and data catalog technologies.
• Integrate enterprise and external data sources across relational and NoSQL databases.
• Build AI-enabled data solutions using Bedrock, RAG pipelines, and vector search technologies.
• Develop infrastructure using CloudFormation, GitHub, Harness, and CI/CD pipelines while leveraging SNS, SQS, and EventBridge.
• Improve reliability, scalability, performance, and maintainability of enterprise data platforms.
• Lead modernization initiatives migrating legacy platforms to cloud-native AWS services.
• Mentor junior engineers and collaborate in an Agile environment to deliver data solutions.

Skills

Databricks

Apache Spark (PySpark)

Python

SQL (PostgreSQL)

dbt

AWS-native data platforms

Education

Bachelor's degree in Computer Science, Engineering, or related technical field

Tools

AWS Glue

Amazon EMR

Amazon MWAA (Airflow)

AWS Lambda

Amazon S3

Redshift

RDS

DMS

CloudWatch

Apache Iceberg

Job description

Attain Talent is seeking a Data Engineer to design, build, and maintain scalable data pipelines and platforms for a GovCon client. You will work with Databricks, PySpark, Python, and SQL (PostgreSQL) to model and transform large data sets in a secure, AWS-based environment.

Responsibilities include evolving ETL/ELT workflows, integrating multiple data sources, and contributing to AI-enabled data solutions with Bedrock and vector search.