Senior Data Engineer

Smart Synergies; McLean (VA); On-site; USD 120,000 - 180,000; Full time

Job summary

Smart Synergies is seeking a high-energy Senior Data Engineer to design, build, and maintain scalable data pipelines on an AWS-based platform. You will own end-to-end delivery from ingestion to analytics-ready datasets, collaborating with data architects, analytics, and business stakeholders to ensure trusted data for decision-making.

You will work on Medallion architecture, CI/CD, and Power BI integration, while applying governance, testing, and optimization across the data lifecycle.

Qualifications

• Bachelor’s degree or equivalent experience in a relevant field.
• 7+ years of data engineering experience with AWS cloud services.

Responsibilities

• Design, develop, and maintain scalable ELT pipelines across diverse sources (ERPs, APIs, vendor files, relational databases).
• Build and optimize AWS-native data pipelines with Medallion Lakehouse architecture.
• Provision and manage AWS infra via Terraform; maintain CI/CD with GitOps, testing, logging, and alerting.
• Create dimensional models, Gold-layer datasets, and scalable vendor file ingestion with SQL, Python, and PySpark.
• Implement automated data validation, anomaly detection, and lineage tracking; support metadata management.
• Troubleshoot data issues and ensure data formats suit Power BI reporting; collaborate with BI developers.
• Partner with Architects and Analytics teams; contribute to code reviews and sprint planning.

Skills

AWS

Python

SQL

PySpark

Terraform

CI/CD

Data modeling

Data warehousing

Power BI integration

UNIX/Linux

Education

Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field

Tools

AWS Glue

AWS Step Functions

AWS Lambda

Amazon S3

Amazon Athena

Amazon SNS

Amazon SQS

Amazon Redshift

Job description

Brief Job Overview

Client is seeking a high-energy, hands-on Data Engineer to design, build, and maintain scalable, production-grade data pipelines across our AWS-based data platform. This role will own end-to-end pipeline delivery—from enterprise system ingestion to creating analytics-ready datasets—while working closely with Data Architects, Analytics teams, and business stakeholders. You will play a key role in ensuring high-quality, trusted data is accessible across the organization to support analytics and strategic decision-making. As part of our core data platform team, you will contribute to the continued evolution of our AWS Lakehouse architecture and help shape best practices.

The Senior Data Engineer has the following responsibilities:

Data Pipeline Development: Design, develop, and maintain scalable ELT pipelines with reusable ingestion frameworks supporting batch and event-driven workflows across diverse sources including ERPs, APIs, vendor files, and relational databases.

AWS Cloud Data Engineering: Build and optimize AWS-native data pipelines implementing a Medallion Lakehouse architecture with a focus on performance and cost efficiency.

Infrastructure, CI/CD & DevOps: Provision and manage AWS infrastructure via Terraform, maintaining CI/CD pipelines with GitOps practices, automated testing, logging, alerting, and recovery mechanisms for production systems.

Data Modeling & Transformation: Design and implement dimensional models, Gold-layer datasets, and scalable vendor file ingestion patterns using SQL, Python, and PySpark, with robust handling of schema drift, audit tracking, and query optimization through partitioning, clustering, and materialization strategies.

Data Quality & Governance: Implement automated data validation, anomaly detection, and lineage tracking within pipelines while supporting metadata management.

Reporting & Power BI Support: Troubleshoot and resolve complex data issues while ensuring pipelines deliver data in formats optimized for Power BI, collaborating with Power BI developers to align data structures with reporting requirements and resolve dashboard-related data issues.

Collaboration & Continuous Improvement: Partner with Architects and Analytics teams to translate business needs into technical solutions, actively contributing to code reviews, architecture discussions, sprint planning, etc.

The successful candidate will have a demonstrated understanding of our mission, commitment to excellence through inclusive and equitable behaviors and practices, ability to quickly build credibility with stakeholders, along with the following competencies and experience:

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent combination of education and experience.
• Minimum 7 years of data engineering experience, with strong expertise in AWS services and hands‑on work with big data and distributed data technologies in the AWS cloud environment.
• Strong proficiency in Python and SQL with experience in PySpark/Spark.
• Hands‑on experience building and operating data pipelines on AWS (Glue, Step Functions, Lambda, S3, Athena, SNS, SQS, Redshift etc.).
• Demonstrated experience with event‑driven pipeline architectures.
• Experience ingesting and processing vendor‑supplied data files on a scale.
• Practical experience implementing Medallion (Bronze/Silver/Gold) architecture in a data lake or Lakehouse environment.
• Experience with Terraform for cloud infrastructure provisioning.
• Experience with CI/CD tooling (e.g., Bitbucket, GitHub, AWS CodePipeline) for data pipeline deployments.
• Strong grasp of data warehousing concepts (star schema, dimensional modeling, SCD).
• Experience in integrating data with Microsoft Power BI or similar BI tools.
• Experience with UNIX/Linux including basic commands and shell scripting.
• Production experience with monitoring, troubleshooting, and on‑call support for data pipelines.
• Experience with Agile engineering practices.

Additional Desired Preferences

• Hands‑on integration experience with Oracle EBS as a data source.
• Experience with data quality frameworks (e.g., Great Expectations, dbt tests).
• Experience with AWS CDK or CloudFormation in addition to Terraform.
• Knowledge of data catalog and data lineage tools (ex. Alation, Collibra).
• AWS certifications (e.g., AWS Certified Data Engineer, AWS Certified Solutions Architect).