Data Engineer

AEM Corporation; Washington (District of Columbia); On-site; USD 120,000 - 160,000; Full time

Job summary

AEM Corporation is seeking a Data Engineer to design, build, and operate scalable data pipelines and lakehouse solutions for Federal clients using Azure services such as Synapse, ADLS Gen2, Functions, and Logic Apps. You will work closely with product owners, analysts, and stakeholders in a Scrum environment.

The role emphasizes hands-on data engineering, data quality and governance, CI/CD, and observability.

Qualifications

• 5+ years of professional data engineering experience and a bachelor's degree (or 3+ years with a related advanced degree).
• Strong Python (pandas, PySpark) and SQL; experience building reproducible, testable data transformations.
• Hands-on expertise with Azure Synapse Analytics (Notebooks/SQL/Spark) and ADLS Gen2.
• Production experience with Synapse or Azure Data Factory, Functions, and Logic Apps for event-driven orchestration.
• Solid data architecture fundamentals (lakehouse patterns, partitioning, schema management) and performance tuning at scale.
• Data quality, lineage, and governance mindset; familiarity with Microsoft Purview or equivalent catalog/lineage tooling.
• Version control (Git) and CI/CD practices (e.g., Azure DevOps) for data solutions.
• Excellent communication skills and a collaborative, help-seeking/help-giving approach in a Scrum team.

Responsibilities

• Design and implement secure, scalable data pipelines in Azure to ingest, profile, transform, improve data quality, and publish trusted data.
• Engineer robust lake storage on ADLS Gen2 using Parquet, Iceberg, partitioning techniques, and performance optimization techniques.
• Establish data quality controls: profiling, validation rules, reproducible transformations, and automated tests with monitoring/alerting.
• Operationalize pipelines with CI/CD, environment promotion, secrets management, and run-time observability.
• Model and serve data for analytics/BI (e.g., Power BI), including row-level security and refresh orchestration.
• Document design decisions, lineage, and data contracts; contribute to catalog/metadata management (Purview).
• Collaborate within a Scrum team—participate in sprint planning, reviews, and retros—partner with product owners/stakeholders to refine user stories.
• Continuously improve team ways of working: propose automation, standardization, and quality practices that increase throughput and reliability.

Skills

Python

SQL

Data engineering

PySpark

Communication

Team collaboration

Git

Azure DevOps

Data governance

Education

Bachelor's degree

Tools

Azure Synapse Analytics

ADLS Gen2

Azure Data Factory

Azure Functions

Logic Apps

Power BI

Microsoft Purview

Git

Azure DevOps

Job description

Data Engineer

Position Summary

Join our team as a Data Engineer and become a key contributor to building trusted, high-quality data products for Federal clients. You will design, implement, and operate scalable data pipelines and lakehouse solutions using Microsoft Azure services including, Azure Synapse, ADLS Gen2, and Azure Functions, and Logic apps. You will collaborate closely with product owners, analysts, and stakeholders in a Scrum environment. The ideal candidate is hands‑on, curious, collaborative, comfortable asking for help, offering help, and proposing ideas that improve team velocity and value.

What You’ll Do

• Design and implement secure, scalable data pipelines in Azure (Synapse Notebooks, Python, Pandas, pySpark, Synapse Pipelines, Azure Functions, Logic Apps, etc.) to ingest, profile, transform, improve data quality, and publish trusted data.
• Engineer robust lake storage on ADLS Gen2 using Parquet, Iceberg, partitioning techniques, and performance optimization techniques.
• Establish data quality controls: profiling, validation rules, reproducible transformations, and automated tests with monitoring/alerting.
• Operationalize pipelines with CI/CD, environment promotion, secrets management, and run‑time observability (e.g., Azure DevOps).
• Model and serve data for analytics/BI (e.g., Power BI), including row‑level security and refresh orchestration.
• Document design decisions, lineage, and data contracts; contribute to catalog/metadata management (e.g., Microsoft Purview).
• Collaborate within a Scrum team—participate in sprint planning, reviews, and retros—partner with product owners/stakeholders to refine user stories.
• Continuously improve team ways of working: propose automation, standardization, and quality practices that increase throughput and reliability.

What You’ll Bring (Required Qualifications)

• U.S. citizen with the ability to obtain a Public Trust background investigation.
• 5+ years of professional data engineering experience and a bachelor's degree (or 3+ years with a related advanced degree).
• Strong Python (pandas, PySpark) and SQL; experience building reproducible, testable data transformations.
• Hands‑on expertise with Azure Synapse Analytics (Notebooks/SQL/Spark) and ADLS Gen2.
• Production experience with Synapse or Azure Data Factory, Functions, and Logic Apps for event‑driven orchestration.
• Solid data architecture fundamentals (lakehouse patterns, partitioning, schema management) and performance tuning at scale.
• Data quality, lineage, and governance mindset; familiarity with Microsoft Purview or equivalent catalog/lineage tooling.
• Version control (Git) and CI/CD practices (e.g., Azure DevOps) for data solutions.
• Excellent communication skills and a collaborative, help‑seeking/help‑giving approach in a Scrum team.

Preferred Qualifications

• Experience integrating with AWS data services (S3, Glue/Glue Data Catalog, Lambda) and/or cross‑cloud data patterns.
• Experience enabling BI/analytics (Power BI, model design, RLS) and supporting analysts with performant datasets.
• Experience building observability for data systems (usage/cost telemetry, data SLAs, pipeline health dashboards).
• Familiarity with MDM and reference data concepts and enterprise governance.
• Relevant certifications: Microsoft Azure Data Engineer (DP‑203), Azure Enterprise Data Analyst, or AWS data certifications.
• Existing Federal Government Clearance.

How We Work

• Scrum‑aligned: contribute to clear user stories, sizing, and acceptance criteria; continually update documentation for work items in progress; participate actively in planning, reviews, and retros.
• Team‑first: seek help early, offer help freely, and elevate team standards through code reviews, pairing, and documentation.
• Outcome‑driven: measure impact (performance, quality, cycle time) and iterate on process and tooling to increase value delivery.

AEM and its affiliates are equal opportunity employers, and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status, or any other characteristic protected by law. - AA M/F/Vet/Disability

Federal law requires AEM and its affiliates to verify identity and employment eligibility with information from your Form I-9. The E‑Verify system is used.

The California Consumer Privacy Act of 2018 ("CCPA") imposes specific obligations on businesses processing personal information of California residents. Pursuant to the CCPA, AEM and its affiliates are required to provide applicants who are California residents a notice, used at or before the point of collection of such personal information, which identifies the categories of personal information that may be collected and why AEM and its affiliates collect such information. Applicants can find information about AEM's privacy policy and CCPA here.