Mid-Level Data Engineer (On-Site in Washington, DC)

Agile5 Technologies, Inc.; Washington (Washington County); Hybrid; USD 68,000 - 152,000; Full time

Job summary

Agile5 Technologies, Inc. in Washington, DC is seeking a Mid-Level Data Engineer to work on-site and support data migration, pipeline modernization, and enterprise lakehouse initiatives.

You will convert legacy Informatica artifacts to Python/PySpark, migrate schemas, and build automated reconciliation pipelines in a secure federal environment. Work closely with senior engineers and database managers, enforcing data quality and version control while contributing to CI/CD, Delta Lake, and

Qualifications

• Proficiency in Python for data transformation and pipeline development.
• Strong SQL skills for query development and schema analysis.
• Experience with ETL/ELT and data migration methodologies.
• Familiarity with cloud data platforms (AWS, Azure, or GCP).

Responsibilities

• Execute daily data migration operations and profile data.
• Convert Informatica mappings to Python/PySpark with preserved business logic.
• Migrate legacy Hive tables to Delta Lake on S3 using Databricks tools.
• Build automated reconciliation scripts and CDC pipelines for synchronization.
• Collaborate in Agile ceremonies and peer code reviews.

Skills

Python

SQL

Data migration

ETL/ELT

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Azure DevOps

Git

Delta Lake

Job description

If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.

Mid-Level Data Engineer (On-Site in Washington, DC)

Washington, DC, US

Salary Range: $68,000.00 To $152,000.00 Annually

About Agile5: Agile5 Technologies, Inc., is a Woman-Owned Small Business (WOSB) and Information Technology (IT) services firm that specializes in the design, development, testing, integration, and maintenance of enterprise software systems. We believe our employees are the company’s most valuable asset. We are invested in seeing our employees grow in their careers, while maintaining a work/life balance. We have an immediate, full-time need for a skilled, energetic, and driven Mid-Level Data Engineer.

Description: The Mid-Level Data Engineer will support data migration, pipeline engineering, and modernization efforts for enterprise data lakehouse architectures. This role involves converting legacy Informatica artifacts into clean Python/PySpark code, migrating database schemas, and building automated data reconciliation pipelines. Working closely with senior engineering leadership and database managers, the ideal candidate will enforce high standards of data quality, data validation, and version control in a secure federal environment.

Mid-Level Data Engineer Job Duties

• Execute daily data migration operations including data profiling, schema mapping, pipeline conversion, and automated reconciliation for Low and Medium complexity Informatica artifacts.
• Convert Informatica mappings into well-documented Python/PySpark code, ensuring all business logic and data quality controls are preserved.
• Migrate legacy Hive tables to Delta Lake format on S3 using Databricks ingestion tools.
• Build and execute automated data reconciliation scripts to validate migration accuracy and establish Change Data Capture (CDC) pipelines for ongoing synchronization.
• Commit all converted code into Azure DevOps with clear documentation and inline comments while maintaining Unity Catalog configurations.
• Perform daily data profiling and side-by-side validation within legacy enclave environments.
• Support Power BI and ESRI integration testing and validation.
• Participate actively in peer code reviews, daily Agile ceremonies, and collaborative data validation sessions.
• Contribute to Data Quality Assessment Reports and support training and knowledge transfer activities.
• Performs other duties as assigned.

Security Clearance Requirements

Public Trust / Tier 4 Eligible: No clearance required to apply; must be a U.S. citizen willing to undergo a background check to obtain a Public Trust / Tier 4 clearance.

Experience Requirements

Minimum experience required varies by degree level: PhD with 0 years; Master's degree with 3 years; Bachelor's degree with 5 years; or High School Diploma with 9 years of relevant experience.

Proficiency in Python for data transformation and pipeline development, as well as SQL for query development and schema analysis.

Experience with ETL/ELT processes, data migration methodologies, and cloud data platforms (AWS, Azure, or GCP).

Familiarity with version control systems (Azure DevOps, Git) and data quality concepts including profiling, cleansing, and reconciliation.

Education Requirements

Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field is preferred (or equivalent combination of education and experience).

Desired Skills / Qualifications

• Experience with Databricks (notebooks, jobs, workspace navigation), PySpark, Apache Spark, and Delta Lake or Apache Iceberg table formats.
• Proven track record converting visual ETL tools (Informatica, Talend, SSIS) to code-based pipelines.
• Experience with Hive, HiveQL, or Hadoop ecosystem components.
• Familiarity with federal IT environments, security requirements, and CI/CD pipelines for data engineering workflows.

Location

Washington, DC

Status

Full time

Schedule

Day shift, Monday–Friday

Physical Requirements

Must be able to remain in a stationary position for long durations of time. Also, must be able to continuously operate a computer and other office productivity machinery.

Travel Required

No

This job description is subject to change at any time.

We are an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability status, protected veteran status, or any other characteristic protected by law.