Data Engineer

Jobtailor

Vienna (VA)

On-site

USD 130,000 - 180,000

Full time

34 hours ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Jobtailor is seeking a senior data engineer to build and optimize batch and real-time data ingestion pipelines in a cloud-first environment. You will design scalable ETL processes across distributed systems and manage data lake architecture for structured and unstructured data.

You will utilize dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, and related services to transform data, orchestrate workflows, and ensure data quality.

Qualifications

Bachelor's degree in Computer Science or a related field; minimum 3 years of relevant work experience.
U.S. citizenship; active security clearance or ability to obtain one.

Responsibilities

Build, optimize, and maintain batch and real-time data ingestion pipelines.
Lead design and implementation of scalable ETL processes across distributed systems.
Develop and manage data lake architecture for structured and unstructured data.
Perform data transformation using dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, and related tools.
Write performant SQL queries and Python/JavaScript scripts for data parsing and cleansing.
Conduct data profiling, linkage, validation, and quality checks.

Skills

ETL Frameworks

Data Ingestion

Data Transformation

SQL Proficiency

Python Programming

Pandas

PySpark

AWS Glue

Azure Data Factory

Databricks

Education

Bachelor's degree in Computer Science or related field

Tools

Dbt

Informatica

Azure Data Factory

Databricks

AWS Glue

AWS EventBridge

S3 Event Notifications

Pandas

PySpark

Azure Blob

Job description

Build, optimize, and maintain batch and real-time data ingestion pipelines, including ETL/ELT processes for structured and unstructured data
Lead design and implementation of scalable ETL processes across complex, distributed systems
Develop and manage data lake architecture for structured and unstructured data
Use dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, AWS EventBridge, and S3 Event Notifications for data transformation, workflow automation, orchestration, and scheduling
Write performant SQL queries and Python/JavaScript scripts for data parsing, ingestion, and cleanup
Conduct data profiling, linkage, validation, and quality checks across diverse sources
Ensure data quality, lineage, governance, and compliance across systems
Enable cloud-based data processing using AWS S3 and Azure Blob and support API integration
Collaborate with data science, engineering, and stakeholder teams to deliver data products and support reporting and model development
Mentor junior engineers and provide technical guidance and peer reviews
Maintain technical documentation for pipelines, data structures, infrastructure, standards, and data product specifications
Support secure data governance practices and performance tuning in modern cloud platforms

Requirements

Must be a U.S. Citizen
Bachelor's degree in Computer Science or a related field, with a minimum of three (3) years of relevant work experience
An active security clearance, or the ability to obtain one, is required
Strong proficiency in Python and SQL
Hands-on experience with Pandas, PySpark, and AWS Glue
Deep understanding of ETL frameworks, data lake design, and cloud-based architecture
Familiarity with real-time and batch data processing tools and methodologies
Experience with data governance, security compliance, and maintaining data quality standards
Must be local to the Vienna, VA area and able to work on-site at the Vienna, VA office (3-5 days/week); hybrid arrangements available at supervisors' discretion

Core Competencies

Demonstrates expertise in building and optimizing ETL processes, managing data lake architecture, and ensuring data quality and compliance in cloud environments. Proficient in Python and SQL, with hands-on experience in data transformation tools and methodologies.

Highest-signal resume keywords