Data Engineer

SteerBridge, Inc.; Vienna (VA); On-site; USD 130,000 - 180,000; Full time

Job summary

Jobtailor is seeking a senior data engineer to build and optimize batch and real-time data ingestion pipelines in a cloud-first environment. You will design scalable ETL processes across distributed systems and manage data lake architecture for structured and unstructured data.

You will utilize dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, and related services to transform data, orchestrate workflows, and ensure data quality.

Qualifications

• Bachelor's degree in Computer Science or a related field; minimum 3 years of relevant work experience.
• U.S. citizenship; active security clearance or ability to obtain one.

Responsibilities

• Build, optimize, and maintain batch and real-time data ingestion pipelines.
• Lead design and implementation of scalable ETL processes across distributed systems.
• Develop and manage data lake architecture for structured and unstructured data.
• Perform data transformation using dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, and related tools.
• Write performant SQL queries and Python/JavaScript scripts for data parsing and cleansing.
• Conduct data profiling, linkage, validation, and quality checks.

Skills

ETL Frameworks

Data Ingestion

Data Transformation

SQL Proficiency

Python Programming

Pandas

PySpark

AWS Glue

Azure Data Factory

Databricks

Education

Bachelor's degree in Computer Science or related field

Tools

Dbt

Informatica

Azure Data Factory

Databricks

AWS Glue

AWS EventBridge

S3 Event Notifications

Pandas

PySpark

Azure Blob

Job description

Build, optimize, and maintain batch and real-time data ingestion pipelines, including ETL/ELT processes for structured and unstructured data
Lead design and implementation of scalable ETL processes across complex, distributed systems
Develop and manage data lake architecture for structured and unstructured data
Use dbt, Informatica, Azure Data Factory, Databricks, AWS Glue, AWS EventBridge, and S3 Event Notifications for data transformation, workflow automation, orchestration, and scheduling
Write performant SQL queries and Python/JavaScript scripts for data parsing, ingestion, and cleanup
Conduct data profiling, linkage, validation, and quality checks across diverse sources
Ensure data quality, lineage, governance, and compliance across systems
Enable cloud-based data processing using AWS S3 and Azure Blob and support API integration
Collaborate with data science, engineering, and stakeholder teams to deliver data products and support reporting and model development
Mentor junior engineers and provide technical guidance and peer reviews
Maintain technical documentation for pipelines, data structures, infrastructure, standards, and data product specifications
Support secure data governance practices and performance tuning in modern cloud platforms

Requirements

• Must be a U.S. Citizen
• Bachelor's degree in Computer Science or a related field, with a minimum of three (3) years of relevant work experience
• An active security clearance, or the ability to obtain one, is required
• Strong proficiency in Python and SQL
• Hands-on experience with Pandas, PySpark, and AWS Glue
• Deep understanding of ETL frameworks, data lake design, and cloud-based architecture
• Familiarity with real-time and batch data processing tools and methodologies
• Experience with data governance, security compliance, and maintaining data quality standards
• Must be local to the Vienna, VA area and able to work on-site at the Vienna, VA office (3-5 days/week); hybrid arrangements available at supervisors' discretion

Core Competencies

Demonstrates expertise in building and optimizing ETL processes, managing data lake architecture, and ensuring data quality and compliance in cloud environments. Proficient in Python and SQL, with hands-on experience in data transformation tools and methodologies.

Highest-signal resume keywords

• ETL Frameworks
• Data Lake Design
• Python Programming
• SQL Proficiency
• Data Governance

ATS Optimization Keywords

Hard Skills

• ETL Processes
• Data Ingestion
• Data Transformation
• SQL Queries
• Python Scripting
• Data Profiling
• Data Quality Checks
• Cloud-Based Architecture
• Real-Time Data Processing
• Batch Data Processing

Soft Skills

• Mentoring
• Collaboration
• Technical Guidance

Certifications & Qualifications

• Active Security Clearance

Industry Keywords

• Data Governance
• Data Quality Standards
• Cloud Platforms
• Data Compliance
• Data Architecture

Tools & Technologies

• Dbt
• Informatica
• Azure Data Factory
• Databricks
• AWS Glue
• AWS EventBridge
• S3 Event Notifications
• Pandas
• PySpark
• Azure Blob