Senior Data Engineer: ETL, Spark & Data Pipelines

VTG

Chantilly (VA)

On-site

USD 120,000 - 170,000

Full time

4 days ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

VTG is seeking an experienced Data Engineer to design, develop, and maintain scalable data processing solutions for complex data integration, transformation, and analytics. The candidate will leverage Python, PySpark, Apache Spark, SQL databases, and ETL processes to transform large volumes of structured and unstructured data into actionable information.

Agile collaboration with customers, developers, and integration partners is expected.

Qualifications

Experience with Python, PySpark and Spark for data pipelines.
Hands-on ETL development and data integration.
Strong SQL skills with relational databases (MySQL/PostgreSQL).
Experience processing structured and unstructured data formats.
Knowledge of RegEx for parsing and transformation.

Responsibilities

Design, develop, and maintain scalable data processing and ETL pipelines.
Extract, transform, and load data from multiple sources into query-ready datasets.
Develop data mapping, transformation, and integration processes.
Process large volumes of logs into reports, metrics, and dashboards.
Optimize data workflows for performance and reliability.
Maintain ETL documentation and data dictionaries.
Collaborate with customers, developers, and integration partners in an Agile environment.
Develop and maintain data models and database queries.