Data Automation Engineer

Compunnel Inc.; Philadelphia (Philadelphia County); On-site; USD 90,000 - 120,000; Full time

Job summary

A leading tech company is seeking an Automation Engineer in Philadelphia to implement and improve data validation and quality assurance for modern data platforms. This position involves designing comprehensive data verification processes and automating data testing solutions. The ideal candidate has strong SQL and Python skills, along with experience in data lakes and pipelines. Join a collaborative team committed to continuous improvement in the banking industry.

Qualifications

• Strong SQL and Python skills focused on ETL/data testing.
• Experience with data lakes, warehouses, and pipelines.
• Exposure to Databricks, Snowflake, or PySpark.

Responsibilities

• Design and execute comprehensive data verification/validation.
• Implement automated data testing solutions using CI/CD.
• Explore new tools to improve data testing processes.

Skills

SQL

Python

ETL/Data Testing

Analytical Skills

Communication Skills

Time Management

Tools

Databricks

Snowflake

PySpark

Job description

We are seeking an Automation Engineer to implement and improve testing, data validation, and quality assurance for modern data platforms and ETL processes. This role focuses on ensuring accuracy, integrity, and reliability of data across systems while driving automation and best practices in testing.

Key Responsibilities

• ETL & Data Testing: Design and execute comprehensive data verification/validation for backend data systems and ETL processes. Perform source-to-target validation and ensure mappings/transformations are correct. Test “data in motion” across multiple systems including data lakes, warehouses, and pipelines. Implement automated data testing solutions using CI/CD pipelines, scheduling, or event triggers.
• Automation & Scripting: Hands-on SQL expertise – strong querying and validation for large datasets. Use Python (Pandas) for automating data quality checks, manipulating JSON/CSV/nested data, and validating structured/unstructured datasets. Work with modern data platforms (Databricks, Snowflake, PySpark) to support scalable validation. Develop solutions to test data visualizations using industry-standard visualization platforms. Share knowledge, assist less-experienced team members, and contribute to a collaborative environment.
• Technical Fluency & Continuous Improvement: Explore new tools, frameworks, and methods to improve data testing processes. Participate in defining best practices across the team. Support departmental and Bank-wide quality initiatives while ensuring compliance with security and policy standards.

Skills & Qualifications

• Strong SQL and Python skills with a focus on ETL/data testing and automation.
• Experience with data lakes, warehouses, and pipelines for quality validation.
• Exposure to Databricks, Snowflake, PySpark or similar data platforms.
• Strong analytical and problem-solving abilities with attention to detail.
• Excellent written/oral communication and interpersonal skills.
• Ability to work independently under minimal supervision, while mentoring peers when needed.
• Strong time management and organizational skills.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Industries

• Banking