Data Engineer: Large-Scale ETL, Spark & AWS

DataVue LLC; United States; On-site; USD 110,000 - 180,000; Full time

Job summary

DataVue LLC seeks a hands-on Data Engineer to build and operate large-scale data workflows, ingesting, cleansing, matching, profiling, and integrating datasets. You will optimize pipelines, automate scheduling, and ensure data quality for analytics and modeling.

Responsibilities include engineering Python/Spark ETL processes in AWS, developing match keys, maintaining data dictionaries, and supporting production data requests. Bachelor’s degree and US work authorization required.

Qualifications

• Advanced Python engineering for large-scale processing and automation.
• Advanced SQL with complex joins and window functions.
• Hands-on Spark/PySpark for distributed data jobs.
• Production AWS experience (S3, Glue/EMR) and secure data access patterns.
• Experience orchestrating multi-step data workflows (Airflow, Dagster, Step Functions, or Glue) including retries.
• Ability to interpret data dictionaries and communicate in a remote team.
• Bachelor's degree; authorized to work in the United States without visa sponsorship.

Responsibilities

• Build and maintain data pipelines and automation.
• Perform data cleansing, matching, and record-linkage across sources.
• Develop attribute profiling and data quality checks.
• Design and optimize staging and downstream tables; manage SQL performance.
• Support historical data capabilities and time-series attributes.

Skills

Python

SQL

Spark/PySpark

AWS

Data pipelines

Data quality

Education

Bachelor's degree

Tools

S3

EMR

Glue

Dagster

Airflow

Step Functions

Parquet

Iceberg

Job description

DataVue LLC seeks a hands-on Data Engineer to build and operate large-scale data workflows, ingesting, cleansing, matching, profiling, and integrating datasets. You will optimize pipelines, automate scheduling, and ensure data quality for analytics and modeling.

Responsibilities include engineering Python/Spark ETL processes in AWS, developing match keys, maintaining data dictionaries, and supporting production data requests. Bachelor’s degree and US work authorization required.