Sr. Data Engineer

TechDigital Group; Detroit (MI); On-site; USD 120,000 - 180,000; Full time

Job summary

TechDigital Group is seeking a Senior Data Engineer to design, build, and optimize scalable data pipelines and platforms supporting analytics, reporting, and AIML use cases. The role emphasizes Snowflake on AWS, Python-based ETL/ELT development, and enterprise scheduling with Control-M, including legacy IBM DataStage ETL experience.

You will collaborate across engineering, analytics, and business teams in an Agile delivery model, delivering robust data products with governance and cost

Qualifications

• Bachelor's degree in CS, Engineering, or related field.
• 9+ years of data engineering experience with enterprise-grade data platforms and production support.

Responsibilities

• Design, develop, and maintain end to end data pipelines batch and near real time using Snowflake, AWS services, and Python.
• Build and optimize data models in Snowflake e.g., dimensional modelling, data vault, or curated data marts for analytics and downstream consumption.
• Develop and maintain ETLELT workflows using Python and IBM DataStage migrate modernize workloads where applicable.
• Implement job scheduling, monitoring, and operational support using Control M alerting, retries, SLAs, and dependency management.
• Ensure data quality, governance, lineage, and documentation standards are met across pipelines.
• Perform performance tuning and cost optimization across Snowflake and AWS query optimization, clustering, warehouse sizing, storage management.
• Partner with stakeholders Data Science AI, BI, Product, and Platform teams to enable data products and Already datasets.
• Participate in Agile ceremonies, contribute to estimation, planning, and sprint execution follow SDLC and change management processes.
• Troubleshoot production issues, perform root cause analysis, and drive preventative improvements.

Skills

Snowflake

SQL

AWS

Python

DBT

DataStage

Education

Bachelor's degree in CS/Engineering

Tools

AWS Glue

Qlik

Control-M

Airflow

Job description

Job Title

Sr. Data Engineer

Job Summary

We are looking for a Senior Data Engineer to design, build, and optimize scalable data pipelines and data platforms supporting analytics, reporting, and AIML use cases. The ideal candidate has strong hand son experience with Snowflake on AWS, Python based ETLELT development, and enterprise scheduling orchestration tools like Control M, along with legacy enterprise ETL experience in IBM DataStage. You will collaborate across engineering, analytics, and business teams in an Agile delivery model.

Experience

6 to 10 Yrs

Primary Skills

Snowflake, SQL, AWS, Python, DBT & DataStage

Secondary Skills

AWS Glue, Qlik, Control-M & Airflow.

Responsibilities

• Design, develop, and maintain end to end data pipelines batch and near real time using Snowflake, AWS services, and Python.
• Build and optimize data models in Snowflake e.g., dimensional modelling, data vault, or curated data marts for analytics and downstream consumption.
• Develop and maintain ETLELT workflows using Python and IBM DataStage migrate modernize workloads where applicable.
• Implement job scheduling, monitoring, and operational support using Control M alerting, retries, SLAs, and dependency management.
• Ensure data quality, governance, lineage, and documentation standards are met across pipelines.
• Perform performance tuning and cost optimization across Snowflake and AWS query optimization, clustering, warehouse sizing, storage management.
• Partner with stakeholders Data Science AI, BI, Product, and Platform teams to enable data products and Already datasets.
• Participate in Agile ceremonies, contribute to estimation, planning, and sprint execution follow SDLC and change management processes.
• Troubleshoot production issues, perform root cause analysis, and drive preventative improvements.

Required Technical Skills

• Snowflake Strong expertise in Snowflake architecture, SQL development, performance tuning, security roles, data loading unloading, and best practices.
• AWS Handson experience with AWS data ecosystem commonly S3, IAM, CloudWatch plus services such as Glue, Lambda, EC2, Step Functions, EMR, or Kinesis as applicable.
• Python Strong Python programming for data engineering ETLELT frameworks, API ingestion, automation, unit testing, logging.
• Control M Experience designing and managing enterprise job scheduling, dependencies, calendars, SLAs, monitoring, and incident handling.
• IBM DataStage Solid experience building and maintaining DataStage jobs, handling complex transformations, and supporting production workloads.
• SQL Advanced SQL skills for transformations, optimization, and data validation across large datasets.
• CICD & Version Control Experience with Git and CICD practices for data pipelines tools may vary.
• Operational Excellence Monitoring, alerting, and production support experience in a 24x7 or business critical environment.
• Good to Have AIML exposure Experience enabling AIML pipelines or feature datasets familiarity with ML lifecycle concepts, feature engineering, or ML Ops tools processes.
• Experience with data governance metadata tools and practices catalogue, lineage, data quality frameworks.
• Exposure to streaming or event driven architectures.

Qualifications

Typical Bachelor's degree in Computer Science, Engineering, or related field or equivalent practical experience. 9 plus years of data engineering experience, including enterprise grade data platform delivery and production support.