Senior Data Engineer

SRM Technologies LLC

United States

Remote

USD 150,000 - 190,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

SRM Technologies LLC is seeking a Senior Data Engineer to design, develop, and maintain scalable cloud data solutions using Python, PySpark, Snowflake, dbt, Iceberg, and AWS. The role focuses on enterprise-scale data platforms, lakehouse architectures, ETL/ELT pipelines, data modeling, and performance optimization.

The ideal candidate will own complex data-engineering components, contribute to technical design decisions, troubleshoot production issues, and mentor junior engineers in a

Qualifications

  • 8+ years of overall Data Engineering experience in enterprise-scale cloud platforms.
  • Hands-on experience with modern lakehouse architectures, ETL/ELT pipelines, and production-grade engineering practices.

Responsibilities

  • Design, develop, test, and maintain scalable ETL/ELT data pipelines.
  • Develop production-quality data-processing solutions using Python and PySpark.
  • Build reusable frameworks for ingestion, transformation, validation, and publishing of data.
  • Process large structured, semi-structured, and distributed datasets.
  • Implement incremental and batch-processing patterns where appropriate.

Skills

Python
PySpark
Snowflake
dbt
Apache Iceberg
AWS Data Services
SQL
Data Lake / Lakehouse
Airflow

Education

Bachelor's or Master's degree in Computer Science / IT / Engineering

Tools

Snowflake
dbt
AWS
Apache Airflow
Spark
Iceberg
Terraform
Docker

Job description

Preferred: AWS Glue, S3, EMR, Lambda, Airflow, Snowpipe/Snowpark, CI/CD, Terraform, Data Modeling

Focus: Cloud Data Engineering, Lakehouse, Data Transformation, Performance Optimization and Production Engineering

Detailed information:

Experience: 8+ years of overall Data Engineering experience , with strong hands-on experience building enterprise-scale cloud data platforms and pipelines.

Primary Skills:
  • Python
  • Snowflake
  • dbt (Data Build Tool)
  • AWS Data Services
  • Advanced SQL
  • Data Lake / Lakehouse architecture
Secondary / Preferred Skills:
  • AWS services such as:
  • S3
  • AWS Glue
  • EMR
  • Lambda
  • Step Functions
  • CloudWatch
  • IAM
  • Apache Airflow or other workflow orchestration tools
  • Snowflake performance optimization and cost optimization
  • Snowpipe / Snowpark
  • Spark performance tuning
  • Data modeling and dimensional modeling
  • Parquet and other columnar data formats
  • Data quality frameworks and automated validation
  • Infrastructure as Code such as Terraform or AWS CDK
  • Docker / containerization
  • Data governance, lineage, security, and access control
  • Agile/Scrum delivery experience
Job Description:

We are looking for a Senior Data Engineer with strong hands-on expertise in Python, PySpark, Snowflake, dbt, Apache Iceberg, and AWS to design, develop, and maintain scalable enterprise data solutions.

The candidate should have strong experience working with high-volume data processing, cloud-based data platforms, modern lakehouse architectures, ETL/ELT pipelines, data modeling, performance optimization, and production-grade engineering practices.

The ideal candidate should be capable of independently owning complex data-engineering components, contributing to technical design and architecture decisions, troubleshooting production issues, and providing technical guidance to other engineers.

Key Responsibilities:
  • Design, develop, test, and maintain scalable ETL/ELT data pipelines .
  • Develop production-quality data-processing solutions using Python and PySpark .
  • Build reusable frameworks and components for ingestion, transformation, validation, and publishing of data.
  • Process large structured, semi-structured, and distributed datasets.
  • Implement incremental and batch-processing patterns where appropriate.
2. Snowflake Development
  • Design and develop scalable data solutions using Snowflake .
  • Develop complex SQL transformations, data models, views, and reusable data structures.
  • Optimize Snowflake workloads for performance, scalability, and cost.
  • Implement appropriate data-loading and transformation patterns between AWS data platforms and Snowflake.
  • Troubleshoot performance and data-quality issues across Snowflake workloads.
3. dbt Development
  • Build and maintain transformation pipelines using dbt .
  • Implement dbt tests and documentation.
  • Follow appropriate development practices for source, staging, intermediate, and business-layer transformations.
  • Support automated deployment and CI/CD practices for dbt projects.
  • Design and implement data-lake and lakehouse solutions using Apache Iceberg .
  • Build scalable table structures for large analytical datasets.
  • Work with partitioning, schema evolution, incremental processing, and table-maintenance strategies.
  • Integrate Iceberg-based datasets with Spark and AWS-based data-processing services.
  • Ensure efficient storage and query patterns for high-volume datasets.
  • Design and implement cloud-native data solutions on AWS .
  • Build data-processing workloads leveraging services such as S3, Glue, EMR and Lambda where appropriate.
  • Implement secure access patterns using AWS IAM.
  • Monitor data workloads and troubleshoot operational issues.
  • Participate in designing scalable, reliable, secure, and cost-efficient cloud data architectures.
6. Performance & Scalability
  • Diagnose and optimize Spark/PySpark jobs , SQL queries, Snowflake workloads, and data pipelines.
  • Identify bottlenecks involving compute, storage, partitioning, data skew, transformations, and queries.
  • Design solutions capable of supporting increasing data volumes without unnecessary infrastructure cost.
7. Data Quality & Governance
  • Implement automated data-quality checks across ingestion and transformation layers.
  • Establish proper logging, monitoring, exception handling, and reconciliation mechanisms.
  • Follow organizational standards for data security, governance, lineage, and access controls.
  • Ensure production pipelines are reliable, auditable, and maintainable.
  • Write clean, modular, reusable, testable, and maintainable code.
  • Perform code reviews and enforce engineering standards.
  • Implement unit, integration, and data-validation testing.
  • Use Git-based version control and CI/CD practices.
  • Create and maintain appropriate technical documentation.
9. Senior-Level Responsibilities
  • Independently drive technically complex data-engineering requirements from design through production deployment.
  • Participate in solution design and architecture discussions.
  • Evaluate alternative implementation approaches and recommend appropriate solutions.
  • Troubleshoot complex production and performance issues.
  • Mentor junior and mid-level data engineers.
  • Collaborate with Architects, Product Owners, Business Analysts, Data Scientists, QA, DevOps, and application teams.
  • Translate business/data requirements into scalable technical solutions.
  • Identify technical risks and proactively recommend improvements.
Core Skills Expected

A strong candidate should demonstrate deep hands-on capability , not merely theoretical exposure, in the following areas:

Area
Python

Large-scale distributed processing, optimization and troubleshooting

PySpark

Large-scale distributed processing, optimization and troubleshooting

Snowflake

Development, modeling, optimization and performance tuning

dbt

Models, tests, macros, documentation and deployment practices

Lakehouse/table design, partitioning, schema evolution and optimization

AWS
SQL

Advanced SQL, query optimization and analytical processing

ETL/ELT, batch/incremental pipelines, data quality and orchestration

Data Lake, Data Warehouse and Lakehouse concepts

Git, testing, code reviews, CI/CD and production support

Preferred Qualifications
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
  • Experience migrating legacy data workloads to modern AWS/Snowflake architectures.
  • Experience working with very large datasets and distributed processing.
  • Knowledge of data security and governance practices.
  • Experience working in Agile delivery environments.
  • AWS and/or Snowflake certification is an added advantage.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Sr. Data Engineer
Sr. Data Engineer

TechDigital Group • Detroit (MI)

On-site
USD 120,000 - 180,000
Lead Software Engineer
Lead Software Engineer

Virtusa Corporation • New York (NY)

On-site
USD 150,000 - 210,000
Snowflake Data Engineer
Snowflake Data Engineer

NewVision • United States

Remote
USD 140,000 - 200,000
Senior Full-Stack AWS Data Engineer
Senior Full-Stack AWS Data Engineer

Virtue America • Pennsylvania

On-site
USD 140,000 - 190,000
Data Engineer
Data Engineer

Tata Consultancy Services Limited • Detroit (MI)

On-site
USD 70,000 - 110,000
Senior Data Engineer
Senior Data Engineer

Anblicks • Dallas (TX)

On-site
USD 100,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Compunnel Inc. • Charlotte (NC)

On-site
USD 120,000 - 150,000
Sr. Data Engineer
Sr. Data Engineer

Veriipro • Boston (MA)

On-site
USD 140,000 - 180,000
Senior Big Data Engineer — AWS, Spark & Snowflake Pipelines
Senior Big Data Engineer — AWS, Spark & Snowflake Pipelines

Compunnel Inc. • Costa Mesa (CA)

Remote
USD 120,000 - 160,000
Data Engineer
Data Engineer

Tata Consultancy Services Limited • Richardson (TX)

On-site
USD 130,000 - 145,000