Data Engineer

Eliassen Group; Dallas (TX); Hybrid; USD 74,390 - 88,166; Full time

Benefits offered by this job

Medical benefits

Dental benefits

401k with matching

Life insurance

Job summary

A consulting firm seeks a Data Engineer to develop modern data platforms using Apache Iceberg on AWS. This hybrid role focuses on building scalable data pipelines and emphasizes strong skills in data governance and distributed systems. Ideal candidates should have 4 to 10+ years of data engineering experience and proficiency with AWS services and Apache Iceberg. Benefits include medical, dental, and 401k with matching, starting at $54/hr. Candidates must be willing to work on a W2 basis.

Qualifications

• 4 to 10+ years of experience in data engineering or similar roles.
• Hands-on experience with Apache Iceberg including table design and evolution.
• Proficiency with distributed engines such as Spark, Flink, or PySpark.

Responsibilities

• Design and build Apache Iceberg–based data lakes.
• Develop scalable batch and streaming pipelines using AWS services.
• Implement CDC pipelines using Kafka Connect.

Skills

Apache Iceberg

AWS data stack (S3, EMR, Lambda, Glue)

Python

Spark

SQL

Kafka

Job description

Hybrid Every Other Week onsite in Westlake, TX

Our client seeks a Data Engineer to build a modern data platform using Apache Iceberg on AWS with Spark, Kafka, and Python. The role emphasizes scalable data pipelines, distributed processing, and lakehouse best practices to support reliable analytics.

Due to client requirements, applicants must be willing and able to work on a w2 basis. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $54.00 to $64.00/hr. w2

Responsibilities

• Design and build Apache Iceberg–based data lakes with ACID-compliant, versioned datasets.
• Implement Iceberg table evolution including schema evolution, partition specifications, and snapshot management.
• Develop best practices for Iceberg governance, metadata compaction, and performance tuning.
• Build scalable batch and streaming pipelines using AWS services such as S3, EMR, Glue, Lambda, and Step Functions.
• Develop ingestion and transformation workflows using Python, Spark, or Flink.
• Implement CDC pipelines using Kafka Connect or equivalent tooling.
• Ensure CI/CD integration with GitHub Actions or similar tooling.
• Design and operate Kafka-based streaming pipelines, including MSK.
• Build Kafka producers and consumers using Python or JVM languages.
• Implement topic partitioning, compaction, schema registry usage, and event versioning patterns.
• Design Iceberg-based data models for analytical and operational use cases.
• Implement automated data quality checks, validation rules, and anomaly detection.
• Build lineage, monitoring, alerting, and pipeline observability.
• Apply AWS security, cost optimization, and data governance best practices.
• Manage IAM, KMS, S3 lifecycle management, networking, and data encryption.
• Operationalize EMR/Glue jobs, containerized workloads, or serverless workloads.
• Partner with analytics, platform, and product teams to deliver high-quality data products.
• Participate in design reviews, architecture discussions, and roadmap planning.
• Mentor junior engineers and contribute to engineering standards.

Experience Requirements

• 4 to 10+ years of experience in data engineering or similar roles.
• Hands‑on experience with Apache Iceberg including table design, evolution, metadata management, and partitioning.
• Deep experience with AWS data stack including S3, EMR, Lambda, Glue, IAM, Step Functions, and CloudWatch.
• Proficiency with distributed engines such as Spark, Flink, or PySpark.
• Fluency in Python for data pipelines, automation, and APIs.
• Expertise building scalable ETL/ELT pipelines and real‑time streaming architectures.
• Strong SQL and data modeling expertise.
• Kafka experience including producers/consumers, schema registry, and partitioning strategies (preferred).

Eliassen Group values transparency in our recruitment practices. Please be advised that Eliassen Group utilizes artificial intelligence (AI) tools as part of its initial application screening process. You may receive email and SMS notifications from the Eliassen Virtual Recruiting Team ([e-mail], 781-808-2924) inviting you to complete a brief voice screening as part of your application process. These tools assist our hiring teams in different ways, including but not limited to, assistance in reviewing application materials to help identify candidates whose qualifications most closely match the requirements of the position. All AI‑assisted evaluations and responses are reviewed by human recruiters before any hiring decisions are made. The use of AI in our process is intended to support fairness, efficiency, and consistency, and Eliassen Group takes measures to prevent bias or discrimination in connection with its hiring practices. By proceeding, you acknowledge, agree, and consent to Eliassen Group’s use of these tools, including AI tools, as part of the application and hiring process.

Skills, experience, and other compensable factors will be considered when determining pay rate. The pay range provided in this posting reflects a W2 hourly rate; other employment options may be available that may result in pay outside of the provided range.

W2 employees of Eliassen Group who are regularly scheduled to work 30 or more hours per week are eligible for the following benefits: medical (choice of 3 plans), dental, vision, pre‑tax accounts, other voluntary benefits including life and disability insurance, 401(k) with match, and sick time if required by law in the worked‑in state/locality.

Please be advised‑ If anyone reaches out to you about an open position connected with Eliassen Group, please confirm that they have an Eliassen.com email address and never provide personal or financial information to anyone who is not clearly associated with Eliassen Group. If you have any indication of fraudulent activity, please contact [e-mail].