Data Engineer

Eliassen Group; Westlake (TX); On-site; Confidential; Full time

Benefits offered by this job

Medical, Dental, and Vision benefits

401k with company matching

Life insurance

Job summary

A leading U.S. financial services organization is seeking a skilled data engineer to design and maintain data pipelines. Located in Westlake, TX, this hybrid role demands experience with Airflow, Spark, and Kubernetes, alongside strong Python programming skills. The ideal candidate will work on cloud platforms (AWS and Azure), implementing Infrastructure as Code using Terraform and deploying containerized applications. A comprehensive benefits package is offered alongside competitive hourly rates.

Qualifications

• Proven experience with Airflow, Spark, and Kubernetes.
• Strong programming skills in Python.
• Experience configuring alerts and monitors for high availability.
• API integration experience preferred.

Responsibilities

• Design, build, and maintain data pipelines and platform services using Airflow and Spark.
• Develop Python-based solutions for data ingestion and transformation.
• Configure and deploy containerized applications on Kubernetes in AWS and Azure.
• Establish CI/CD pipelines with tools like GitHub and Jenkins.

Skills

Python programming

Airflow

Spark

Kubernetes

Terraform

RDBMS concepts

CI/CD tools

AWS

Azure

Problem-solving skills

Education

Public cloud certification in Azure or AWS

Tools

Terraform

CloudFormation

Jenkins

GitHub

Artifactory

Job description

Hybrid every other week onsite in Westlake, TX.

Our client is a leading U.S. financial services organization serving individuals and institutions with a broad portfolio of investment products and technology-driven solutions. Headquartered in Boston with major hubs across the U.S., the company operates at scale and invests in innovation across enterprise infrastructure, data, and customer platforms.

Due to client requirements, applicants must be willing and able to work on a w2 basis. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $63.00 to $73.00/hr. w2

Responsibilities

• Design, build, and maintain data pipelines and platform services using Airflow and Spark.
• Develop Python-based solutions to enable data ingestion, transformation, and orchestration.
• Configure and deploy containerized applications on Kubernetes in AWS and Azure.
• Implement Infrastructure as Code for platform engineering using tools such as Terraform, CloudFormation, or ARM templates.
• Apply RDBMS concepts to design and optimize data models and queries.
• Establish CI/CD pipelines with tools such as GitHub, Artifactory, and Jenkins.
• Configure monitoring and alerting to ensure a highly available and reliable platform.
• Collaborate with cross-functional teams to address evolving requirements and deliver scalable solutions.
• Communicate technical decisions and influence best practices within the team.

Experience Requirements

• Proven experience with Airflow, Spark, and Kubernetes.
• Strong programming skills in Python.
• Working understanding of cloud platforms, preferably AWS and Azure.
• Hands-on experience with Infrastructure as Code using Terraform, CloudFormation, or ARM.
• Solid grasp of RDBMS concepts and hands-on experience.
• Strong knowledge of configuring and deploying containerized applications in public cloud environments.
• Experience with CI/CD tools such as GitHub, Artifactory, and Jenkins.
• Experience configuring alerts and monitors for high availability.
• Strong problem-solving skills and experience working with evolving requirements.
• Strong communication and influencing skills.
• API integration experience (preferred).
• Public cloud certification in Azure or AWS (preferred).
• Typescript programming experience (preferred).

Education Requirements

• Public cloud certification in Azure or AWS (preferred).