Senior Data Engineering Lead - Spark & Python

LTM; Irving (TX); On-site; USD 125,000 - 180,000; Full time

Benefits offered by this job

Comprehensive Medical Plan

Disability Coverage

401(k) Plan with Company match

Life Insurance

Vacation Time, Sick Leave, Paid Holyd?

Paid Parental Leave

Job summary

LTIMindtree in Irving, Texas is seeking a Senior Big Data Developer to design and optimize scalable data processing solutions within the Databricks SparkSQL Python ecosystem. The ideal candidate has deep expertise in Python, Spark SQL, and Spark-based pipelines and can translate complex business requirements into robust data architectures.

You will build and maintain pipelines using Apache Spark with Python and Scala, collaborate with data engineers and scientists, and optimize jobs for

Qualifications

• 10+ years of experience in Python and Spark SQL.
• Experience designing scalable big data processing solutions in Databricks ecosystem.
• Strong collaboration with cross-functional teams including data engineers and data scientists.

Responsibilities

• Design, develop and deploy complex Spark-based data pipelines.
• Optimize Spark jobs for performance and scalability in distributed environments.
• Mentor junior engineers and conduct code reviews.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Ensure data quality across diverse data sources.

Skills

Python

Spark SQL

Scala

Databricks

Big data processing

Tools

Apache Spark

SparkSQL

Databricks

Job description

LTIMindtree in Irving, Texas is seeking a Senior Big Data Developer to design and optimize scalable data processing solutions within the Databricks SparkSQL Python ecosystem. The ideal candidate has deep expertise in Python, Spark SQL, and Spark-based pipelines and can translate complex business requirements into robust data architectures.

You will build and maintain pipelines using Apache Spark with Python and Scala, collaborate with data engineers and scientists, and optimize jobs for