Associate Principal - Data Engineering

LTM; Irving (TX); On-site; USD 125,000 - 180,000; Full time

Benefits offered by this job

Comprehensive Medical Plan

Disability Coverage

401(k) Plan with Company match

Life Insurance

Vacation Time, Sick Leave, Paid Holyd?

Paid Parental Leave

Job summary

LTIMindtree in Irving, Texas is seeking a Senior Big Data Developer to design and optimize scalable data processing solutions within the Databricks SparkSQL Python ecosystem. The ideal candidate has deep expertise in Python, Spark SQL, and Spark-based pipelines and can translate complex business requirements into robust data architectures.

You will build and maintain pipelines using Apache Spark with Python and Scala, collaborate with data engineers and scientists, and optimize jobs for

Qualifications

• 10+ years of experience in Python and Spark SQL.
• Experience designing scalable big data processing solutions in Databricks ecosystem.
• Strong collaboration with cross-functional teams including data engineers and data scientists.

Responsibilities

• Design, develop and deploy complex Spark-based data pipelines.
• Optimize Spark jobs for performance and scalability in distributed environments.
• Mentor junior engineers and conduct code reviews.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Ensure data quality across diverse data sources.

Skills

Python

Spark SQL

Scala

Databricks

Big data processing

Tools

Apache Spark

SparkSQL

Databricks

Job description

Job Summary

Seeking a Senior professional with 10 years of experience in Python and Spark SQL to design and optimize scalable big data processing solutions within the DatabricksSparkSQLPython ecosystem

Role Description

Job Title: Senior Big data Developer

Job Location: Irving, Texas

Job Summary: Seeking a Senior professional with 10 years of experience in Python and Spark SQL to design and optimize scalable big data processing solutions within the DatabricksSparkSQLPython ecosystem

Job Description: Develop and maintain scalable data processing pipelines using Apache Spark with Python and Scala. Utilize SparkSQL to analyze large datasets and extract meaningful insights supporting datadriven decision making. Collaborate with crossfunctional teams to integrate big data solutions into existing systems. Optimize Spark jobs for performance and efficiency in distributed computing environments. Ensure data quality and consistency across various data sources. Stay updated with the latest advancements in Spark Scala Python and related big data technologies to continuously improve processing capabilities.

Roles and Responsibilities: Design develop and deploy complex data processing workflows leveraging Apache Spark Scala Python and SparkSQL. Write efficient reusable and maintainable code for Spark applications. Troubleshoot and resolve performance bottlenecks in Spark jobs and data pipelines. Collaborate with data engineers data scientists and stakeholders to translate business requirements into technical solutions. Conduct code reviews and provide mentorship to junior team members on best practices and coding standards. Participate in architectural discussions and contribute to the evolution of big data platforms. Monitor and maintain the health availability and reliability of Spark clusters and data services. Karat Interview mandate

Other Details

Variable Compensation: This position includes a bonus or variable payout component based on individual and organizational performance. Actual compensation within the range will be dependent upon the individual's skills, experience, performance and internal equity.

Benefits And Perks

• Comprehensive Medical Plan Covering Medical, Dental, Vision
• Short Term and Long-Term Disability Coverage
• 401(k) Plan with Company match
• Life Insurance
• Vacation Time, Sick Leave, Paid Holidays
• Paid Paternity and Maternity Leave

The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation.

Disclaimer

The compensation and benefits information provided herein is accurate as of the date of this posting.

Equal Opportunity Statement

LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.