Big Data Consultant

EXL; Dallas (TX); Hybrid; USD 120,000 - 180,000; Full time

Job summary

EXL is seeking an experienced Data Engineering Lead to drive architecture decisions and mentor distributed teams. You will own end-to-end data pipelines, focusing on on-prem Hadoop ecosystems, Spark processing, and effective scheduling with CA7/Control-M.

The role requires strong SQL, ETL capabilities, and a passion for AI/ML concepts, with a focus on scalable data platforms that accelerate business insights.

Qualifications

• 7-10 years of experience in data engineering and technical leadership.
• Hands-on Hadoop ecosystem (Hadoop, Hive, Impala, HDFS) in on-prem environments.
• Solid Apache Spark experience with large-scale jobs.
• Hands-on with job scheduling/orchestration (CA7 or Control-M).
• Strong SQL, ETL skills and data architecture ability.
• Interest in AI/ML concepts and willingness to learn.

Responsibilities

• Lead design and architecture decisions for data platforms and pipelines.
• Mentor and guide development teams through implementation and best practices.
• Own end-to-end data engineering pipelines from ingestion to consumption.
• Collaborate with analytics and product teams to optimize data flows and performance.

Skills

Data engineering

Big Data / Hadoop

Apache Spark

ETL

SQL

Distributed systems

Mentoring

AI/ML awareness

Tools

CA7/Control-M

Job description

Our passion at EXL is enabling organizations to achieve a measurable, competitive advantage through Data Management. We partner with our clients to design and implement scalable, accessible, and innovative solutions. Our goal is to make sense of data to drive client’s business forward through data strategy, data governance and building cloud data platforms. EXL is a leading data analytics and digital operations and solutions company that partners with clients to improve business outcomes and unlock growth. By bringing together deep domain expertise with robust data, powerful analytics, cloud, artificial intelligence ("AI") and machine learning ("ML"), we create agile, scalable solutions and execute complex operations for the world’s leading corporations in industries including insurance, healthcare, banking and financial services, media, and retail, among others. Focused on driving faster decision-making and transforming operating models, EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect. Headquarters in New York, our team is over 44,500+ strong, with more than 50 offices spanning six continents.

Employment Type: Full-Time

Mode - Hybrid (2-3 days from Office)

Job Summary:

• 7-10 years of overall experience in data engineering, with a proven track record in technical leadership (design ownership, mentoring, guiding development teams).
• Big Data / Hadoop ecosystem (Hadoop, Hive, Impala, HDFS) - deep, hands-on experience in on-premises environments.
• Apache Spark - solid experience developing and tuning large-scale distributed data processing jobs.
• Job scheduling / orchestration - hands-on experience with CA7 or Control-M (or comparable enterprise schedulers).
• Strong understanding of data structures, ETL processes, and SQL.
• Extensive experience with large-scale data processing and distributed systems.
• Demonstrated ability to make sound architecture/design decisions and to mentor and support other developers.
• Exposure to AI/ML concepts or tools, with a strong willingness to learn and grow in this space.