AWS Databricks Lead

LTM; Hartford (CT); On-site; USD 110,000 - 150,000; Full time

Benefits offered by this job

Comprehensive Medical Plan Covering

Short/Long-Term Disability Coverage

401(k) Plan with Company match

Life Insurance

Vacation Time, Sick Leave, Holidays

Paid Paternity and Maternity Leave

Job summary

LTIMindtree is seeking a Databricks-focused data engineering role for our Hartford, CT onsite team. The candidate will implement Databricks-based solutions using AWS services such as S3, Lambda, and Glue, with PySpark and Python libraries to handle data engineering tasks.

The role emphasizes ETL concepts, data pipeline architecture, and end-to-end data warehousing. Collaboration with business stakeholders and mentoring of junior engineers are key responsibilities.

Qualifications

• Experience delivering Databricks-based solutions using AWS services.
• Proficiency with PySpark and Python libraries for data engineering tasks.
• Strong SQL development skills including stored procedures and data marts.

Responsibilities

• Collaborate with business users to understand challenges and provide Databricks-based solutions.
• Research and evaluate new Databricks features; conduct POCs and document best practices.
• Provide support for Spark-related issues and participate in code reviews with recommendations.
• Lead and mentor junior team members in development and troubleshooting activities.
• Oversee mid-sized teams and ensure quality and timely delivery in an Agile environment.
• Contribute to solution design discussions and produce design documents.
• Drive job cost optimization and performance tuning to enhance system efficiency.
• Support CI/CD processes and infrastructure automation to streamline deployments.
• Actively participate in Agile ceremonies and improve team collaboration and productivity.

Skills

Databricks

AWS services

PySpark

Python libraries

SQL development

ETL concepts

Agile SDLC

CI/CD

Communication skills

Tools

Terraform

GitHub

Job description

Role Description

AWS services and Databricks

Location Hartford CT onsite

Databricks Features

• Delta Lake
• Unity Catalog
• Scalable data processing frameworks

AWS Cloud Services

• Amazon S3
• AWS Lambda
• Amazon SNS
• AWS Step Functions
• AWS Glue

Job Cost Optimization Techniques

• Minimize runtimes
• Improve system performance

Python Core Libraries & PySpark

• Python core libraries
• PySpark for data engineering tasks

SQL Query Development

• SQL query development
• Stored procedures
• Creating data marts and views in RDBMS

ETL Concepts & Data Pipeline Architecture

• ETL concepts
• Endtoend data pipeline architecture
• Data warehouse and reporting layers

CICD Pipelines & Deployment Tools

• Terraform

Version Control Systems

• GitHub or equivalent platforms

Development Methodologies

• Agile methodology
• Software Development Life Cycle SDLC

Excellent communication and presentation skills with the ability to collaborate effectively with stakeholders

Roles Responsibilities

• Collaborate with business users and stakeholders to understand challenges and provide effective Databricksbased solutions
• Research and evaluate new Databricks features conduct Proof of Concepts POCs and develop best practice documentation
• Provide support for Sparkrelated issues and participate in code reviews offering recommendations for improvement
• Lead and mentor junior team members guiding them in development and troubleshooting activities
• Manage midsized teams and oversee delivery of work ensuring quality and timely completion
• Take ownership of assigned tasks and contribute effectively in an Agile environment
• Participate in solution design discussions and prepare designdocumentation artifacts based on requirements
• Drive job cost optimization and performance tuning initiatives to enhance system efficiency
• Support CICD processes and infrastructure automation to streamline deployments
• Actively participate in Agile ceremonies and contribute to continuous improvement of team collaboration and productivity

Other Details

Variable Compensation: This position includes a bonus or variable payout component based on individual and organizational performance. Actual compensation within the range will be dependent upon the individual's skills, experience, performance and internal equity.

Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree ("LTIM").

Benefits And Perks

• Comprehensive Medical Plan Covering Medical, Dental, Vision
• Short Term and Long-Term Disability Coverage
• 401(k) Plan with Company match
• Life Insurance
• Vacation Time, Sick Leave, Paid Holidays
• Paid Paternity and Maternity Leave

The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation.

Disclaimer: The compensation and benefits information provided herein is accurate as of the date of this posting.

LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.