Sr. Databricks Developer

Cognizant; Mississauga; Hybrid; CAD 90,000 - 120,000; Full time

Job summary

A leading technology services company is seeking a skilled professional in Artificial Intelligence and Analytics to develop scalable data processing solutions using Spark in Scala. The role involves collaborating with teams, mentoring junior developers, and utilizing Databricks technologies. Candidates should possess strong backgrounds in Python, Databricks SQL, and Delta Lake while being able to work effectively in a hybrid model. This is an excellent opportunity for those who are passionate about data-driven decision-making and innovation.

Qualifications

• Strong background in Spark in Scala and Databricks technologies.
• Proficiency in Python programming for data processing tasks.
• Experience with Databricks Unity Catalog Admin and Databricks CLI.
• Expertise in Databricks SQL and Delta Lake for data management.
• Knowledge of PySpark for efficient data processing.
• Ability to work in a hybrid model.
• Excellent problem-solving skills and attention to detail.

Responsibilities

• Develop and implement scalable data processing solutions using Spark in Scala.
• Administer and manage Databricks Unity Catalog for data governance.
• Utilize Databricks CLI to automate workflows.
• Write Python scripts for data integration and transformation.
• Design Databricks SQL queries for improved reporting efficiency.
• Implement Databricks Delta Lake for data reliability.
• Create Databricks Workflows for automating data pipelines.
• Collaborate with teams to translate business requirements into solutions.
• Troubleshoot complex data processing issues.
• Provide mentorship to junior developers.
• Stay updated with industry trends.
• Ensure alignment with company standards.

Skills

Spark in Scala

Databricks technologies

Python programming

Databricks SQL

Delta Lake

PySpark

Problem-solving

Attention to detail

Job description

Practice - AIA - Artificial Intelligence and Analytics

About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all intelligent, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to design the future—a future in which trial-and-error business decisions have been replaced by informed choices and data-supported strategies.

By applying AI and data science, we help leading companies to prototype, refine, validate, and scale their AI and analytics products and delivery models. Cognizant’s AIA practice takes insights that are buried in data and provides businesses a clear way to transform how they source, interpret and consume their information. Our clients need flexible data structures and a streamlined data architecture that quickly turns data resources into informative, meaningful intelligence.

Key Responsibilities

• Develop and implement scalable data processing solutions using Spark in Scala to enhance data analytics capabilities.
• Administer and manage Databricks Unity Catalog to ensure efficient data governance and security.
• Utilize Databricks CLI to automate workflows and streamline data operations.
• Write efficient Python scripts to support data integration and transformation processes.
• Design and optimize Databricks SQL queries to improve data retrieval and reporting efficiency.
• Implement Databricks Delta Lake to ensure data reliability and consistency across platforms.
• Create and manage Databricks Workflows to automate data pipelines and improve operational efficiency.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Troubleshoot and resolve complex data processing issues to maintain system performance and reliability.
• Provide technical guidance and mentorship to junior developers to foster skill development and knowledge sharing.
• Stay updated with the latest industry trends and technologies to continuously improve data solutions.
• Contribute to the company’s mission by developing innovative solutions that enhance data-driven decision-making.
• Ensure all development activities align with company standards and best practices.

Qualifications

• Possess a strong background in Spark in Scala and Databricks technologies.
• Demonstrate proficiency in Python programming for data processing tasks.
• Have experience with Databricks Unity Catalog Admin and Databricks CLI.
• Show expertise in Databricks SQL and Delta Lake for data management.
• Exhibit knowledge of PySpark for efficient data processing.
• Display the ability to work effectively in a hybrid work model.
• Have excellent problem-solving skills and attention to detail.

*Please note, this role is not able to offer visa transfer or sponsorship now or in the future*