Azure Data Engineer

Cognizant; Sydney; On-site; AUD 120,000 - 160,000; Full time

Job summary

Cognizant is seeking an experienced Azure Data Engineer with strong hands-on expertise in PySpark, Azure Synapse Analytics, SQL, and Big Data technologies. You will design, develop, optimize, and support scalable data pipelines and analytics solutions across Microsoft Azure and Big Data platforms.

Responsibilities include building ETL/ELT workflows, supporting Azure Data Lake Storage Gen2, Azure Data Factory, Databricks, and related services, and ensuring data governance and production support.

Qualifications

• 3+ years of practical experience with Azure Synapse Analytics and Azure cloud data services.
• Hands-on experience in PySpark, Spark SQL, Python and SQL for data pipeline development.
• Experience with Big Data tools such as Spark, Hadoop, Hive, Kafka, Sqoop, HBase.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and Azure Synapse Analytics.
• Build and optimize ETL/ELT workflows for large-volume data.
• Develop and support Azure Synapse pipelines, notebooks, SQL scripts, and data transformation logic.
• Collaborate with architects, analysts, QA, DevOps and client stakeholders to deliver robust data solutions.

Skills

PySpark

Azure Synapse Analytics

Python

Tools

Spark SQL

Azure Data Factory

Azure Databricks

Azure Data Lake Storage Gen2

SQL

Hadoop

Job description

Our Culture:

Your passion, integrity and experience are integral to Cognizant's success. You will join a dynamic and expanding global leader in IT and Business consultancy where you will be valued for who you are. We take pride in our partnership with our clients, so your ability to add value and provide exceptional service to our clients is fundamental to your success. In return, you will be presented with opportunities to develop your career and collaborate with talented colleagues in a supportive, diverse environment.

At Cognizant we recognize that companies that are open and welcoming to a multi-culturally diverse workforce will thrive with fresh perspectives and collaborative knowledge. Cognizant focuses on promoting & increasing gender diversity and providing a workplace which encourages great participation and an equal playing field, where merit and accomplishment are the only criteria for success.

Job Summary:

We are looking for an experienced Azure Data Engineer with strong hands-on expertise in PySpark, Azure Synapse Analytics, SQL, Big Data technologies, and modern cloud data engineering practices. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines, distributed data processing frameworks, and enterprise-grade analytics solutions across Microsoft Azure and Big Data platforms.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, Python, SQL, and Azure Synapse Analytics.
• Build and optimize ETL/ELT workflows for large-volume structured, semi-structured, and unstructured data processing.
• Develop and support Azure Synapse pipelines, notebooks, SQL scripts, stored procedures, views, and data transformation logic.
• Work with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure SQL Database, and related Azure data services.
• Develop and support Big Data solutions using technologies such as Apache Spark, Hadoop, HDFS, Hive, Sqoop, Kafka, HBase, and related distributed data processing components.
• Translate business requirements and functional specifications into detailed technical designs and reusable data engineering components.
• Implement data ingestion, transformation, validation, reconciliation, and publishing processes across enterprise data platforms.
• Perform performance tuning of PySpark and SQL workloads by optimizing joins, partitioning, caching, file sizing, indexing, and query execution plans.
• Optimize Big Data jobs and distributed workloads by tuning Spark configurations, partitions, memory usage, execution plans, storage formats, and cluster resource utilization.
• Develop reusable frameworks for metadata-driven ingestion, incremental loads, error handling, audit logging, and data quality checks.
• Support unit testing, system testing, UAT, deployment, production monitoring, incident resolution, and post-production support.
• Collaborate with architects, business analysts, QA teams, DevOps teams, platform teams, and client stakeholders to deliver robust data solutions.
• Ensure compliance with data governance, security, lineage, audit, access control, and operational standards.
• Mentor junior data engineers, review code, provide technical guidance, and promote engineering best practices.

Required Skills:

• Experience in data engineering, Big Data platforms, distributed data processing, cloud data solutions, and enterprise data processing.
• Hands-on experience in PySpark, Spark SQL, Python, SQL, and data pipeline development.
• Hands-on experience with Big Data ecosystem tools such as Spark, Hadoop, HDFS, Hive, Kafka, Sqoop, or HBase.
• Minimum 3+ years of practical experience working with Azure Synapse Analytics and Azure cloud data services.
• Proven experience in designing, developing, testing, deploying, and supporting production-grade data engineering solutions.
• Ability to independently own technical design, development, defect resolution, deployment, and production support activities.

Benefits:

Joining Cognizant will give you the opportunity to learn and collaborate with some of the most talented people in the industry, while having your finger on the pulse of emerging industry trends and working on the cutting edge of technology in your field of expertise.

We recognize that our people perform at their best when they feel valued as significant contributors and that is why at Cognizant, taking care of our employees is a priority:

• You can pursue innovative career tracks and opportunities here
• You can enhance your professional development through education and dedicated training
• We'll give you the skills you need to keep pace with the changing workplace while our compensation, benefits and wellness packages help you stay healthy and plan for the future.