Intermediate Data Engineer

Trinnex Inc.; Bengaluru, Chennai District, Pune District; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

CDM Smith is seeking a Data Engineer to join our Digital Engineering Solutions team. This role focuses on building scalable data pipelines, using Azure, Databricks and Spark to deliver actionable insights for AEC projects.

You will collaborate with data scientists and engineers, implement data quality, governance, and observability, and help advance our data platforms across the firm.

Qualifications

• Bachelor's degree; 3 years of related experience, travel may be required.

Responsibilities

• Design and develop data pipelines and workflows to extract, transform, and load (ETL/ELT) data from various sources into target systems.
• Automate workflows to improve efficiency, scalability, and reliability in data integration processes.
• Support the optimization and management of data storage solutions, including data lakes and warehouses, for performance and scalability.
• Ensure data quality by implementing processes to validate completeness, accuracy, and consistency of data.
• Ensure compliance with data governance policies and maintain data privacy and security standards.
• Collaborate with Data Scientists, AI engineers, Analytics Engineers, and stakeholders to understand business requirements and deliver reliable data solutions.
• Support Data Scientists and AI engineers by preparing and delivering trusted datasets and data pipelines that enable advanced analytics and AI/ML solutions.
• Assist in implementing monitoring and observability tools for data pipelines to ensure reliability and issue detection.
• Stay abreast of the latest developments and advancements, including new and emerging technologies & best practices and new tools & software applications and how they could impact CDM Smith.
• Assist with the development of documentation, standards, best practices, and workflows for data technology hardware/software in use across the business.
• Perform other duties as required.

Skills

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

Python

SQL

Apache Spark

Hadoop

Kafka

CI/CD

DevOps

SDLC

Data governance

Education

Bachelor's degree

Tools

Databricks

Hadoop

Azure

Kafka

Job description

Company Overview

Check out this video and find out why our team loves to work here!

Join Us! CDM Smith – where amazing career journeys unfold.

Imagine a place committed to offering an unmatched employee experience. Where you work on projects that are meaningful to you. Where you play an active part in shaping your career journey. Where your co-workers are invested in you and your success. Where you are encouraged and supported to do your very best and given the tools and resources to do so. Where it’s a priority that the company takes good care of you and your family.

Our employees are the heart of our company. As an employer of choice, our goal is to provide a challenging, progressive and inclusive work environment which fosters personal leadership, career growth and development for every employee. We value passionate individuals who challenge the norm, deliver world‑class solutions and bring diverse perspectives. Join our team, and together we will make a difference and change the world.

Job Description

CDM Smith is seeking a Data Engineer to join our Digital Engineering Solutions team. This individual will be part of the Data & AI Technology group within the Digital Engineering Solutions team, helping to support strategic Architecture, Engineering and Construction (AEC) initiatives using modern data technologies and analytics to deliver actionable business insights and robust solutions for AEC professionals and client outcomes.

The Data & AI Technology group will help lead the firm in AEC‑focused Business Intelligence and data services by providing scalable data platforms, strong engineering practices, and reliable data pipelines to enable analytics and AI capabilities across the organization. The Data & AI Technology group will specifically utilize advanced analytics, data science, and AI/ML to give our business and our products a competitive advantage. It includes understanding and managing the data, how it interconnects, and engineering data pipelines and data assets to enable self‑serve BI and BA opportunities.

This position is for a person who has demonstrated strong foundational data engineering capabilities, is experienced with data technologies and processes, and enjoys framing a problem, contributing to solutions, and supporting implementation efforts. As a member of the Digital Engineering Solutions team, the Data & AI Technology group will also engage in research and development and provide support to AEC practices at CDM Smith, participating in product research, testing, and the incubation of data technology‑related ideas that arise from around the company.

Key Responsibilities

• Design and develop data pipelines and workflows to extract, transform, and load (ETL/ELT) data from various sources into target systems.
• Automate workflows to improve efficiency, scalability, and reliability in data integration processes.
• Support the optimization and management of data storage solutions, including data lakes and warehouses, for performance and scalability.
• Ensure data quality by implementing processes to validate completeness, accuracy, and consistency of data.
• Ensure compliance with data governance policies and maintain data privacy and security standards.
• Collaborate with Data Scientists, AI engineers, Analytics Engineers, and stakeholders to understand business requirements and deliver reliable data solutions.
• Support Data Scientists and AI engineers by preparing and delivering trusted datasets and data pipelines that enable advanced analytics and AI/ML solutions.
• Assist in implementing monitoring and observability tools for data pipelines to ensure reliability and issue detection.
• Stay abreast of the latest developments and advancements, including new and emerging technologies & best practices and new tools & software applications and how they could impact CDM Smith.
• Assist with the development of documentation, standards, best practices, and workflows for data technology hardware/software in use across the business.
• Perform other duties as required.

Skills and Abilities

• Experience with the Software Development Life Cycle (SDLC) and Agile Development methodologies.
• Experience with Microsoft Azure cloud services, including Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
• Proficiency in building and maintaining data systems using modern frameworks like Apache Spark and Databricks.
• Proficiency in programming languages such as Python, SQL.
• Knowledge of data modeling and designing scalable ETL/ELT processes.
• Familiarity with real‑time streaming solutions, such as Kafka or Azure IoT Hub.
• Hands‑on experience with distributed computing tools, including Apache Spark and Hadoop ecosystems.
• Familiarity with CI/CD pipelines and DevOps practices for data solutions.
• Knowledge of monitoring tools and techniques for ensuring pipeline observability and reliability.
• Strong problem‑solving and critical thinking skills to identify and address technical challenges effectively.
• Excellent interpersonal and presentation skills to collaborate with colleagues, stakeholders, and partners.
• Ability to effectively communicate technical concepts to both technical and non‑technical audiences.
• Detail oriented with the ability to execute complex or specialized technical tasks.

Skills & Abilities

• Possesses solid foundational knowledge of Cloud ETL tools and deployment methodologies.
• Strong problem‑solving and critical thinking skills.
• Strong verbal and written communication and collaboration skills.
• Ability to work effectively across all levels in a highly dynamic environment.
• Possesses strong attention to detail.
• Experience in using programming languages to develop, test, and deploy data applications and systems.

Qualifications

Bachelor's degree. 3 years of related experience. Equivalent additional directly related experience will be considered in lieu of a college degree. Domestic and/or international travel may be required. The frequency of travel is contingent on specific duties, responsibilities, and the essential functions of the position, which may vary depending on workload and project demands.

Preferred Qualifications

• Experience working with Databricks or modern Lakehouse architectures.
• Experience supporting data pipelines used by advanced analytics or AI/ML solutions.
• Experience working with large‑scale enterprise data platforms and cloud‑based data environments.

Amount of Travel Required

No Travel is required

Background Check and Drug Testing Information

CDM Smith Inc. and its divisions and subsidiaries (hereafter collectively referred to as “CDM Smith”) reserves the right to require background checks including criminal, employment, education, licensure, etc. as well as credit and motor vehicle when applicable for certain positions. In addition, CDM Smith may conduct drug testing for designated positions. Background checks are conducted after an offer of employment has been made in the United States. The timing of when background checks will be conducted on candidates for positions outside the United States will vary based on country statutory law but in no case, will the background check precede an interview. CDM Smith will conduct interviews of qualified individuals prior to requesting a criminal background check, and no job application submitted prior to such interview shall inquire into an applicant's criminal history. If this position is subject to a background check for any convictions related to its responsibilities and requirements, employment will be contingent upon successful completion of a background investigation including criminal history. Criminal history will not automatically disqualify a candidate. In addition, during employment individuals may be required by CDM Smith or a CDM Smith client to successfully complete additional background checks, including motor vehicle record as well as drug testing.

Additional Pay Range Information

CDM Smith is committed to fair and equitable compensation practices. This pay range is a good faith estimate representative of all experience levels for the geographic location assigned to the position. In addition to geographic location, a candidate’s salary is determined by several other factors including, but not limited to, the role, function and associated responsibilities, relevant work experience, skills, required certifications, and education/training.

Additional Compensation

All bonuses at CDM Smith are discretionary and may or may not apply to this position.

Visa Sponsorship Available

No - We are not open to sponsoring for this position