Client is seeking an experienced Data Engineer to design, develop, and maintain data pipelines, data models, and data quality solutions that support business analytics and data-driven decision-making.
The ideal candidate will have strong programming skills in Python or SQL, hands-on experience with Google Cloud Platform (GCP), and expertise in data integration, data transformation, data modeling, and pipeline optimization. This role requires the ability to work independently, solve complex data challenges, implement data security best practices, and mentor junior data engineers.
Key Responsibilities
- Design, develop, and maintain data pipelines to collect and process data from multiple sources and systems.
- Integrate, consolidate, clean, transform, and structure data for analytical and business use.
- Design, implement, and optimize complex datasets and data models to improve performance and reliability.
- Apply data characteristics and delivery patterns to develop data quality rules and monitor data processes.
- Develop and implement data quality checks, identify data anomalies, troubleshoot issues, and ensure data accuracy and consistency.
- Leverage Google Cloud Platform (GCP) services and related technologies to build scalable and reliable data engineering solutions.
- Stay up to date with emerging technologies, industry trends, and best practices in GCP and data engineering.
- Evaluate and recommend new technologies and solutions to improve data processing, performance, and operational efficiency.
- Apply data management, maintenance, reporting, governance, and security best practices.
- Implement security best practices across data pipelines, cloud infrastructure, and data processing workflows.
- Use Git and CI/CD pipelines to support version control, automated deployments, and efficient development workflows.
- Troubleshoot and resolve issues related to data ingestion, transformation, pipeline execution, and downstream data consumption.
- Identify process inefficiencies and recommend continuous improvements to existing data engineering solutions.
- Provide technical guidance and mentorship to junior data engineers.
- Review data models, dataset implementations, and code developed by junior team members to ensure quality and adherence to best practices.
- Collaborate with technical teams and business stakeholders to align data engineering solutions with business objectives.
- Communicate technical concepts, data insights, risks, and proposed solutions effectively to technical and non-technical stakeholders.
Required Technical Skills
- Data Engineering: Strong experience in data engineering environments, data processing, and analytical data preparation.
- Programming: Advanced proficiency in Python or SQL.
- Google Cloud Platform (GCP): Hands-on experience with GCP services and general cloud computing concepts.
- Data Pipelines: Experience developing, maintaining, troubleshooting, and optimizing data pipelines.
- ETL/ELT and Data Integration: Experience with data ingestion, transformation, integration, and processing from multiple sources.
- Data Modeling: Experience designing, implementing, and optimizing data models for performance and scalability.
- Data Quality: Knowledge of data quality frameworks, validation rules, anomaly detection, and data integrity.
- Version Control: Advanced proficiency in Git.
- CI/CD: Experience integrating CI/CD pipelines into development and deployment workflows.
- Cloud Security: Understanding of cloud security best practices and secure data pipeline implementation.
- Data Governance: Knowledge of data management, maintenance, reporting, and governance practices.
- Other Cloud Platforms: Basic knowledge of cloud environments beyond GCP.
Required Experience and Qualifications
- Bachelor's degree in Science, Technology, Engineering, Mathematics (STEM), or an equivalent qualification.
- Master's degree is highly preferred.
- Minimum of 5+ years of experience as a Data Engineer or in a closely related role.
- Hands-on experience developing and maintaining moderately complex data pipelines.
- Experience designing and optimizing data models and supporting downstream analytical use cases.
- Ability to work independently with limited supervision and handle complex technical challenges.
- Experience using Git and CI/CD integration pipelines.Strong problem-solving, analytical, communication, and collaboration skills.
- Cloud certification is highly preferred.
- Technical Communication and Collaboration: Ability to communicate technical concepts and data insights clearly to different audiences.
- Operational Excellence: Ability to identify inefficiencies, document processes, and recommend improvements.
- Influence and Decision-Making: Ability to evaluate technical options and explain their benefits, risks, and trade-offs.
- Strategic Thinking: Understanding of how data engineering solutions support business objectives.
- Problem-Solving and Innovation: Ability to develop reliable, scalable, and efficient solutions to complex data challenges.
- Leadership and Mentorship: Ability to guide junior team members and review technical implementations.