Cloud Data Lead: ETL, Pipelines & Big Data

Pyramid Consulting, Inc; Mexico; On-site; PHP 5,618,000 - 8,115,000; Full time

Job summary

Pyramid Consulting, Inc. is seeking an experienced Big Data Lead to design, develop, and support scalable data engineering solutions in a modern cloud-based environment.

You will work across data integration, ETL/ELT, and data pipelines to enable reliable analytics and decision making. The role emphasizes distributed data processing, SQL proficiency, and programming with Python/Java/Scala, plus hands-on experience with cloud platforms and data lake architectures.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field; equivalent practical experience may be considered.
• 5+ years of experience developing and supporting data integration, ETL/ELT, or data engineering solutions.
• Proven experience working with large data volumes in distributed or cloud-based environments; strong SQL skills.

Responsibilities

• Design, develop, and support scalable data integration, ETL/ELT, and data engineering solutions.
• Process and manage large data volumes across distributed and cloud-based environments.
• Develop efficient data pipelines using SQL and programming languages such as Python, Java, Scala, or Spark SQL.
• Build and manage data workflows using orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Step Functions.
• Design resilient and fault-tolerant pipelines with logging, monitoring, retry mechanisms, idempotency, and failure recovery.
• Work with relational and non-relational databases, data warehouses, and data lake architectures.
• Troubleshoot data pipeline and integration issues and implement effective solutions.
• Collaborate with engineering, application, and business teams to deliver reliable data solutions.
• Contribute to data architecture, modeling, governance, metadata management, and master data management initiatives.
• Support containerized deployments, CI/CD pipelines, and Infrastructure as Code practices.

Skills

Data integration

ETL/ELT

Distributed data processing

Cloud platforms

SQL

Programming (Python/Java/Scala)

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field

Job description

Pyramid Consulting, Inc. is seeking an experienced Big Data Lead to design, develop, and support scalable data engineering solutions in a modern cloud-based environment.

You will work across data integration, ETL/ELT, and data pipelines to enable reliable analytics and decision making. The role emphasizes distributed data processing, SQL proficiency, and programming with Python/Java/Scala, plus hands-on experience with cloud platforms and data lake architectures.