Lead Data Platform Architect

Good co India; United States; Remote; USD 130,000 - 170,000; Full time

Job summary

Good co India is seeking a data engineering lead to design scalable, secure data architectures and modern data platforms on cloud providers. You will drive standards, build batch and streaming pipelines, and mentor engineers across teams to deliver data-driven solutions.

You will work with Python/SQL, Spark, Kafka, and orchestration tools, ensuring robust data delivery, governance, and cost-effective cloud infrastructure across AWS, Azure, or GCP.

Qualifications

• 5 to 10 years of experience in data engineering or distributed data processing, with demonstrated technical leadership and architecture ownership.
• Strong proficiency in Python and SQL, with experience in Scala or Java.
• Extensive hands-on experience with Apache Spark, PySpark, Kafka, Hadoop, and large-scale data processing frameworks.

Responsibilities

• Lead the design and implementation of scalable, secure, and high-performance enterprise data architectures.
• Define data engineering standards, architectural guidelines, and best practices across data platforms and pipelines.
• Design and optimize batch and real-time data processing solutions using Apache Spark, Kafka, and other distributed computing technologies.
• Architect and manage modern data platforms, including data lakes, data warehouses, and lakehouse solutions.
• Drive cloud-based data engineering initiatives across AWS, Microsoft Azure, or Google Cloud Platform (GCP).
• Develop and optimize complex ETL/ELT pipelines to ensure reliable, efficient, and high-quality data delivery.
• Establish data governance, security, privacy, quality, and compliance frameworks across enterprise data systems.
• Lead technical design reviews, evaluate emerging technologies, and recommend solutions aligned with business objectives.
• Mentor and guide data engineers, promote engineering excellence, and support technical capability development.
• Collaborate with data scientists, analytics teams, architects, product managers, and business stakeholders to deliver data-driven solutions.
• Optimize data platform performance, scalability, availability, and cloud infrastructure costs.
• Implement automated testing, CI/CD, monitoring, orchestration, and infrastructure-as-code practices.
• Troubleshoot complex production issues and drive continuous improvement in data reliability and operational efficiency.

Skills

Python

SQL

Scala

Java

Apache Spark

Kafka

Hadoop

Databricks

Snowflake

Delta Lake

Airflow

CI/CD

Infrastructure as Code

AWS

Azure

GCP

data lakes

data warehouses

lakehouse

ETL/ELT

data governance

security

monitoring

Education

Bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related technical discipline

Tools

Databricks

Snowflake

Delta Lake

Apache Airflow

Job description

Good co India is seeking a data engineering lead to design scalable, secure data architectures and modern data platforms on cloud providers. You will drive standards, build batch and streaming pipelines, and mentor engineers across teams to deliver data-driven solutions.

You will work with Python/SQL, Spark, Kafka, and orchestration tools, ensuring robust data delivery, governance, and cost-effective cloud infrastructure across AWS, Azure, or GCP.