Lead Data Engineer

Amaze Systems; Houston (TX); On-site; USD 180,000 - 230,000; Full time

Job summary

Amaze Systems in Houston, TX seeks a Lead Data Engineer to architect and lead scalable Azure-based data lake initiatives. You will drive Databricks Delta Live Tables pipelines, oversee data governance, and mentor a team delivering end-to-end analytics solutions.

Strong leadership and cloud data experience required. You will collaborate with executives to translate technical challenges into actionable insights, ensuring performance, security, and budget adherence across large-scale data

Qualifications

• Bachelor’s or Master’s degree in IT or a related discipline.
• 15–17 years of experience delivering cost-effective cloud-based data analytics solutions.
• Proven track record in building data lake architectures and optimizing performance in big data environments.
• Azure Data Engineering or Databricks certification preferred.

Responsibilities

• Architect, design, and implement scalable Azure-based data lake solutions.
• Lead the development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming.
• Provide technical leadership and ensure adherence to best practices and quality standards.
• Establish roadmaps for capability-building across technologies like Qlik Replicate and Unity Catalog.
• Oversee data governance, security, and metadata management.
• Mentor and develop a high-performing team skilled in Azure and data streaming.

Skills

Azure Data Platform

Databricks

Delta Live Tables

Unity Catalog

SQL

Python/Scala/Spark-SQL

ETL pipelines

Data governance

Leadership

Data migration

Education

Bachelor’s or Master’s degree in IT or related field

Tools

PowerBI

QlikSense

Azure Synapse

Databricks

Job description

Role - Lead Data Engineer

Location – Houston, TX

Employment type - W2

Key Responsibilities

• Architect, design, and implement scalable Azure-based data lake solutions.
• Establish roadmaps for capability-building across technologies such as Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog.
• Provide technical leadership, ensuring adherence to best practices and quality standards.
• Lead the development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming.
• Optimize Databricks resources for high-volume data processing without exceeding budget.
• Oversee the use of Databricks Unity Catalog for data governance, lineage, and metadata management.
• Develop and enforce robust data governance policies, ensuring compliance, security, and data quality.
• Monitor data pipeline performance and proactively resolve production issues.
• Collaborate with cross-functional teams to align project objectives with business goals.
• Translate technical challenges into actionable insights for executive teams.
• Mentor and develop a high-performing team skilled in Azure, Databricks, and data streaming technologies.
• Facilitate knowledge transfer sessions to empower internal teams with cutting-edge methodologies.

Education And Experience

• Bachelor’s or Master’s degree in IT or a related discipline.
• 15-17 years of experience in the IT industry with expertise in delivering cost-effective cloud-based data analytics solutions.
• Proven track record in building data lake architectures and optimizing performance in big data environments.
• Certification in Azure Data Engineering or Databricks (preferred).

Technical Skills

• Strong knowledge of Azure technologies, including ADF, ADLS, Databricks, Stream Analytics, Delta Live Tables, and Synapse Analytics.
• Advanced proficiency in SQL and programming languages such as Python, Scala, or Spark-SQL.
• Experience in designing and developing scalable ETL pipelines and optimizing data workflows.
• Deep understanding of data migration from on-premises to cloud, cloud to cloud.

Leadership And Soft Skills

• Proven ability to lead diverse teams and manage end-to-end analytics projects.
• Exceptional communication and collaboration skills for engaging executives and stakeholders.
• Strong sense of ownership, accountability, adaptability, and commitment to results.
• Expertise in identifying risks and implementing effective mitigation strategies.

Nice To Have

• Familiarity with big data tools like Hadoop, Kafka, and Spark.
• Experience with visualization tools such as PowerBI or QlikSense.
• Knowledge of AI/ML, NLP, and GenAI concepts.
• Familiarity with AGILE and KANBAN methodologies.

Shivani | Sr. Talent Acquisition Specialist

Amaze Systems Inc

E: |