Technical Architect — Data and Analytics (AWS)
Experience: 12–16 years
Location: Hyberabad / Bengaluru
Role Summary
The ideal candidate should be able to handle the complete architecture, Solutioning and handle client calls effectively.
What You’ll Do
- Technical Leadership
- Drive end-to-end technical solutions, ensuring performance, scalability, and reliability.
- Establish coding standards, best practices, and development guidelines.
- Conduct code reviews, provide technical mentorship, and resolve complex technical issues.
- Data Platform Architecture
- Architect and implement large-scale data ingestion, transformation, and storage layers.
- Design Lakehouse architecture using Delta Lake, Unity Catalog, Medallion model.
- Ensure data quality, lineage, and governance across data assets.
- Collaborate with cloud architects to integrate platform services across Azure and AWS.
- Solution Development
- Develop optimized Spark jobs (PySpark/Scala) with strong performance tuning practices.
- Implement ETL/ELT pipelines using Databricks notebooks, Workflows, and Glue Jobs.
- Integrate data from multiple sources into ADLS/S3 with appropriate security and metadata management.
- Build CI/CD pipelines using Azure DevOps / GitHub Actions.
- Cloud Infrastructure & Security
- Define IAM roles, managed identities, ACLs, and storage permissions for secure operations.
- Work with Terraform/CloudFormation for infrastructure automation.
- Monitor cloud resource utilization and optimize cost.
- Cross-Functional Collaboration
- Interact with stakeholders, analysts, architects, and business teams to translate requirements into technical designs.
- Present architectural recommendations and solution proposals to leadership.
- Coordinate with DevOps and Cloud Engineering for deployment and operational readiness.
Must-Have Skills
- 10+ years of experience in Data Engineering, Big Data, or Cloud Data Platforms
- 8+ years of hands‑on experience with Databricks (PySpark/Scala, Delta Lake, Lakehouse architecture)
- 8+ years of experience with Databricks (RBAC, ACLs, hierarchical namespaces)
- 6+ years of experience designing data ingestion and transformation pipelines using Spark‑based frameworks.
- 4+ years of experience in AWS Glue (Glue ETL jobs, Workflows, Crawlers, Glue Catalog).
- 4+ years of experience in Athena, Spark and Python scripts.
Good to Have
- Experience with version control systems with GitHub.
- Exposure to GitHub pipelines.
- Knowledge of performance optimization techniques.
- Support the overall development and unit testing, system testing, and deployment of the solution.
- Ensure smooth handover to operations and support teams.
- Ensure proper integration with existing systems, data sources, and third‑party services.
Tech Stack
AWS Glue (Glue ETL jobs, Workflows, Crawlers, Glue Catalog), Databricks (PySpark/Scala, Delta Lake, Lakehouse architecture), SQL