Big Data & Cloud Engineer
We are seeking a Big Data & Cloud Engineer with hands-on experience in Python, Apache Spark, Databricks, Snowflake, Azure Cloud, ETL/Data Pipelines, and Microservices. The ideal candidate should be capable of understanding business requirements, proposing technical solutions, collaborating with stakeholders, and developing scalable data engineering solutions. Exposure to AI-assisted development, MCP/Agentic AI, and Azure CloudOps is highly desirable.
Key Responsibilities
- Understand business and technical requirements and translate them into scalable solutions.
- Design, develop, and optimize ETL/data pipelines using Python, Apache Spark, Databricks, and Snowflake.
- Build and maintain REST APIs and microservices for data integration.
- Develop entity/profile matching solutions using ML and graph algorithms.
- Deploy and manage cloud-native solutions on Microsoft Azure.
- Support Azure CloudOps activities, including networking, identity, security, monitoring, and observability.
- Optimize SQL queries and data processing performance.
- Leverage AI-assisted development tools (Cursor, GitHub Copilot) to improve productivity.
- Collaborate with cross-functional teams, stakeholders, and architects throughout the development lifecycle.
- Troubleshoot production issues and continuously improve system performance and reliability.
Requirements
- 3?5 years of experience in Data Engineering or Cloud Engineering.
- Strong Python programming skills.
- Apache Spark (PySpark).
- Databricks.
- Snowflake.
- ETL/Data Pipeline development.
- REST APIs and Microservices.
- Microsoft Azure.
- SQL and query optimization.
- Strong analytical and problem-solving skills.
- ML or Graph Algorithms for Entity/Profile Matching (preferred).
- Azure CloudOps (Networking, Identity, Security) (preferred).
- Monitoring & Observability (preferred).
- MCP / Agentic AI (preferred).
- AI-assisted development (Cursor, GitHub Copilot) (preferred).
- JavaScript/jQuery knowledge (preferred).
Work Arrangements
- Hybrid work mode (2?3 days/week in office).
- Location: Bengaluru, Karnataka, IN.
- Full-time position.
- Experience: 3?5 Years.
Education and Benefits
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Strong analytical and logical reasoning.
- Excellent communication and stakeholder management skills.
- Ability to understand requirements and recommend technical solutions.
- Self-driven with a willingness to learn new technologies.
- Strong collaboration and teamwork skills.