Spark, Hive, Scala, SQL ,Python scripting ,data visualization
Key Responsibilities:
- Lead and mentor a team of data engineers, fostering a culture of collaboration and continuous improvement.
- Oversee the planning and execution of data projects, ensuring alignment with business objectives and timelines.
- Provide technical guidance and expertise to the team, promoting best practices in data engineering.
Data Architecture and Development:
- Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and store large volumes of data.
- Architect and implement robust data models and schemas to support analytics and business intelligence needs.
- Ensure data quality, integrity, and security across all data assets.
Collaboration and Communication:
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet their needs.
- Communicate complex technical concepts to non-technical stakeholders in a clear and concise manner.
- Collaborate with other engineering teams to integrate data solutions with various applications and systems.
Technology and Innovation:
- Stay up-to-date with emerging technologies and trends in data engineering and big data.
- Evaluate and implement new tools and technologies to improve data processing, storage, and analysis capabilities.
- Drive innovation and continuous improvement in data engineering practices.