Specialist - Data Engineering

LTM; Pune District; In loco; INR 900.000 - 1.500.000; Tempo pieno

Descrizione del lavoro

LTM in Pune is seeking a Specialist - Data Engineering to design, develop, and maintain robust data solutions in the Azure cloud, enabling analytical insights. You will architect scalable data pipelines, data lakes, and data warehousing components while working with data scientists and analysts.

The ideal candidate has 5–8 years of Azure data engineering experience, deep expertise in Azure Data Factory, Data Lake, SQL, and Databricks, and a Bachelor's degree in a related field.

Competenze

• 5-8 years of professional experience in data engineering with a focus on Microsoft Azure cloud services.
• Proficiency in Azure Data Factory for orchestrating and automating data pipelines.
• Databricks experience for big data processing, analytics and ML initiatives.

Mansioni

• Design, develop, and implement scalable data pipelines using Azure Data Factory for ingestion and transformation.
• Manage and optimize data storage in Azure Data Lake, ensuring data integrity and accessibility.
• Develop and maintain complex data processing workflows using Databricks for analytics and ML initiatives.
• Implement and manage relational databases in Azure SQL for high performance and reliability.
• Optimize data storage and retrieval, including Azure Blob storage usage.
• Design and build data warehousing solutions leveraging Azure Synapse Analytics for BI and reporting.
• Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
• Monitor data pipeline performance, troubleshoot issues, and optimize data flow.
• Ensure adherence to data governance, security, and compliance standards across engineering activities.
• Provide technical guidance and mentorship on Azure data engineering best practices.

Conoscenze

Azure Data Factory

Azure Data Lake

Azure SQL

Databricks

Formazione

Bachelor's degree in Computer Science / Engineering / IT or related field

Strumenti

Azure Synapse Analytics

Azure Blob Storage

Descrizione del lavoro

We are seeking a highly skilled Specialist - Data Engineering to join our team in Pune. This role involves designing, developing, and maintaining robust data solutions within the Azure cloud ecosystem, ensuring efficient data processing and availability for analytical insights. The ideal candidate will possess deep expertise in Azure data services and a proven track record of delivering scalable data platforms.

• 1. Design, develop, and implement scalable data pipelines using Azure Data Factory for data ingestion and transformation.
• 2. Manage and optimize data storage solutions within Azure Data Lake, ensuring data integrity and accessibility.
• 3. Develop and maintain complex data processing workflows utilizing Databricks for advanced analytics and machine learning initiatives.
• 4. Implement and manage relational databases in Azure SQL, ensuring high performance and reliability.
• 5. Optimize data storage and retrieval mechanisms, including the effective use of Azure BLOB storage.
• 6. Design and build data warehousing solutions leveraging Azure Synapse Analytics for business intelligence and reporting.
• 7. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver appropriate solutions.
• 8. Monitor data pipeline performance, troubleshoot issues, and implement optimizations to ensure efficient data flow.
• 9. Ensure adherence to data governance, security, and compliance standards across all data engineering activities.
• 10. Provide technical guidance and mentorship to junior team members on best practices in Azure data engineering.

Skills and Qualifications:

Required Skills:

• 5-8 years of professional experience in data engineering, with a strong focus on Microsoft Azure cloud services.
• Proficiency in Azure Data Factory for orchestrating and automating data pipelines.
• Extensive experience with Azure Data Lake for scalable data storage and processing.
• Strong command of Azure SQL for database management and query optimization.
• Demonstrated expertise in Databricks for big data processing, analytics, and machine learning.

Desired Skills:

• Experience with Azure BLOB storage for unstructured data management.
• Familiarity with Azure Synapse Analytics for integrated data warehousing and analytics.
• Bachelors degree in Computer Science, Engineering, Information Technology, or a related field.

Excellent problem-solving abilities and a proactive approach to technical challenges.

Strong communication and collaboration skills to work effectively within a team environment.