Senior Azure Data Engineer/Architect | Immediate Joiners | Remote

Enable Data Incorporated; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

An innovative firm is seeking an experienced Data Engineer to design and maintain cloud-based data solutions. This role involves utilizing Apache Spark and Databricks to develop robust data pipelines and ensure data quality and security. You will collaborate with cross-functional teams to implement data models and optimize performance while providing guidance to junior engineers. If you are passionate about data engineering and eager to work with cutting-edge technologies, this opportunity is perfect for you. Join a dynamic team and contribute to the continuous improvement of data processes in a rapidly evolving field.

Qualifications

• 10+ years of experience as a Data Engineer focusing on cloud solutions.
• Strong knowledge of SQL and experience with relational and NoSQL databases.

Responsibilities

• Design and maintain scalable data solutions in the cloud using Apache Spark and Databricks.
• Build and optimize data pipelines for ingestion, processing, and integration.

Skills

Azure DataBricks

Azure Data Factory

Pyspark

Python

SQL

Spark

Education

Bachelor's degree in Computer Science

Master's degree in Engineering

Tools

Apache Spark

Databricks

Apache Airflow

Job description

Experience Required: 8+ Years

Mode of work: Remote

Skills Required: Azure DataBricks, Azure Data Factory, Pyspark, Python, SQL, Spark

Notice Period: Immediate Joiners / Permanent / Contract role (Can join within April)

Responsibilities

• Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and Databricks.
• Gather and analyze data requirements from business stakeholders and identify opportunities for data-driven insights.
• Build and optimize data pipelines for data ingestion, processing, and integration using Spark and Databricks.
• Ensure data quality, integrity, and security throughout all stages of the data lifecycle.
• Collaborate with cross-functional teams to design and implement data models, schemas, and storage solutions.
• Optimize data processing and analytics performance by tuning Spark jobs and leveraging Databricks features.
• Provide technical guidance and expertise to junior data engineers and developers.
• Stay up to date with emerging trends and technologies in cloud computing, big data, and data engineering.
• Contribute to the continuous improvement of data engineering processes, tools, and best practices.

Requirements:

• Bachelor's or master's degree in computer science, engineering, or a related field.
• 10+ years of experience as a Data Engineer with a focus on building cloud-based data solutions.
• Strong experience with cloud platforms such as Azure or AWS.
• Proficiency in Apache Spark and Databricks for large-scale data processing and analytics.
• Experience in designing and implementing data processing pipelines using Spark and Databricks.
• Strong knowledge of SQL and experience with relational and NoSQL databases.
• Experience with data integration and ETL processes using tools like Apache Airflow or cloud-native orchestration services.
• Good understanding of data modelling and schema design principles.
• Experience with data governance and compliance frameworks.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills to work effectively in a cross-functional team.

Interested candidates can share your resume or refer a friend to [e-mail] for a quick response.