Principal Data Engineer (Azure)

Tiger Analytics; Richardson (TX); On-site; USD 90,000 - 150,000; Full time

Job summary

Join a dynamic and innovative firm at the forefront of the AI revolution as a Principal Data Engineer. This role offers the chance to work with cutting-edge technologies on Azure, designing and implementing robust data solutions that will enhance the lives of millions. You will collaborate with talented teams, pushing the boundaries of what's possible while developing scalable data ingestion pipelines and supporting data science initiatives. With significant career development opportunities in a fast-paced environment, this position is perfect for those eager to make a real impact in the world of analytics and data engineering.

Benefits

Great Place to Work Certification

Flexible Work Arrangements

Career Development Opportunities

Collaborative Work Environment

Qualifications

• Hands-on experience with Azure and Databricks for data solutions.
• Strong programming skills in SQL, Python, or Scala/Java.

Responsibilities

• Design and build scalable data ingestion pipelines.
• Collaborate with teams to deliver analytical solutions.

Skills

Data Lake Implementation

Azure Data Factory (ADF)

PySpark

Databricks

SQL Programming

Debugging Skills

Big Data Technologies

REST API Development

Agile Methodologies

DevOps Processes

Education

Bachelor's Degree in Computer Science or related field

Certifications in Data Engineering (e.g., DP-203)

Tools

Azure Synapse Analytics

Event Hub

Cosmos DB

Hadoop

Spark

Airflow

Kafka

Git

Jenkins

Azure DevOps

Job description

Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you’ll find our delivery centers across the globe and offices in multiple cities across India, the US, UK, Canada, and Singapore, including a substantial remote global workforce.

We’re Great Place to Work-Certified. Working at Tiger Analytics, you’ll be at the heart of an AI revolution. You’ll work with teams that push the boundaries of what is possible and build solutions that energize and inspire.

Curious about the role? What your typical day would look like?

As a Principal Data Engineer (Azure), you would have hands-on experience working on Azure as cloud, Databricks, and some exposure/experience on Data Modelling. You will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure.

Key Responsibilities:

1. Design and build scalable & metadata-driven data ingestion pipelines (For Batch and Streaming Datasets).
2. Conceptualize and execute high-performance data processing for structured and unstructured data, and data harmonization.
3. Schedule, orchestrate, and validate pipelines.
4. Design exception handling and log monitoring for debugging.
5. Ideate with your peers to make tech stack and tools-related decisions.
6. Interact and collaborate with multiple teams (Consulting/Data Science & App Dev) and various stakeholders to meet deadlines, to bring Analytical Solutions to life.

What do we expect?

1. Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database.
2. A comprehensive foundation with working knowledge of Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB, and Purview.
3. A passion for writing high-quality code that is modular, scalable, and free of bugs (debugging skills in SQL, Python, or Scala/Java).
4. Enthusiasm to collaborate with various stakeholders across the organization and take complete ownership of deliverables.
5. Experience in using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic Search.
6. Adept understanding of different file formats like Delta Lake, Avro, Parquet, JSON, and CSV.
7. Good knowledge of building and designing REST APIs with real-time experience working on Data Lake or Lakehouse projects.
8. Experience in supporting BI and Data Science teams in consuming the data in a secure and governed manner.
9. Certifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE) are a valuable addition.

Note: The designation will be commensurate with expertise and experience. Compensation packages are among the best in the industry.

Job Requirements:

• Mandatory: Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database.
• Optional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB, and Purview.
• Strong programming, unit testing & debugging skills in SQL, Python, or Scala/Java.
• Some experience using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic Search.
• Good understanding of different file formats like Delta Lake, Avro, Parquet, JSON, and CSV.
• Experience working in Agile projects and following DevOps processes with technologies like Git, Jenkins & Azure DevOps.
• Good to have:
• Experience working on Data Lake & Lakehouse projects.
• Experience building REST services and implementing service-oriented architectures.
• Experience supporting BI and Data Science teams in consuming the data in a secure and governed manner.
• Certifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE).

This position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility.