Data Engineering- SSEII

First American; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

First American (India) is implementing a Databricks Lakehouse to unify data storage and enable efficient processing across the organization. You will design data pipelines, build API integrations, and implement robust ETL processes with a focus on data integrity and quality.

Collaborating with data engineers, analysts, and scientists, you will optimize data workflows using Spark, Kafka, and Airflow, ensuring high performance and observability in a containerized environment.

Qualifications

• Development experience with Python, Scala, Java, or C#. Python preferred.
• 5+ years with Databricks, Snowflake, BigQuery, Spark, Hive, Hadoop, Cloudera, or Redshift.
• Proficiency in high-performance data pipeline design and observability.
• Containerization experience with Docker, Kubernetes, or similar.
• Experience with Apache Airflow or similar data orchestration tool.
• Bachelor’s degree in CS/IT or equivalent; minimum 15 years formal education.

Responsibilities

• Operationalize a Databricks Lakehouse for unified data storage and analysis.
• Design, develop and maintain scalable data pipelines and API integrations.
• Implement ETL processes, ensure data quality and integrity.
• Utilize Spark, Kafka, and Airflow for large-scale data processing; monitor and troubleshoot.

Skills

Python

Data pipelines

Big data processing

Containerization

Education

BSc/BE in Computer Science or IT

15 years of formal education

Tools

Databricks

Snowflake

BigQuery

Apache Spark

Airflow

Docker

Kubernetes

Job description

Company Summary First American (India) is a GCC (Global Capability Center) of the First American Financial Corporation (NYSE: FAF) family of companies. FAI is a proud member of the FORTUNE 500 companies and has been amongst the Fortune 100 Best Companies to Work For® list for eight consecutive years. First American Financial Corporation provides comprehensive title insurance, closing/settlement, property data and technology solutions. First American (India) creates quality solutions for its customers by combining software, back office, and knowledge processing operations to fulfill First American's business requirements. Our priorities are our employees, customers, and shareholders - in that order. First American (India) has been ranked amongst India's Best Companies To Work For™ 2023: Listed amongst the Top 100 by Great Place To Work® India, FAI is also certified Best Workplaces for Women and Workplace with Inclusive Practices. Software Services helps build First American's product suite that encompasses the best in class Title Insurance, Settlement and Mortgage solutions platforms. Leverages technology product stack across Microsoft platform predominantly to develop, enhance and maintain the best in class applications. The R & D division delivers solutions for the title insurance industry leveraging the best of NLP, AI and ML.

Job Summary

What You’ll Do: Operationalize our Databricks Lakehouse: Develop and maintain a unified data platform using Azure Databricks, enabling efficient data storage, processing, and analysis across the organization. Build Data Pipelines: Design, develop, and maintain scalable data pipelines; establish API integrations for efficient data transfer. Implement ETL processes and ensure data integrity and quality. Deploy Data Technology: Utilize technologies such as Spark, Kafka, and Airflow to manage large-scale data processing. Test, monitor and troubleshoot systems to ensure smooth operation. Collaborate: Work closely with data engineers, analysts, and scientists to understand their needs and optimize the performance of data workflows

What You’ll Bring:

• Development Experience: 5+ years with Python, Scala, Java, or C#. Experience with Python is preferred.
• Data Engineering Tools: 5+ years with Databricks, Snowflake, BigQuery, Apache Spark, HIVE, Hadoop, Cloudera, or RedShift. Experience with PySpark on Databricks is preferred.[TN1]
• Data Architecture: Proficiency in high-performance data pipeline design and mastery of pipeline guarantees, such as Idempotency, At-Least-Once Processing, Exact-Once Processing, Fault Tolerance, Eventual Consistency, Streaming Consistency, Transactional Consistency, and Observability.
• Containerization: Experience developing in a containerized environment like Docker, Rancher, or Kubernetes.
• Data Orchestration: Experience with Apache Airflow (or similar tool) for orchestrating data processing jobs.
• Education: Bachelor's degree in Computer Science (or related field) or equivalent combination of education and experience.
• Educational Qualification and Experience: Minimum of 15 years of formal education - Graduate / Post Graduate in Computer Science / Information Technology

FAI is committed to create an environment that respects, supports and inspires all individuals. We do not discriminate on the basis of color, religion, sex, gender identity, sexual orientation and age. At FAI, we celebrate diversity and believe that an inclusive workforce benefits employees, the organization and our community. We are an Equal Opportunity Employer.

For more information about our company and dedication to putting People First, check out https://firstam.wd1.myworkdayjobs.com/faicareers.