Data Engineer

PropMarker; Ernakulam; On-site; INR 1,200,000 - 1,800,000; Full time

Benefits offered by this job

Real AI product in PropTech space

Collaborate with data science & prod

Growth opportunities

Job summary

PropMarker seeks a Data Engineer to design, build and maintain large-scale data systems and pipelines. You will collaborate with data scientists, analysts and stakeholders to translate data needs into robust solutions.

You will develop architectures, implement web scraping pipelines, and design ETL/ELT workflows with a focus on performance, security and scalability. A strong background in Python, SQL and modern data tooling is required.

Qualifications

• Bachelor's or Master's in CS/IT or related field.
• 25 years of experience as a Data Engineer or similar role.
• Strong Python and SQL skills; hands-on PySpark, Airflow, Snowflake, MongoDB and Databricks.

Responsibilities

• Develop, construct, test and maintain architectures such as databases and large-scale processing systems.
• Design, build and maintain data pipelines using Python, PySpark, Airflow, Snowflake, MongoDB and Databricks.
• Build and maintain web scraping pipelines using Scrapy and Playwright/Selenium, including handling anti-bot measures, scheduling and data cleaning.
• Design ETL/ELT workflows and data warehouse models, using dbt where suitable.
• Design and optimise database schemas, queries and storage for performance, reliability and cost.
• Add data quality checks, validation, monitoring and alerting to our pipelines.
• Implement data security, privacy and access control measures.
• Work with stakeholders to understand data requirements and deliver solutions that meet them.
• Collaborate with data scientists and analysts to support their analytical and machine learning needs.
• Prepare data for AI features, including LLM and RAG use cases.
• Develop and maintain clear technical documentation.

Skills

Python

SQL

PySpark

Airflow

Snowflake

MongoDB

Databricks

Scrapy

Git

Data pipelines

Education

Bachelor's or Master's degree in Computer Science/IT

Tools

dbt

Playwright

Selenium

Job description

Role & responsibilities

You will develop, build, test and maintain the databases and large-scale data processing systems that store, retrieve and analyse our data efficiently. You will work closely with data scientists, data analysts and other stakeholders to understand their data needs and turn them into reliable solutions.

• Develop, construct, test and maintain architectures such as databases and large-scale processing systems
• Design, build and maintain data pipelines using Python, PySpark, Airflow, Snowflake, MongoDB and Databricks
• Build and maintain web scraping pipelines using Scrapy and Playwright/Selenium, including handling anti-bot measures, scheduling and data cleaning
• Design ETL/ELT workflows and data warehouse models, using dbt where suitable
• Design and optimise database schemas, queries and storage for performance, reliability and cost
• Add data quality checks, validation, monitoring and alerting to our pipelines
• Implement data security, privacy and access control measures
• Work with stakeholders to understand data requirements and deliver solutions that meet them
• Collaborate with data scientists and analysts to support their analytical and machine learning needs
• Prepare data for AI features, including LLM and RAG use cases
• Develop and maintain clear technical documentation

Preferred candidate profile

• Bachelor's or Master's degree in Computer Science, Information Technology or a related field
• 25 years of experience as a Data Engineer or in a similar role
• Strong programming skills in Python and strong SQL
• Hands-on experience with PySpark, Airflow, Snowflake, MongoDB and Databricks
• Proven experience in web scraping (Scrapy)
• Experience designing, developing and maintaining data pipelines
• Strong knowledge of database design, data warehousing and ETL processes
• Familiarity with Git and version control
• Strong problem-solving and analytical skills
• Excellent communication and collaboration skills
• Able to work independently and in a team

Perks and benefits

• Work on a real AI product in the PropTech space
• Collaborate directly with data science and product teams
• Learning and growth opportunities.
• Good office environment in Kochi .