Data Engineer- Databricks + AWS

InfoCepts; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

InfoCepts invites a Data Engineer specialized in Databricks and AWS to develop, support and steer end-to-end BI solutions. You will work on PySpark code and data pipelines to enable analytics use cases for the client, collaborating with architects and cross-functional teams.

The role requires 5+ years of DataEngineering experience, strong SQL, warehousing concepts, and hands-on Databricks knowledge including Unity Catalog and Autoloader.

Qualifications

• Bachelor's degree in computer science, engineering, or related field.
• Demonstrated continued learning through technical certifications or related methods.
• 5+ years of relevant experience in DataEngineering.
• Relevant experience in media domain.
• Thorough understanding of data warehousing concepts and technologies.

Responsibilities

• Work with account/project architects to understand and translate requirements into technical implementations.
• Develop, implement, and maintain PySpark code through Databricks UI that enables data and analytics use cases for the client.
• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.

Skills

Databricks

PySpark

Python

SQL

Data warehousing

Unity Catalog

Notebook

Autoloader

Genie

AWS - S3/Lambda

Education

Bachelor's degree in Computer Science/Engineering

Job description

Position:DataEngineer- Databricks + AWS

Purpose of the Position:

Develop,support and steer end-to-end businessintelligenceusingDatabricks

Location:Nagpur/ Pune/ Bangalore/ Chennai

Key Responsibilities:

• Work withaccount/projectarchitects tounderstand and translaterequirements into technical implementations.

• Develop, implement, and maintainpysparkcode through Databricks UIthat enables data and analytics use casesfor the client.

• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.

Roles and Responsibilities

Work and Technical Experience:

Mandatory Skills:

• Overall 5+ and 3+Hands onExperience with Databricks,PySparkand Python

• Thorough understanding ofDatabricksUnityCatalog, Notebook,Autoloader,Data apps,Genieand other key services/features

• Proficiencyin SQL and data manipulation skills

• Understanding of data warehousing concepts and technologies.

Desirable Skills:

• Understanding ofAWS platform and its key services like S3, Lambda etc.

• Familiarity with ETL processes and tools for data extraction, transformation, and loading.

• Familiarity withAI/BI dashboards in Databricks, Agentic workflows in Databricks

Qualifications:

• Bachelor's degree in computer science, engineering, or related field

• Demonstrated continued learning through one or more technical certifications or related methods

• 5+years of relevant experiencein DataEngineering

• Relevant experience inmediadomain

Qualities:

• Self-motivated and focused on delivering outcomes for a fast-growing team and firm

• Able to communicate persuasively through speaking, writing, and client presentations

• Able to consult, write, and present persuasively

• Able to work in a self-organized and cross-functional team

• Able to iterate based onnew information, peer reviews, and feedback

• Able to work with teams and clients in different time zones

• Research focused mindset