Data Engineer

Responsibilities:

• Work closely with data product team and business end-users to implement and support data platforms using best-of-breed technology and methodology.

• Work closely with product vendors to design, deploy and integrate different data platforms with the organization’s ecosystem.

• Ensure that the deployed platforms follow the IT security and compliance policies.

• Establish processes and develop monitoring mechanisms to detect the availability of the platforms.

• Act as the solution architect and develop the end-to-end onboarding of projects onto the data platforms deployed in the organization for different BI needs such as data ingestion, visualization, data governance.

• Perform testing on the deployed data platforms.

SKILLS:

• At least 5 years’ experience of working as a platform support engineer or a software engineer or a data engineer.

• Solid working knowledge of implementing the optimal data structures and algorithms to create efficient and scalable applications in Java or Python.

• Strong hands-on experience and working knowledge of application implementation to implement new platforms.

• Familiar with application integration using RDBMS.

• Hands-on experience of using Linux (or Unix-like OS) as the development environment, and familiar with Shell scripts and command line tools.

• Knowledge of data integration and data visualization platforms such as Informatica, Tableau, and Power BI.

• Exposure and knowledge in the following technologies will be advantageous:

  • Java, Python

  • Snowflake, Oracle, MS SQL, Denodo

  • AWS services

• Experience in the Systems Development Life Cycle (SDLC) implementation methodology and/or Agile methodologies like Scrum and Kanban.

• Good understanding and able to apply good industry practice of code versioning, testing, CICD workflow, and code documentation.

• Good team player, with strong analytical skill and enjoy complex problem solving with innovative ideas.

• Good communication/people skills and able to interact with data analysts, business end-users and vendors to design and develop solutions.