Opening For Data and AI Engineer

Tekskills; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Tekskills is seeking an experienced data engineer to build end-to-end data pipelines and analytics solutions. You will collaborate with design and business teams to assemble large data sets, ensure data model integrity, and deliver APIs for analysts and data scientists.

The role demands 7+ years in DWBI with Azure, Databricks, SQL, Spark, Python and Scala skills, plus familiarity with GenAI concepts to support AI use cases.

Qualifications

• Total experience of 7+ years in IT industry in DWBI.
• Experience in Azure cloud platforms.
• Ability to define and develop data integration patterns and pipelines.
• Strong knowledge of data modelling, data warehousing, and BI concepts.
• Experience with Data Lake, Databricks, Data Factory, BI dashboards/BI implementation.
• At least 4 years experience in SQL/PostgreSQL and BI solutions with 3rd party integrations.
• Experience with ERP, Spark, Scala, Python, SQL scripting, NoSQL (e.g., MongoDB) and cloud tech (Azure).
• Familiar with GenAI concepts (RAG, LangChain, LlamaIndex) to collaborate with data scientists/ML engineers.

Responsibilities

• Build end-to-end data pipelines including collection, transformation, quality and integration for analytics solutions.
• Collaborate with solution design and business teams to identify data requirements and assemble large data sets.
• Design and fine-tune analytics solutions to meet business/technical needs.
• Ensure data model integrity with the client data architect.
• Create data tools for analytics and data scientist teams to optimize models.
• Support continuous operation with DevOps for data/analytics solutions.
• Develop APIs with data analysts/scientists.
• Provide status reports to client and Infosys managers.
• Stay curious about emerging AI trends in data engineering.

Skills

Databricks

Azure Data Factory

SQL

Spark

Scala

Python

ADLS

GenAI concepts

Education

Bachelor/Master in CS/IT

Job description

Description

Main Responsibilities

• Build end-to-end data pipelines including data collection, transformation, quality and integration solutions to facilitate Client data & analytics solutions.
• Collaborate with solution design and business requirements teams to identify data requirements and assemble large, complex data sets that meet the requirements
• Design, implement and fine-tune analytics solutions that meet business and technical requirements
• Work collaboratively with the Client data architect to ensure data model integrity
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
• Create data tools for analytics and data scientist teams that assist them in building and optimising the data models and solutions
• Work with the DevOps engineer to support the consistent operation of data & analytics solutions
• Work closely with the data analysts & data scientists to design and develop APIs
• Weekly and need basis Status reporting to Client and Infosys managers
• Self-motivated, able to work independently, and attention to details
• Curious about emerging AI trends and their implications for data engineering

7 - 12 years

Academic Qualification

• Bachelor or Masters degree in a related field (e.g. computer science, information technology, etc.)

Professional Experience

• Total experience of 7+ years in IT Industry in DWBI.
• Experience in Azure cloud platforms and technology
• Ability to define and develop data integration patterns and pipelines
• Strong knowledge of data modelling, data warehousing, and BI concepts
• Experience with Data Lake, Databricks, Data Factory, BI Dashboard, and BI implementation projects
• At least 4 years experience in SQL/PostgreSQL, data and BI solutions with integration to 3rd party tools
• Experience working with application server software (e.g. ERP), Spark, Scala, Python, SQL scripting languages, relational databases (e.g. SQL DB/DW), NoSQL platforms (e.g. HBase, MongoDB, Cassandra), cloud technologies (e.g. Azure)
• Highly experienced with processing large and complex datasets and building end-to-end data pipelines using on-premise or cloud-based data platforms
• Experienced in coding in data management, data warehousing or unstructured data environments
• Experience in the energy sector or other asset-intensive industries will be highly regarded
• Experience in using coding assistants (e.g. GitHub Copilot) for uplifting productivity in development
• Familiar with GenAI concepts like RAG, orchestration frameworks like LangChain and LlamaIndex, vector databases etc. to collaborate efficiently with data scientists and ML engineers in AI use cases

Mandatory skills

Databricks, Azure Data Factory, SQL, Spark, Scala, Python, ADLS, Familiar with GenAI concepts