QA Automation +ETL Testing

Tek Ninjas; Hyderabad; On-site; INR 1,500,000 - 2,100,000; Full time

Benefits offered by this job

null

Job summary

Tek Ninjas seeks an experienced ETL/Data Engineer to contribute to the Content Externalization Team. The role involves hands-on development of end-to-end data pipelines, pipeline modernization to Spark, and collaborating across globally distributed teams.

You will leverage Python/Scala, Spark, Hive, and AWS, ensuring data quality, governance, and efficient data processing. Strong communication and agile practices are expected for success.

Qualifications

• 5+ years in technology development and production support.
• 5+ years building pipelines that extract, transform, and load data into an information product.
• 5+ years developing and supporting ETLs using Python/Scala on Databricks and Spark.
• Experience with Python, Spark, and Hive; understanding of data warehousing and data modeling techniques.
• Knowledge of visualization tools (e.g., Tableau, R).
• Strong data engineering skills with AWS cloud platform.
• Experience with streaming frameworks such as Kafka.
• Core Java, Linux, SQL, and scripting language.
• Experience with relational databases, preferably Oracle.
• Experience in CI/CD, containers, and orchestration technologies.
• Experience working in an Agile development environment.
• Ability to quickly understand and work with new technologies.

Responsibilities

• Own analysis, design, architecture, development and support of ETL pipelines.
• Assist in migrating legacy ETL pipelines to Spark framework.
• Collaborate with global teams and cross-functional stakeholders.
• Follow agile best practices and release management.

Skills

Python/Scala

Spark

Hive

SQL

Linux

Tools

Databricks

Kafka

Tableau

Oracle

Job description

Experience : 6+

The Impact: As a member of the Content Externalization Team, you will be responsible for analysis, design, architecture, development, and support of ETL pipelines using Spark framework in Databricks platform. The ideal candidate should have expertise with cutting edge technologies and a desire to drive change through all alignment across the enterprise. The role requires the candidate to be a hands‑on problem solver and developer helping to extend and manage the ETL pipelines.

Whats in it for you:

• Exposure to work on Latest cutting‑edge Technologies.
• Opportunity to grow personally and professionally.
• Exposure to cutover legacy ETL pipelines to Spark framework.
• Responsibilities:
• Demonstrate a strong sense of ownership and responsibility with release goals. This includes
• understanding requirements, technical specifications, design, architecture, implementation, unit
• testing, builds/deployments, and code management.
• Ensure compliance through the adoption of enterprise standards and promotion of best practice /
• guiding principles aligned with organization standards.
• Build and maintain the environment for speed, accuracy, consistency and ‘up’ time
• Hands‑on position requiring strong analytical, architecture, development and debugging skills that includes both development and operations.
• Attaint in-depth Functional knowledge of the domain that we are working on.

More Details:

• Understand Incident Management, Change Management and Problem Management, root cause
• analysis.
• Ensure data governance principles adopted, data quality checks and data lineage implemented in
• each hop of the Data.
• Be in tune with emerging trends cloud technologies & Big Data.
• Should drive and execute complex technical requirements.
• Demonstrate excellent verbal and written communication skills.
• Collaborate with team members across the globe.
• Interface with cross functional teams and downstream applications as needed.
• Be a self‑starter that is also an excellent team player.
• Follow agile best practices. Experienced in working with global teams.
• Regularly evaluate cloud applications, hardware, and software.
• Work closely with cyber security team to monitor the organization’s cloud policy.
• Focus on building a team culture that is based on trust. Inspire your team member so that they
• focus on continuous growth.

What We’re Looking For

• Minimum 5+ years of working experience in Technology (application development and production support).
• 5+ years of experience in development of pipeline that extract, transform, and load data into an
• information product that helps the organization reach its strategic goals.
• Minimum 5+ years of experience in developing & supporting ETLs and using Python/Scala on both Databricks and Spark platform.
• Experience with Python, Spark, and Hive and Understanding of data-warehousing and datamodeling techniques.
• Knowledge of industry-wide visualization and analytics tools (ex: Tableau, R)
• Strong data engineering skills with AWS cloud platform
• Experience with streaming frameworks such as Kafka
• Knowledge of Core Java, Linux, SQL, and any scripting language
• Experience working with any relational Databases preferably Oracle.
• Experience in continuous delivery through CI/CD pipelines, containers, and orchestration technologies.
• Experience working in an Agile development environment.
• Experience working with cross functional teams, with strong interpersonal and written communication skills.
• Candidate must have the desire and ability to quickly understand and work within new technologies