Data Engineer Pleno

Tata Consultancy Services; Barueri; Presencial; BRL 20.000 - 80.000; Tempo integral

Resumo da oferta

A leading company in IT services is seeking a Senior Data Engineer to design and implement complex ETL processes and manage data pipelines on cloud platforms. The ideal candidate will leverage extensive experience in data warehousing, strong SQL knowledge, and familiarity with cloud technologies like Azure and GCP. This role demands excellent communication skills and the ability to work collaboratively with global teams.

Qualificações

• 5+ years of ETL and data warehousing development experience.
• 3+ years designing ETL and data lakes on cloud platforms.
• Experience in implementing scalable data hubs.

Responsabilidades

• Collect, store, process large datasets for ETL processes.
• Develop tools and frameworks for the developer community.
• Provide technical support throughout the project lifecycle.

Conhecimentos

ETL

Data Governance

SQL

Data Analysis

Cloud Technologies

Formação académica

Bachelor’s degree in computer science or related field

Ferramentas

Spark

Azure

GCP

Data Bricks

Snowflake

Descrição da oferta de emprego

We are looking for a Senior Data Engineer with a track record of designing and implementing large and complex technology projects at a global scale. The ideal candidate would have a solid foundation in hands-on ETL and analytical warehouse development, understand complexities in managing end to end data pipelines and in-depth knowledge of data governance and data management concepts. To be successful in this role, the candidate would require a balance of product-centric technical expertise and navigating complex deployments with multiple systems and teams. This role requires interaction with technical staff and senior business and IT partners around the world. This position is also responsible for ensuring operational readiness by incorporating configuration management, exception handling, logging, end-to-end batch, and real-time data pipeline operationalization for getting data, managing and processing into the hub.

Key Responsibilities:

• Collect, store, process and analyze large datasets to build and implement extract, transfer, load (ETL) processes

• Create frameworks and developer tools, design training materials, conduct developer training. Evangelize new ideas, standards, best practices and solutions with the developer and tech community.

• Develop quality code with thought through performance optimizations in place right at the development stage.

• Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.

• Ingesting huge volumes data from various platforms for needs and writing high-performance, reliable, and maintainable ETL code

• Interact with business analysts and functional analysts in getting the requirements and implementing the ETL solutions.

• Provide technical support to project teams throughout project lifecycle around technology platforms, solution design, security, debugging, profiling, performance tuning etc.

Essential Business Experience and Technical Skills:

Required:

• 5+ years of ETL and data warehousing development experience

• 3+ plus years of experience designing ETL and data lakes on cloud or big data-based platforms

• Demonstrated experience with implementing, and deploying scalable and performant data hubs at global scale

• Demonstrated experience in cutting-end database technologies, and cloud services such as Azure, GCP, Azure, Data Bricks or Snowflake; deep experience in technologies such as Spark (Scala/python/Java), ADLS, Kafka, SQL, Synapse SQL, Cosmos DB, Graph DBs

• Hands on expertise in: Implementing analytical data stores in Azure platform using ADLS/Azure data factory /Data Bricks and Cosmos DB (mongo/graph API)

• Strong analytic skills related to working with unstructured datasets

• Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.

• Eagerness to learn new technologies on the fly and ship to production

• Demonstrated experience in Configuration Management, DevOps automation

• Excellent communication skills: Demonstrated ability to explain complex technical content to both technical and non-technical audiences

• Experience working enterprise complex large scale, multi-team, programs in agile development methodologies

• Experience in Solution implementation, performance testing and tuning; ADLS, Synapse SQL database or GCP Big Query and GCS management and Performance tuning - Partitioning / Bucketing.

• Bachelor’s degree in computer science or related field

About the company

Tata Consultancy Services is an Indian multinational information technology services and consulting company headquartered in Mumbai, Maharashtra, India with its largest campus located in Chennai, Tamil Nadu, India.

Notice

Talentify is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.

Talentify provides reasonable accommodations to qualified applicants with disabilities, including disabled veterans. Request assistance at [e-mail] or 407-000-0000.

Federal law requires every new hire to complete Form I-9 and present proof of identity and U.S. work eligibility.

An Automated Employment Decision Tool (AEDT) will score your job-related skills and responses. Bias-audit & data-use details: www.talentify.io/bias-audit-report . NYC applicants may request an alternative process or accommodation at [e-mail] or 407-000-0000.