Lead Bigdata Engineer

Job Description

Skill Set : Total Experience : No of Openings : Job Post Date : Job Expiry Date : Domain :

Location

Job Reference No : Recommend to friend

Job Summary

• Skill – Lead Hadoop Spark Developer
• Primary Skill – Python, Pyspark, Spark, SQL, Unix Shell Scripting, Airflow
• Exp – 10-15 Yrs
• Location – Chennai – (Sholinganallur SEZ) / Bangalore – (RMZ Ecoworld – Bellandur)
• Hybrid Mode – 3 Days a week as per organization policy

Data Engineering Hadoop Developer (10-15 Years Experience)

Job Summary

Seeking a Hadoop Developer with 8 years of experience in big data engineering to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies. The role involves batch/stream processing, data integration, performance tuning, and close collaboration with business teams.

Key Responsibilities

• 1. Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
• 2. Build and optimize large-scale data processing jobs using Spark and/or Hive.
• 3. Develop workflows using Airflow.
• 4. Write complex SQL/HiveQL for data transformation and reporting needs.
• 5. Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
• 6. Ensure data quality, lineage, and governance standards are followed.
• 7. Monitor, troubleshoot, and tune jobs for performance and scalability.
• 8. Create technical documentation and follow coding best practices.

Required Skills

• 1. 8 years of hands-on experience in Hadoop/big data development.
• 2. Strong knowledge of HDFS, Hive, Spark,
• 3. String knowledge in Python/Scala
• 4. Strong SQL and data modelling fundamentals.
• 5. Experience with workflow orchestration tools (Airflow).
• 6. Good understanding of Linux, shell scripting, and distributed systems.
• 7. Familiarity with version control (Git) and Agile delivery practices.

• Buddy Referral policy

• All Buddy referrals will be through Buddy Portal and Referrer must provide all relevant details like Passport Details, Date of Birth & Email ID of Buddy without any ambiguity in the Buddy portal. Profiles referred through any other means will not be eligible for consideration under the Buddy Scheme.

• ELIGIBILITY/APPLICABILITY TO PROCEED WITH BUDDY REFERRAL.

• a) Buddy referrals can be submitted against all Lateral open demand in bands U2 to E1 and P2/E1/RG1

• b) Candidature of a Buddy will remain valid for consideration against any opening for a maximum period of three months (90 days) from the date of screening by the Recruiter (Applicant id creation Date) Thus, associate would be eligible for Referral Fee even if the Buddy doesn't get selected against the opening for which buddy was referred initially but gets selected subsequently (within this six months validity period) for another opening and joins Tech M.

• An associate must fulfill following additional criteria to be eligible for Referral Fee: -

• i. Referred candidate should have been registered in the Buddy Portal with an Applicant Id.

• ii. Referrer should have provided correct details pertaining to Buddy being referred including details like IT PAN Card No., Passport Details, Date of Birth, Email ID and ex-Tech M ID details without any ambiguity in the Buddy Tool. Profile/s uploaded with incorrect details will not be eligible for the referral.

• iii. Buddy bonus payout will be subject to applicable tax deductions: -

• iv. The buddy bonus is payable after the Buddy completes 90 days in TechM subject to fulfillment of all other criteria mentioned in this policy with the ensuing payroll-cycle.

• v. At the time of payment of Referral Fee, Buddy/Referrer should not be Serving Notice Period/ Resigned/ Separated.

• vi. The Background Verification (BV) should have been successfully completed and the BV Status should be Green/Green from Red/Green from Amber. for Buddy and BV of the referral associate should Green/Green from Red/Green from Amber/NA (ie Rebadging/Acquisition/Group co) .

• vii. Candidate(s) being referred as Buddy should not have been an employee of Tech M or its subsidiaries any time in a period of 12 months immediately preceding the subscription of candidature as Buddy.

• viii. Candidate/Buddy should not be a member of family of associate referring his/her candidature. Here, the term Family encompasses Spouse, Children including legally adopted ones and Siblings .

• ix. Referrer should not have been on Selection Panel for the position for which the referred candidate is being considered for, at any stage .

• x. Referrer should not have reporting relationship with Buddy till the time payment is made of buddy joining.

• Please read Privacy Notice using this URL :

• https://careers.techmahindra.com/Terms-of-use.aspx

• (Kindly Open URL in New Tab of Browser. Right Click on Link > Open in New Tab)

• Buddy Bonus

• Band of Buddy

• India (INR)

• USA (USD)

• UK (USD)

• Europe (USD)

• ANZ (USD)

• Japan (USD)

• Indonesia & China (USD)

• ROW (USD)

• DOU Mexico (USD) *Payable as per Policy

• U2/U3

• 18,750

• 1,125

• 975

• 1,125

• 750

• 1,500

• 500

• 375

• 1,000

• U4

• 37,500

• 1,875

• 1,875

• 1,875

• 1000

• 2,250

• 750

• 525

• 1,125

• P1

• 52,500

• 2,250

• 2,250

• 2,250

• 1,500

• 3,000

• 1,125

• 750

• 1,125

• P2 / E1 / RG1

• 75,000

• 2,250

• 2,250

• 2,250

• 1,500

• 3,000

• 1,125

• 750

• 1,500

• I agree the term and condition.

• Confirmation