Senior Specialist - Data Engineering
LTIMindtree · IT Services & Consulting
- Bengaluru, India
- On-site
- Posted today
- Data & AI
About the job
Role: AWS Data Engineer
Responsibilities:
Design develop and optimize largescale data processing pipelines using Apache Spark
Build and maintain ETLELT workflows using AWS Glue Amazon S3 and Amazon Athena
Orchestrate endtoend data pipelines and processing workflows using AWS Step Functions
Develop ingestion frameworks to handle multivendor data schemas including Ericsson and Oracle datasets
Standardize transform map and integrate heterogeneous vendor data into common data models
Implement data validation reconciliation error handling and dataquality controls across ingestion pipelines
Optimize Spark jobs Glue workloads storage structures queries and underlying datastore performance
Monitor pipeline execution troubleshoot failures perform rootcause analysis RCA and improve processing reliability
Support schema evolution metadata management partitioning and performance optimization for large datasets
Collaborate with architecture platform analytics and downstream application teams to ensure scalable and reliable data delivery