…

Lead II - Data Engineering

UST · IT Services & Consulting

  • Hyderabad, Bangalore
  • On-site
  • Posted today
  • Data & AI

About the job

we have a opening for big data developer • 7-10 years of professional experience in data engineering. • Minimum 4 years building and operating production data pipelines on AWS. • Strong programming ability in Python or Scala, with production-quality code standards. • Expert-level SQL, including window functions, CTEs, and query plan analysis. • Deep, hands-on Apache Spark experience — not just usage, but performance tuning and debugging. • Demonstrated ability to diagnose Spark data skew, shuffle bottlenecks, and memory issues. • Production experience across the AWS data stack: EMR, S3, Athena, Redshift, and Lambda. • Streaming data experience with Amazon Kinesis or Apache Kafka. • Workflow orchestration with Apache Airflow (MWAA) or an equivalent scheduler. • Solid data modelling fundamentals, including dimensional modelling and slowly changing dimensions. • Working knowledge of data lake table formats such as Apache Iceberg, Hudi, or Delta Lake. • Experience operating what you build — CloudWatch, structured logging, ing, and on-call participation. • Familiarity with version control and collaborative development workflows using Git. • Ability to work directly with non-technical stakeholders and communicate trade-offs clearly