Data Engineer 4

Comcast · Telecom

India - Chennai, Comcast India Engineering Cent
On-site
Posted yesterday
Apply by 7 Oct
Data & AI

About the job

Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and move at an amazing pace, thanks to our remarkable people, who bring cutting-edge products and services to life for millions of customers every day. If you share in our passion for teamwork, our vision to revolutionize industries and our goal to lead the future in media and technology, we want you to fast-forward your career at Comcast.

Job Summary

This role concentrates on advanced data structures and pipelines development and maintenance, upholding quality and integrity in diverse storage solutions. It involves optimizing data flows, mentoring junior engineers, and contributing to internal tool development. The role serves as a key expert, providing strategic guidance and innovative solutions in data engineering.

Job Description

Freewheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can insert advertisements around the world.

We are looking for Data Engineer 4 to help shape the next phase of FreeWheel’s Audience Ingestion platform. This role is focused on evolving today’s multi-stage ingestion landscape into a more unified, observable, scalable, and migration-friendly pipeline for audience membership and taxonomy processing across Audience Manager and Buyer Cloud use cases.

The engineer in this role will work across the full ingestion lifecycle: partner file receipt, validation, normalization, raw and curated data processing, Snowflake-based transformation layers, Aerospike serving paths, migration controls, and operational observability. The role is expected to influence both current delivery and future-state platform design, especially around workload isolation, partner onboarding, migration stages, and automation.

Key Responsibilities

Design and enhance audience data pipelines that ingest partner files from cloud storage, validate schema and business rules, and load data into raw, foundation, and master data layers for downstream processing.

Build and maintain scalable ingestion patterns for multiple file formats and ingestion modes, while ensuring deterministic processing, replicability, and strong operational reliability.

Implement and optimize Snowflake-based data processing using external tables, CDC streams, tasks, dynamic tables, and merge patterns for incremental refresh and historical tracking.

Develop high-quality data models and storage patterns for raw ingest, append-only history, snapshot/master tables, and feature-serving layers that support both analytics and low-latency downstream use cases.

Partner with application and platform teams to support Aerospike-serving workflows, migration compatibility modes, and future cutover to more efficient replace-based update patterns.

Drive data quality across the pipeline through schema validation, reject handling, ingest status logging, file-level metadata capture, anomaly detection, and freshness/SLA monitoring.

Improve platform observability by exposing processing, freshness, backlog, and operational metrics through standardized monitoring and alerting patterns.

Contribute to workload isolation and partner-level scalability strategies so that volume, SLA, and priority can be managed independently by audience partner, client type, or pipeline category.

Support migration from legacy and parallel pipelines into the unified membership pipeline, including routing logic, partner configuration, staged cutover, and reconciliation.

Help build future-ready onboarding and provisioning capabilities for new audience partners, including configuration-driven setup of storage, processing, and data platform resources.

Collaborate with architects, product, and downstream consumers to simplify fragmented workflows, reduce operational overhead, and improve self-service visibility into data movement and pipeline health.

Mentor junior engineers and raise the bar on design quality, coding standards, testing discipline, and data platform best practices across the team.

Must-Have Skills

Advanced SQL, including strong experience with incremental processing, merge logic, CDC-driven transformations, and performance-aware data modeling.

Strong Python proficiency for data engineering, automation, and platform tooling.

Deep hands-on experience with Snowflake in production, including Streams, Tasks, Dynamic Tables, external tables, and warehouse/workload management.

Strong experience with AWS-based ingestion patterns, especially S3-centric file pipelines and cloud-native event-driven processing.

Experience designing ingestion frameworks for structured and semi-structured file formats such as CSV, TSV, and PSV, with robust schema validation and reject handling.

Experience building reliable batch and near-real-time data pipelines with a strong focus on replay, idempotency, lineage, and operational supportability.

Strong understanding of data quality, observability, SLA management, and file/job-level operational metrics.

Experience with data modeling for both historical and serving-oriented datasets, including snapshot, append-only, and feature-serving patterns.

Experience working on cloud infrastructure and deployment automation, with exposure to Terraform, schema change, or similar infrastructure/database change management tooling.

Ability to work across current-state systems and future-state platform transitions, balancing delivery, migration safety, and long-term simplification.

Good to Have

Experience with Aerospike-backed serving systems or other low-latency key-value serving layers.

Experience with Kafka, EKS/Kubernetes workers, autoscaling consumer patterns, or queue-driven distributed processing.

Familiarity with ingestion patterns or file-to-serving architectures.

Experience with Datadog or similar monitoring platforms for data pipeline alerting and operational dashboards.

Exposure to Databricks/Delta-based ingestion design, especially in environments evaluating multiple implementation paths for unified ingestion.

Experience with partner onboarding automation, region-aware data processing, and privacy/compliance-aware storage design.

Experience Profile

8+ years of experience in data engineering, with strong ownership of production data pipelines and platform-scale systems.

Proven experience leading complex technical initiatives that span ingestion, transformation, storage, and serving layers.

Demonstrated ability to shape engineering direction, not just execute tasks especially in migration-heavy, multi-system environments.

Experience mentoring engineers and influencing architecture, operational excellence, and engineering standards across teams.

Employees at all levels are expected to:

Understand our Operating Principles; make them the guidelines for how you do your job.

Own the customer experience think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services.

Know your stuff be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences.

Win as a team make big things happen by working together and being open to new ideas.

Be an active part of the Net Promoter System a way of working that brings more employee and customer feedback into the company by joining huddles, making call backs and helping us elevate opportunities to do better for our customers.

Drive results and growth.

Support a culture of inclusion in how you work and lead.

Do what's right for each other, our customers, investors and our communities.

Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.

We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools that are personalized to meet the needs of your reality—to help support you physically, financially and emotionally through the big milestones and in your everyday life.

Please visit the benefits summary on our careers site for more details.

Education

Bachelor's Degree

While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.

Certifications (if applicable)

Relevant Work Experience

7-10 Years

Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.

Apply on Comcast