Data Ingestion & Pipeline Development
•Build and maintain data ingestion pipelines from telemetry and usage data sources.
•Develop scalable ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL.
•Support both batch and event-driven data ingestion patterns.
•Ensure data pipelines are reliable, performant, scalable, and cost-efficient.
Snowflake Data Engineering
•Create and maintain curated usage datasets in Snowflake .
•Design efficient data models, tables, views, and processing layers for downstream billing use cases.
•Optimize SQL queries, warehouse usage, partitioning strategies, and performance.
•Support analytics, validation, and reconciliation use cases on Snowflake.
Ratings Engine & Billing Transformation
•Support implementation of rating logic for usage-based billing.
•Prepare billing-ready datasets for invoice processing.
•Generate data outputs and feeds required for SAP invoicing .
•Work on pricing, usage calculation, billing rules, and financial data transformation logic.
•Ensure raw telemetry data is transformed into accurate, validated, and invoice-ready records.
Data Quality, Reconciliation & Audit Controls
•Design and implement data quality checks across ingestion, transformation, rating, and billing layers.
•Build reconciliation controls to validate data completeness, accuracy, and consistency.
•Implement audit trails for billing data movement and transformation.
•Create exception-handling and error-management processes for failed or inconsistent records.
•Support root cause analysis for billing discrepancies and data issues.
CI/CD, Monitoring & Operations
•Build and support CI/CD pipelines for data engineering deployments.
•Work with tools such as GitHub Actions, GitLab CI, Jenkins, AWS CodePipeline , or similar.
•Implement operational monitoring, alerts, dashboards, and pipeline health checks.
•Maintain technical documentation, runbooks, and support procedures.
•Collaborate with engineering, finance, billing, and operations teams to ensure smooth delivery.
Required Skills & Experience
Technical Skills
•Strong hands-on experience in data engineering , ETL/ELT development , and large-scale data pipeline design.
•Experience with: - AWS Glue
•PySpark
•Python
•SQL
•Cloud-based data processing
•Strong experience working with Snowflake as a data warehouse or enterprise data platform.
•Advanced SQL skills for: - Data analysis
•Validation
•Troubleshooting
•Query optimization
•Performance tuning
•Experience with batch and event-driven ingestion patterns .
•Understanding of: - Data modelling
•Partitioning
•Performance tuning
•Cost optimization
•Scalable pipeline design
Billing & Financial Data Skills
•Good understanding of one or more of the following: - Usage-based billing
•Rating engines
•Invoice feeds
•SAP invoice integration
•Financial data processing
•Billing reconciliation
•Experience transforming usage/telemetry data into billing-ready outputs will be strongly preferred.
Data Quality & Governance
•Experience building: - Data quality checks
•Reconciliation frameworks
•Audit controls
•Exception handling processes
•Error logging and monitoring workflows
•Ability to ensure data accuracy, traceability, and reliability across the billing lifecycle.
DevOps & Engineering Practices
•Hands-on experience with CI/CD tools such as: - GitHub Actions
•GitLab CI
•Jenkins
•AWS CodePipeline
•Similar deployment tools
•Familiarity with Git-based workflows and release management.
•Ability to write clean, maintainable, and well-documented code.
•Experience working in Agile delivery environments.
