Lead Data Platform Engineer

Lead Data Platform Engineer

Data platform, client data intake and delivery

Title Lead Data Platform Engineer

Location: Singapore or Vietnam (remote-friendly, with occasional travel as required)

Team: Data / Product Delivery; working closely with Product, Engineering, Client Success and senior client stakeholders

Role type Full-time

About Company

Out client is a data and AI-led retail technology business helping enterprise retailers make better commercial decisions across category, assortment, space and store execution. We work with leading retailers across APAC and global markets, combining data engineering, analytics, product capability and practical client delivery.

About the Role

We are looking for a hands-on Lead Data Platform Engineer to own the practical data layer between client systems, Our platforms and internal and external services. This is not a narrow pipeline-only role. You will architect reliable data intake processes, transform complex client data into trusted models, and work directly with client-facing and engineering teams to ensure delivery is both commercially useful and technically robust. The right person will be ambitious, accountable for outcomes and comfortable moving between architecture, technical design and hands-on execution. You will take ownership beyond your individual tasks, step into difficult technical problems when required, and use AI pragmatically to improve engineering speed, quality and repeatability.

Role Mission

• • Architect scalable platforms and processes for client data intake, validation, transformation and downstream consumption.
• • Create reliable, repeatable data pipelines that support internal products, client-facing services and analytical outputs.
• • Raise the quality bar across coding standards, deployment strategies, data modelling rules and data reliability controls.
• • Provide delivery leadership across project roadmaps, task ownership, technical feedback and solution evaluation.
• • Bridge client requirements and engineering execution so Omnistream captures the right data, models it properly and delivers trusted outputs.
• • Identify and implement practical AI-enabled workflows that accelerate delivery without compromising quality, security or governance.

What You Will Own

• • Client data architecture: design the target approach for ingesting, storing, transforming and serving client data across Omnistream use cases. Omnistream | Lead Data Platform Engineer
• • Data intake and transformation: build and improve repeatable processes that move client data from source systems into analytics-ready datasets.
• • Data modelling standards: define clear rules for data structures, naming conventions, logic, assumptions, lineage and model handover.
• • Pipeline reliability: ensure core pipelines are available, monitored, maintainable and resilient enough to support client commitments.
• • Data quality and consistency: manage missing values, incorrect data, duplicates, anomalies, schema changes and business logic issues that undermine trust in outputs.
• • Technical standards: establish practical coding, testing, deployment and review expectations so delivery does not depend on individual judgement alone.
• • Project roadmap execution: break down technical work, establish clear ownership, track dependencies and surface delivery risks early.
• • Technical design and solution review: evaluate proposed solutions, challenge weak assumptions and get into the detail where needed to drive the right outcome.
• • Client and engineering alignment: work across clients, Client Success, Product and Engineering to determine what data is required, where trade-offs exist and what is needed for acceptance.
• • AI-enabled engineering productivity: identify opportunities to automate repetitive work, accelerate analysis, improve documentation and strengthen engineering workflows using AI and LLM-enabled tools.
• • Documentation and knowledge transfer: document data flows, decisions, assumptions, QA checks, dependencies and operating playbooks so knowledge is not locked with one person.

Key Responsibilities

• Design, build and maintain production-grade data pipelines across cloud storage, warehouses, databases and workflow orchestration tools.

• Develop reusable data intake templates, validation checks and transformation patterns for new and existing client deployments.

• Use SQL and Python to clean, reconcile, model and transform structured and semi-structured data.

• Create automated checks for missing values, incorrect values, schema drift, data mismatches and business logic inconsistencies.

• Troubleshoot pipeline failures and data quality issues quickly, with clear communication on root cause, impact, ownership and remediation.

• Maintain safe deployment practices so code, data models and workflow changes move reliably across environments.

• Translate client questions and operational requirements into data requirements, acceptance criteria and delivery milestones.

• Review technical designs with Engineering and Product teams and ensure solutions are fit for scale, maintainability and client delivery.

• Support client onboarding, deployment, go-live and post-deployment optimisation.

• Use AI-assisted tools responsibly to speed up development, testing, documentation, troubleshooting and solution exploration.

• Escalate risks early where data quality, unclear ownership, client dependencies or technical debt could impact delivery.

• Take accountability for the end outcome, even where delivery depends on multiple teams.

Technology Stack: Jira AWS Redshift AWS S3, Azure Blob Storage PostgreSQL Dagster Python, pandas, SQL.

What We Are Looking For

• 5+ years of experience across data engineering, analytics engineering, platform engineering, data solutions or implementation-focused data roles.

• Strong SQL capability, including complex joins, CTEs, window functions, reconciliation checks, warehouse-aware design and query performance considerations.

• Strong Python capability for data processing, automation, scripting and production-quality workflow support.

• Hands-on experience with cloud data environments, ideally AWS Redshift, AWS S3 and/or Azure Blob Storage.

• Experience with orchestration and production data workflows; Dagster experience is highly valuable.

• Strong understanding of data modelling, data contracts, data quality frameworks and analytics-ready dataset design.

• Ability to set standards rather than only follow tickets, including coding conventions, deployment approach, modelling logic and QA expectations.

• Confidence reviewing technical proposals, challenging weak assumptions and giving constructive feedback to engineers.

• Ability to work with clients and client-facing teams to determine the actual data required to solve business problems.

• A high ownership mindset, with accountability for outcomes rather than activities and the judgement to remove blockers across team boundaries.

• Comfort operating in a lean, fast-moving environment where roles evolve and ambiguity is part of the operating model.

• The ability to articulate how you use AI in your day-to-day work and where AI should not be relied upon. Highly Valued Experience

• Retail, grocery, pharmacy, logistics, supply chain, category management, merchandising or store operations data experience.

• Experience building data platforms or delivery frameworks used by both internal teams and client-facing services.

• Experience coordinating technical project roadmaps, assigning ownership and tracking dependencies.

• Experience improving data reliability through monitoring, testing, validation and documented operating processes.

• Exposure to BI tools, data products, SaaS deployments, AI-enabled workflows or decision-support platforms.

• Demonstrated use of AI or LLM-enabled tools to improve engineering productivity, automate repetitive work or accelerate delivery while maintaining quality and governance.

• Fluent English communication