Senior Data Engineer

Senior Data Engineer Senior Data Engineer at K3 Advisory Group: build governed data foundations, scalable pipelines and semantic layers that drive AI-enabled, compliant advisor workflows. About K3 Advisory Group K3 Advisory Group is a UK professional services group with 18 trading subsidiaries and more than 1,200 staff. The Group spans corporate finance, tax, restructuring and insolvency, legal, financial planning, and technology-enabled advisory services. All technology delivery must balance the pace required to exploit AI, data and automation with strict expectations around governance, regulatory obligations, security-by-design, audit trails, and client confidentiality. Group Technology builds shared platforms, data foundations and AI-assisted products that scale across these businesses while accommodating local variation. Engineers in this team work close to commercial outcomes, with direct visibility of advisors, partners and clients. Role Purpose Design and build the governed analytical data foundations, semantic contracts and data pipelines that underpin K3's secure products, embedded analytics and AI-assisted workflows. This is a senior role and carries a clear quality bar: data flowing through the platform supports advisor, partner and (in places) client-facing decisions across multiple subsidiaries, including FCA-regulated entities. Tenancy, lineage, quality and auditability are not aspirations — they are baseline expectations. The role also helps set the standard for how acquired businesses are integrated onto the Group's data platform without compromising those standards. Key Responsibilities Data Platform & Modelling • Build and maintain scalable data pipelines, models and serving layers across cloud data platforms. • Define clean analytical schemas and semantic contracts for product, dashboard and AI consumption. • Implement ELT/ETL pipelines using SQL, Python, DBT and modern orchestration patterns. • Design data models that support tenant/client separation, permission-aware access and audit requirements from the ground up. • Apply lakehouse/medallion patterns (bronze/silver/gold) where they earn their place, avoiding architectural complexity that does not serve a clear use case. Quality, Reliability & Observability • Build quality checks, reconciliation controls and monitoring for data freshness, completeness and accuracy. • Instrument pipelines with logging, metrics and alerting so that failures are detected and triaged quickly. • Define and maintain SLAs/SLOs for critical datasets in collaboration with product and business owners. • Own incident response and post-incident learning for the data domains assigned to the role. Integration & Acquisition Support • Integrate operational systems, finance systems, CRM systems, external APIs and reporting platforms where needed. • Help onboard acquired businesses onto the Group's data platform with a repeatable, documented pattern rather than bespoke effort each time. • Work with AI/ML engineers to expose approved data functions without leaking raw platform details or unauthorised data access. • Support embedded analytics and native dashboard development by providing governed metric definitions and performant data views. Engineering Standards & Mentoring • Champion CI/CD, code review, documentation, naming standards and data governance practices. • Mentor less experienced engineers and help set the quality bar for data engineering delivery. • Contribute to platform decisions : tooling selection, modelling conventions, lineage approach and metric definition processes. • Drive a culture of "build it once, use it many times" so that delivery teams benefit from clear source mappings, metric definitions and data contracts. Required Experience & Skills • 5+ years commercial data engineering experience, including time spent in senior or lead capacities. • Strong SQL and Python skills. • Production experience with Microsoft Fabric, Databricks, Snowflake or a similar cloud data platform. • Experience with DBT, medallion architecture, lakehouse/warehouse modelling and data quality frameworks. • Experience designing semantic layers, metric definitions or governed analytical contracts. • Experience with APIs, system integrations, monitoring, error handling and pipeline observability. • Strong understanding of secure multi-tenant or permission-aware data access patterns. • Experience supporting analytics, dashboarding, AI/ML or data product teams. • Able to work effectively with UK stakeholders and translate business needs into data architecture and delivery plans. Desirable Experience • Exposure to professional services, financial services or legal data domains (client/matter/case, finance, risk, pipeline). • Experience supporting AI/LLM use cases through curated semantic layers, metric stores or retrieval datasets. • Familiarity with Azure-native data tooling, including Microsoft Fabric, Synapse, Data Factory and Purview. • Experience with data contracts, data products or domain-oriented data ownership (e.g. data mesh-style patterns) applied pragmatically. Success Measures • Reliability : Data pipelines are reliable, monitored and recoverable, with clear ownership and documented runbooks. • Reuse : Analytical contracts support product, dashboard and AI use cases without bespoke rework for every client or workflow. • Quality : Data quality issues are detected early and resolved with clear ownership; reconciliation against source systems is routine, not heroic. • Governance : Data access patterns preserve tenancy, permissions and auditability across all subsidiaries, including FCA-regulated entities. Working Environment • Reporting line : Group Technology leadership, working day-to-day within a cross-functional product squad (engineering, data, AI, design, product). • Stakeholders : UK-based business leaders, partners, operational teams, compliance and risk functions, and client-facing advisors across multiple subsidiaries. • Delivery model : Iterative, product-led delivery with short feedback loops, paired with the governance discipline appropriate to a regulated professional services environment. • Tooling baseline : Modern cloud platform (Azure-first), Git-based source control, CI/CD pipelines, infrastructure-as-code, observability tooling and a documented engineering handbook. • Ways of working : Code review, pairing, design reviews, threat modelling for sensitive features, and lightweight architecture decision records (ADRs). Governance, Security & Compliance Expectations Every engineer in Group Technology is expected to treat the following as non-negotiable foundations, not optional extras: • Confidentiality : Client, matter, and case data is highly sensitive. Need-to-know access is the default; broad access is the exception and must be justified. • Security by design : Threat modelling, secure defaults, secrets management, dependency scanning and least-privilege access are built into features from day one. • Auditability : User actions, data access and administrative changes are logged in a tamper-evident, queryable form suitable for internal audit and regulatory review. • Responsible AI : Where AI is used, model behaviour, prompts, tools and data access are versioned, evaluated and monitored. Human oversight is preserved for material decisions. • Regulatory awareness : For features touching FCA-regulated entities (e.g. Pareto, Luna), additional controls apply around record-keeping, client communications and data handling. Engineers are expected to flag uncertainty early. • Data protection : UK GDPR and Group data protection standards apply across all subsidiaries; data minimisation, lawful basis and retention controls are part of normal design. Development & Progression • Clear engineering career path with senior, lead and principal levels, plus a parallel route into architecture or engineering management. • Exposure to acquisitions, integrations and greenfield product builds across multiple professional services disciplines. • Supported learning budget, certifications relevant to the role, and time allocated for proof-of-concept work and tooling improvements. • Direct line of sight to commercial outcomes — engineers see how their work changes how advisors and clients actually operate. Person Specification • Seniority : Sets standards, mentors others, and is trusted with material decisions about data architecture. • Outcome-focused : Optimises for the right answer reaching the right user safely — not for technology choices in isolation. • Governance-aware : Sees tenancy, lineage and auditability as enablers of speed, not blockers of it. • Pragmatic : Knows when a simpler model serves the business better than a more sophisticated one. • Collaborative : Works closely with AI engineers, full-stack engineers, analysts and business stakeholders, and translates fluently between them. Division K3 Advisory Group Department K3 Advisory Group - IT Locations Kuala Lumpur Remote status Hybrid Yearly salary MYR138,650 - MYR149,321 Employment type Full-time Employment level Executive/Senior Level About K3 Advisory Group With over 1,200 employees across the Group, 25 offices in the UK, and international bases in Malaysia and Cyprus Founded in 1998 Co-workers 1200+ Turnover 113M