Senior Data Engineer

About UBDS UBDS Group empowers organizations to achieve remarkable digital transformations through innovative technology solutions We work with clients across the public and private sector to design and build products, platforms and services that genuinely make a difference to users. We’re looking for a Senior Data Engineer who combines deep technical capability with a strong consulting mindset. Someone who enjoys ambiguity, cares about users, and can turn messy business problems into robust, production-grade data solutions. The Role As a Senior Data Engineer at UBDS, you’ll work closely with clients, designers, analysts and engineers to shape and deliver modern data solutions end to end. You’ll be trusted to lead technical thinking, make pragmatic design decisions, and support teams in delivering high‑quality outcomes that stand up in the real world. This is a hands-on role with influence: you’ll be building, guiding others, and helping clients understand what good looks like. What You’ll Do Turn ambiguous business problems into clear data, analytical and technical solutions. Design, build and operate production-grade data pipelines, models and products using SQL, Python and modern data engineering frameworks. Architect and work across data warehouses, lakehouse and event-driven patterns, including strong semantic models for analytics and visualisation. Deliver solutions on cloud data and AI platforms (AWS/Azure), applying solid software engineering, testing and CI/CD practices. Embed analytics, experimentation, AI/ML integration, governance, security and data quality best practices by default. Create clear data visualisations, narratives and consultant-grade materials that help clients understand and act on insight. Work closely with users and stakeholders in Agile, client-driven environments, learning from live services and real-world feedback. Apply strong critical thinking, attention to detail and relevant industry and government standards to deliver solutions that are easy to hand over, maintain and evolve. What We’re Looking For Core Data & Technical Skills Ability to translate ambiguous business problems into clear data, analytical and technical requirements. Strong SQL and Python skills for data modelling, transformation, automation and analysis. Experience designing data models and architectures across data warehouses, lakehouse and event-driven patterns, or strong semantic modelling in Power BI or other visualisation tools. Hands-on experience with modern data engineering frameworks such as Spark / PySpark, Pandas and orchestration tools. Experience working with cloud data and AI platforms on AWS or Azure (e.g. Databricks, Redshift, Bedrock, Microsoft Fabric). Solid software engineering fundamentals, including Git, testing and QA, CI/CD pipelines and modular, maintainable design. Strong analytics and statistical reasoning, including metric design, experimentation and interpretation of results. Practical AI/ML literacy, including integrating model APIs, evaluation approaches and prompt orchestration. Experience applying data governance, security and quality best practices, such as access control, data catalogues, lineage and GDPR awareness. Ability to communicate insight through data visualisation and storytelling using tools like Power BI, Tableau or Python-based charts. Consulting & Delivery Mindset A user-centred approach: curiosity about who will use the solution, why it matters, and the impact it will have. Confidence communicating clearly and working effectively in ambiguous, client-driven environments. Ability to produce high-quality, consultant-grade slides and written materials. Strong critical thinking, with a clear and logical approach to defining and delivering data solutions. Proven experience delivering production-grade solutions to real users, supporting live services and learning from real-world feedback. High attention to detail, including readable code, consistent naming conventions and clear technical and user documentation. A product-driven mindset, with experience delivering features and outcomes iteratively in Agile teams. Familiarity with relevant industry and government standards, and a habit of seeking out appropriate standards at the start of delivery rather than reinventing solutions.