Lead Data Engineer (ADB)

Lead Data Engineer (ADB) Apply We are looking for Lead/Senior Data Engineer (ADB) About the Client Our client is one of the Big Four accounting firms and the world’s largest professional services network. Headquartered in London, they operate in 150+ countries with 460,000+ professionals delivering excellence in audit, tax, consulting, and advisory. The team you will join designs, develops, and deploys innovative enterprise technology and AI-driven tools to support the delivery of tax services. It is a dynamic group combining expertise in tax, software engineering, change management, and project management, working on initiatives from tool design and deployment to training development and engagement management. Project Tech Stack Azure Cloud, Microservices Architecture, .NET 8, ASP.NET Core services, Python, MongoDB, Azure SQL, Angular 18, Kendo, GitHub Enterprise with Copilot, LangGraph, LangChain, RAG Pipelines, Multi-modal LLMs What You’ll Do Define and enforce best practices and coding standards across the project Conduct thorough code reviews to ensure adherence to established guidelines and maintain high code quality Working both independently and in close collaboration with others in the team Communicating clear instructions to team members and helping manage the flow of day-to-day operations Communicating with the client regularly Design, develop, and maintain robust and scalable Spark applications Write clean, maintainable, and efficient code following best practices and coding standards Optimize code for performance and scalability, ensuring efficient data handling Work closely with cross-functional teams to deliver high-quality software solutions Identify and resolve technical issues, ensuring the reliability and performance of applications Create and maintain comprehensive documentation for code, processes, and workflows What You Bring 5+ years of hands-on experience in software development Practical experience developing SQL stored procedures and migrating them into Spark SQL or PySpark environments Extensive expertise using PySpark on Databricks , encompassing Delta tables, cluster management, and workflow automation Proven track record of optimizing Spark performance within production-level projects Strong proficiency in Python , specifically for complex data manipulation utilizing libraries like Polars or Pandas Solid understanding of columnar data storage formats, particularly Parquet, with practical experience using Delta Tables Proven expertise in data processing, analysis, and transformation workflows Strong analytical and problem-solving abilities with a detail-oriented mindset Practical and pragmatic approach to balancing standardized processes with flexibility to meet project goals effectively Solid understanding of microservices architecture and its implementation in scalable systems Nice to have Experience working with Azure Cloud services (or other major cloud platforms), including a range of SaaS offerings such as Service Bus, Data Lake, Blob Storage, Redis, and more Familiarity with FastAPI Expertise in containerization and orchestration tools such as Docker and Kubernetes English level Advanced 📩 Ready to Join? We look forward to receiving your application and welcoming you to our team! Department Data Engineering Locations Multiple locations Remote status Fully Remote About Bonapolia For job seekers, BONAPOLIA offers a gateway to exciting career prospects and the chance to thrive in a fulfilling work environment. We believe that the right job can transform lives, and we are committed to making that happen for you.