Data Engineer

Data Engineer Collinson is the global, privately-owned company dedicated to helping the world to travel with ease and confidence. We use our expertise and products to craft customer experiences which enable some of the world’s best known brands to acquire, engage and retain the most demanding and choice-rich customers. In particular, our unique expertise and insight into high earning, frequent travellers allows us to create products and solutions for our clients that inspire greater customer engagement to drive more profitable relationships, enrich their travel experiences, protect what matters and assist in in times of need. While specialising in Financial Services, Travel and Retail, we also support clients in multiple sectors. We have worked with over 90 airlines, 20 hotel groups and more than 600 financial institutions and banks, with clients including Accor Hotels, Air France KLM, American Express, British Airways, Cathay Pacific, Diners Club, Mandarin Oriental, Mastercard, Radisson Hotel Group, Sephora, Visa and Vhi. We take our 30 years’ experience working with these kinds of household names in over 170 countries, and help our clients to deliver the smarter experiences it takes to differentiate their propositions, and help them win deeper devotion with their customers. Collinson is a privately-owned entrepreneurial business with 2,000 passionate people working in 20 locations worldwide. Our solutions include Priority Pass, the world’s best known airport experiences programme, while we are also the trusted partner behind many of the leading financial services, airline and hotel brand’s reward programmes and loyalty initiatives. Purpose of the job The Full-Stack Data Engineer will play a critical role in Collinson's efforts to deliver customer-centric global solutions with a focus on hands-on execution. As a key member of the data engineering team, you will be responsible for designing, building, and maintaining data pipelines and infrastructure in both AWS and other cloud environments. Your hands-on experience in delivering real-world solutions will be invaluable as the team works to modernize its data landscape. In this hands-on role, you will be tasked with leveraging your technical expertise to deliver customer centric solutions that meet the unique needs of internal and external stakeholders across the Collinson group's global footprint. Additionally, you will have the opportunity to bring innovation to the team, using cutting-edge data engineering frameworks and techniques. As a Full-Stack Data Engineer, you will be expected to not only deliver on technical projects, but also to mentor junior members of the team and help them get up to speed quickly. Strong communication and interpersonal skills are essential, as you will be working closely with stakeholders and clients to understand their data needs and help bring their data-driven initiatives to life. Key Responsibilities - Design, develop, and maintain data pipelines for collecting, transforming, and loading data into various data stores. - Build and maintain data warehousing and data lake solutions - Develop and deploy data models that support various business requirements - Write efficient and scalable code in languages such as Python, Scala, or Java - Lead the design of data solutions with quality, automation, and performance in mind - Own the data pipelines feeding into the Data Platform ensuring they are reliable and scalable - Ensure data is available in a fit-for-purpose and timely manner for business and analytics consumption - Work with Data Governance team to ensure solutions are compliant with regulations such as GDPR and CISO policies and data quality is baked-in to pipelines - Maintain and optimise existing data pipelines to improve performance and quality, minimising impacts to business - Collaborate with cross-functional teams to understand data requirements and provide support for data-driven initiatives - Set and embed standards for systems and solutions, and share knowledge to keep the team engaged and skilled in the latest technology - Prototype and adopt new approaches, driving innovation into the solutions - Work closely with the Data Product Manager to support alignment of requirements and sources of data from line of business systems and other endpoints - Effectively communicate plans and progress to both technical and non-technical stakeholders - Develop and implement the Data roadmap for strategic data sets - Communicate complex solutions in a clear and understandable way to both experts and nonexperts - Mentor and guide junior members of the team to help them get up to speed in a short amount of time - Interact with stakeholders and clients to understand their data requirements and provide solutions - Stay up-to-date with industry trends and technology advancements in data engineering - Promote the Data Platform and Data & Analytics team brand throughout the business and represent the interests of data engineering in cross-functional forums - Champion the importance of modern data solutions across the business and support the education of colleagues on the business value of obtaining good quality data Knowledge, skills and experience required - Extensive experience leading AWS and cloud data platform transformations - Proven track record of delivering large-scale data and analytical solutions in a cloud environment - Hands-on experience with end-to-end data pipeline implementation on AWS, including data preparation, extraction, transformation & loading, normalization, aggregation, warehousing, data lakes, and data governance - Expertise in developing Data Warehouses - In-depth understanding of modern data architecture such as Data Lake, Data Warehouse, Lakehouse, and Data Mesh - Strong knowledge of data architecture and data modelling practices - Cost-effective management of data pipelines - Familiarity with CI/CD driven data pipeline and infrastructure - Agile delivery approach using Scrum and Kanban methodologies - Ability to scope, estimate, and deliver committed work within deadlines, both independently and as part of an agile team - Supporting QA and user acceptance testing processes - Innovative problem-solving skills and ability to provide clear recommendations - Understanding of the impact of changes to business rules on data processes - Excellent communication and influencing skills, especially in regard to data solutions and outcomes - Experience managing and leading a small team of data engineers - Self-driven and constantly seeking opportunities to improve data processes - Strong knowledge of how data drives analytical and reporting activities, including automated marketing and personalization capabilities Python, PySpark, SQL, NoSQL DBs (Mongo), Bash Scripting, Snowflake, DBT, Kafka, Airflow, Glue, Github, shell Scripting, AWS, Kinesis, Terraform, APIs, and Lakehouse. Person Specification • Self-motivator with a desire to learn new skills and embrace new technologies in a constantly changing technology landscape • Ability to thrive in a fast moving environment • Ability to show initiative, innovation and work independently when required • Ability to work at pace and tackle project challenges in a collegiate, collaborative way • Goal and outcome orientated • Thoroughness and attention to detail • Good communication skills (ability to present, inform and guide others) • Ability to bridge communications between technical and business focused groups • Comfortable working with people at all levels in an organisation Division Technology & Data Locations Cape Town Remote status Hybrid Cape Town About Collinson We use our expertise and products to craft customer experiences. Our range of services helps global brand acquire, engage and retain choice-rich customers. © 2023 Collinson International Limited. Registered in England & Wales under registration No. 2577557 Registered address : 3 More London Riverside, London, SE1 2AQ, United Kingdom. Co-workers 2000