Data Engineer

At Brook Green Supply we are an independent B2B energy supply company, creating risk management and energy supply solutions for the UK’s I&C sector. We are reshaping the I&C energy supply sector by providing expert consultation and risk management services. Our vastly experienced team, who have operated in the energy industry for many years, are looking to provide a fantastic career opportunity for an ambitious, proactive individual, with personal development and progression aligned to the continued rapid growth and success of the business. We are seeking a Data Engineer to join our growing data engineering team. This role is perfect for someone passionate about building robust, scalable data infrastructure using open-source technologies. You'll work in an environment that values innovation, technical excellence, and collaborative problem-solving. Key Responsibilities • Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools • Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements • Develop data models and schemas that support analytics, business intelligence and product needs • Monitor, troubleshoot, and optimise data pipeline performance and reliability • Collaborate with stakeholders, analysts and product team to understand data requirements • Implement data quality checks and validation processes to ensure data integrity • Participate in architecture decisions and contribute to technical roadmap planning Required Skills and Experience Technical Skills: • Great SQL skills with experience in complex query optimization • Strong Python programming skills with experience in data processing libraries (pandas, NumPy, Apache Spark) • Hands-on experience building and maintaining data ingestion pipelines • Proven track record of optimising queries, code, and system performance • Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow) • Knowledge of distributed computing concepts and big data technologies • Experience with version control systems (Git) and CI/CD practices • Experience with relational databases (PostgreSQL, MySQL or similar) • Experience with containerization technologies (Docker, Kubernetes) • Experience with data orchestration tools (Apache Airflow or Dagster) • Understanding of data warehousing concepts and dimensional modelling • Understanding of cloud platforms using infrastructure-as-code (IaC) approaches • Knowledge of streaming data processing and real-time analytics • Experience with data quality and monitoring tools Analytical and Soft Skills: • Strong analytical mindset with attention to detail • Excellent problem-solving and debugging skills • Ability to work independently and manage priorities • Strong communication skills and ability to explain technical concepts to non-technical stakeholders • Experience working in agile development environments • Passion for continuous learning and staying current with emerging technologies Preferred Qualifications: • Bachelor's degree in Computer Science, Engineering, Mathematics, or related field • 2-5 years of experience in data engineering or related roles • Experience working in the Energy industry Data Engineer • Experience with columnar databases (ClickHouse or similar) • Experience with open-source data catalogue and metadata management tools • Experience with performance tuning • Experience with data visualisation tools (Tableau) • Experience with GraphQL Why you’ll love working here This is a role where you will have genuine opportunities for growth and development. You will gain hands-on experience in data analysis, the UK energy market and gas and electricity invoicing while working in a collaborative, supportive team that values initiative and curiosity. You will have the flexibility to balance office and remote work, and you will be part of a business that trusts its people and rewards contribution. We also offer a comprehensive benefits package, including: • 25 days annual leave plus Bank Holidays • Flexible working arrangements • Private health care • Company pension scheme • Life and medical insurance, eyecare scheme • Employee Assistance Programme • Cycle to Work scheme • Family-friendly policies • Recruit and Reward scheme • Access to Perkbox benefits platform A note from us Even if you don’t tick every box, we would still love to hear from you. If you are detail-oriented, analytical, and motivated to grow your career in the energy industry, we will give you the training and support to succeed. Brook Green Supply is committed to ensuring equal opportunities, fairness, dignity, and respect for all employees and applicants. Diversity makes us stronger, and we welcome applicants from all backgrounds.