This application is now closed
This job is part of an Advanced Research + Invention Agency-funded project.
Dovetail is an agent foundations research group. We’ve recently received an ARIA grant to fund more team members. This application is for a 10-week fellowship in graduate-level mathematics, with the possibility of extension (funding permitting).1
Due to funding conditions, applicants must be based in the UK. You must be resident and eligible to work in the UK. This application will be open until the 17th May, but we will be interviewing and accepting on a rolling basis so we strongly encourage you to apply earlier.
You can read more about agent foundations and our research agenda on our website.
What the role might be like
Alfred and Alex will be leading a group of roughly 4 to 6 other people all engaged in mathematical AI safety research. We’re open to experience levels from undergraduates who are enthusiastic about maths to post-docs who are looking for a way to transition into AI safety.2 (We also welcome applicants outside academia!)
Some group members might work together, while others might do solo projects. All group members will have regular one-on-one check-ins with us, and we’ll also hold regular group meetings. We’ll spend our time finding useful formal definitions of relevant concepts, formulating and proving theorems about them, and communicating our ideas and results. We’ll find and share relevant papers, host read-through meetings, and discuss ideas with external researchers.
Here are some of the basic parameters of the fellowship.
- £62,400 full-time equivalent, pro-rated (£30 per hour)
- Full- to half-time. We can be flexible with hours if you need to fit this job around other work or study.
- 10 weeks, starting 1st June 2026 at the latest. Due to conditions of our funding, the fellowship must be completed by mid August, meaning that the latest you could start is 1st June. We would be happy if you wanted to start earlier. It seems to take people roughly about 6 weeks to get up to speed on enough agent foundations concepts to get a sense of how they relate to the research problems.
- Remote-first (but happy to meet in person). Dovetail is an international, distributed team, so we don’t have an office. But Alfred and many of our group members will be in the UK, so it may be possible to organise in-person meetings.
- Meeting 2-5 times per week. Especially in the beginning, we’d like to do a pretty large amount of syncing up. It can take a long time to convey all the aspects of the research problems. We also find that real-time meetings regularly generate new ideas. That said, some people find meetings worse for their productivity, and so we’ll be responsive to your particular work style.
- An end-of-term write-up. It seems to take longer than 10 weeks to get big results in the types of questions we’re interested in, but we think it’s good practice to commit to producing a write-up of some kind from the initial period. We also encourage research fellows to give a talk on their research at the end of the fellowship.
The research problems
As with a lot of research in agent foundations, it’s quite difficult to concisely communicate what exactly we work on. Probably the best way to tell if you will be interested in our research problems is to read our research wiki or our write-ups on LessWrong, or the example problems below and then have a conversation with us about it. A running theme in our research is the nature of `world models’ in agentic systems. All our research is purely mathematical,3 rather than experimental or empirical. None of it involves machine learning per se, but the resulting theorems should apply to ML systems.
The domains of mathematics that we’ve been focusing on include: dynamical systems of all kinds, probability theory, information theory, algorithmic information theory, measure theory, ergodic theory. Things we’re interested in but less knowledgeable about include: singular learning theory, computational mechanics, abstract algebra, reinforcement learning theory.
Here are some short write-ups of projects you could work on. Click on the problem titles to be taken to a pdf of a 1-page problem write up.
- Abstract Algebra of World Models. If the world is described by an algebraic structure, we can form coarse-grained versions of this structure which act as models of the original. Can we apply tools from abstract algebra to understand how this process occurs when we model the world?
- Deterministic Natural Latents. Natural latents were introduced by John Wentworth and David Lorell in this paper. Can we prove whether the existence of a stochastic natural latent implies the existence of a deterministic natural latent with comparably low error? Some previous Dovetail work on this problem is described in this blog post.
- Multi-step Touchette and Lloyd Theorem. Take theorem 10 from this information theory paper by Touchette & Lloyd and extend it to the case with multiple timesteps. We have written a blog post about why we are interested in this theorem here.
- Algorithmic Touchette & Lloyd Theorem. Again, this project relates to theorem 10 from this information theory paper which we have written about here. We are interested in proving a similar result but using algorithmic information instead of Shannon information.
These are just examples of problems which have a reasonably concise and well-defined mathematical framing. Some of our projects are less well-defined and a big part of the work is developing them to become mathematically formal. If there is something else you are excited to work on, we are also open to hearing other project proposals, provided that they are within the scope of our research program.
Application process
If you’re interested, fill out this application form! You’re also welcome to email us with any questions. If you are unsure whether to submit an application, we encourage you to err on the side of applying. After that, the rest of the application steps are;
- A short, conversational interview (20 min)
- A longer interview (1h) where we talk about your and Dovetail’s research interests in more detail, and come up with some potential concrete projects.
- Then, you go off and do some thinking & reading about your project ideas, and write a more detailed proposal. We’ll pay you £200 for this part, and you should spend roughly 6-10 hours on it.
After we have read your proposal, we should have a pretty good sense of whether we would work well together, and we’ll make a decision about whether to offer you the 10-week fellowship (or whatever else we may have negotiated). If you are successful, we expect that the time between the first interview and the job starting will be two weeks.
- We’re flexible with some specifics, if you have existing obligations on your time. ↩
- To give some more detail on these bounds; we expect applicants to be ready to start research at the level of graduate mathematics. We’ve met many undergraduates at this level, but they are invariably enthusiastic about mathematics and have already developed their skill level beyond what’s learned in a STEM degree. That said, there’s no essential knowledge beyond that. No one is an expert in agent foundations, so we’re really seeking what is sometimes called “mathematical maturity”. On the other side, if you have far more mathematical expertise than either of us and still want to join, we’re happy to discuss the details of your situation to see if it makes sense for both parties. ↩
- More specifically, the desired results are mathematical. Many of the ideas are “pre-mathematical”, in that the first part will be to translate the ideas into the appropriate formalisms. ↩