general-sep29
Responsible Release of AI-Generated Mathematics
Spots 
At present, some frontier AI labs are testing
At present, some frontier AI labs are testing advanced mathematical problems on proprietary models that remain inaccessible to the broader scientific community. Our recommendations are formulated with this practical context in mind. However, ideally, they would not do so. We want to state clearly from the start: we do not endorse this practice, and we ask them to stop testing advanced mathematical problems on proprietary models.
The mathematical community has long-standing norms concerning the
The mathematical community has long-standing norms concerning the dissemination and peer review of results. These norms have been essential for the reliability, trustworthiness, and effectiveness of mathematical work. One of the most important scholarly norms in mathematics is that the authors of a paper should understand the mathematical argument of that paper, have verified its correctness themselves, and take full responsibility for the content. Moreover, in the mathematical community, authors of works that significantly advance the field regularly give seminars at other institutions and talks at conferences, explaining their new developments and answering questions from colleagues. This all works towards developing the deepest possible human understanding of mathematics, which is one of the crowning glories of millennia of human development.
However, it is now the case that AI
However, it is now the case that AI can output mathematical arguments in situations without the human who prompted it being able to understand the arguments, verify them, or take responsibility for them. We believe that human understanding of mathematics remains of paramount importance. How, in this new era, can we work towards a new paradigm that includes human understanding of mathematics as part of responsible scholarly output?
We asked the mathematical community for feedback about
We asked the mathematical community for feedback about what it would mean for AI labs to responsibly release mathematical results and received over 600 replies (the survey asked about a specific situation in which OpenAI announced the existence of many results without giving details). Informed by these responses, we arrived at a set of recommendations, supported by a clear plurality of respondents, aimed at any AI lab whose models are likely to have a significant impact on mathematics. The overarching principles underlying these recommendations are the following.
If AI labs produce significant mathematical results, they
If AI labs produce significant mathematical results, they should responsibly release the results, as outlined in this document, as soon as possible.
AI labs that release substantial mathematical output without
AI labs that release substantial mathematical output without immediate accompanying human understanding must take responsibility for ensuring that human understanding will follow. In particular, AI labs should provide significant support, including funding, to help develop this understanding.
The development of human understanding must remain organic
The development of human understanding must remain organic and community led. It should not be directed by AI labs, even when the labs have produced the results.
We present our recommendations themselves in Section 2
We present our recommendations themselves in Section 2, and in Section 3 we make some comments about access to powerful model. 2. Responsible release of results generated by AI labs We recommend two possible courses of action, depending on the level of human understanding that accompanies a result. 2.A. Papers that a human understands
Papers for which there is a mathematician responsible
Papers for which there is a mathematician responsible who fully understands the content should follow the academic mathematical community’s traditional norms: the mathematician(s) concerned should post a preprint, submit a paper for peer review at a journal, and give talks to explain the work to other mathematicians. 2.B. Papers that are not yet understood by anybody