The SCILA won a Russian Science Foundation grant to study the use of large language models for solving differential equations

On November 28, 2023, the results of the Russian Science Foundation competition were announced: the application submitted by the Laboratory of Social and Cognitive Informatics received support for the year 2024.

Under the guidance of Sergei Koltsov, the following tasks are planned within the research framework:

1. Creating a database of differential equations and their analytical solutions in the form of text sequences. This task comprises the following subtasks: 1) implementing a procedure for extracting differential equations and their solutions from electronic handbooks in PDF format; 2) implementing a procedure for extracting differential equations and their solutions from electronic handbooks in DjVu format; 3) developing a procedure for converting obtained equations into LaTeX format; 4) creating a database of equations and solutions in LaTeX format.

2. Generating embeddings of differential equations and solutions based on their textual representations.

3. Developing neural network models for predicting solutions of differential equations based on the Seq2Seq model (Sequence-to-Sequence). This task includes the following subtasks: 1) training the Seq2Seq model based on RNN, LSTM, and GRU models and their combinations; 2) fine-tuning the Seq2Seq model based on Transformer models (large language models: Bert, XLNet, T5).

4. Designing neural network models for predicting solutions of differential equations based on the Seq2Seq model, considering the developed embeddings of differential equations. This task consists of the following subtasks: 1) training the Seq2Seq model based on RNN, LSTM, and GRU models and their combinations; 2) fine-tuning the Seq2Seq model based on Transformer models (large language models: Bert, XLNet, T5).

5. Analyzing existing quality metrics for the performance of neural networks in generating textual data and adapting these metrics for predicting solutions of differential equations.