Online notes: a real-time speech recognition and machine translation system in university lectures

Abstract

Online Notes is a system for automatic speech recognition and real-time translation at the level of university lectures which has been developed at the University of Ljubljana. The system is primarily intended for students of the University of Ljubljana who do not understand Slovenian well or do not understand it at all, as well as students with sensory impairments. Within the project, we have developed two speech recognition models and a web application consisting of four portals. So far, 29 pilots have been carried out, among them a semester pilot, where we followed two exchange students of the Erasmus+ program who, during the entire semester, attended the Spatial Statistics course at the Faculty of Civil Engineering and Geodesy at the University of Ljubljana in Slovenian using the Online Notes system. In addition to the real time use, they also had automatically generated subtitles in English available as learning material, which were created on the basis of corrected Slovenian transcripts. The two students successfully completed all their study obligations. They pointed out that the system is very useful for students who do not understand Slovene, as it makes it possible to follow lectures, and they also pointed out certain shortcomings, such as the cognitive complexity of monitoring the system, the lecturer in the table at the same time, and the poor quality of translations in lectures in alive.