Until now, tennis training sessions have mainly taken place on the tennis court. The development of a virtual tennis trainer will make it possible to also train at home in future. The partners, TU Wien and VR Motion Learning, have been working on the development of their virtual tennis trainer since March 2020. Whereas up to now the main focus has been on analysing and automating the recognition and evaluation of movement sequences, the team is now concentrating on the didactic component in order to provide users with helpful feedback. The project is funded by the Österreichische Forschungsförderungsgesellschaft FFG (Austrian Research Promotion Agency), which announced the project extension by the end of January.

Training of the AI

With the developed "Tennis Simulator" (Tennis Esports), users can experience virtual tennis and perform physically correct strokes. The tennis trainer based on this is aimed at people who are new to the sport or want to improve their technique. When played correctly, tennis is not only more fun, but injuries can also be prevented. In order for prospective tennis players to learn from a virtual trainer, the latter must first be equipped with the appropriate skills. For the purpose of analysing how different strokes such as forehand topspin, backhand slice or serve are correctly executed, the team first invited experienced tennis players to record their movements – as well as the guidance of the racket – with cameras. The resulting database of different strokes could then be analysed and compared with a specially developed artificial intelligence.

"At the end of the first phase of the project, the virtual tennis trainer was able to record and evaluate new movements of the players in a fully automated way, based on the strokes recorded as correct," says Hannes Kaufmann, head of the Virtual and Augmented Reality group and professor for Virtual and Augmented Reality at TU Wien. "The challenge we are now facing is the implementation of a feedback function. After all, the simulator should also contribute to improved game performance." The feedback function not only has to be implemented technically, it is also necessary to consider how learning and training effects will be greatest. "One advantage that the virtual environment offers us is that the same situation can be experienced several times. The feedback on a stroke can therefore be implemented directly," explains Peter Kán from the Computer Graphics research department at TU Wien. When training virtually, the players stand on a (virtual) tennis court on which they can move freely.

On the court or at home?

The programme can be seen as an extension to the training on the tennis court. The idea is to give human coaches a tool to enable players to train off the court, alone or in pairs, at different locations. "In order to be able to train at home in the future, we are developing a version in the coming project phase that will analyse the strokes exclusively on the basis of existing VR sensor data. An additional camera system for motion detection will then not be necessary," reports Peter Kán. The players would then only need a VR headset and a tennis racket – the previously required depth camera would become superfluous. Training is already possible on an area of 2x2 metres, which can be extended to 10x10 metres.

By developing a version for home use, the team is responding to obstacles such as the lack of a training partner or a lack of time flexibility. It is important to the development team to create an application for people who are interested in the sport of tennis, not primarily in e-sports. The virtual tennis trainer is expected to be available for the Oculus Quest by the end of the year.

Contact

Prof. Dr. Hannes Kaufmann
Research Area Computer Graphics
TU Wien
+43 1 58801 18860
hannes.kaufmann@tuwien.ac.at

Dr. Peter Kán
Research Area Computer Graphics
TU Wien
+43 1 58801 188645
peter.kan@tuwien.ac.at 

Text: Sarah Link