Hannes Kaufmann in interview
Hannes Kaufmann, opens an external URL in a new window, Professor of Virtual and Augmented Reality, knows the developments in VR and AR from 20 years of research in this field. The focus of his research is to develop meaningful applications for the benefit of humanity. In this interview, he talks about current research projects, the problems caused by the lack of cooperation with international technology companies, studying VR at university and job prospects for students.
Professor Kaufmann, what projects is your research department currently working on?
Hannes Kaufmann: We are working on haptics, the sense of touch, and we are using a mobile robot to create the illusion of haptic sensations. The robot travels alongside a VR user as a personal assistant. As soon as the user wants to touch a virtual wall, for example, the robot arm holds a real object (e.g. a board) in the right place, making us believe that we are standing in front of a wall (more on this in part 1 of the interview with Hannes Kaufmann, opens an external URL in a new window). In addition, a proposal for a TU Wien-wide Mixed Reality Lab, involving all faculties and five academic partner organizations, has just been funded. In this Mixed Reality Lab, which will start this year, we will also experiment with room sounds: 192 loudspeakers will be installed in the room to acoustically simulate different environments and reproduce correct room sounds for several visitors. We are also currently working on two projects on VR training systems for emergency services (fire and rescue) and two other projects on the use of VR and AR in the construction industry.
Where is development and research in the field of virtual and augmented reality taking place and in what direction is this research going?
HK: At universities, we do basic and applied research. In Austria, VR research has been going on for more than 25 years, also very intensively in the area of AR, which is why large companies have set up research centers here. In general, we work on new methods, algorithms, software components and develop innovative VR/AR systems that are made of existing hardware components. We rely heavily on hardware that is available on the market. This is where we are lacking in Europe, as we depend on a few manufacturers from the US and Asia. In Europe there are only two companies developing VR glasses. This leads to problems in our research, as we often have to wait for the latest devices and are subject to software restrictions. We also have no influence on important hardware adaptations or changes.
Does this mean that even as a top researcher in this field, you have no opportunity to get in touch with hardware manufacturers directly?
HK: That is out of the question. Chinese manufacturers are not interested in the European market, there is no contact. First, they produce for the Chinese market, then maybe for the US. The situation with American products is similar. If we are lucky, the development comes to us one year later – for example, the Microsoft HoloLens, opens an external URL in a new window, which was initially only available on the American market, and we only got it after six months through American colleagues. This is a big problem for research because we have to publish and work with these devices. American colleagues for example can develop new methods, do user studies and publish before we even get the devices. Sometimes devices come on the market only in a few European countries where the market is big enough, e.g. hardware that is sold in Germany but not in Austria.
Why is there a lack of interest on the part of manufacturers to work with the leading people in research?
HK: The way companies think is different. Research is done behind closed doors and know-how is not allowed to leak out. Microsoft has about 1,000 researchers working on HoloLens. No university research group has that kind of staff capacity. Researchers are poached from universities and bought; instead of publicly accessible scientific publications, patents are written for companies. The technology company Qualcomm, for example, is only interested in major customers, which means that new chips are sold directly to producers. We are talking about quantities of thousands, tens of thousands of chips. In short, the main interest of large corporations lies in sales figures and the mass market; universities and university research are not at all interesting to them. The Covid pandemic has made the issue more pressing, as there are still problems with supply bottlenecks. The high investment costs speak against such a reestablishment of production facilities in Europe, although the EU would like to invest here according to current reports.
This is the opposite of university research, where new knowledge is generated in a comprehensible and reproducible way to enable new applications. But the knowledge researched by companies remains private and is monetized.
HK: That’s the way it is. We can’t contribute to certain areas because the companies can invest enormous amounts of money where they see a profit. One example is the big graphics card manufacturer Nvidia, which does a lot of research on computer graphics for games. Nvidia can have 50 people working on certain problems – we just don’t have the resources. So we go into areas where the “big guys” don’t get involved, areas perceived as less suitable for the masses.
Research for society
We have many research topics: for example, we are investigating applications such as training for firefighters and other emergency services. This is not an area of interest to large manufacturers, but it is very important for society. We do a lot of research in cooperation with small and medium-sized enterprises in Austria (SMEs), which provide us with exciting challenges. We are also working with industry in the field of “digital twins”: In large, walk-in virtual environments, employees can be trained on machines before they actually exist. We have also developed product representations of large baking machines (which would not fit in a showroom for space reasons alone). Personally, it is important to me to deal with serious applications; I also see this as my contribution to the good of society. I find it very appealing to learn about different application areas and research questions. When we develop applications for rescue workers or in the medical field, we learn a lot just by talking to the users. For example, we have developed a tracking system for tunnels. This allows for the accurate tracking of the position of machines in tunnels over long distances. Of course, the same optical positioning technology can be used for other purposes, for example in rehabilitation to record and analyze patients’ movements.
How large is the group working on a research project in your research department?
HK: Usually one to two, at the most it’s three people working on a project. It’s a question of funding. Once we had a requested research project with six people working on it, but only for half a year.
Is the situation internationally comparable?
HK: In research, yes. Typically, the budget of funded research projects allows for the employment of one to three project staff on average for up to four years. There are a few larger research initiatives that allow more people to be employed for up to seven years. In general, highly competitive short-term budgets have made it difficult to address larger topics that require long-term research.
How do students get into the field – at what point can they specialize?
HK: Virtual and augmented reality courses are an integral part of the Visual Computing and Media and Human-Centered Computing Masters, opens an external URL in a new window’ programs. In addition, the basics are taught in the “Multimedia” lecture in the Bachelor’s program. The new Bachelor’s program “Informatik”, opens an external URL in a new window (from October 2023) will allow students to specialize in “Visual Computing” from the second semester onwards. We are very happy about all students who are interested in VR/AR. In our courses you will be able to work hands-on with VR glasses and VR hardware. The new mixed reality lab will also be used in future courses. In general, I would like to add that the job prospects for students in VR/AR, as well as in the whole visual computing environment, are excellent. We constantly get job requests from companies for graduates. To quote a colleague: “You can get a job within three hours”.
Discussing with Hannes Kaufmann, Jan 27
For those who would like to discuss with Hannes Kaufmann, he will be available for questions and answers on TU Wien’s social media channels Instagram, opens an external URL in a new window and Facebook, opens an external URL in a new window. When? Friday, January 27, 09:00-11:00
This interview was conducted as part of a feature for TUW Magazine’s Science Fiction themed issue.
The Revolution of Thought Transmission, opens an external URL in a new window is part 1 of the interview with Hannes Kaufmann about the current state of development of VR and AR applications, ethical issues and taboos, and the social responsibility of research.
Hannes Kaufmann ist Professor an der TU Wien für Virtual and Augmented Reality, opens an external URL in a new window und leitet den gleichnamigen Forschungsbereich. Seine Schwerpunkte sind u.a.: mobile Computing, 3D User Interface Design, Bildung in Mixed Reality, VR, Bewegung und Tracking, Augmented Reality.
Interview: Edith Wildmann