Why is Microsoft opening a new laboratory in Zurich of all places?
Marc Pollefeys: The ecosystem is perfect here. The laboratory will focus on computer vision and artificial intelligence, which means teaching machines how to see. And in the field of computer vision, Zurich is a global leader alongside places like the Bay Area. It is not difficult here to find local talent and recruit new ones. Another contributing factor is no doubt that I had spent two years on a research visit at Microsoft in the US, where I helped further develop HoloLens, Microsoft’s mixed reality glasses.
What can computer vision contribute to mixed or augmented reality?
The essence of mixed Reality is to seamlessly combine Virtual elements with the real world. For this it is critical to know precisely how the glasses move through the real world and understand the environment around the user. This problem is solved with computer vision. Similarly, computer vision is also used to observe the gestures of the user to let him interact with the virtual content.
In which other fields do you see computer vision as a key technology?
Computer vision is also a key technology for machines that need to move or act in an environment that is not fully controlled. This applies to drones or other mobile robots and self-driving cars, but also to cars with advanced driver assistance. In fact, most new cars and many mobile phone apps now use computer vision technology. Cameras are often the most effective sensors for machines to perceive the world around them – just like humans who rely on their eyes to understand their surroundings.
Can you explain why Zurich is so far ahead in this complex field?
An ecosystem has developed in recent years with research institutions, startups and established companies, but the real trigger was no doubt ETH Zurich. Experts from various fields in engineering and computer science – all disciplines that are highly regarded at ETH Zurich – work here on matters related to machine learning. Contributing to this is also the fact that in addition to basic research, applied research and cooperation with industry are very important at ETH Zurich.
The Disney Research Lab in Zurich is also proof of this. Was that your role model?
I have advised Disney Research Lab on scientific issues and have seen how well these structures work. It may well be that we will work together in the future.
More concretely, what will you mainly work on in your laboratory?
These are projects with a long time horizon: if we have a vision of what the mixed reality glasses HoloLens should be able to accomplish in a few years’ time, then we will try to make this a reality. For us, it is about how HoloLens can concretely help people improve how they solve difficult problems, for example in complex repair work. We will then work on how the glasses can better perceive their environment or how different versions of HoloLens can communicate with each other.
How is the laboratory being organized?
We have been working since July, and six of the fifteen full-time positions have already been filled. We are focusing on recruiting scientists from computer vision and related fields such as machine learning and robotics. There will also be positions for ETH doctoral candidates. We are currently looking for premises close to ETH Zurich. I work 50 per cent at the lab and 50 per cent at ETH Zurich.
What do you appreciate about the parallels between university and lab work?
I enjoy brainstorming with innovative students on fundamental problems and working independently. But I also want to create something that will change the world a little bit. In Zurich, I can combine the best of both worlds with Microsoft and ETH Zurich, and, in my opinion, all sides can benefit from this.
Biography:
Marc Pollefeys has been a professor of computer science at ETH Zurich since 2007 and, since 2018, director of Microsoft’s Mixed Reality & AI Lab in Zurich. During his two-year research visit at Microsoft, he contributed to the development of the mixed reality glasses HoloLens. He has already received prestigious awards for his work and is involved in several startups.
