Keynote
Craig Mundie: Good afternoon everyone! It is wonderful to be back in China and in Beijing and to have this opportunity to talk to you this afternoon. We have two parts to the session this afternoon. First, I am going to share with you some of the vision we have for how technology may change education in the future. All of you, being students, have an appreciation for the challenges that you face, and mastering many of the new ideas and technologies. Even some of the projects we just saw are great examples of how creativity can be applied to solving many problems. One of our issues and opportunities is to think of how technology can help to change education itself. So let me begin with a video that will provide some context of why I think this will be so important in the future. Please show the video.
(Video)
So this video clip was taken from a presentation that we did about one year ago at the TED Conference – technology, entertainment and design in the US, when we launched the world-wide telescope. And importantly, the man talking was Roy Gould, one of the world’s leading astronomers and astrophysicists. And this tool showed that even for the world’s leading scientists, the ability to bring these things together to create a new model of interacting with this large-scale data was an important concept and a powerful one even for the world leaders. What we’ve done since then is actually…I brought here an example of that same thing, which is, what you are looking at now is a camera looking down at a surface computer. So in this environment, I now have the ability with just my hands to essentially zoom in and zoom out, to do the same kind of manipulations that historically we only did with our personal computers. This allows many, many more people to become story tellers, as Roy Gould said, the ability to pass down information from one generation to another, from people who have special gifts and insights to people who are less understanding. One of the things that was so powerful to me, as we developed the world-wide telescope and as we began to launch it, was the potential not to just provide tools or technologies to the world of leading scientists, but to bring many of these mysteries of the universe into focus for a much larger audience. One of the challenges that we all face in the future is getting enough kids to be involved in science and engineering. The world is going to have an ongoing set of technical challenges so we have to get more and more people who are experts in all fields of science and engineering. And yet globally we actually see a shrinking of the population who are excited about these important technologies. So when this technology was under development – and in the final stages of the testing – they not only gave it to Roy Gould and some of the astrophysicists at Harvard University, but Curtis Wang who developed it at Microsoft Research in Redmond, also had a friend in Toronto and he gave that technology to him and said, “Hey, I’d like you to play with it.” So now I would like to play a guided tour on this same surface that was developed by someone who lived in Toronto and was a young student. The person who did this completely by himself was only six years old. So, here’s his story.
(Demo)
I can barely remember when I was six years old, but I am sure I wasn’t giving tours of the Ring Nebula. Back then, 50 years ago, kids had a very limited exposure to technology and the world’s knowledge that was available to them. Today, children age six are growing up with access to essentially all of the world’s information. But Instead of having them in giant libraries of books – which were very difficult to process – we are using information technologies to make this more and more accessible to them. So when I look at the power of these tools and the excitement that a child like Benjamin has at age six and his ability to manipulate this technologies the way that he does, it makes me very, very proud, and also very, very hopeful that on a global basis we will be able to get these kinds of technologies into the hands of virtually every student from a very young age.
So now I want to talk a little bit about how we rethink computing, in order to be able to move in that direction. Computing has evolved for the most part over the last 50-60 years, and it has gone through a series of waves of evolution. In the beginning, the machines were very large, they were used for dedicated purposes. Eventually we put terminals on them and made them into mainframes and terminal based processors. And then we began to miniaturize them a little bit, and we made the mini computer era. But the real phenomena that made computing more accessible came with the arrival of the microprocessor and the personal computer. And in that time period – since then, we have gone through several eras in evolution of personal computing. The DOS era, which was near the beginning of Microsoft was a time when we had very simple text-based interface and very limited capability. But even so, it was a powerful change for people who historically only had access to calculators on their desks as a way to do any kind of significant computing. They had no way to really build models. But even this was too difficult for most people to use. And the real next breakthrough came with the introduction of what we call the GUI or the graphical user interface. Microsoft’s core businesses grew out of our ability to take the concept of word processing and spreadsheets, and to marry them up into this graphical model of man-machine interaction. That expanded the universe of people dramatically to use these technologies, and do interesting things. That has persisted and grown into a popular computing environment that all of you use and think nothing of these days. About 10 or more years ago, the core Internet technologies also began to emerge. And there was a set of applications that were very important in making the use of Internet popular. The Internet as a technical medium had been with us since the 1970s, but like the DOS, it was too hard to use. So the next graphical user interface was created in the form of the World Wide Web and the web browser as a tool or application that had brought appeal and a lot simpler model of interaction for people; and coupled with electronic mail, these two applications by themselves made, the use of the Internet globally on top of personal computers, a real phenomena. Today we find ourselves, 15 years later, looking at what is going to change, how the technology will evolve. And I think we can now already see the emergence of two succeeding waves. The next and most immediate will be the combination of the evolved client environment, the personal computer and its diversification into many other devices - the mobile phones, the car the game console, the surface table; all of these will become part of the aggregate client computing environment. And then the cloud, which will be the programmatic extension of the model of publishing that we know as the World Wide Web. When you take these together and think of them as one giant scale distributed concurrent programming environment, you get what I believe will be the world’s next programming platform. So we are all now searching for the class of applications that will be empowered by the cloud plus client architecture. And I think it will be an exciting time, in the next few years, when we try to develop these applications. But this incredible performance increase that will come at both ends of the communication system also create the opportunity for us to extend the computers into a new class of experience, which we call the natural user interface. Just as DOS was too complex for many people to use, even the World Wide Web and pointing and clicking is too complex and too localized a way of presenting computing for most of the world’s people. Today the world has about 6.5 billion people and only about 1.5 or 2 billion have any direct use of computers. So there are two-thirds of the world today, with another 2-3 billion coming down the road in the future, who do not get the benefit of these technologies. And we think that that has to be resolved. And one key way to do that is to create another advance in the way that people use computing. In fact, the business of software for many years has been about increasing the level of abstraction, that we present computing to the user. In the very early days, those users where really the programmers, and the introduction of programming languages and higher-level language were waves of abstracting the hardware; The introduction of operating system was a way of abtrating away the complexity of the underlying differences in all the hardware and peripheral systems. And that made the system more approachable. And then as we created a higher level of applications, the tool became more and more powerful for people to use. So, today as we see on the slide, each element of the computing system we know today is about to change in a way that this abstraction level is going to increase. The platform itself as I said will become this “cloud + client” architecture. There will be new ways required to get parallel performances, to get correct programs, and distributed, concurrent execution environments, at every scale – from a local or an individual device to applications that span the entire range of computing capacity from the cloud down to those devices. Computing will be everywhere – we won’t just to the computer to get the benefits of that technology. The computer will be more and more all around us. We’ll see the arrival of many new display technologies that make it more ever present in our environment. We will have the models of touch as a way of direct manipulation of the interface, as we see today on the surface or things like mobile phones. I think we will also see a world where networking continues to get more and more capable, whether on the wire or through the air. So more and more we will assume that networking is a continuous function, not something that we only do occasionally. I think another big change is that more people will start to look at higher levels of modeling as the way that they get to describe the problems that they want their computers to solve. When we created the spreadsheet, what we were really doing was creating the first tool, a general modeling tool that the average person could use to create a description of the problems that they wanted the computer to help them solve. And a couple of new models of data that have a lot more semantic richness than those we use today, plus an explosion in the way we use modeling techniques, I think it will again create a much more powerful way for people to get benefit from all these computers. I think another one of the other biggest changes that will take place in the next few years is this transition where computers will become less and less a tool, where if you are an expert, and you did an apprenticeship, and you understood the tool you could create some incredible things. But still too few people are able to master these tools in a way that help them to solve problems. I think the next generation of computing will be one where the computer works on your behalf. It becomes a great assistant to you, whether in your work life, or at your home. And less and less, you will have to master the tool; and more and more the computer will step forward and provide help for you without you having to direct it in a very explicit way. You will be able to speak to it and ask questions at a much high level, and use much higher levels of modeling techniques to solve problems. For many years, Microsoft Research has developed individual technologies that we think are at the core to help in the transition from tools to assistance. And of course, many of the important changes here are to move beyond typing and pointing into a more natural way of interacting between people and the computer system. And so in the last year one of my favorite demonstrations that has come out of the work of Microsoft Research was done by taking a whole family of our technologies. One of them was the new robotic toolkit that we have, which creates a model for composing large-scale distributed concurrent systems. I think this is a very important technology because as I said the future platform will almost always now contain this requirements for parallel execution and distributed operations. That is different from the way most applications were developed in the past. But they did an interesting thing, which was to say, “how we can use this technology of robotics not to create a controllable machine, or a physical robot, but how can we create a virtual robot that would integrate many of our historical research technologies in a way that would qualitatively change people’s perception of what it would be like to be able to interact with a machine.”
I would like to show you a video, but I would like to first explain it with a slide. On this graph you see several different segments. On the top left, there is a graph of the CPU performance of this particular processor. It is the fastest individual PC that we can build today, it has 8 processors. And even when this application is doing almost nothing, the processors are about 40 percent utilized. Because it becomes a continuous computing problem, instead of just an occasional computing problem and so this is a way of showing, even as crude as this example is, as the microprocessor evolves, in the next four to five years, they will become 50 or 100 times more powerful. And if built this way, these applications will just get faster and faster, and more and more powerful. That is a motivational factor for how you think about developing these applications. The next pane down one the left is actually the face of the robot. In this application, we named her “Laura.” Today we only have a very rough implementation of her features, we only try to produce her facial features, and animate her face in a crude way. But as you know from looking at video games like Xbox, if you apply enough computing power, it is very possible now to get very, very compelling presentation of individual features; and I think you see this in the movies today. But they have the opportunities of a very specific contexts and a lot of off-line processing. So, again as computing becomes more powerful, the quality of the animated avatar becomes much, much richer. You will see the benefits of that. The bottom panel on the left shows what the people are interacting with. So, Laura today is presented just as a face on a screen. But of course you could dream about a future where this is a fully anthropomorphized robot and people could interact with it in a way which was in fact a whole lot more human like. The pane on the right shows representation of all the simultaneous types of processing that is going on. Laura sees with a wide-angle, high-definition video camera, and she hears with an array microphone, which are represented with the green circles at the bottom. Digital signal-processing techniques are used to look at where the sound is coming from and to localize it so that she can begin to associate items in the visual scene with the spoken words that are produced. In real time they annotate the speakers with what the system is learning about them; The system examines their clothing and in fact will interact with people in different ways, based on whether they are dressed casually or formally. The first one was built as a receptionist working to work in the lobbies of Microsoft to arrange for shuttle buses, to move from one building on the campus to another. You can see the normal vector to determine which way they’re looking. The red dots you will see moves around as a representation of the eye gaze of the robot so she looks and seeks to make eye contact with the person she is talking to. So more and more the goal is to make the robot have the same type of cultural sensitivity and interaction that any two people will have if they met and interacted in the lobby. So you will see for the first time here the ability to bring together in real time – machine visions, machine learning, speech recognition, speech synthesis, image processing, and 3-D graphics modeling simulation, all happening in real time. One of the things that is important about this application is that the latency requirement among all of these different interacting parts is extremely high tolerances. Human beings have evolved over many, many years, and are very sensitive in the brain to how these different things – these different audio and visual cues interact. If you get them off a little bit it breaks your ability to believe in the interaction. And so while this is not perfect, you can see in our first attempt now to use these technologies to bring together a complete man-machine interaction built around these technologies. So lets see the video and you’ll see the interaction.
(Demo)
I’ll show you one more video now, where we decided to take Lora out of the lobby of the build, and see if she could interact spontaneously with people. So she was set up in one of the research buildings, so that people walking by could stop, she would approach them and ask if they wanted to play a trivia game. So in this quick clip you will see what it is like when Laura starts to engage people and in a freeform way start to play trivia games.
(Demo)
Every time I show these demos, I say that my dream, which is represented by this demonstration, is that as the computer becomes more powerful, and the expert system that we embed into them, are not just about making shuttle reservations or making registrations in the lobby, or even playing question and answer games. My dream is that in a few years we will be able to build systems where this could be Professor Laura or Dr. Laura. And that these kinds of technologies are really at the heart of being able to improve education and healthcare for billions of people in the world today who have no access to these types of experts, no ability to get this level of personal counseling or tutoring. So I think this shows that there is a path-forward that we can use. This kind of addressees many of the technical disciplines, but I want to show you one more video as a way of introducing a demonstration about the actually evolution of technology in education. This is a demonstration of how the Microsoft Research people have worked here in China in an e-Heritage project to use the technology broadly to try to preserve the Chinese culture and history. One of the student teams talked about their own work to use the computer and modeling for just one aspect of architecture, and here you will see that our ability to apply this more broadly in the field of arts and culture has a lot of promise too.
(Demo)
Now, I would like to move on to talk very specifically, to close the talk, about how we see applying technology much more broadly in the field of education. Our goal is to essentially enable a lot more personalized learning than what people get today. To nurture these 21st-century skills that will be important for people in workplace and in their normal daily lives in the future. To build powerful communities of learning, today the Internet allows us to bring together people with common interests and to do that without the bounds of geography. We want to support a lot of different kinds of technologies in the educational process, and we want to extend the reach of these technologies to provide high-quality education for billions of people today who do not have access to it. So the way that we dreamed about this is to use of a number of technologies, like this tablet technology, as a way to make technology-assisted learning a lot more approachable. What I have here is a tablet computer, which one can buy anywhere in the store today, and we built new technology applications to begin to build a prototype of what it might be like to be a student, like you are today, some years in the future where everybody has grown up using tablet technology, and they become an integral part of how you learn and collaborate with other students. So let start by assuming that you are taking a course in human anatomy, think what it might be like to be able to look at these where you have computer-assisted models to assist with each aspect of this. So now I can build up a model of the human systems, looking first at the skeleton system. Then I can basically pick the circulatory system and look at that and say how does that fit. Tell me how muscles get laid up in there. These are things that would have been impossible for most people to see, and certainly to understand in a layered way how they come together. You can impose the nervous system and see where they are. In a sense, this becomes a graphical way of navigating all of the world’s information. For example, we designed this application with the belief that little icons, like in this case the green dots and other color dots you will see in the future, represent places where specific additional information available; so as students, you can chose to explore these dots. For example, if I pick one of these I choose look at a specific citation by Henry Gray, a famous author of a textbook on the anatomy of the human body. In this, I can now not just see flat 3D pictures but I can see 3D models that allow us to bring these things to life. I can take the model, right now I have just zoomed in on the hands, and I can essentially zoom up all the way up to the head and so now I have a complete model of the body. I can use this to pick it and animate and turn it around and examine different parts of it. Each time I do this, the computer system is starting to do things on my behalf. I don’t have to tell it each time to go out and get some information and bring it back. I do not have to explicitly do searches for examples. Each of these things is done automatically in order to be prepared to help me with the learning process. So let’s say that now I want to go and look up more information and learn about the brain. So I touch the brain, and it goes out and examines and brings back a lot more information and we delve in and I really want to understand synapses and how they work. I know that this is some kind of biochemical process, in this case the model shows me a photo micrograph of the actual synapses and I can look at a computer animation of how the synapse actually fires where it is essentially a biochemical process. So if you show me that animation, you can see how it actually might work, and I can get more information. One of the things you grow up with today is this idea of social networking – the ability to text each other, and to collaborate with one another. But today that does not happen, perhaps with the exception of computer clubs, broadly as a part of a normal curriculum. But we do not see any reason why these social computing techniques won’t find their way into a real model of learning and collaboration. For example, I can close that, and click on another message that pops-up; in this case the instant message is one that says one of your classmates has been studying the same part of the synaptic firing process. He actually has some insights and he developed a sketch of his own to show how the process works. And he is sharing it with his classmates. Now the ability to have real-time presence information and real-time communication becoming part of the learning process is fully integrated into this environment. Having done that, I can decide I want to go on and look deeper into what other resources are available. So I can ask the computer what has it discovered for me. On the bottom, I can see a long list of different citations that I could study. Here, though we have actually had the computer analyze each of these relative to the subject matter we are studying, and to color-code them, where the green ones are the most relevant, the orange ones are thought to be perhaps relevant, and the red ones may be a bit exploratory or not directly related. There are a lot of citations, but of course these are things that could apply to almost everyone. So now I can have simple ways of filtering. So instead of that, everyone, I can say really show me citations that relate specifically to the work of my study group. So if I click on that, I get a much smaller set. Here I can explore different things, and could look directly at the most relevant materials. Let’s say I feel a bit more fun-loving and I decide to go explore caffeine in my brain just to learn a little something out of ordinary. So here it says that caffeine affects the brain, and you can find out thought a simulation what parts of the brain are affected. We have figured this out by using functional MRIs scans and if you want to see it, you can look at my simulation tool. So I click on the simulator here. It comes out, and you see the model of brain, and you can see where, near the frontal lobes, that caffeine actually produces this stimulating effect, it gives me a slider, where I control to see how much caffeine produces different levels of activity in the brain – from very small amounts to drinking a cup of coffee. You can see how each of these different kinds of drinks or things that you might each has some effect on the brain. Each of the citations is color-coded with a set of boxes, and each box is something that represents another subject that I could explore. So if I just use my hands and scroll across these, you can see little pointer. I will just click on the last one, and this actually begins to show me more information about how semantic analysis becomes a big part of it.
One of the things we recognized today is that today the Internet is largely just a big library; it is a repository and you don’t know when you do searches whether or not it really is meaningful for you and how the computers may find those of things of greatest interest. So I say “show me a systematic analysis on all of these citations”; and so you can see the ones that I looked at, and you can see all the different concepts that are reflected in all of these different documents. Having done that, I can use this to pick things in a more focused way that I wanted to explore and learn about more. So in this case, I want to see more specifically about the text in this environment, and what the citations are. So here not only do I get to see the document, but the computer can actually read the documents for me. Just as you do as students today – when you would read it and you would highlights it in order to determine for later reference which parts are the relevant, well here the computer can again highlight it in a color-coded way. So I can say what about that citation here in green – what does that actually look like. So I can zoom into the document I can read the documents. More and more clearly as you sit here in China you see people that are multi-lingual, and of course we have a more and more worldwide connected research community. So frequently, you want to see the documents in your native language, or if you are multi-lingual, you want to read in a different language. So we expect by this time that machine translation will be possible for all of these things. So now I can go back and see who was the original author is. In this case it was a doctor in Japan, and I can say I just want to look at this in Japanese, so now you can see a real-time translation of that. Having decided that it is actually what you want you can then drag this citation directly into your digital notebook, so it becomes directly incorporated. So your ability to combine freely all the different information that you encounter whether it is presented in a textbook or presented in a classroom or whether developed by your student friends, and to have a consolidated representation of that we think will be more and more possible. Our dream is that these types of technologies – digital repositories will be available to all students from a very young age. So in fact, since this is a dream about the future, let’s sweep back in time to when this person was in the seventh grade. That was the first time they studied anything about biology and these were their notes about the central nervous system at the time that they were studying this in the middle school. So they can say ok, there is some consistency between what I have learned through the years as I have studied these different topics. So I can essentially zoom back out to the present day. Here we are back looking at a schedule of the type of things that are going on in the anatomy course, but also the other activities that we have underway in my other courses. One of the things we expect to be more and more important is the ability to collaborate with people – not just in a remote way, but also in a more direct and personal way and we expect devices to interact with each other more directly..
Here I have the same surface table. You might get one of these in the future – in your dorm room, or in your library, or your coffee shop. And the ability for the computer vision system to recognize these other devices and for them to integrate becomes better and better. So if I put the device on the table, it immediately will go and extract for my use some of the documents that I was collecting there. If another friend comes up who has been working with you and might have collected some things on the cell phone, they can place their cell phone on the surface and you spill out the documents related to that. And you can begin to look at these documents and essentially zoom them in and turn them around and share information with each other. In this case, you say ok, some of these documents are related to the brain. One of us could have gone to the biology lab and gotten a computer assisted model of what the brain looks like. So when I put it down, I can essentially use it as a way of navigating and referencing additional information. So now my ability to organize these things to sort them out and figure out which ones actually might be the most relevant. The direct manipulation interface, we think will become increasingly important, and we can have a group we want to publish them to. So at the top right corner we have a folder where all the classmates assemble what they think are the most important documents. So you can essentially just flip them in there and when you are done, you essentially have a collection of things that you need together that you believe are most relevant to the subject matter at hand. One might ask, well, that’s all great but these things are expensive, not everyone could have access to these kinds of technologies. What is going to happen to make that better?
So in closing, I want to show you one more piece of technology. This is an active display, we took one segment of this demo and we have built it in to this prototype. Today we have a little technology like a piece of a cell phone on the back. It is actually an active display; it is changing and showing different parts of that hand moving as I was giving the demonstration. What is interesting about this is that it is extremely flexible. So in the future, you can see, it is only less than 1 mm thick. The idea that everybody will have a lot of potential for surface technology with them all the time is going to be better and better. So, you can think in the future as a student you may just have one of these in your backpack. And if you want to have a quick conversation with somebody or you need more display technology or surface than you can get on out of your cell phone. You will just have a wireless connection between this and your phone, or between this and your tablet computer. And you can just take it out and set it down and essentially have a direct manipulation interface just like that big surface table. So we are very optimistic that as technology continues to evolve we are going to have more and more capability to enhance the educational experience and to have the computer work on your behalf not just be a tool but as a research assistant.
In closing, we think there are incredible opportunities now to take all of these evolving technologies – the cloud plus client computing architecture, the many-core processing evolution, semantic analysis, machine translation, social computing and networking, and the natural user interfacing, and all the whole world’s knowledge can be brought together in expert systems that can reach out and help you more. With that, I think we can not only help the generations of students that will follow you in these great school here, but hopefully we will find that it is a technologies to allow billions of students who today have no access to great teachers and great learning, and to be able to bring to them a world of knowledge and dreams in the future too.
Thank you, very much!
Student Q&A Session:
Q: Hello, I am an intern at Microsoft Research Asia. My question is what is your expectation for Microsoft Research Asia in the next 10 years?
Craig Mundie: Well, I have several expectations. One, I expect it continue to grow and be one of the largest research operations that we have in the world. Two, I think that many of these fundamental technologies that I talked about today will be on the research agenda for the group here. Three, I expect that as the Asia market becomes a greater and greater part of the world’s total consumption of these intelligent technologies, that the research group in Asia will have to take the lead in providing the cultural adaptation of these concepts to the unique market, social, and legal environment that exist in the Asian environment. The traditional models of localization of products really don’t work as we begin to embed them deeper and deeper into the ways people live and work. So I think that the Microsoft Research Asia will have to play a role in each of the dimensions over that 10-year period.
Q: Mr. Mundie, I am master’s student from Tsinghua University. I know Microsoft Research Asia has lots of collaborations with universities in the Asia-Pacific Region. How will the economic downturn affect the investment of Microsoft in this field?
Craig Mundie: I don’t think it will have much effect at all. When Microsoft’s management looks at the current economic crisis globally, we believe that it will be sort of reset and a slow growth out, and in that environment, we made the decision that the thing that we will preserve the most will be our investment in research and development. We think that in previous economic downturns, ever since the 1800s, the companies that have been in the lead in the emergence from the economic downturns are the ones that have made the greatest investments in new technology development during the financially weak periods. And so Microsoft will sacrifice other activities in order to be able to sustain our research and development activities. So while there may be some small adjustments as we adjust many things around the world, I don’t think there will be any material impact on our research and development activities here in China, and we certainly don’t want to slow down our collaboration with many of the academic institutions around the world.
Q: I have a question. Many students want to know whether Microsoft Research Asia will recruit new staff in the coming years.
Craig Mundie: Yes, we will continue to add staff. You know, even in this economically weak period, we will add as many as we would in very strong years, but even now the company is continuing to add people, and we are maintaining all of our intern programs in Microsoft Research worldwide. So no slowing-down on internships and some modest slowing-down in hiring of new research members.
Hsiao-Wuen Hon: In the next few months, there will be more than 10 fresh graduates that will join Microsoft Research Asia. So we will continue to do that. I also want to add one point. If you remember, when we started the Microsoft Research Asia 10 years ago in 1998, it was in the middle of the Asian Financial Crisis. So I think I agree with what Craig said that during the financially difficulty time, we are going to make smart investment in R&D, and we will gradually end up being the winner.
Q: Good afternoon, Mr. Mundie. Just now you showed us a lot of new technologies, and you concluded that our lives would be changed by these technologies. I believe that the changes will actually happen only when there is some evolution in hardware instead of in software. So I want to know if Microsoft, as a great software company, will do any research and development concerning hardware.
Craig Mundie: Microsoft Research, for many years has actually done hardware research too. Mostly we think about doing it at the architecture level as oppose to the actual manufacturing of the computer itself. But we partner closely with many of the leading companies that manufacture hardware systems in order to make sure that there is an optimal match between what we want in the software and what they are able to build in the hardware. I have had groups in both of research organization and in our incubation activities for the last seven years working closely with the hardware companies in the architecture of these evolving high-core count microprocessors. We have people who are working on all of the data parallel extensions and transformation of graphic pipelines and many of the other things. So while we don’t generally produce hardware ourselves, we are very involved in the evolution in the architecture of those hardware machines. Today it is very clear that there will be many, many hardware changes in the next three to five years that will actually create the biggest change in software that we have seen in the last 30 or 40 years. So I think there is no shortage of the evolution in hardware.
Hsiao-Wuen Hon: Craig, do you want to say a few more words about the surface devices that are not sold in China?
Craig Mundie: The surface I demonstrated is our initial product where we are trying to take a number of technologies around machine vision, and direct manipulation of interfaces, and begin to think about a day where many of the surfaces in your world will be place of interaction and display – just like the last flexible display I showed you. And I don’t know exactly which form we will decide to bring these forward here in China or in other markets, but we do see a time not too distant in the future where many, many display surfaces will be present both in the home and in the workplace, and that will add more interaction capabilities than the current display model that we have today. So surface is the first step in that direction.
Q: Good afternoon, Mr. Mundie. Cloud computing is becoming closer and closer to us. Our next goal is natural user interfaces. But I think the first step towards natural user interface is that we can have access to the so-called “cloud operating system.” That means we don’t have to install a huge operating system locally on our hard drives. We just turn on our computer and we can have access to an operating system on the cloud. How far do you think it is from us?
Craig Mundie: I actually completely disagree with your statement. I think you have two completely separate questions, and I will answer them separately. One is “what is the future of how software is essentially developed and installed on all devices.” So I think that the old model of installing software on any local devices is going to change quite dramatically. The software is not going to get any less complicated. It is actually getting more complicated. But what that means is that we need get better ways to develop it, and we need better ways to maintain it and install it. And I think increasingly if there is a cloud component, that the introduction of software into these increasingly intelligent client devices will be completely automated. You will not have a model of separate installation of the either operating systems, or the applications, it will be a bit more like a “click-to-run” model, like what you have on the Internet where you click on something and you see it running. So I think, one, it is important to recognize that that will happen no matter what. As to the question of natural interface, I actually don’t believe that the demo I showed you of the robotic receptionist – that can’t be actually built in the cloud. The reason is that the communication latencies between the devices where you present this and where it gets computed are too long to get this coordinated real-time interaction. And the amount of computing that is required is just huge. You have to remember that the “cloud” is built out of exactly the same components as the client. So the only reason that the cloud is an economical computing environment is that it is a low-duty cycle computing environment like time-sharing. And these natural user interfaces have a completely opposite property to the point-and-click interfaces. Point and click uses very little computing and so the machines are mostly idle. Natural user interface is continuous computing of a much more intensive nature and with incredibly tight in tolerances in terms of timing. So it is my technical belief that these types of computations are too expensive and too tightly controlled from latency point of view to be computed in the cloud. In addition, even if you think that the latency is not an issue, the cost of communication will go up dramatically. If you tell me that the robot is sitting there with high-definition cameras watching the scene, and that you are moving the HD video up to the cloud to be processed continuously so that she knows when you walk up and want to interact, it is completely not economical to build that up there. So that is why I say that the future platform is this cloud-client combination type of platform, and you want to assign appropriate tasks to the right ends of the wire, and you want to get computing as close to the data as you possibly can. So that is why I think it important not to think the next era of computing is cloud. The next really important era is this cloud-client hybrid, and then you have to think about writing these applications in a way that span this technology spectrum in a best possible way.
Q: So this student comes from Beihang University. What is your view of software piracy, and what’s Microsoft’s strategy for it?
Craig Mundie: The challenge of getting an effective intellectual property regime in China and in other countries remains a big problem. Great progress has been made in China. More work remains. But I also think that this future that I described today, where we would have the ability to have a component of our offering in the cloud, as well as a component of our offering on the client, creates some new opportunities to offer the consumer different ways for paying for the software that is not an upfront licensing model, the way that has been for many years. And in many of these countries where people culturally and morally want to do the right thing, we have to make sure that they have a business model and a product offer that conforms with their ability to pay. And so the services give us a way to create high-value products that don’t actually have as high piracy opportunities as our historical technologies used to have. If you look at other software-related businesses in China, for example, some of the gaming companies, they have high-value software, but it is 100-percent a service. So there is no piracy because you cannot steal the service. I think that in the future there will be an opportunity to blend these models together even in Microsoft’s expanded business, and I am hopeful that plus steady progress in China and our their attention to the question of intellectual property protection will both improve the business for Microsoft here in China but will also create favorable environment for the emergency for many, many more software businesses in China. And in the discussion I had over the years with Chinese leaders, it was their realization a few years ago that China needs to move beyond being a manufacturing economy to being a knowledge economy. Anywhere in the world, a knowledge economy in the future will have more and more of its valuable knowledge represented in software, and so even if you did not care at all about products of Microsoft or any other multinational, China herself will require a vibrant software business, and that is why the leadership in the country has been increasing their emphasis on the importance of intellectual property protection. So I think there are two things to make it better. One is the traditional model of software sales and distribution will come to be become better protected, but I think that the monetization through services and other payment architectures will also make more software available at more attractive pricing terms, and that will also help.
Hsiao-Wuen Hon: Today’s event is being broadcast live, so we also have questions from our online audience. I would like to ask the first question on behalf of the netizens – Now that you have taken over Bill Gates’ role as the technical leader, will you change Microsoft’s global strategy? For instance, will you integrate Microsoft’s various businesses, and/or branch into new areas in order for Microsoft to survive the economic downturn?
Craig Mundie: Well, I think it is important to realize that it is almost three years since we announced that Bill would retire in the summer of 2007. So we had a two-year period where we divided Bill’s responsibilities to me and to Ray Ozzie. So, I don’t make the total business strategy for Microsoft. My focus is to try to look at where the new business opportunities are, and where the new technology evolution requires us to think differently about creating these business opportunities. So I do focus a lot on the much longer term issues. Hopefully the economic downturn is not something that will be with us for many, many years. We expect that the economic weakness will persist into calendar year 2010, and then there will be a slow return to increasing economic performance. And frankly, there is a lot of that return will be led by the higher-growth economies like China and India. And so, we want to make sure that we have the right strategy to grow in those countries and also grow elsewhere. I don’t think that my work will be directly involved in trying to pick an individual structural change. Steve Ballmer and the Presidents of our business divisions are the ones to think about the right operating structure for the company. My job is to make sure that Microsoft Research and our new business development activities prepare us well for the longer-term future. That is where my focus is.
Hsiao-Wuen Hon: Can you comment on the new areas of health and education?
Craig Mundie: Over the last three years there have been three areas that we felt warranted creating new business groups in the company. One of them is in healthcare area, because health is the largest single expenditure of virtually any governments in the world. And it is of course a basic need of all people. And so we decided that with the transition to digital health, and the involvement of the individuals in their own health and well being in the future, that Microsoft could play a role, so we created new businesses there. Education is another area - it is the No.2 expenditure by most governments, and also a basic need for all citizens. And in both health and education, we see two thirds of the planet who are disenfranchised right now relative to access to these key things. So we started that business. The third new business area is focused at the global emerging middle class. We see certainly in China but in many other countries, more and more people are being given the opportunities to have greater income through technology advances, and we want to be able to ensure that we have products that are tuned to the business and individual needs of these people who have less disposable income. You know China is no different from the US when you look at the top demographic group – there are wealthy people, they can afford all the technologies they need, whether in their personal life or their business life. The difference between some of these countries is just how big their emerging middle class is and to what extent they have disposable income. So we created another business group that is now working to identify what the technological needs are of these people who have very moderate disposable income but who want to take advantage of technologies for their families, in their homes, or in their small businesses. And so we built a group focusing on that as well.
Q: Good afternoon, Mr. Mundie. I am a student from Peking University. My question is, you have showed a lot of technologies including Laura. You have mentioned that Microsoft is creating business in health and in education, and in many areas. Of course, Microsoft is leading in areas, but also you have mentioned that these services are very expensive. Of course they will not come to ordinary people in a short time. So, as Mr. Bill Gates has retired to help the poor, and to help people with AIDS, I want to ask have you got a plan to continue such charity deeds to help more people?
Craig Mundie: For many years there has been a very careful distinction between what Bill Gates and his wife did in their philanthropy, and specific philanthropic activities at Microsoft. We don’t blend them together. So, you know the company is doing many things to help people who can’t afford to access the technologies. We have a program now for a number of years called “Partner in Learning” where we basically make all of our technologies available for free to students around the world who cannot afford to buy any of these products. And now we have these Green Spark programs where all of our development tools we have are made available to every university student in the world - and now every high school students in the world at zero cost. So the scale of our philanthropy is not the size of the Gates Foundation, but the Company has for many years had a big focus on trying to make technologies available to people who have interest and don’t have the ability to pay. I think it important to recognize that companies like Microsoft cannot by themselves – no matter how philanthropic we were – really do much to help with the scale of global problems to help the poor. On the other hand I think our technologies are absolutely essential to dealing with the problems of the poor globally. And so what we have been focused on is how Microsoft can work with various components of society who deal with the welfare requirements of the people who don’t have disposable income – whether it is governments, NGOs, philanthropies like the Gates Foundation, Microsoft invests both in coordinating our work with those organizations, and in making sure that when we can, we make our technologies available at very low or even zero cost to those welfare-based operations in order to promote their use more broadly. So that is how we are approaching it today.
Q: Good afternoon, Mr. Mundie and Mr. Hon. I come from the Institute of Software of the Chinese Academy of Sciences. As we all know, two days ago, one of the biggest competitors of Microsoft, Oracle successfully acquired Sun. Some analysts point out that through this acquisition, Oracle can improve its database, ERP, and middleware, which will enhance its competitiveness against Microsoft. My question is, as the Chief Research and Strategy Officer of Microsoft, from your standing point, what does this acquisition mean to Microsoft: is it a chance or a threat? Will Microsoft make strategic adjustment in response to this acquisition?
Craig Mundie: The answer is “Yes” to all of those things. Clearly when any consolidation occurs between major players in the industry, it represents both a threat and an opportunity. So any good business people will look at that situation and try to determine what the relative components are. You know I cannot really speak to what Larry Ellison and the Oracle management think is their principal motivations for the acquisition itself. Certainly we can speculate, just like the press speculates, on what it may or may not bring to their competitiveness and position. Sun has always been a provider of some big machines that run enterprise database products and in fact I think Sun equipment probably runs more Oracle enterprise databases than any other kinds of machines. But Oracle depends heavily on the sales of the same thing through HP, Dell, IBM, and others. And now they may end up in the hardware business which creates a competitive dynamics with those companies which could change the dynamics in that industry. Historically Oracle has not been a company very focused on the open-source community, and now through Sun they acquired some of the major open-source activities such as Open Office, and Java, and so it remains to be seen how the community will related to Oracle and its approaches to many of these technologies. So I think it is really too early to tell how this will evolve, but clearly any good business people will look at both sides of the coin and make individual business decisions to adjust appropriately. We will do the same thing.
Q: I am a student from the High School Affiliated to the Remin University of China. It is an honor to be here, and as a high school student, maybe I am more concerned about the future development of a person. So my question is that is it better for an individual to explore in more aspects of scientific areas, or to focus on only one specific area? Or does a person in the 21st century need to handle a lot of skills involved in different areas in science?
Craig Mundie: I think the answer has several parts. If you are a young student still, I think it is important to explore as many fields as you can. Because the luckiest people in the world – and I consider myself one of them – are people who discover fairly in early life what their passions and interests really are, and find a way to make a living doing that. And so you know it is important that you explore enough in the early years of your education. That you don’t close your opportunity to discover what it is that you really love. On the other hand, I think there is another change happening in the world today, in academia and broadly in business, which is more and more of the challenges that we face need to be solved by multi-disciplinary teams. That is, less and less likely today that any significant problem is going to be solved only by a team of specialists who were narrowly educated in that one discipline. Or if they are specialists, they will have to be complimented more and more by people who have broader skills. One of the student presentations today talked about the fact that they have brought people together in the student club from many different parts of the university. They have literary students; they have technical students, math students. And I think this multi-disciplinary orientation is extremely valuable. Certainly at the graduate school level, the world’s elite universities today recognized the need to develop more balance in their students, more general ability to collaborate with people beyond their own disciplines. And many universities today are restructuring their departments or restructuring the way they sponsor their researches in order to promote this multidisciplinary collaboration. So I think for all of you who are students today – whether at high school level or above – that you should be on the lookout for the things that you really love, and are passionate about. That to the extent that you focus around those things, and pay great attention to the need to collaborate with your classmates, and the ability to do your work in some way that brings different viewpoints forwards. I think this will be a hallmark of the people who make the biggest differences in the world in the future, and I recommend that approach to all of you, no matter what ages you are in your studies today.
Hsiao-Wuen Hon: This is the second question from our netizens. This is a personal question. What will you do after you retire from Microsoft?
Craig Mundie: I have no plan to retire. That is a sort of an important statement in its own light. In July this year I will be sixty years old, and yet I still feel quite young. And medical technology has continued to evolve in a way where I think life spans are going to extend. You know, the most aggressive people like Ray Kurzweil, who is a futurist, and many of you probably have read his books like “Singularity”. I tend to agree in principal with some of the things that Ray advocates, which is that we are at a point of time where the technological advances are so rapid that in the course of your lifetimes, it is very hard to predict exactly how it will occur, but it is quite likely that we will have the ability to build replacement parts for people. And so I think we have a huge societal challenge ahead as a species, when in fact people will live longer and longer lives. And so, the traditional models of growing old and retiring, I think, must be updated in the relatively near future. My personal goal is to continue to help with important problems. And I think Microsoft, for me, has been a very good platform to go out on a worldwide basis and look to bring technologies to bear on society’s most important challenges. Today the world has many challenges – like the ones we have talked about – healthcare and education of course. But energy and environment and others. I think that these technologies are at the heart of being able to reduce these challenges. So I don’t sit still very well. My wife does not think it would be very good to have me in the house all the time. And so as long as my health is good, I will continue to work and hopefully be allowed to continue to make technological advances that will bring solutions to many of the world’s difficult problems.
Q: I am from Peking University. We saw some impact that you have shown how software will change our way of learning. But what if not just change the ways, but change the structure of knowledge, and also the way of thinking. You know education is influential when they grow up. For example, the six-year-old boy, when he grows up his first imagination of the universe will not be the stars up in the sky, but the screen of Microsoft software. So I think it is really a big change. So what do you think of that?
Craig Mundie: When I think of Benjamin and what he does, he sees the screen that was architected by people whose goal was to make it possible for a six-year-old to navigate in the sky to make a story. The fact that it happens to come from a Microsoft researcher I don’t think will be material in his life as an adult. The reason I showed it today is that I agree with your statement that these technologies change the way people think. That is really the power of it, and the fact that any one application or presentation comes from Microsoft or somebody else won’t be as important as the fact that at the age of six he has such capability. And all of you today grow up, and your brain is wired differently than my brain. And the reason is that you have been exposed to a set of technologies and opportunities from a much younger age than I was. I just had my first granddaughter and I know that as she grows up now, she will be exposed to things in a fundamentally different way. Madame Liu, or State Councilor Liu visited me at Microsoft in Redmond on a Sunday morning. She and I were talking and I told her that the efforts to explore Chinese culture and language were important and in fact in my own family, my seven-month-old granddaughter is now going to classes to learn Chinese – mandarin. The reason is that the neural scientists now believe that at a very early age, even before you can understand any words or speak any words, that the neural pathways form based on the sounds it is exposed to. And so if you want to be an expert or a natural speaker of other languages, the earlier you are exposed to them, the better off it is. So her present for Christmas this year, when she was four months old, my wife and I gave her Chinese lessons. You know, I just think that is an example of how as technologies and society evolve, we recognize that it is more and more important to expose people to these things very, very early. So today when you grew up, you think game consoles and cell phones and personal computers, they are just normal. When I grew up, they did not exist. When my granddaughter grows up, she will have things like Benjamin has by the time when she is two years old. And I think that her brain will actually be physically wired differently than mine and yours. And of course, that is the thing that allows our species to evolve, not just in physical ways, but in intellectual ways, and I think the role of technology in allowing that to happen more and more globally and ultimately for more and more people is a very powerful and important thing. I think many times olden people came to me and they say they are so worried about people like you. I say “Why are you worried about them?” They said, “I watch my teenage kids – they talk on the cell phone, text, and look at the computer, watch TV, and write their term papers simultaneously. How could they possibly do any of these things well?” And I said, “You don’t understand your kids look at you and ask ‘why can you only do one thing at a time?’” And the difference is how your brains are wired. And so I think that in the future we have to recognize that each generation is given a different set of gifts and experiences, and we need to make sure that we fully take advantages of them, not to think they create new problems.
Q: Hi, Craig. Thank you for your presentation. It is really amazing. I am a student of the School of Software at Tsinghua University. I am interested in software engineering and in knowledge engineering. The problem I am now facing is how can get a group of experts in different domains to work together and share their knowledge effectively. But I really feel excited by your demonstration and that to build such a demonstration, I think you need software engineers, computer scientists, and also experts from medical domains to work together. Do you have very good examples or ideas to share with us about this question?
Craig Mundie: In one way this relates to my earlier comments about multi-disciplinary approaches. I think that as we raise the abstraction level of the software tools, our ability to directly incorporate the expertise of people in other disciplines will get better and better. In my era, if you want to have someone who understood heat transfer and metals, and figure out how to represent that, you either had to teach them how to write FORTRAN Programs, or you had to have a programmer who was willing to learn enough about the thermal dynamics of heat transfer to write the program on their behalf. As we get higher- and higher-level tools, I think it becomes more and more a direct process of giving people who have special knowledge the ability to capture that. I believe that the next big stage of this will be – what I think of it as “expert systems” - where people who have domain knowledge come together and find a way to encapsulate that in ways that more and more people are able to describe their problems, and have the expert system come forward and assist them and organize and solving the problem. And that the amount of computational facilities we have, and our ability to model more and more physical phenomena, will make it a more and more direct process for people to get these kinds of solutions. So whether it is in biology or the other physical sciences, I think that things will get better and better. And I also think that the models of visualization and the presentation of information will come to a higher level, too. Again, if you look at one of the students’ presentation where they take the live maps and basically they are doing mashup on that in real time. If you think back just a few years ago, that would have been completely impossible – I mean the ability to have the maps, and the ability to operate on them in real time – it was just not possible. Now that you give these higher-level capabilities to people, it becomes a lot easier for them to build them. I think of this mashup on the maps as a very simple and fairly informal example of composition. And I think one of the biggest challenges in software as a discipline is that we are not yet graduated – beyond the art forms – into a true engineering discipline where we can formally compose systems at very large scales. I think this is our next great challenge, and with it comes solutions to concurrency and complexity problems and potentially correctness. And so our ability to reason about the correctness of programs will become an important thing that we have to do in the future. So focusing on ways that allow domain experts to compose their knowledge into these software tools, I think, will be a way that we begin to move forward, and make progress in that direction.
Q: Does Microsoft have this kind of research programs or research projects related to this specific problems like having a lot of domain experts working together in this knowledge exchange?
Craig Mundie: Yes, we have some research in that area. We have a number people who are looking at this question generally of composition. And I think somewhere there may be some advances.
Q: Hi, Mr. Mundie, I am from the No.101 Middle School, and I want to ask a question. Do you think that these amazing products will have some negative influences on teenagers in the future? And what will Microsoft do with that?
Craig Mundie: No, I don’t think they will create negative influences.
Q: I think these teenagers’ minds will be easier because of these products?
Craig Mundie: Easier – you mean weaker? I don’t think so. You know, these tools should allow you to apply your mental capacities to higher order of problems, not to make you mentally lazy. I met with some of our interns here before this. I told them when I was a young kid, to me the tools that I had to help with computation was a slide rule. And when we were all in engineering school and we had slide rules people said to us that – they invented the calculator – and people said, “Oh, calculators! They are going to make all the students so lazy because the calculators will keep track of the decimal point and with a slide rule we had to keep track of the decimal in our heads.” Now, you’ve grow up, and not only did you have the calculator but you also had the personal computers. I don’t think you are mentally weak because of that. And so I think what has happened is that you are able to use your intellect on higher-order problems. And I think this will be always true for people who have the right approach to using these tools. So I think that the students in the future will be very lucky because they will have such powerful tools for support.
Q: Hi, Craig. I am a student from Xi’dian University. It is my honor to hear your speech, and I admire hearing about modern and future technologies. In your speech, we witnessed the advantages of future technologies, and we witnessed the fact that the technologies can change our daily lives, especially the way we think. However, after we see some advantages, I see some disadvantages of technology we cannot ignore. For instance, the environmental influence, the technologies of computer sciences and engineering. I think some waste of current environment is influence is by these technologies. At the hardware level, the waste of client is popular in many network companies. At the software level, the software programmers make a great deal of wastes in significant resource of time and memory. My question is, how to deal with this problem?
Craig Mundie: I think that society always chooses to find, at moment of time, a balance between making things easier and wasting some assets, whether those are intellectual assets, energy assets, and physical resource assets. Because an attempt to completely control all of these at any moment over constrains the system. On the other hand, computing as a discipline, without question, has improved people’s lives. And now that there is increasing focus on energy and environmental issues, we can now turn more of our attention to optimizing the way in which the computing world is using these increasingly scarce resources. So when I look at the machines we will build in the next few years, they will be incredibly more conservative of electricity in the way they operate than in the past. The reason is people care about it now, and people did not care about it in the past. Companies must focus on the aspects of product design which creates the value proposition that the market place rewards at any moment. So in the last 30 years, people didn’t reward companies in the computing business who said, “my computer uses less power.” Today, the market place is saying, “Hey, I am worried about that.” Or government regulations may tell them that they must worry about that. When there is an economic rewards for solving these problems, we will solve those problems. So whether it is at the level of individual machines, or the data centers in the cloud, there is a huge research and development effort now to making these things more energy-efficient. Similarly in software, one could argue that all the new machines have more capacity than people need. If everyone were just to go back to writing in assembly languages, and would be very careful to optimize every single instruction that was used by their machine – maybe they could use fewer machines. The problem is that the number of people who could write useful software, if confined to the using assembly languages, is a tiny, tiny percentage in the population. So we waste computer power in order to increase simplicity so that more people could get the benefit. So at macro level, the society is always finding a re-balancing of its investment around each of these problems. I think most importantly, to recognize in this – I say energy field, is that … and Microsoft itself has had to think deeply about this in the recent year or two. If you look at the carbon footprint of Microsoft itself, it is actually fairly small. So in an environmental sense, the company is not a big carbon producer that is affecting the atmosphere. On the other hand, if you look at what we could do both by providing better tools to help people with bigger carbon footprints, or what we could do to write software that could lower the energy consumptions of all the world’s computers. The leverage is much higher on doing the ladder two than on just on trying to lower our own carbon footprints. So as a business we always have to look at what is the ultimate objective and what we are willing to do about it. But you know I am quite optimistic that many of these computer technologies will not only be more efficient themselves, but that in the future, they will be the key to solving the management of energy consumption broadly, and may in fact to be the key tools to find zero-carbon energy resources in the future.
Q: Good afternoon Mr. Mundie. My question is short. If I find your surface and the interface is not good enough or not customized, do I need an expert to change the interface?
Craig Mundie: I think more and more people will be able to modify the way they interact with a computer themselves. Today, when you think about the computer interface, it is mostly about the graphic user interface. There are some aspects of that need to be rigid in order to get uniformity. There is always a tradeoff between personalization and the economy of bringing lots of people forward to use it. And so I think we always want to provide the freedom for people to change what they can without creating so much cacophony that no one has a common view of how the thing works. That is always a delicacy balance. As we move beyond the PC with the keyboards, the mouse, and the fixed graphical user interface – into more natural user interfaces, then I think the flexibility for you to interaction with the machine in ways you are more comfortable with will get better and better. And the idea to have customized each aspect of it will be less of a question.
Hsiao-Wuen Hon: This concludes our event today. I know many of you like me don’t want this to end, I promise you on behalf of Craig we will do it again in the future. So once again, let’s give another round of applause to our distinguished speaker Craig Mundie.
Craig Mundie: Thank you. Thank you everyone.
