Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
To the edge of the universe and back again

How a kid’s dream changed how we see the heavens—
and our world

Imagine flying through the universe, faster than the speed of light, visiting planets, nebulae, stars, and asteroids. Or sit back, and let someone else drive while being shown around. Now, imagine you can do this free, in the comfort of your living room, while wearing your pajamas. Sound like fun?

Here’s how Microsoft Research built the ultimate virtual telescope to let you do just that—and more.

Curtis Wong
Curtis Wong

Start with a boy’s stargazing imagination

Growing up in Los Angeles, Curtis Wong would gaze up at the sky and wonder what stories the stars and planets told. Even filtered through the smog, the night sky captured his imagination. Today, when Wong contemplates the night sky, he consults the application he helped create: the WorldWide Telescope.

A Sidewalk Astronomer (Documentary Sample)

Part of his inspiration came from John Dobson, the iconoclastic physical chemist who later became a monk and a world-famous amateur astronomer. Dobson longed to share his knowledge of the universe with others, and for much of his life, he could be found on a street corner in San Francisco, setting up his huge homemade telescope and captivating listeners with stories about what they were seeing in the heavens.

Wong began working with Dobson in 1993, assisting him in the eponymously named John Dobson’s Universe, a CD-ROM project that centered on building a story-driven exploration of the universe for the public. Unfortunately, the project was canceled before completion, but for Wong, its goals remained alive.

The Fourth Paradigm: Data-Intensive Scientific Discovery book cover
In The Fourth Paradigm: Data-Intensive Scientific Discovery, a collection of essays expands on the vision of pioneering computer scientist Jim Gray for a new, fourth paradigm of discovery-based, data-intensive science and offers insights into how it can be realized.

Add lots of data

In 2002, Wong, by then working for Microsoft Research, began to collaborate with another Microsoft researcher, Jim Gray. Gray was one of the first to see how big data—that is, large and complex data sets—would have a major impact on scientific research. He predicted the emergence of a “fourth paradigm” in scientific methodology—one in which researchers gain new insights from the analysis of large data sets. And nowhere were data sets getting larger than in astronomy, where improved electronic sensors were gathering ever-growing data while the volume and higher resolution of digital photography began to gobble up space on our computers. New efforts such as the Sloan Digital Sky Survey gathered terabytes of information on the red shift, spectra, and imagery from faint galaxies, enabling astronomers to map the large-scale structure of the universe. Moreover, astronomers from around the world were beginning to share their data online, instead of stockpile them in inconvenient, hard-to-locate locations.

Even filtered through the smog, the night sky captured his imagination.

To Gray, this wealth of information was the perfect laboratory in which to accelerate the searching and querying of enormous data sets. Gray realized that this immense data collectively represented the output of what he called a worldwide telescope. During a talk, Gray asked for visualization people to join him in the effort to use this enormous store of data.

Wong saw this as an opportunity to create the astronomy tool he’d been imagining: not just a collection of images, but a visualization of the universe that would give every kid what he’d dreamed of as a boy in smog-shrouded Los Angeles—a clear view of a dark sky, through a gigantic telescope, with a professional astronomer providing a guided tour of what you’re seeing. Wong then proposed the Universe Project, which he saw as both an educational tool of profound value and an opportunity to enable researchers to visualize large data sets. He was convinced that such work, as Gray had predicted, would become an increasingly important aspect of scientific research.


TEDxCaltech: WorldWide Telescope: The interactive sky on your desktop

In 2005, Wong recruited Jonathan Fay to join the team. Wong recognized Fay for his prowess as a developer but also knew he was an amateur astronomer—likely the only Microsoft researcher who has built a fully functional domed observatory in his backyard—and an engineer who has experimented with high-performance visualization simulations. Likewise, Fay knew of Wong’s critically acclaimed work with interactive media for education, but oddly enough, their early work together had nothing to do with the Universe Project. Instead, Fay began his association with Wong by working on an interactive, cloud-based photo display.

The challenge was to turn a cool demo into a working application, one that not only showed the universe in 3-D, but also let you fly through it.

Jonathan Fay
Jonathan Fay

On the side, however, Fay continued tinkering with a 3-D viewer that he’d begun years earlier while working on an Earth-mapping project with Jim Gray. Fay had since turned his attention to using the viewer to create 3-D maps of celestial objects, such as the moon and Mars, and his 3-D viewer had begun garnering attention during Microsoft technical conferences. Wong saw the potential of adapting Fay’s viewer to his vision of creating the ultimate virtual telescope, a tool that would organize and visualize astronomical data and provide contextual, interactive stories about the universe.

Jim Gray
Jim Gray

Sadly, Jim Gray was lost at sea in 2007 and never got to see the full visualization of the astronomy data. In his honor, Wong renamed the Universe Project to WorldWide Telescope, using the terminology Gray had coined.

The Jim Gray eScience Award was also named in his honor.

Mix in beautiful, smooth imagery

Fay immersed himself in the Universe Project, and today, he remains the primary technical force behind the WorldWide Telescope’s astronomy aspects. He began by porting Palomar Digital Sky Survey data into the 3-D viewer, reprocessing it all to make it the largest composite image of the night sky. Soon, the astronomy viewer had not only Wong’s support, but that of Gray as well, and it was demonstrated in 2007 during TechFest, Microsoft Research’s annual technology showcase. The viewer displayed the night sky on 18 20-inch monitors tiled together to create a wall-sized display.

The TechFest display was a crowd pleaser, but the challenge was to turn a cool demo into a working application, one that not only showed the universe in 3-D, but also let you fly through it. Perhaps most importantly to Wong, he wanted to let users create narrated, interactive, rich-media guided tours, and enable them to access further information from the web. Consequently, everything Fay and his colleagues built inside the visualization engine was designed to enhance storytelling, by adding interactive authoring capabilities for cinematography, annotation, animation, narration, and music.

Eighteen 20-inch monitors tiled together gave a wall-sized display of Fay's viewer at TechFest 2007.

They also focused on rendering the imagery in its full beauty at the highest resolution. Unlike previous planetarium software, which relied on illustrations or low-resolution images of the sky and was cluttered with grids and labels, WorldWide Telescope used real images of the night sky at the highest resolutions available, as captured by major telescopes from every area of the spectrum: optical, X-ray, radio—you name it. The result was and is awe-inspiring and does not sacrifice the information provided by those intrusive, old-style text notations. In WorldWide Telescope, you simply hover over an object, and its name and associated data appears.

Christophe Poulain
Christophe Poulain
Hugues Hoppe
Hugues Hoppe

But even that was not enough for Fay, who wanted to eradicate the visible boundary lines created by stitching together 3,600 plates of photographic images. This led to the Terapixel project, which eliminated the checkerboard effect by reprocessing all the underlying data to create new images and a smooth panorama of the entire sky. In essence, it puts all the images through a processing pipeline on a supercomputing platform, which enabled Fay and his collaborators—Dean Guo, Christophe Poulain, and Hugues Hoppe—to produce the seamless view of the heavens in WorldWide Telescope.

All that work paid off: You can start with a naked-eye image of the sky and then seamlessly zoom into a Hubble Telescope image, all the while maintaining context to understand where you are and the scale of what you’re seeing. The interface is designed for both the casual user, who wants to zip around the universe, and the serious student, who wants to examine specific celestial details.

Planetariums that use WorldWide Telescope

Planetariums can use WorldWide Telescope to create breathtaking exhibits that provide two things they’d never had before—views of the sky based on real imagery and a 3-D universe.

Adler Planetarium

  • Renowned facility in Chicago.
  • Has the highest-resolution digital dome in the United States, at 80 million pixels.
  • Cosmic Wonder is an interactive planetarium show about the mystery and vastness of the universe.
 

American Museum of Natural History, Hayden Planetarium

  • Based in Central Park West, New York.
  • The Hayden has installed WorldWide Telescope in its dome and has been experimenting with its possibilities.

California Academy of Sciences, Morrison Planetarium

  • San Francisco’s new, reconceived museum, aquarium, and planetarium.
  • Has installed WorldWide Telescope and had produced an interactive show on earthquakes.
 

University of Washington Planetarium

  • Smaller, teaching planetarium in Seattle.
  • Uses WorldWide Telescope to create a state-of-the-art display at a fraction of the price of using conventional equipment.

Bring it back to Earth

As stellar—no pun intended—as the WorldWide Telescope’s celestial stories are, the utility of this application is hardly limited to astronomy. Shortly after its launch, Wong began looking for opportunities to share the WorldWide Telescope’s data-visualization and narrative potential with Microsoft product groups. To Wong, one obvious opportunity was to help business people extract intelligence from the mountains of Earth-bound data locked in their spreadsheets. That’s how WorldWide Telescope became the inspiration behind a business-intelligence product contribution to Microsoft Excel 2013, a project codenamed GeoFlow, that rolled out for public preview as an Excel add-in in the spring of 2013.

“GeoFlow is like Flight Simulator for your data.”

— Scott Ruble, principal program manager lead, Microsoft Office Division

GeoFlow turns location-based data into powerful visualizations that provide new insights by plotting data on a globe over time. For example, the GeoFlow team took ticket sales from a Picasso exhibit at the Seattle Art Museum to create heat maps, bubble charts, and bar displays that show where the tickets were sold and whether the sales were made in person or online. By adding the dates of the sales, the visualizations are animated on the globe across time. Focusing on sales in the Seattle metro area, a user can see when and where sales spiked and could create an annotated, guided tour that explores contributing factors such as the impact of targeted advertising, the influence of blogs, or the availability of servers for online sales.

To make GeoFlow a reality, Wong connected with Scott Ruble, a principal program manager lead in the Microsoft Office Division, to persuade the leadership of the Startup Business Group and the Office Excel group of the potential for 3-D spatial and temporal, interactive data visualization. The WorldWide Telescope, Ruble says, provided a “tangible manifestation” of what GeoFlow could become.

Wong prototyped some of the data visualizations on Earth in a special version of the WorldWide Telescope, enabling Office program managers and developers to see firsthand the functionality GeoFlow could deliver for business analytics. This working prototype helped the Office development team and Wong to design and build GeoFlow from the ground up—in the Office code base—by using the latest technology and tightly integrating it with Excel to optimize its performance with potentially millions of rows of data.

In another demonstration of GeoFlow’s capabilities, the Microsoft account team in Dallas used residential data to visualize energy consumption across homes of varying sizes and values. The team members used publicly available property data to obtain information about the size, age, and location of Dallas-area houses. They next generated simulated monthly kilowatt-hours consumption data on a per-household basis, using national averages on seasonal consump­tion and their own personal household energy-consumption information.

Sample GeoFlow reportSample GeoFlow report from Dallas-area data on housing and energy consumption.With all these data sets in place in an Excel workbook, the team then generated a variety of interactive reports, including bubble charts, pie charts, maps, and line and bar graphs that visualized and analyzed the hypothetical energy-consumption patterns. For example, the team produced heat maps that visualized household electricity consumption according to the location, age, size, and market value of homes. By taking advantage of the Play feature in GeoFlow, they then could show how the map of electricity consumption changes over time, so that a customer might see its ebb and flow over a three-year period. They also zoomed in on specific neighborhoods to illustrate differences in consumption patterns among them.

GeoFlow Preview for Excel 2013

GeoFlow has the potential to democratize access to such powerful business analytics, making it available to millions of people who have these sorts of geospatial and temporal data. Reactions to the public preview—which launched in the spring of 2013—have been gratifying to Ruble and his team.

“I’ve lost track of how many ‘wow—this is amazing’ comments we have gotten,” he says. “Customers are using GeoFlow to find unique insights in their data and tell the story behind it while having an enjoyable experience. GeoFlow is like Flight Simulator for your data.”

It may seem improbable that the same research project that started with visualizing and telling stories about the universe could inspire a product that will enable enterprises to use data visualization to enhance their businesses, but that’s exactly what happened with the WorldWide Telescope and GeoFlow. Who’d have thought a boy’s stargazing would take him to the edge of the visible universe and then lead him back to Earth to help millions of people see, understand, and share stories about data to increase their understanding of the world?