Imagine if you will a life-size Tyrannosaurus Rex standing in front of you snapping his sharp, sharp teeth in your direction. No, scientists have not succeeded in using DNA extracted from a mosquito in fossilized amber to create dinosaurs. This is curatorial knowledge and technical expertise making a science fiction movie into an educational reality. Your ipad can act as a window through which you can see what a dinosaur skeleton would look like with flesh. Doesn’t it sound awesome?! Let me tell you from firsthand experience, it is. And so is its potential for all sorts of museums – including yours.
Want to see what I’m talking about? Watch this video.
What is Augmented Reality and How Do You Do It?
Helen Papagiannis gave a talk at TedxDubai 2011where she defines Augmented Reality nicely.
“We can think of it in terms of virtual reality. In virtual reality we’re closed off from our physical world. We’re completely immersed in a computer generated environment. Now in Augmented Reality we have the opportunity to still be in our physical surroundings in our actual space and now hold up an augmented reality device, such as a Smartphone or an iPad, and be able to see different information layered on top of our reality. So how does this work? Well the augmented reality device will have a software on it and the software when it recognizes a symbol, an object or an image will then place additional content.”
If you are like me you are now asking how can I create this!? To find out I spoke with Ken Reddick, Creative Director at Meld Media – the company that worked with the ROM to create the Augmented Reality (AR) as well as a number of other really cool digital creations for their temporary exhibition Ultimate Dionsaurs: Giants from Gondwana.
Ken told me that there are two options in creating AR. You can create a Custom AR application or you can go with a user platform such as Junaio or Layar. The user platforms are easy and quick to use but you can do more if you decide to create a custom AR experience (which of course will be more expensive).
Hardware, Software and Logistics
Ken created two custom AR experiences for the ROM. The first is for the ROM’s ad campaign to promote the Ultimate Dinosaurs exhibit and is a true AR experience. Ken and his team also created an AR-like experience called the Dino Viewer.
Around Toronto and within the exhibit there are posters that are part of the ad campaign promoting the exhibit. Visitors can download the ROM’s Ultimate Dinosaur App and use it to activate the AR. Users point their iPhone at the marker (the black square that forms the border of the poster) which causes the dinosaur to jump out of the poster on their iPhone screen.
At the end of the exhibit there are three HUGE dinosaur skeletons on display – the Tyrannasurus Rex, the aptly named Gigantosaurus and the Carnotaurus. Each dinosaur has three iPads installed in front of them – the Dino Viewers. These iPads point towards the skeleton and when activated with touch show the dinosaurs as they would look with flesh. The team animated the dinosaurs in consultation with the curators to create accurate portrayals.
The iPads can be moved (swiveled on their stand) to scan different parts of the skeleton. Visitors can touch the dinosaur on the screen to learn more about it. When touched the body part will start moving to show you how it would have looked in action and a text bubble pops up with educational information about that body part.
Ken told me they decided to install the iPads because it meant they could create a more realistic looking dinosaur. This is because with installed devices, Meld Media could make higher resolution dinosaurs knowing that the iPad 3 (the latest iPad hardware) could handle it. If they had created the Dino Viewer for people to use on their own devices, Ken and his team would have had to create lower resolution dinosaurs so they could be viewed on the lowest common denominator – the slowest device (the iPhone 3GS).
Meld Media created the AR and AR-type experience using a number of different elements. The AR from the ad campaign was created using tracking and animation. String is used for tracking – it allows the image marker to be recognized and then launches the AR. The 3D dinosaur animation, which in this case is the AR that is launched, is created using a gaming development application called Unity.
Unity was also used for the Dino Viewer (once again to create the 3D dinosaur animation) but String was not – it is the reason why the Dino Viewer is not strictly an AR experience. The image marker works for the two dimensional poster but when creating AR on top of a 3D object (the dinosaur skeleton) using a marker would be a bit more of a challenge (imagine markers all over the dinosaur skeleton or wallpaper on the wall behind the skeleton – not ideal). So instead, Ken used the gyroscope and compass on the iPad and created a spherical image. A gyroscope is a device used to measure and maintain orientation; this in conjunction with a compass allows the iPad to know where the dinosaur should be.
The background of the museum space on the Dino Viewer, which really makes it seem that the dinosaur has emerged from the computer generated environment and into our physical one, is made by taking pictures from the exact viewpoint of where the iPad is installed. The pictures are taken and then stitched together to create a spherical image. This places the 3D animation into the same space as the viewer. But we aren’t actually looking at a live view of the real world using this image sphere, just the illusion of it, another reason why the Dino Viewer is not a true AR experience, rather an AR-like experience.
A new tracking library software has emerged since the creation of the Dino Viewer called Vuforia which works across platforms meaning you don’t have to decide iOS or Android but can create one AR experience that will work on all devices and operating systems.
Cool AR Applications
There’s some really cool stuff happening with AR in other fields that Ken pointed me to. This Word Lens AR would allow museum visitors who speak different languages from those used in a museums interpretive text panel to be able to read them! The Ikea Catalogue is already awesome but look at how much cooler it becomes with AR. One of the features of this is that the app gives you x-ray vision to see inside furniture. Can you imagine that applied to a museum object? Think of the possibilities! How about using AR to remove objects? Check out this video to see how it works. This could take a game where objects disappear from the museum to a whole new level! Okay I know that there are a lot of exclamation points but I’m really excited about what AR could do to make museums more engaging!
If you want to know more Helen Papagiannis writes about AR in her blog Augmented Stories.
To see more ROM videos about their exhibit visit the ROM’s Dino Channel.
Stay tuned for part two of my Augmented Reality series – Does Augmented Reality Augment Learning?