Augmented Reality for education and engagement in Retail

[av_layerslider id=’23’ av_uid=’av-456bs8′] [av_textblock size=” font_color=” color=” av-medium-font-size=” av-small-font-size=” av-mini-font-size=” av_uid=’av-2gvjmw’ custom_class=” admin_preview_bg=”]

My car’s engine just popped out of my bonnet!

As Melbourne, Australia hosts the latest Formula 1 Grand Prix, Mercedesme hosted an exclusive event for members of the Australian and international media. Participants met two of Mercedes-Benz’s newest cars, with a twist.

Digital content was superimposed on vehicles so that a 3D engine emerged from the bonnet of the AMG GT R before exploding into component parts and retracting to show labels. On its rear spoiler a small digital version of the AMG GT R sat spinning on a virtual turntable, changing colour and size at a touch.

Participants wore Epson’s Moverio Augmented Reality (AR) headsets while inspecting the new Mercedes-Benz A-Class and AMG GT R, which enabled them to view interactive 3D virtual models, videos, and text. A viewer sitting in the A-Class could look around and see key new features highlighted with digital labels.

When visiting a store to look at a car, a washing machine, or even a box of cookies, prospective customers are no longer reliant on printed material. They can choose their own focus, they depth of information, and even their language, and they can interact with it.

 “Applications like virtual training of expensive and dangerous scenarios, or integrating back-end data to streamline operations, are rewarding problems to solve.” said Vivek Aiyer, CEO at Appearition, which created the AR software application, “However it’s consumer facing apps like this one that enable everybody to have a little creative fun, bringing cinematic AR closer to reality.”

Retailers are beginning to realise that the technology exists for anybody to use AR and VR through their own smartphones and tablets or dedicated high-end devices like the Moverio headsets. Products can be customised, and purchases can then be made with a salesperson or in an App.

Looking beyond current capabilities, as makers of headsets like Epson increase usability and bring down prices, we are likely to see people wearing them every day, even in the street, making the experience both hands-free and immersive.

“Success comes from collaboration. This taste of AR in retail celebrates Epson’s sponsorship of Mercedes in this year’s racing season. For an Australian-owned business like Appearition, it’s gratifying to be invited to work with respected global partners like Epson and Mercedes.

Collaboration comes from persistence. We initiated a partnership with Epson two years ago. The software was built using Unity, another partner with whom Appearition co-launched an Australian Centre of Excellence in 2018, focusing on Immersive Technologies for the Enterprise sector.” said Vivek.
[/av_textblock] [av_video src=’https://www.appearition.com/wp-content/uploads/2019/04/App-Epson-MercedesGo-Event-March-2019-1.mp4′ mobile_image=” attachment=” attachment_size=” format=’16-9′ width=’16’ height=’9′ conditional_play=” av_uid=’av-ju94y0xk’ custom_class=”]

Read More

Staff Blog: Mucking about with 360 VideoSphere VR (& boats) Part 4

[av_image src=’http://appearitionwebsite-1862015318.ap-southeast-2.elb.amazonaws.com/wp-content/uploads/2017/03/BannerVideSphereArticle4.jpg’ attachment=’4477′ attachment_size=’full’ align=’center’ styling=” hover=” link=” target=” caption=” font_size=” appearance=” overlay_opacity=’0.4′ overlay_color=’#000000′ overlay_text_color=’#ffffff’ animation=’no-animation’][/av_image] [av_one_full first min_height=” vertical_alignment=” space=” custom_margin=” margin=’0px’ padding=’0px’ border=” border_color=” radius=’0px’ background_color=” src=” background_position=’top left’ background_repeat=’no-repeat’ animation=”] [av_textblock size=” font_color=” color=”] This is the last in a series sharing lessons learnt producing a short VideoSphere (360 Virtual Reality) film for the Melbourne Fringe Festival.

In the last update, the team completed our shoot on Melbourne Regatta Day aboard the tall sailing ship Enterprize, capturing content from a 19th century costumed musician, to a fire-fighting boat shooting jets of water into the air.

We were about to discover the differences between editing regular footage and editing in a 360 VR environment.
[/av_textblock] [/av_one_full][av_heading tag=’h3′ padding=’10’ heading=’EDITING 360 FOOTAGE WILL HAVE YOU IN STITCHES’ color=” style=’blockquote modern-quote’ custom_font=” size=” subheading_active=” subheading_size=’15’ custom_class=”][/av_heading] [av_textblock size=” font_color=” color=”] VideoSphere (360 VR) film is captured on multiple cameras running at the same time. Unlike regular footage and even computer generated (CG) virtual environments, this footage from different cameras must be “stitched” together to create a sphere.

Stitching usually requires specialist software which in this case came with the Samsung Gear 360, so the camera owners (the festival) managed this. If using a different camera check this out before you buy. You may notice that live 360 video on Facebook and YouTube doesn’t seem to need it – some two-lens devices can stitch as they go.

The greater the number of lenses your device has, the higher the quality of the end-product. The periphery of a fisheye may be distorted, and there may be a noticeable difference in exposure if adjacent lenses face dramatically different light levels. E.g. direct sun versus full shade; see the images below.
[/av_textblock] [av_one_full first min_height=” vertical_alignment=” space=” custom_margin=” margin=’0px’ padding=’0px’ border=” border_color=” radius=’0px’ background_color=” src=” background_position=’top left’ background_repeat=’no-repeat’ animation=”] [av_image src=’http://appearitionwebsite-1862015318.ap-southeast-2.elb.amazonaws.com/wp-content/uploads/2017/03/Body_Image_1_Article_4-484×400.jpg’ attachment=’4474′ attachment_size=’portfolio’ align=’center’ styling=” hover=” link=” target=” caption=” font_size=” appearance=” overlay_opacity=’0.4′ overlay_color=’#000000′ overlay_text_color=’#ffffff’ animation=’no-animation’][/av_image] [av_textblock size=” font_color=” color=”]

Showing the difference in exposure between two fisheye cameras

[/av_textblock] [/av_one_full] [av_one_full first min_height=” vertical_alignment=” space=” custom_margin=” margin=’0px’ padding=’0px’ border=” border_color=” radius=’0px’ background_color=” src=” background_position=’top left’ background_repeat=’no-repeat’ animation=”] [av_hr class=’invisible’ height=’50’ shadow=’no-shadow’ position=’center’ custom_border=’av-border-thin’ custom_width=’50px’ custom_border_color=” custom_margin_top=’30px’ custom_margin_bottom=’30px’ icon_select=’yes’ custom_icon_color=” icon=’ue808′ font=’entypo-fontello’] [av_image src=’http://appearitionwebsite-1862015318.ap-southeast-2.elb.amazonaws.com/wp-content/uploads/2017/03/Body_Image_2_Article_4-845×684.jpg’ attachment=’4475′ attachment_size=’gallery’ align=’center’ styling=” hover=” link=” target=” caption=” font_size=” appearance=” overlay_opacity=’0.4′ overlay_color=’#000000′ overlay_text_color=’#ffffff’ animation=’no-animation’][/av_image] [av_textblock size=” font_color=” color=”]

Showing the difference in exposure, and overlap of alignment, between two fisheye cameras

[/av_textblock] [/av_one_full] [av_hr class=’invisible’ height=’50’ shadow=’no-shadow’ position=’center’ custom_border=’av-border-thin’ custom_width=’50px’ custom_border_color=” custom_margin_top=’30px’ custom_margin_bottom=’30px’ icon_select=’yes’ custom_icon_color=” icon=’ue808′ font=’entypo-fontello’] [av_textblock size=” font_color=” color=”] You’ll be dealing with some very large files – think Gigabytes. After exchanging DropBox files and physical USB sticks we had the stitched footage ready to edit.

I went through all our footage, and wrote a rough narrative sequence featuring the best shots and sound, as a starting point for Nadia and Andrew. We bought a one-month subscription to Adobe Premiere Pro (it is memory intensive so check the system requirements first to be sure your computer can handle it).
[/av_textblock] [av_heading tag=’h3′ padding=’10’ heading=’AUDIOSPHERE – DIRECTIONAL SOUND AS A NARRATIVE TOOL IN VR AND 360 VIDEO’ color=” style=’blockquote modern-quote’ custom_font=” size=” subheading_active=” subheading_size=’15’ custom_class=”][/av_heading] [av_textblock size=” font_color=” color=”] Whether it’s CG VR or VideoSphere, with immersive 360 you will need to think carefully about how you cut. In conventional video you can start the action at the same point on the screen that the last shot ended on, because you know exactly the field of view.

With 360 and VR, each user will have a different field of view when a shot transitions. They’ll be facing forward, backwards, even the ground beneath them. So, if you cut straight to action you cannot be sure they’ll see it.

360 AudioSphere sound (see the last two posts) can be used to call attention to the action. However, even without 360 sound, ambient noise such as music can be used to create continuity across shots.

Our narrative was a simple sequence designed to flow cohesively rather than a story, so the lack of directional sound was not a deal-breaker. Andrew and Nadia carried music from the ship’s band from one shot into the next to ease the viewer across cuts.
[/av_textblock] [av_heading tag=’h3′ padding=’10’ heading=’PLAIN SAILING IS PEACEFUL, BUT OBSTACLES ARE GREAT TEACHERS’ color=” style=’blockquote modern-quote’ custom_font=” size=” subheading_active=” subheading_size=’15’ custom_class=”][/av_heading] [av_textblock size=” font_color=” color=”] With luck the most painful part of our experience will benefit you.

Some great footage was captured by strapping the camera to a yard-arm high above the deck. The Bolte Bridge appears to slide over the Enterprize as she sails below. Passengers and crew can be seen from above and one climbs the rigging.

Because the camera had to be mounted sideways for this shot so that the thick yard-arm did not obscure the ship below it, the footage is also sideways. You might expect the Samsung Gear 360 to automatically orient the footage, the way that the screen on a Samsung mobile device does… you’d be wrong.

Adobe Premiere is a great tool, but VideoSphere and the concepts of editing in Virtual Reality worlds are still new. Frustratingly it could not rotate the footage. There is specialist software available but it required a full year’s subscription at around US$250. For a minute of footage on a non-profit endeavour this is as much overkill as shooting an albatross with a ship’s cannon.

Then the festival organisers offered to rotate it for us, problem solved
[/av_textblock] [av_heading tag=’h3′ padding=’10’ heading=’FOUL WEATHER BEFORE SUNSHINE’ color=” style=’blockquote modern-quote’ custom_font=” size=” subheading_active=” subheading_size=’15’ custom_class=”][/av_heading] [av_textblock size=” font_color=” color=”] Problems rarely turn up without bringing a friend. The festival was more work than expected for the organisers’ technical staff and rotating our footage was low on their list of priorities. As the start of the festival approached they advised they were unable to follow through on the offer. This was a dilly of a pickle because the delay meant it was now too late to reshoot the footage. We beat the social media bushes to flush out a solution

Marianna Makówka, the festival General Manager, was tenacious. She worked tirelessly to find a solution by asking around her network. Ed Bellamy from Staples VR, came to our rescue and rotated the footage, a generous gesture given this was on top of his paid workload.

After a few storms, we were safe in harbour. It was hard work but a great experience. VideoSphere is here to stay, so why don’t you give it a try?
[/av_textblock] [av_heading tag=’h3′ padding=’10’ heading=’USEFUL LINKS’ color=” style=’blockquote modern-quote’ custom_font=” size=” subheading_active=” subheading_size=’15’ custom_class=”][/av_heading] [av_textblock size=” font_color=” color=”]

  1. Part 1 of this Series: http://appearition.com/mucking-about-with-360-videosphere-vr/
  2. Our final 360 VideoSphere of the Tall Ship Enterprize: https://youtu.be/4ktGy2X8W3E
  3. For an immersive experience and ease of use try using a Google Cardboard headset and selecting this icon in YouTube:youtub

CONTACT US

[contact-form-7 404 "Not Found"] [/av_textblock]

Read More

Staff Blog: Mucking about with 360 VideoSphere VR Part 3

Interested in 360 VideoSphere (360 Virtual Reality)? This series shares what I learnt producing a short film aboard an old Sailing Ship for the Melbourne Fringe Festival[1].

In the last update, my team of volunteers were looking forward to shooting our first 360 VR short film. Then we hit a hurdle. Our cinematographer was unable to do any test shots or editing, due to incompatibility with his computer and the festival’s practice camera.

AWFUL AUDIO, OR AUDACIOUS?

The hurdle was followed by a face-first tumble into the mud when our sound engineer had an overseas job at the same time as the shoot. We could not reschedule; Melbourne Regatta Day aligned with our shooting window, and was too good to miss.

Plan B. We contacted other specialists, and they were keen to try 360 VideoSphere production, but there was not enough time to line up people and equipment. So we fell back to Plan C; use the Samsung Gear 360 camera’s inbuilt microphone. This would not be great if you’re recording a concert, or producing a narrative that requires directional sound to direct audience attention. Still, for our purposes we were pretty pleased with the quality.

body_image_1_article_3

NEW TECHNOLOGY? IF WISDOM FAILS TRY TENACITY

Conventional wisdom with emerging technology is test it early and iron out the inevitable problems. It’s wisdom for a reason.

Unfortunately, the festival’s production units were unavailable in advance. So, we could give up, or improvise and manage the risk. We decided to go for it, and as expected, encountered problems right away.

It is not possible to get behind a 360 camera and look through a viewfinder, but with the Samsung Gear you can use your phone as a remote viewer. A nifty feature, unless as in this case, Samsung block access to the app because you’re in a country where the Camera has yet to be officially released. The festival hadn’t identified this issue because like us, this was their inaugural spin on the 360 dancefloor.

Short of trying solutions like IP masking to make it look like we were in South Korea (where the camera was bought) we would have to shoot blind – so that’s what we did. For example, we climbed the mast and out onto a yardarm to attach the camera. Then, we recovered it after 10 minutes to physically connect it to my laptop and review the footage.

Sometimes even workarounds need a workaround. The case around the camera’s USB port was too small for our cable, and modifying the borrowed camera was out of the question. Our resident inventor, Andrew, borrowed the skipper’s knife and whittled away his own USB cable’s superfluous housing. I admit, I was sceptical but it fit neatly into the Camera’s port.

IN THE NEXT UPDATE

So after a long day shooting we had plenty of good footage.

However, 360 VideoSphere (360 VR) film is captured on multiple cameras. So, this composite footage must be “stitched” together before it can be edited. The results create some unique trials in the editing suit!

USEFUL LINKS

 

For an immersive experience and ease of use try using a Google Cardboard headset and selecting this icon in YouTube: google_cardboard_logo

 

CONTACT US

[contact-form-7 404 "Not Found"]

Read More

Staff Blog: Mucking about with 360 VideoSphere VR Part 2

All Goes Well, Until It Doesn’t

Interested in 360 VR (VideoSphere)? This series shares what I learnt producing one aboard an old Sailing Ship for the Melbourne Fringe Festival[1].

I was lucky enough to come across the opportunity at a VR (Virtual Reality) meetup to produce a VideoSphere short film, with the camera and expert guidance supplied[2]. I have produced traditional video, and was already intrigued after seeing two staff from ABC TV talk about their experience at an earlier Mixed Reality meetup[3].

When Amy Nelson and Astrid Scott explained how they produced the ABC’s first 360 production, I was struck by how accessible they made it[4]. They faced the challenge of placing their camera on a pole over an angry bull in the middle of a rodeo in outback Queensland. For other shots they had the camera operator hiding behind a barrel. Not because of the bull, but because hiding the crew behind the camera is not an option, when there is no “behind” the camera.

They were candid about accepting mistakes. They knew that many of the rules learned over the last century do not apply to this medium so new practices must be developed through experimentation.

So given this chance, my first thought was ‘brilliant!’ which is my reaction to anything related to Virtual and Augmented Reality. But I had no team, no 360 experience, and no time. So, it had everything going for it but common sense.

1830’s Cultural Heritage Meets 360 Video

The festival required a proposal. Cultural Heritage (i.e. history) kinda rocks and everybody loves old fashioned sailing ships, whatever their opinion of Johnny Depp’s (over)acting in Pirates of the Caribbean. However, not everybody can spend time aboard one, let alone to climb out onto a yard-arm high above the deck. I have been a volunteer aboard the Enterprize, an educational tourism ship, for the last few years, and this was a way to share the experience.

body_image_1_article_2

The first step was to create a storyboard (a sequence of shots, like panels in a comic book) and get the festival and the ship’s management on board.

The next step was to put together a team. For the cinematography I called on Andrew Gotts, an old friend who has worked in video production. He enjoys experimental technology, and has a good head for heights. He suggested an editor, Nadia Nusatea, so that made three. We still required an audio specialist so I approached Darius Kedros who runs a VR Audio special interest group[5].

There were a few short weeks to learn the technology, shoot and edit. But we now had a plan, a team, and something to film.

Cameras and Audio Equal Trouble and Strife

Then, bad news. The practice camera provided was incompatible with Andrew’s hardware, and we could not borrow the production camera until less than a day before the shoot. Our choices were to quit, or go in blind.

Worse news. Darius would be overseas for an extended period. Understandably he did not want to risk his very expensive audio capture equipment with somebody else; particularly when it would be suspended over salt water.

Next week

In next week’s edition: All At Sea But Problem Solving: We improvise to solve our production problems and capture our footage, but even editing 360 creates its own obstacles

Useful Links

Many of these are Melbourne (Australia) based, but you can find similar resources wherever you are.

  1. Part 1 of this Series: http://appearition.com/mucking-about-with-360-videosphere-vr/ 
  2. Nathan Beattie’s VR Developers’ Meetup: https://www.meetup.com/Melbourne-VR/
  3. Leah Bunny and Emily Harridge’s Real World VR Meetuphttps://www.meetup.com/Melbourne-VR-Virtual-Reality-Meetup/
  4. ABC TV’s first 360 VideoSphere production: http://mobile.abc.net.au/news/2015-12-03/vr:-the-life-of-a-bronc-rider/6966832
  5. Darius Kedros and Sally Kellaway’s VR Audio Group: http://www.meetup.com/Melbourne-AR-VR-Audio-Meetup/

 

CONTACT US

[contact-form-7 404 "Not Found"]

Read More

Staff Blog: Mucking about with 360 VideoSphere VR Part 1

I looked about Antarctica in all directions. From the snowy landscape rising towards the centre of the continent to the waves breaking against the beach; a beach made of large pebbles, clearly visible at my feet.

VideoSphere (360 video) is real world footage that enables you to look in all directions including up and down even on a regular computer. Use a VR (Virtual Reality) headset and it will fill your field of view and move with your head. This means you are no longer limited to a window predefined by somebody else. I’m a history geek so I used it for Cultural Heritage. However, if it is broadcast live, at your sister’s wedding or a music concert, you can be 1,000 miles away, but see exactly what you would see sitting in the audience.

My first experience was with its sibling, PhotoSphere (which are 360 still images). I was “standing” on the Antarctic beach, while physically located in my kitchen in Australia, thanks to Paul Pichugin[1].

screen-shot-2016-11-24-at-2-34-14-pm

360 Video in Practice 

If you have not come across VideoSphere it is only a matter of time. Mainstream news organisations, tourism and sporting bodies are already using it as more than a novelty. You can even produce it yourself for family and friends.

At the entry level, you can pick up a decent camera with respectable sound capture, such as the Samsung Gear 360[2], for under US$500 (plus software such as Adobe Premiere[3] and a computer with enough grunt to run it. If you’re a serious professional and have US$45,000 handy, you can pick up Nokia Ozo, a bargain down from its original US$60,000 price tag[4]

Another factor is sound. Capturing ambient noise is not difficult. If you would like to capture directional sound and edit layers of sound (e.g. voice, action, music, and ambient) to a high quality then you may need a sound technician with a directional sound rig, editing desk, and software.

Using VideoSphere Yourself 

If you’re interested in producing your own VideoSphere, having one done professionally, or are just curious about the technology, you can read this series. I’ll cover the lessons (and mistakes to avoid) that I learnt by producing this short film aboard an old-fashioned Sailing Ship[5] for the Melbourne Fringe Festival. You can watch the video below on Youtube with a Google Cardboard headset and selecting the google_cardboard_logo symbol. 

A quick aside on the furious debate – VR or not VR

You may encounter an argument that 360 should not be referred to as VR because you can look but not touch. I.e. you can’t interact the way you can with responsive computer generated (CG) content. It’s true, but it’s like arguing that a spider is not an insect. most people just don’t care. So, for the sake of simplicity I am bundling this into the Mixed Reality family.

Next week

Tune in next week for my next article: All Goes Well, Until It Doesn’t: Putting together a production in record time with technology so new it has not yet been released in the country has its risks

Useful Links
    1. Paul Pichugin’s Antarctica: http://immersiveimages.com.au/tours/antarctica/#s=pano10
    2. Samsung Gear 360 Camera: http://www.samsung.com/global/galaxy/gear-360/
    3. Adobe Premiere video editing software: https://helpx.adobe.com/premiere-pro/system-requirements.html
    4. Nokia Ozo 360 Camera: https://ozo.nokia.com/# 
    5. Melbourne’s Tall Ship Enterprize: http://www.enterprize.org.au/

 

CONTACT US

[contact-form-7 404 "Not Found"]

 

Read More