Posts

Using Machine Learning to leverage the power of Augmented Reality

Augmented Reality (AR) in its current definition is the overlay of digital information on a real-world view. In practical terms, it is the process of recognising specific physical objects in a device’s camera and superimposing digital content such as video, audio or 3D models.

Visual recognition is one aspect of AR which encompasses image, object, scene and facial recognition. Computer vision technology is used to identify shapes and patterns through a complicated set of mathematical models. These models and processes are all facets of Machine Learning (ML) that drive Artificial Intelligence (AI).

ML is the science of “teaching” the system to look for commonalities and patterns and assessing the probability that a match is found. Effectively, with a set of mathematical models in place, the system is fed a collection of information that represents a positive match. For instance, if we want to teach the system to identify a cat, we provide thousands of images of cats and let the system process and find common visual patterns across all the images.

This is known as deep learning where the outcome is a system that can recognise and track almost any pattern. With this capability, we can inject a virtual projection into the area that is being recognised and tracked to deliver, what is called, an augmented reality experience.

The power of AI and ML is being able to make decisions based on the real-world scenario. Let’s consider its application in a security surveillance system. A machine that has been trained to detect weapons, such as knives and guns, can be used to observe CCTV security vision. In real-time, it can look for patterns in the scene that resemble its definition of a weapon. If identified, a notification alarm could be raised for someone to act.

Pattern recognition is not limited to visual only. Auditory, gesture and other data patterns can also be “taught” using ML. Continuing with our security surveillance example, a trained machine could be used to listen to sounds in the environment and detect patterns of shouting or offensive language being used.

The challenges

One of the hurdles in training a machine to identify patterns is sourcing enough material that is deemed a “positive match”. In these cases, systems are designed with feedback loops to allow machines to “learn by experience”. If for some reason the machine fails to detect what it is supposed to, it can be taught what was missing in the initial dataset and be trained to act on it the next time it occurs. All this is supported by an aspect of ML called “convolutional neural networks”. Different nodes that perform specific mathematical functions on the dataset are interconnected to achieve the specified outcome.

The opportunities

In a time when vast amounts of information is available at our fingertips, being able to recognise the world around us and decipher what is relevant will become critical. Whether at work, at home or in a social setting, successful real-world augmentation will rely on AI and ML observing and recognising our environment and adapting information to match our situation.

As hardware technology improves and wearable, handsfree devices become a reality, ML and AR will become an integral, yet ambient part of our lives.

Simon Galanakis is a passionate advocate of effective AR experiences and is currently Appearition’s Platform Architect and Senior Solution Designer.

Frontier technologies – Future beckons

Mouli Ganguly, Member, Board of Advisors, talks on Industry 4.0, Digital Transformation, Frontier Technologies and more:

  1. What are some technologies that you think will play a huge role in Digital Transformation and why?

All of them play a role in terms of Digital Transformation. So, let us take an industrial example. Dump trucks used in mining have huge tyres. Their worth is about a million odd dollars and some of their tyres cost up to $100k. Now it is important to see the level of productivity an application can deliver. The benefits are already clear & compelling.

Every modern piece of truck has got transponders all over the place. That kicks in the IoT element. This enables us to track what it is doing by tracking the signals received. With AI and insights, what we can do with the data coming through is, collect and put them in a place. We can decide here is what I want to get the data or do something about it.

With AR, VR and MR, you render the whole application to how the user can best use it, with data and insights. For example, you could present the same information that you’re collecting via transponders to the truck drivers in a manner where he can see the whole environment around him. You can set it up, such that language is not a barrier. In India, for instance, truck drivers driving cross-country speak different languages. Irrespective of whatever language one speaks, they can all see an image, interpret & take decisions. Every driver knows what a boulder is. Or what are the factors that may damage the dumpster or damage the tyres or something else. Whilst sitting inside the dumpster, they may not be able to view everything through naked eye.

“Seeing the information presented through Virtual or Augmented reality enables him to make better decisions.”

Another way to grasp all this up – you see a control room can observe the movements of a dumpster. They receive the information from transponders and visualise the data and interpret the implications that arise from it. They can then come up with solutions like specific risk issues or actions that need to be taken. A team from the customer & Appearition has to plan this out.

In terms of Blockchain, it makes sure that a certain event, transaction or data, is stamped as authentic. It is not widely used yet. There are restrictions on its usage or application. But for transactions of high value, this technology not only gives you an experience but also makes the transaction very safe and auditable. It also facilitates to build a record as to who did what at exactly when. This can also act as an audit trail for any purpose required. It is a foul-proof way to keep track of things. The transaction can be financial or otherwise.

So that is how all the technologies can play a role in Digital Transformation. Often a combination of these technologies that answers / solves a business problem.

  1. What will be the road-map of early adapters vs cautious players?

Typically, early adapters are an organisation’s visionary. They are in the very senior position. Also, in a generational sense, say between gen X, gen Y or the middle aged people, there are several technology gaps. Kids born after 2000 basically grew up with screens. Just like millennials grew up with the PCs. Every time a new technology is introduced, you think of things in a very different way. For example, in my generation of baby boomers, , people would write text for a power point presentation. Whereas, these days, a presentation is far more experiential.

In an organisational context, the gap is often at thinking through the process change or adaption for an idea.  If the plan is not backed by experience or relevant subject matter expertise, it will fall through.

As an early adapter, the visionaries come up with ideas. The cautious players are the ones who go through the practical things. So, they go through factors like business use case, defining what will change, how will it work, the change management and integration part of it, how to get data out of organisational system that exists, make sure that the data is accurate, quality is good.

Early adapters are analytical thinkers/dreamers. Cautious people on the other hand, achieve dreamers’ dreams. This kcan take a very long time. Now, where an organisation is driven by a visionary, often, you get things moving simply because he has got an idea that he wants to make it happen. And he (CEO/COO) will tell somebody typically that, ‘look I just want to make it happen. Work as a team and make it happen’.

Facebook phenomenon

For an early adapter enterprise, if you get the vision right, you get a first mover advantage over others. For instance, the Facebook phenomenon. They have created an entirely new business. Whereas sometimes, early adapters who did not plan well have gone bust. Because, they have placed some bets that have gone awfully wrong. And this has sort of impacted their existing business, customers and whole the lot. Cautious players tend to be thoughtful followers. They let others experiment and use those learnings in their business. They try to keep their profitability high and ensure quality, risk management & repeatability, so thatcustomer engagement does not take a hit.

  1. In this process, how can an enterprise build its internal and external stakeholders’ digital IQ?

There are whole lot of methodologies. In my view really simple. The agenda very much is a COO/CFO/CIO/CMO/CPO idea. The CEO says this is what we need to do. The CMO then says, this is how we want to market it. This is the experience we want to provide. The CPO Says here is the culture or context in which I want my people to operate. Those are the three elements – the customers, the culture, the strategic direction and somewhere in between there is an elephant in the room, the digital space. Compliance, security, all those kinds of things. All this has to hang together be funded effectively and provide return to the investors, That is why you need a special type of CFO for the digital businesses. That is very important in terms of creating the digital IQ.

You drive the innovation, which is the business agenda. Secondly, you need to be agile. Agility is where you can own up to your mistakes quickly when needed and at a low cost.  The whole team, not just some champions  must work together on an agile manner. We need collaboration within the organisation and with the suppliers & customers. Within the compliance ecosystem the regulatory authority, and any number of agencies are there, because the laws haven’t caught up in Digital yet. And the next comes the partners. Because no one organisation can do everything by themselves. It is very, very important for the partners to work together. Again, look at what Appearition is doing. They work with a whole lot of partners to deliver what must be delivered. And, at another level, the IQ is built through great culture of innovation and risk taking.

The first question you ask is why are you doing it? What is the innovation? Why am I doing it? What difference will it make? And the ability to execute, in my mind is the other side of the equation. The CIO, the guy whom implements the technology. The CFO, the guy who checks out the money, burning millions of dollars, putting things at risk. Those kinds of functions become responsible of every action. So, the whole thing is a question of balance.

The ability to execute the vision is possibly the biggest challenge of the Exec team.

  1. What will the role of data analytics be in this revolution?

If you take the first thing that comes out in the digital world, it is data. There is so much data in every aspect of our life, every moment. We are being absolutely bombarded with data. Our memory spans have reduced. In today’s mindset, I don’t think people have the patience to read a booklet. This data is compressed and made consumable through AI and Analytics. This is turned into information. But then, to turn it into information, you need to know, what information you were after. You need to take the data in a certain way. From information, comes knowledge. Then you personalise it. Then you look at the information and say, I’ve read the report, what I’ve learnt out of the report is this, this and that. And then, beyond that comes very, very content specific, decisions.

For instance, if you are an insurance company and you want to know the information on who are the people who do not have an insurance and are most likely to become my customers? And if there are certain attributes that you identify, you may decide whether it is important or not and do something with it.

2/2 of his exclusive interview. Click here to read the first part.

Industry 4.0 – Time to reassess business plans

Mouli Ganguly, Member, Board of Advisors, talks on Industry 4.0, Digital Transformation and more:

  1. How has market reacted to digital transformation so far? What are some factors that seem to aid it/ factors that act as a barrier?

Introduction of computers in the 80s and internet in the 90s were digital transformations (DT). These transformations speed up an existing process in their own way.

Currently, with Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), Artificial Intelligence (AI), Internet of Things (IoT), Big Data etc, we are heading to the third wave in digital change. With this change, content reach has become faster and more specific to the user’s individual & specific context.

There is a question of what is the hype and what is the reality? The hype is often an expectation that Everything will turn digital from day one. But the reality is it takes a long, long time.

Digital disruption is when the whole user experience changes. When you look at the current technologies like AR-VR, you are in an immersive environment. The second thing that changes in a business is convenience. It gets faster and better. Take for instance YouTube, people are not wholly dependent on it for video services. We have now got other video streaming options. From being costly and time-consuming to produce a video, now they’ve become so simple.  Within a span of five years, existing technologies have changed so much.

In the residential space, AR-VR is already there. If you look at smartphones or movies, almost every movie has a VR effect in it. Whereas in the enterprise phase, it is happening far more slowly due to change management issues. There are a whole lot of processes, policies, rules that play out. These multi-factor influences act as barriers, at times.

Take a look at this scenario:

On a personal level, if you see a 3-D glass that you fancy, you just go ahead and buy it and try it and see. But in the enterprise phase, we cannot change what we do on a day-to-day basis, immediately. It is a gradual process to adopt changes and involves formal change management. Enterprises don’t have individual decisions like personal buyer choices. A new technology adoption must act in a certain business standard. In an enterprise space, one person cannot just act on it or start using something. There are other people involved. This is a huge barrier in adopting or initiating digital transformation.

  1. If you were to chart a blueprint for an enterprise’s digital transformation, what actions would it typically involve?

Adopting new technologies to businesses happen if there is a clear benefit. Unlike personal choices, a business cannot adopt a technology or a process without a defined benefit. There should be a strategy or a growth plan that cites the benefits.

Benefits can be divided into three: Either the customer or staff experience the product or service delivers is much superior leading to increased loyalty and willingness to adopt from them. Since organisations measure customer satisfaction, this is an inclusive environment.

Next one on the list is Product or process improvement. Take for instance claims processing in an Insurance company. It is quite difficult organising in person, especially if the person has availed trauma insurance. In a digital world, you can have a number of different tools from virtual assistance to people working in collaborative environments to remote diagnostics with telemedicine, all of these improves the overall process.

Enterprises must include the market & customer in its product development. It is easier to create a digital product. One can quickly create a product and launch it and test it as a POC (proof of concept), before scaling up or canning it all together.  Therefore, the whole product cycle in the market is faster than before. Through failure, you learn, Fail fast and fail without incurring a high cost or brand damage and you will innovate with successful offerings.

Finally comes the cost reduction factor. Basically, several things drive digital adoption. You grow the experience, generate customers, grow staff loyalty and therefore get more market opportunities. You get market opportunities to get new products by reducing the cycle time to market, your process has changed, you change the format quickly. And as your cost reduces, you become more productive. This sort of drives some of the opportunity.

Now, what is the big risk that you face while doing so?

Benefits, use cases & training/change management needs are not clear. For instance, when we say I’ll improve the claims processing experience in an insurance company, one needs to clearly articulate what is the use case. What is the customer roadmap of the journey? How the customer uses the new digitised transformational services that you provide. Then one has to map it all the way. At each point, you have to state what is different. How or where will I be delivering this service and how will the business be using it. It is not a technology or process issue. Business must very clearly define the benefits they will get.

Now, how do you do it?

Firstly, create a simple proof of concept. And these concepts have been around for a long time, but very, very quickly you spin something up like Appearition is doing at the moment, we can create an experience for the customer to show them the actual benefits very quickly.

The next thing that comes is change management. You have to say, if we are going to do things differently, digital is to be applied on a day to day basis, how do you make sure that when either our customers or suppliers or staff use it, how’d they be trained, accept it and use it. And most importantly, everybody in the eco-system, i.e. the staff, customers, suppliers should be able to deal with the change. This change management leads to a fully functional prototype. Proof of concepts at a very low price, say 30-50k dollars, roughly, you show them how it is done, from there we move to a fully functional prototype. From the prototype, we get a use case development.

In our design, we use tools like machine learning, artificial intelligence, data from the Internet of things and analytics to continually improve the experience.

The last one, which must be designed in terms of the blueprint, is integration. Every organisation has a whole lot of legacies. Because nobody is working in a vacuum. The new systems have to interwork with the legacy in most cases. This becomes a difficult, costly & cumbersome process.

3. What are some of the opportunities and risks that enterprises will face whilst adopting it?

Whatever digital services one provides, one must draw the data from those existing legacies. One of the elements to add to the risk is realisation. Because, sometimes, some firms do not have a clear strategy on how they want to go digital. They just dump and start an initiative without a strategy. So, if you get an application up and running, it will look very nice and appealing to whoever designed it and they will think this will change the world. But if there is no change management to support it, the customers, the staff are not wedded to it. It hasn’t been well thought through.

 

Follow this space for Part-2 of this interview.

Augmented Reality – The future of retail industry

Home delivery, telephone order and e-commerce. What do these things have in common? They changed the way we shopped. Introduction of these technology/ideas, enhanced our shopping process. Similarly, Augmented Reality (AR) is set to alter the face of retail industry.

AR is highly scalable and has multiple uses across this industry. A look at some interesting ways in which AR is expected to grow:

Trial/testing of a product:

AR helps to bridge the gap between a customer’s perception of the product and the reality. This is the quintessential need for physical shopping or trial for a product.

Furniture: In 2013, IKEA – the Swedish retailer tested the ropes of AR, by launching an AR-based app. It used AR-overlay of 3D models of IKEA’s products on the real-time feed of the camera. It helps visualise how a product would look at a given space.

Shoes: Sneaker-maker Converse created an AR app in 2010. When one points a camera at their feet, the app overlays a projection of the shoe on their feet.

Clothing: Ever stood in a never-ending queue ahead of the trial room only to find the dress doesn’t fit you? Well, AR is about to change the game in clothing trials using overlay.

Japanese retail store – Uniqlo tried to address this concern by enabling an augmented trial room. This room had a mirror with an LCD screen that let you choose the apparel you wish to try. The app then overlays different colors of the clothing to help you make the best choice possible.

Virtual makeup trial: Sephora, the makeup retailer launched an app – ‘Sephora Virtual Artist’ that enabled overlaying different makeup looks. Once the customer scans the face using a camera, the app detects the different organs like nose, lips, eyebrows, skin etc and allows you to try a shade.

 

Product information:

When shopping at a busy super-market, we are often sceptical of a product’s ingredients. The aisles are long and nearest assistant is pre-occupied. What if one had a personal assistant to share further information on every product? AR does just that, acting as your personal shopping assistant. Several retail big-wigs like Walmart, Tesco, Carrefour have been testing and experimenting with AR in shopping.

Chinese e-commerce retailer – Yihaodian, opened virtual stores across the country. This enabled consumers to shop on the go. The app has virtual shopping aisles; customers can select and arrange for a home delivery.

AR is already booming in several such retail verticals. Add the magic of AR to your business. To know more, drop us a message.

This blog is a re-written excerpt from an article first published in the Augment.com

Image Courtesy (in the order of images used): 1, 2, 3, 4

Intern series: How to land an internship in a tech company

Vivienne Zhu is a Commerce & Law student at Monash University currently interning with Appearition’s head office in Melbourne.

How’d I get land my first internship? By telling Tushar about my lonely yet, solution oriented solo hike on the snow covered Bukhansan Mountain in Korea. This is only half true. My passion for marketing, previous work in a NFP and willingness to step out of my comfort zone shone through against many other candidates.

Landing an internship in a tech company without tech knowledge?

As someone who has never learnt about IT, augmented reality or coding, it’s quite different and interesting to work in this environment. Often others in the office use a lot of technical terms that I don’t necessarily understand, but I’m always free to ask (or Google in my spare time). However, the technical jargon that comes along in a tech space naturally becomes more comprehensible.

As a marketing intern, my first few weeks consisted primarily of competitive analysis and events research. However, I was soon trusted to start my own social media campaigns on Twitter, Facebook, and LinkedIn. I constantly put my hand up to take on opportunities within my capabilities – e.g. filming and editing a video for a client. This allowed my peers to see that I was capable of more than what they initially thought.

screen-shot-2016-11-24-at-3-27-54-pm-1

In my first three months, I’ve had opportunities to work on new projects and learn more practical marketing skills (social media analysis, Google Adwords and driving B2B Marketing). Whilst I am not majoring or studying marketing, I am continuously developing my skills in this digital age of marketing. Creating content and following trends is very important. Whilst I’ve learnt so much about marketing, I’ve also learnt about the intricacies of augmented reality.

What is augmented reality?

Before this internship. I had no idea what augmented reality was. Prior to my interview, I was furiously researching about the industry. AR is still quite new and my only touch point was my Snapchat obsession (sorry to disappoint but I never got into the Pokemon Go fad). I’ve experimented with Appearition’s apps, and now understand the solutions they provide for businesses. AR goes far beyond consumer capabilities, but is likely to have a great impact in the workplace in the future.rgea

I look forward to the coming weeks and potential projects I’ll be involved in, and I know this is just the beginning for my personal augmented reality journey.

By the way, I wasn’t completely alone on the mountains. I strategically followed three senior Korean hikers who took me on the hike of a lifetime. At the end of the day, it’s all about strategy, passion and drive.

If you’d like to follow my work – like our Twitter, Facebook, and LinkedIn pages!

 

CONTACT US