I Robot, you shop

The future shopping experience is almost here.  Microsoft and SoftBank Robotics have announced a strategic collaboration in the field of cloud robotics. This will see the “Pepper”, SoftBank Robotics’ humanoid robot cloud enabled and delivering an experience that consumers  will never forget.

The collaboration has been divided into two stages.  The first stage will see Microsoft and Softbank cloud enable “Pepper” linking it to the Azure cloud-based IoT suite. The second will be the creation of a next generation in-store solution that will see Pepper combined with the Microsoft Surface Hub large screen collaboration device and Surface 2 in 1 devices delivering a retail experience to customers.

he field testing will begin in late March, between 24th and 30th at Softbank Mobiles “Pepper-Packed Mobile Shop” (Minato-ku, Tokyo) with the intention that customers will only be served by the robots. The aim is to be ready to go to market with a product by Autumn 2016 in Japan

Should we be concerned about the rise of the robot

When Pepper was developed by Aldebaran Robotics and Softbank Mobile its purpose was to read people’s emotions from expressions and voice tone. It was introduced to the world in June 2014 and has been showcased at Softbank mobile phone stores in Japan since then. It finally went on sale for 198,000 yen in June 2015. To date Pepper has greeted and amused customers in the Softbank mobile stores, however this announcement sees a huge leap forward in terms of human robot interaction. This is another step on the interaction between human and robot and there will be concerns.

While the world is not about to be invaded by self reproducing Pepper robots, nor are they being imbued with artificial intelligence, they may potentially taking jobs out of the service industry especially for younger people.  This is not going to happen in the short term and there is still a long way to go before robots replace humans in many stores.  Where information is critical to a sale though, such as mobile phones, it may indeed find a home. It will be interesting to see what other applications Microsoft turns to next.

Future retail

This is not just about the robot however. In the collaboration Microsoft will be collating large volumes of data in store. This can include customer behaviour and point-of sale purchases that can be integrated with similar data from online smartphones and other e-commerce sites. What is interesting is that as Pepper can identify emotions displayed by people it will not just be collecting data about what people ultimately buy but also their opinions about other devices, colours and equipment.

For example if shown a newly launched tablet in two separate colours, it may be possible to determine what colour the buyer prefers, even if they choose not to purchase the device.  This data is invaluable and as it is tracking human emotions it is far deeper than any online data collection will have gone before.

Pepper, and potentially the Azure IoT platform behind it will be able to tailor suggestions to specific customers, and although Microsoft currently states this will be in line with previous buying behaviour it could easily be extrapolated to include buyer impressions for non purchase. Suggestions will be made to shoppers via the surface Hub or the terminals which each of the Pepper robots carries. This takes machine learning to a new level and as the systems become more complex and accurate it will be interesting to see just how farthe process can be taken.

With access to the inventory platform as well, Pepper will be able to make knowledge known such as reducing inventory.  The statement “This is the last one we have available”, suddenly assumes an honesty that might not have been previously possible and without having to phone up one can assume that in the future pepper will be able to state when the restocking will occur with some accuracy.

In addition to having customer service skills, Pepper will also facilitate efficient inventory management by leveraging real-time feedback from stores. Customers visiting the stores will be able to use Surface Hub or Surface to browse products visually, and then easily make purchases based on Pepper’s recommendations for them.

Microsoft will also deploy Cortana, though it is unclear whether this will be within Pepper or within the tablets that Pepper carries, the advantage of this is that Microsoft translation based services, including multi lingual support will enable Pepper to serve customers from different nationalities. It will be interesting to see how the trial goes and how this story develops.

Pepper not the only machine learning tool to use emotions

At InterConnect 2016 in Las Vegas, IBM announced a set of emotion APIs for Watson. These are available to developers via IBM’s Bluemix Platform as a Service and the company is already showing applications where they can be used. While Microsoft may have taken a lead through this collaboration with SoftBank Robotics, IBM is determined not to be far behind.

From an IoT perspective this opens up a lot of challenges for regulators. Is it acceptable to have devices reading human emotions and feeding that data back to sales people? Will we see the technology deployed in police stations and courthouses? How long before a court decides that the jury can be shown evidence from a computer that a defendant or even a witness may be not telling the truth?

For watchers of Sky TV series Lie To Me, this could have some significant consequences. At present the technology is often superseded by human oversight. Once embedded into technology that oversight goes away and while this is just a sales channel at the moment, how far will the public be prepared to accept it?

Conclusion

Would you want be served by a Robot, while there are certain stores the answer is a definitive “NO”, there are some stores where it could be an advantage. In technology stores especially having a shop assistant that you can talk to in a language that is understood, and delivering the correct information back about the capabilities of a device would be refreshing. Too often shop assistants knowledge only goes so far and the ability to bring the full array of technical specifications could be important. However Pepper is a long way from being able to serve and offer advice in a clothes store.