Enabling Physical Analytics in Retail Stores Using Smart Glasses

ACM Mobicom 2014 |

Published by ACM - Association for Computing Machinery

We consider the problem of tracking physical browsing by users in indoor spaces such as retail stores. Analogous to online browsing, where users choose to go to certain webpages, dwell on a subset of pages of interest to them, and click on links of interest while ignoring others, we can draw parallels in the physical setting, where a user might “walk” purposefully to a section of interest, “dwell” there for a while, “gaze” at specific items, and “reach out” for the ones that they wish to examine more closely and possibly purchase.

As our first contribution, we design techniques to track each of these elements of physical browsing using a combination of a first-person vision enabled by smart glasses, and inertial sensing using both the glasses and a smartphone. We address key challenges, including energy efficiency by using the less expensive inertial sensors. Second, during gazing, we present a method for identifying the item(s) within view that the user is likely to focus on based on measuring the orientation of the user’s head.

Finally, unlike in the online context, where every webpage is just a click away, proximity is important in the physical browsing setting. To enable the tracking of nearby items, even if outside the field of view, we use data gathered from smart-glasses-enabled users to infer the product layout using a novel technique called AutoLayout. Further, we show how such inferences made from a small population of smart-glasses-enabled users could aid in tracking the physical browsing by the many smartphone-only users.

ThirdEye: Tracking User’s Physical Browsing in Indoor Spaces

We have built a system that would enable the tracking of physical browsing by users in indoor spaces such as retail stores. In a retail store, tracking the physical aspects of a shopper’s behaviour like walking, dwelling, gazing, reaching out, etc. might give us deep insights into the shopper’s interests. Using a combination of a first-person vision enabled by smart glasses, and inertial sensing using both the glasses and a smartphone, we have designed techniques to track each of these physical aspects of the shopper’s behaviour. Apart from retail stores, our system can help track physical browsing in other indoor spaces say during trade shows, conferences etc.