How Emerging Technology Is Evolving The Way Field Sales Merchandising Delivers Value Today And.....What’s Next?

by Scott Lester - July 21, 2022

Have you heard the story where the field sales merchandiser walks into a retail store to do his visit, has to manually take a ‘proof of visit’ picture and upload it to his HQ database, then has to ask the store owner to ‘imagine’ how a free standing display unit (FSDU) will look in his store (including the dimensions, design and store position), then heads to the shelf to take more pictures to upload to HQ, then gets out a pen and paper to write down current stock and re-orders required, whilst noting down areas that could be improved to achieve a ‘perfect store/shelf’ status….

This isn't a field sales rep visitation efficiency horror story from 1996, this is a real example of the findings of a recently commissioned investigation and analysis that eyekandy conducted based on; ‘How field sales staff conduct visits, what are the workflows of a visit, what data is collected and how can emerging technology optimise the process’.

Ensuring that the ‘perfect shelf’ is in place, all the time, and that any POS is as impactful as possible, is the name of the game in this business.

eyekandy works with brands, retailers and merchandisers to deploy AR, VR and Video solutions to optimise these workflows via our proprietary Field Sales Platform (FSP). As we followed our assigned field sales representatives around for a few days, it became clear that it’s a supply chain workflow ripe for a digitisation overhaul to drive profits, efficiency and ROI of the human effort.

My last company was acquired by one of the largest retail merchandising companies in the United States, Advantage Solutions. Their business has successfully grown over the last 30 years by placing ‘in-aisle brand advisors’ into thousands of stores, such as Walmart and Costco, to assist the brand with product sampling, surveys, coupon distribution, POS display design, plano-graming store layouts (POG) and with shelf compliance (making sure the shelves are stocked and stacked to promote the brand) .

In addition to these services, merchandisers such as Acosta and Exsellence, deliver traditional retailer support services including; optimising sales displays, assisting with product rotation and placement and ensuring accurate voids and out-of-stock corrections.

Whilst, I’ll focus on the ‘what’s next’ part of this article, it’s worth providing an update on technologies that we are deploying now.

1 Predetermined location geo-fencing

There was one thing that all the field merchandisers had in common, they all had a smartphone with them. This is the turnkey moment that enables everything that follows to happen and opens up the industry to digitisation, so let’s start with the manual documentation of store visits.

By way of example, typically a field sales merchandiser is asked for a ‘proof of visit’ this is usually a manually taken photo of the store, on the day. Using the agent's smartphone in a more efficient and cost effective way can use GPS to verify the representative's location automatically as they begin the process of filling in the forms with a geo-location triggered ‘check-in’ function that is automated and secure. A geo-fence is pre-populated around each store and every time a representative enters the ‘fence’, a visit is recorded.

In addition, in many cases there are repetitive questions where the answer would seldom change, such as; how many fridges in store, length of shelves, number of shelves, how many shelves on entry, how many shelves on exit? This information about the store could be held in the database. After the first visit to a new store the data would be stored. Subsequent visits would load this information and the representative would only be asked to verify it, rather than manually entering it each time. This data can all be pulled automatically once a rep enters the ‘predetermined location geo-fence’.

Illustration of Google Maps

2 Free Standing Display Unit Sales

In addition to the sale of products, representatives have the opportunity to sell in FSDU units. These units require time to assemble and the practicalities of doing this for each store visit, mean they are unlikely to be sold. Store owners are unlikely to want to take the time for this to happen and representatives are unlikely to assemble them on each store visit. Without the store owner being able to see a fully assembled FSDU they will be unable to judge the space required in their store to accommodate it.

This is already being addressed with the production of AR models of the FSDU units. Including these models into the smartphone process will enable the representatives to clearly show the store owners the FSDU in context in their store. Enabling them to see quickly the space required and make a decision without the need for the representative to fully assemble the unit.

The eyekandy Field Sales Platform (FSP) improves the time and effort challenge of getting approval for design, location approval and dimension approval of instore Free Standing Display Units (FSDU’s). The eyekandy team deployed both Avatar and Static FSDU’s to help streamline this process. See example below;

“When the field merchandisers came in and showed this new technology, I couldn't believe it, when you demonstrated it in store and showed what my fixture could look like, I was blown away. After you completed your work with the FSDU’s and moved my biscuits around, my sales have increased. Such an innovating piece of technology, no other reps that visit have shown me anything close. My biscuit fixture looks amazing.”Harvey Kapoor of Nisa Blackwater

3 The Perfect Shelf

As part of the collaboration eyekandy also produced a best in class ‘perfect shelf’ AR model, this experience was developed to allow the in field staff to visually showcase to each retailer the design, stock and layout that provides the highest increase in revenue and sales.

Interactive elements were overlaid onto the perfect shelf allowing the user to tap and explore key facts about each product line whilst also providing key information on sales figures and average, expected revenue.

“We saw a step change in our retailer engagement with the retailers stepping from behind their till to discuss the category with our team at the fixture. The AR prompted them to be more inquisitive on how to do more in their store. Since launching the AR we have seen over 10,000 interactions take place in the first few weeks and the team activating more merchandising solutions where previously the retailer would not have been engaged.”Huw Davies MD of Exsellence

4 Live Video Platform

Finally, if a picture is worth a thousand words, what’s a video worth? Building an ‘always on’ live video access through the eyekandy FSP for the store owners is proving very useful. There’s always the question that they wished they’d asked when the field merchandiser was on the visit, there is clarification and requests on products and product placement and the ability for the store owner to show, via video, what he’s seeing and thinking in the store, is a game changer. In addition, the eternal frustration caused to any field sales rep with the dreaded, ‘buyer not available’ status when they arrive in store is resolved with the live video utility, which enables the buyer to catch the rep (face to face), and vice-versa, with a much more flexible schedule of access. The eyekandy FSP sets each field sales reps phone specifically to be available, or unavailable, to match their work hours and timezone meaning there are no unwelcome calls out of hours.

Flipz video demo

So, if that’s the now…the future starts here..

1 Store Shelf Scanning and Archiving

Use of Lidar shelf scanning technology that revolutionises the collection and analysis of real time on shelf data. In addition barcode scanning offers some innovative opportunities to reduce human effort/cost.. The latest IOS devices have a Lidar (Light Detection and Ranging) sensor included on the higher end models. This sensor along with the high quality camera is capable of generating full 3D models of the interior of a store or specific shelf layout and location within a store. The rep could essentially ‘scan’ the interior of each store and save a true to scale 3D model of that store to the pladis database (visible by all leadership and staff if required).


By collecting this 3D data periodically you could reference the models to see how the store has changed over time. With accurately scaled interior models of a store, new FSDU units could be trial placed throughout the store digitally or bespoke layout recommendations could be proposed and trialled for each location digitally. This would allow representatives to assess their size and fit prior to a store visit. This data could also prove very valuable as a real time up to date d model of each retailer can be referenced to gain averaging data points about how products are presented across each region.

2 Multi-Image Product Recognition

Mobile device cameras now have an incredibly hi-resolution camera installed in them. Using these cameras we can scan an entire shelf then using machine learning and AI, identify what products are present on the shelves.

We are probably six months away from releasing this machine learning technology. Although the latest phones can handle the computing power needed to process the images, the machine learning AI for image recognition is not quite there yet.. Training the platform to recognize all the products in their various varieties, orientations and under different lighting conditions is a tough one for us to crack, we need a few more months on this one..

The goal is to improve the accuracy of the data and speed compared to the current process of a field representative manually scanning the shelf. Real time data points could also then be overlaid over each recognised product showing key information to the rep and store owner (controlled centrally at platform level).

3 Digital Measurement

IOS devices with a LIDAR sensor are capable of measuring distances with an accuracy of 1⁄2 inch today. As part of our research, field merchandisers are asked to fill in the length and number of shelves in the stores.

Changing the measuring system to a digital scan, using the LIDAR sensor, would enable the build up of a digital footprint of the shelves in the store and have reliable input data all contained within one centrally managed platform. This data would be far more accurate and contain the ability to compare shelf positioning in a store, over time.

Virtual measurement

4 Product Recognition

Using a mobile device, the field merchandiser scans products on the store shelves and finds up to date information, in real time.  This is achieved through the developments of real time image recognition, AI and machine learning. This information could be comparisons to sales in nearby stores, or latest offers on the product. It could be used to inform store owners as to what products are selling well in the surrounding area and suggest relocating certain products to a more visible location.

Product scanner

5 Barcode Lookup

In addition to image recognition, barcode scanning on mobile devices is becoming increasingly popular in the market. Mobile phone cameras can easily scan product barcodes, which reference the eyekandy FSP, to lookup and display content. The same system is also being utilised to scan products and display additional information that the product labelling doesn't show or is hard to find. This includes items like ingredients that people may be allergic to, or if the product is Gluten Free or Vegan.

This same technology can be adapted to help the store owner to answer any customer questions, or for a Field Merchandiser to answer store owners questions surrounding a product.

It could also be used as an alternative solution to the Product Recognition option above. Using a barcode instead of image recognition would mean products that go through a rebrand or redesign would not need to go through the image recognition AI training, as the barcode design printed on the packaging would still remain the same.

Additional value add cases can be obtained in the near future by integrating a barcode scan function into the reps headsets, ordering, additional info, offers, revenue stats can all be presented to the Rep once the barcode has been identified by the platform.

6 Product Location Identification

Utilising the LIDAR sensor on the latest IOS devices in conjunction with Hi-Res cameras, products within the store can be scanned, identified and digitally measured in relation to the floor, other products, the entrance or checkout. This data can be gathered and processed for use in advising the store owner where best to relocate products based on analytical data.

It also gives the FIeld Merchandiser a digital representation of where each product is in the store. A forensic level view of the store. This can be utilised on a single product to analyse why a certain product is selling poorly/well in one area compared to another. The results of this data could also be fed back in realtime to the platform enabling the representative to suggest alternative locations for products based on data about successful stores.

Grocery store scanner

7 Wearable FSDU placement

As technology evolves, wearable glasses will be hitting the market this year. With companies like Snapchat / Facebook / Apple all having glasses in development. Currently none of these technologies possess the ability to overlay AR content through clear lenses, undoubtedly this will happen. Facebook’s Oculus Go has all the technical capabilities needed to overlay content onto the real world and currently has a passthrough technology allowing content to be viewed over the real world albeit in VR overlaid on the passthrough camera feed.

When this technology is released it will enable the user to be far more immersed in the AR content. The ability to place FSDU units while wearing AR glasses, will give the Field merchandisers and the store owners a significantly more immersive experience, which equates to faster decision making.

Wearable tech

8 Wearable Product Scanning

While the idea of combining a camera into a user's glasses is controversial and has raised privacy issues. Cameras have already been integrated into Rayban and Snapchats devices. The technology will advance to a stage where these devices contain transparent screens rather than just glass. Utilising a combination of the camera feed, advanced machine learning and image detection will enable the devices to overlay content directly into the user's line of sight in the glasses. This type of technology might seem a long way off. However, Magic Leap headsets, and Microsoft hololenses are already capable of the majority of this. It's by no means unimaginable that the costs of these devices will continue to fall to a level that becomes obtainable to the general public.

As eyekandy continues to innovate the Field Sales Platform (FSP) it future proofs this type of incorporation with little, to no, disruption to the field sales team.

Wearable devices

9 Hand Gesture Product Information

There have been many advances in hand gesture recognition technologies. Pioneers in the industry like Leap motion have developed laser scanning devices to accurately track individual fingers almost 10 years ago. More modern solutions to this problem rely on using advanced AI to just interpret the video from a forward facing camera. While this does pose issues of tracking hands that are at the side or back of a person, Oculus and HTC Vive have already integrated multiple cameras pointing alternate directions to solve these problems.

Future devices will no doubt incorporate these technologies allowing users to freely walk around the aisles of a store using nothing but hand gestures to select products. Glasses will then be able to overlay information not listed about that product on the packaging.

In the commercial world this will be adverts, videos, discounts etc. However for the world of Field merchandising this information could be any, or all of the AR enhancements listed above, and any product information, that would currently be viewable through a tablet or smartphone.

Wearable tech


Sometimes with technology, as in business, as in life - you just have to get on the train with it - even if you're not 100% sure where it's going, you know that if you don't get on, you're 100% going to be left behind.

The eyekandy Field Sales Platform (FSP) not only improves much of the existing merchandiser experience store visitation processes, but also includes new feature enhancements like barcode scanning, image recognition, digital lidar measurement and shelf scanning capabilities to push forward the value the rep can offer in store..

As AI advances with features, such as faster on-device processing, the solutions will be available instantly, all via the merchandisers smartphone. By improving the on site interaction and data capture / process, we can also improve leadership visibility and data capture, ordering process and analytic insights for our partners.

Of course, it’s never as simple as ‘out with the old and in with the new’ when is comes to process and technology, it’s a journey, it’s a blend of legacy databases and processes for a while. Our partners trust eyekandy to navigate the hype, from the deployable technologies, that drive value.

Much of the data inputs don't change, they simply get captured on a more automated, digitised, measurable, accurate, efficient basis that enables the real human effort to be focused on the data and the insights.

As the perfect storm for AR/VR and data heavy technologies arrives, with the combination of ubiquitous 5G, with more capable mixed reality (MR) handset devices from the likes of Apple and Samsung for the mainstream user, the strategic decisions to redirect human effort to more intelligent tasks and integrate emerging technology into the supply chain now, will separate the winners from losers in the next couple of years.

© 2024 EYEKANDY Limited. All rights reserved.
Registered in England #09482425
Registered LLC in USA #7520144