Technology & Collaboration: The Winning Formula to Defeat Shopping Bots

December 28, 2020 | by Ameya Talwalkar

As an innovator, software is always the first thing I think about when addressing a problem. But, in a recent blog about shopping bots and the holiday season, Sandy Carielli at Forrester reminded me that you’ve got to think about holistic solutions – comprised of software/technology, people and process – in order to best address our hardest issues. Sandy highlights the importance for security teams to be in lockstep with e-commerce teams in order to adequately block those pesky shopping bots which create headaches for retailers and frustrations for shoppers who are left with empty carts or incomplete orders. These are great, and often overlooked recommendations. However, bots are not just a security problem they’re also an e-commerce problem. In my opinion, open communication between security and e-commerce teams is the key to success in retail.

Let’s explore some other ways to combine both great technology and great cross-team collaboration to keep your loyal customers happy.

Identify “Normal” and Know When to Expect Peaks

From the collaboration side, Sandy calls out the importance of great communication from the e-commerce and marketing teams over to the security and operations teams so they know when to expect high volumes of traffic. Plan for high volumes when sales start, product inventory is replenished and when marketing efforts like email and social media promos go live.

On the technology side, it’s important to model out what known “human” traffic looks like. Where does the traffic come from? How does the traffic interact with your website or app? Knowing what good looks like helps you identify bots faster so you can take proactive actions to block them. During peak shopping periods, especially with high-in-demand items, a surge in human traffic can trick basic bot mitigation solutions to flag it as illegitimate traffic resulting in the blocking of real users from shopping online. This leads to customer dissatisfaction, social media outrage, brand loss and eventually a decline in revenue.

Know the Hot Targets

It seems obvious that security and operations should be made aware of the hot products for the shopping season, but sometimes communication gets missed. Or, like the Atari ET game of the 80s sometimes the hot products are flops, while sleeper products suddenly become hot, not unsimilar to the great toilet paper shortage of early 2020 or the looming outdoor heaters restaurants are now scrambling to find. It is important to communicate the popularity projections so that everyone can best anticipate and be prepared to manage the shopping bots.

With projections in hand, it’s critical that security teams use bot mitigation to ensure that as much of the hot product as possible is purchased by human shoppers. In high volume/high-value sales, we often see retailers putting all shoppers in “waiting rooms” in an effort to ensure site reliability and to give human buyers a chance. The drawback to this approach is the loss of revenue from the rest of the merchandise on sale. “Waiting rooms (aka queueing)” take a broad stroke at trying to control the shopping bot problem, when in reality you need scalpel-like precision. Most advanced shopping bots have built-in techniques that allow the bots to get out of the “waiting rooms” ahead of the humans, leading to a scenario we often see – “Bots are shopping, while the humans are waiting”. And no one likes hanging out in the waiting room, as evidenced by the backlash on social media when legitimate buyers lose out to automated buyers.

Manage Multiple Threats

Sandy questions in her blog “Are we only concerned about hoarding?” and of course, the answer is, “No!” But, defending against automated shopping bots on top of gift card fraud, ATO, and inventory spinning all at once can be quite the juggling act. It’s important to build out plans to identify and respond to all these threats and optimize your security technology according to your plan.

Retail customers are hit with a wider range of automated attacks than any other industry, with the entire spectrum of the OWASP Automated Threats being found in our retail customer deployments. Many of the attacks go beyond classic credential stuffing and Account Take Over (ATO), using advanced techniques to evade first-generation bot prevention tools and commit scalping (aka automated shopping), denial of inventory and scraping attacks. It’s important to balance bot prevention with customer satisfaction while ensuring that your security strategy takes into account these additional threats.

Build Out Your Defense Playbook

Having a playbook is an important part of a good defense, and it’s a good idea to also test out scenarios before you have to utilize them in the heat of an attack. Tabletop exercises will help everyone on the team get familiar with the plans, and may also help you identify areas that need to be fleshed out further, or perhaps even identify new tools that are needed.

Again, flexibility in your bot mitigation tools is important as you probably want to have a different response to each of the different plays. Slowing down account aggregators for reward points during peak periods, blocking competitive scrapers while allowing search engine bots and price comparison partner bots are good strategies. Make sure you choose a bot mitigation strategy that has flexibility and precision to implement your version of the Defense Playbook. It’s also important that the technology implemented to stop bots doesn’t create unnecessary friction for your legitimate shoppers.

The steps that Sandy identifies are exactly the kinds of proactive actions that our Threat Monitoring team takes to support our retail clients. With all of the pandemic’s uncertainties where every day some new threat or challenge seems to arise, having open lines of communications and extremely flexible and open bot mitigation is key to providing excellent service and a great shopping experience for your customers.

Ameya Talwalkar


Ameya Talwalkar

President, Chief Executive Officer & Founder

Additional Resources