Shopping sites ratchet up battle against bots
Skip to main content
Sister Publication Links
  • Automotive News Canada
  • Automotive News Europe
  • Automotive News China
  • Automobilwoche
AN-LOGO-BLUE
Subscribe
  • Subscribe
  • Account
  • login
  • HOME
  • NEWS
    • Dealers
    • Automakers & Suppliers
    • News by Brand
    • Cars & Concepts
    • Final Assembly
    • China
    • Shift
    • Mobility Report
    • Special Reports
    • Digital Edition Archive
    • This Week's Issue
    • The 1965 Mustang Shelby GT350 reigned supreme in the Ford Frenzy tournament.
      Shelby GT350 gets its one shining moment
      Dan Parker will pilot his Chevrolet Corvette with a custom guidance system that provides audible feedback.
      Blind racer gets Cruise support, but he'll drive
      You may have won! But no
      The course was planned for a Vermont ski resort, until neighbors shouted the idea down.
      Off-road course for Broncos? Not in my backyard
    • Huawei
      Huawei to invest $1 billion on auto technology
      GAC Motor Co. launched sales of a new full electric crossover, the Aion Y, in March.
      New-vehicle sales rebound 75% in March from virus-hit 2020
      Wuling launched sales of its first pickup, the Zhengtu, on March 18.
      GM Q1 sales rebound but fall short of pre-COVID level
      The Ford Equator SUV arrived in the Chinese market on March 28
      Ford Q1 sales surge 73% over virus-hit 2020
    • VW unit is pumping funds into mobile charging robot
      Companies work smarter, not harder, with logistics-focused robots
      Other industries combine for more bot orders than automakers, suppliers
      Gatik capitalizes on growing demand for shorter-distance deliveries
    • Mobileye Udelv
      Intel's Mobileye teams with startup Udelv on automated delivery
      Nuro Domino's
      Domino's, Nuro to start robot pizza delivery in Houston
      Cruise Origin
      Cruise to deploy robotaxis in Dubai from 2023
      Seattle, Washington
      Seattle drafts blueprint to electrify transportation by 2030
    • Elon Musk interview
      Tesla’s Model Y, on sale since March, is a showcase of EV technologies competitors will benchmark.
      Technologies of Electrification
      Cadillac’s Lyriq EV will be unveiled Aug. 6.
      Future Product Pipeline
      A CALL TO ACTION
    • Ad process evolves with pandemic
      Pandemic adds resonance to Garff effort
      Microchip illustration
      The latest numbers on the microchip shortage: GM production pummeled
      Moreno sets sights on Washington
    • Access F&I
    • Fixed Ops Journal
    • Marketing
    • Used Cars
    • Retail Technology
    • Sales
    • Best Practices
    • Dealership Buy/Sell
    • NADA
    • NADA Show
    • Automakers
    • Manufacturing
    • Suppliers
    • Regulations & Safety
    • Executives
    • Talk From The Top
    • Leading Women Network
    • Guide to Economic Development
    • PACE Awards
    • Management Briefing Seminars
    • World Congress
    • Aston Martin
    • BMW
      • Mini
      • Rolls-Royce
    • Daimler
      • Mercedes Benz
      • Smart
    • Ford
      • Lincoln
    • General Motors
      • Buick
      • Cadillac
      • Chevrolet
      • GMC
    • Honda
      • Acura
    • Hyundai
      • Genesis
      • Kia
    • Mazda
    • McLaren
    • Mitsubishi
    • Nissan
      • Infiniti
    • Stellantis
      • Alfa Romeo
      • Citroen
      • Chrysler
      • Dodge
      • Ferrari
      • Fiat
      • Jeep
      • Lancia
      • Maserati
      • Opel
      • Peugeot
      • Ram
      • Vauxhall
    • Renault
    • Subaru
    • Suzuki
    • Tata
      • Jaguar
      • Land Rover
    • Tesla
    • Toyota
      • Lexus
    • Volkswagen
      • Audi
      • Bentley
      • Bugatti
      • Lamborghini
      • Porsche
      • Seat
      • Skoda
    • Volvo
    • (Discontinued Brands)
    • Virtual reveals (Sponsored)
      • MITSUBISHI: 2022 Outlander
      • NISSAN: 2022 Pathfinder and 2022 Frontier
      • GENESIS: 2021 GV80
      • KIA: 2021 K5
      • LEXUS: 2021 IS
      • NISSAN: 2021 Rogue
      • TOYOTA: 2021 Venza and 2021 Sienna
    • Auto Shows
    • Future Product Pipeline
    • Photo Galleries
    • Car Cutaways
    • Design
  • OPINION
    • Blogs
    • Cartoons
    • Keith Crain
    • Automotive Views with Jason Stein
    • Columnists
    • China Commentary
    • Editorials
    • Letters to the Editor
    • Send us a Letter
    • Mach-E
      For Ford, 3rd-party charging network must improve to reap Mustang Mach-E potential
      VW logo with reflection
      VW lied to sell diesels; now it lied to sell EVs
      265725_XC90_Inscription_in_Birch_Light.jpg
      Volvo targets top talent in U.S., China with generous parental leave policy
      Fisker gets a rare second chance to build his own car company
    • In case of supply chain woes, break glass
      Quick! The Biden phone
      VW's Joke Is On You
      Joke Is On You
      In case of supply chain woes, break glass
      view gallery
      3 photos
      Leo Michael Cartoons - Q2 2021
      Leo Michael: There's Nothing You Can Do!
      Nothing You Can Do
    • Shifting gears away from the stick shift
      SEMA still a wonderful circus
      Penske still has plenty of races to win
      Ford's turn in the hot seat
    • April 6, 2021 | For Volkswagen, authenticity is the only path forward
      March 30, 2021 | Tesla “changes” car industry
      March 23, 2021 | Remote work will continue after the pandemic
      March 16, 2021 | Industry offers warning to Apple
    • Ford's Blue Advantage tool shows it's ‘listening' to dealers
      Jamie Butters
      EV rebates must work at retail
      Tracy Fred
      Dealers deserve access to their data
      Dan Shine
      Service drive can lift dealerships as source of used-vehicle inventory
    • Why Xiaomi holds promise of becoming formidable EV player
      Geely’s new EV push is decidedly new age
      Charging poles installed outside a Tesla store in downtown Shanghai.
      Unreliable public EV charging sites prompt automakers to take wheel
      VW Group's playbook to dominate EV market
    • A step in right direction for Nissan dealers
      ‘Voltswagen' April Fools' gag far from funny
      Less is more in lineups as well as on lots
      Electric vehicle charger
      Electric vehicle hype opens door to short sellers
    • ‘Voltswagen' was clever and fun
      With EVs, bias for traditional carmakers
      There goes VW recycling again
      Mixed-use plan a winner
  • DATA CENTER
  • VIDEO
    • AutoNews Now
    • First Shift
    • Special Video Reports
    • Weekend Drive
    • AutoNews Now: Biden eyes $50B for chip production, research
      AutoNews Now: EV battery decision looms for SK Innovation, LG Chem
      AutoNews Now: Ford skipping summer shutdown at many U.S. plants
      Mercedes leads Lexus, BMW in tight luxury race
    • First Shift: Ford dealers plan standalone Bronco stores
      First Shift: Hyundai to idle more output due to chip shortage
      First Shift: Biden's EV plan said to include $100B for consumer rebates
      First Shift: Stronger auto emissions rules coming by July, EPA head says
    • How a Seattle-area dealer is teaching, retaining auto techs 
      Virtual sales manager Jay Barger
      'Glued to a phone': Acura store's virtual sales boss lifts deliveries, efficiency
      One man’s journey from homelessness to thriving dealer
      Bert Ogden Auto Group
      How a Texas group is cutting costs, saving millions
    • Why the pickup is the auto industry's 'battleground'
      Carlos Ghosn's quest to restore his reputation
      Why Ford must execute to avoid 'deep trouble'
      Why Honda is 'locked and loaded' for 2020
  • EVENTS & AWARDS
    • Events
    • Awards
    • Congress Conversations
    • Retail Forum: NADA
    • Canada Congress
    • Europe Congress Conversations
    • Fixed Ops Journal Forum
    • Shift: Mobility at a Crossroads
    • 100 Leading Women
    • 40 Under 40 Retail
    • All-Stars
    • Best Dealerships To Work For
      • Register for the 2021 Best Dealership To Work For
    • PACE Program
    • Rising Stars
    • Europe Rising Stars
  • JOBS
  • AN Solutions
  • +MORE
    • Leading Women Network
    • Podcasts
    • Webinars
    • In the Driver's Seat
    • Publishing Partners
    • Classifieds
    • Companies on the Move
    • People on the Move
    • Newsletters
    • Contact Us
    • Media Kit
    • RSS Feeds
    • Shift: A Podcast About Mobility
    • Special Reports Podcasts
    • Daily Drive Podcasts
    • AAM
    • Cars.com
    • DealerPolicy
    • Gentex
    • IHS Markit
    • Remember Group
    • Reputation.com
    • Ricardo: Dave Shemmans
    • Ricardo: Marques McCammon
    • Allstate: Want more from your F&I?
    • Ally: Navigating the future of automotive retailing
    • Amazon Web Services: Any place, any time, any channel
    • Amazon Web Services: The power of the cloud
    • Amazon Web Services: Universal translator: Harnessing sensor data to build better automotive software
    • Capital One Auto: Dealerships remain vital to an increasingly digital car shopping journey
    • DealerSocket: 5 steps to modernizing the buyer's journey
    • Epic Games: Real-time tech is the next frontier of automotive
    • Epic Games: Transforming the auto industry with digital assets
    • FTI Consulting: Crisis as a catalyst for change
    • Google: 5 trends shaping the auto industry's approach to a new normal
    • Google: Google's dealer guidebook helps dealers navigate today's digital landscape
    • IHS Markit: Automotive loyalty in the wake of the COVID-19 recession
    • IHS Markit: COVID-19: The future mobility delusion
    • Level5: 2020 Automotive E-Commerce Report
    • Motormindz: Toward hyperconnectivity: 5 ways to position your business to profit from connected car
    • Naked Lime: Bring social reputation together as part of big-picture marketing
    • Wells Fargo Auto: Switching gears from LIBOR to SOFR
    • Ally: Do It Right
    • DealerSocket
    • Deloitte: Cyber everywhere: Preparing for automotive safety in the face of cyber threats
    • Facebook: The road to a zero-friction future
    • Guide To Economic Development
    • PayPal Credit: How consumer financing helps drive sales for online auto parts retailers
MENU
Breadcrumb
  1. Home
  2. Retail
July 13, 2014 01:00 AM

Shopping sites ratchet up battle against bots

Vince Bond Jr.
  • Tweet
  • Share
  • Share
  • Email
  • More
    Print
    BLOOMBERG
    For shopping sites such as Cars.com, TrueCar and AutoTrader, whose listings of vehicles in dealer inventories attract millions of legitimate consumers a month, it's become a constant battle to weed out shady players that can compromise traffic data.
    Traffic jam

    Car shopping sites AutoTrader, Cars.com and TrueCar have to detect suspicious bot activity amid a torrent of Web traffic each day. Their hands are full.

    • AutoTrader: Averaging more than 18 million unique monthly visitors in 2014; topped 16 million unique monthly visitors last year

    • Cars.com: Around 13.5 million unique monthly visitors in 2013 across desktop and mobile platforms

    • TrueCar: Tallied 3.9 million unique visitors a month in the first quarter of 2014

    Cars.com site analysts knew something was up about seven years ago when they saw traffic suddenly surge on BMW listings in Albany, N.Y.

    A little digging found that the activity was no cause for celebration. It wasn't coveted in-market car shoppers who were driving up page views on dealers' inventory listings but rather Internet bots -- snippets of nefarious software code masquerading as human Web users -- that had invaded the site and skewed its traffic metrics.

    It was the opening salvo in a cyberwar that Cars.com is still fighting today, against an enemy that is only becoming more sophisticated.

    "That's when we started doing something about it," William Swislow, Cars.com's chief information officer, said of the bot menace. "We spend a lot of time and creativity basically recognizing when someone who appears to be accessing the site is a real person or a bot."

    The risk goes beyond skewed numbers. Site managers say bots can swipe inventory listing data for reuse by outside parties and can even skim off ad revenue that's generated from online clicks.

    For shopping sites such as Cars.com, TrueCar and AutoTrader, whose listings of vehicles in dealer inventories attract millions of legitimate visitors a month, it is critical to weed out shady players and protect the integrity of the data they're reporting in order to sustain their relationship with dealers. Any distortion of the metrics could confound dealerships that are seeing heavy activity on the sites but not getting a proportionate number of leads.

    The sites say the reports they provide to dealers -- with metrics such as how many e-mail leads and phone calls were generated by a listing and how many times vehicle detail pages were clicked -- are scrubbed clean of bot activity. But figuring out what the bots are after and how to beat them is a constant challenge.

    "It's hard to know who they are in many cases and what they want to do with the data," said Scott Hernalsteen, AutoTrader's senior director of enterprise analytics.

    "If you look at the success of AutoTrader over the years, we've built up a large number of listings," he said. "They could be trying to acquire [the information], accumulate it to repackage it and sell it to others. At the end of the day, we typically don't know."

    Scott HernalsteenAutoTrader

    "It's hard to know who they are in many cases and what they want to do with the data."

    Precisely how much online traffic is generated by bots -- and how many marketing dollars are wasted on these bogus users -- isn't known either. Estimates of the wasted ad spending range from $1 billion a year to nearly $12 billion.

    Solve Media, a digital advertising and security firm, said its research found that suspicious Web traffic rose 40 percent in 2013, with bot traffic peaking at 61 percent of all traffic during the fourth quarter.

    A common template for bot activity is criminals creating a network of bogus Web sites. They can be one-page junk sites, or more sophisticated sites that look legitimate because they've copied content from other sources, said Jeffrey Tognetti, product development team lead at DealerX, a digital marketing company.

    The site operators create a bot and send it to car shopping or dealer sites with hopes that it will be recognized as a potential buyer. If that happens, an ad retargeting platform -- a service that shows ads to people on the Web sites they visit based on their online shopping history -- could key in on the bot and follow it to the fake site, where it would display an ad.

    As a result, "the person who owns the real estate, who owns that Web site, gets paid … for the impression," Tognetti said. "And then often the bot itself clicks on the ad and gets paid for the click."

    As the efforts to detect them have become more sophisticated, so have the bots themselves. AutoTrader's Hernalsteen said his site, which uses an array of filters if bots make it past their initial wave of defense, receives tens of thousands of attempts to copy content each day.

    The bots sometimes originate in foreign locales, but their creators can disguise them as U.S.-based users to fool Web sites that are equipped to block foreign traffic.

    "Early on we've seen different sources where you'd have international IP addresses hitting a publisher's Web site. Those were easy to sniff out," said Scott Cunningham, vice president of technology and ad operations for the Interactive Advertising Bureau. "Over time, they've gotten really good at masquerading."

    One clear warning sign for shopping sites is browsing speed: Seeing one IP address, or computer identification number, hit hundreds of pages within a minute is a dead giveaway that a bot is at work. But some bots are being designed to closely mimic human behavior by browsing slower to avoid getting flagged.

    John Williams, TrueCar's senior vice president of platform operations, says the site counters these bots with anomaly detection software developed in-house. The software identifies patterns in the traffic, pointing out in some cases whether similar IP addresses appear to be working in concert. If addresses appear to be bots, Williams said, the site can block them, or impose strict rate limits on them when browsing the site.

    A rate limit means there is a maximum rate at which a given set of addresses can load pages, Williams says. An error is returned to the user when the limit is reached.

    TrueCar also spots suspicious patterns through data analysis when looking at factors such as how many car makes a user looks at before buying.

    "We often have these things pop up where [we say] 'Hey, there are some small percentages who look at 300 different makes and they never buy anything.' That's not a person," Williams said. "We just go ahead and block those also."

    Cars.com requires every e-mail lead submitted to go through an automated system that tries to discern if it's legitimate. Plus, Swislow said the fraud team manually reviews any lead that looks suspicious. He said his team has made significant progress against bot traffic over the past several years.

    But even with dedicated security teams and technologies working to spot bots, he says, the enemy can't be eradicated.

    "The nature of the Internet is every time you take steps to defend your site against unauthorized use, there are going to be people who will try to get around it," Swislow said. "I wouldn't make the claim [that] we have no issues whatsoever, but we think we have a pretty good handle on it."

    RECOMMENDED FOR YOU
    Baker Auto Group expands into Florida
    Letter
    to the
    Editor

     

     

    Send us a letter

    Have an opinion about this story? Click here to submit a Letter to the Editor, and we may publish it in print.

    Recommended for You
    Baker Auto Group expands into Florida
    U.S., European consumers warm to EVs, but remain wary of price - survey
    U.S., European consumers warm to EVs, but remain wary of price - survey
    Ford digital used-car tool lifts sales as chip shortage pinches new inventory
    Ford digital used-car tool lifts sales as chip shortage pinches new inventory
    Sponsored Content: VIDEO: Cars.com CEO talks state of the industry, dealer innovation
    Sign up for free newsletters
    EMAIL ADDRESS

    Please enter a valid email address.

    Please enter your email address.

    Please verify captcha.

    Please select at least one newsletter to subscribe.

    See more newsletter options at autonews.com/newsletters.

    You can unsubscribe at any time through links in these emails. For more information, see our Privacy Policy.

    Digital Edition
    Automotive News 4-12-21
    THIS WEEK'S EDITION
    See our archive
    Fixed Ops Journal
    Fixed Ops Journal 4-12-21
    Read the issue
    See our archive
    RETAIL TECHNOLOGY NEWSLETTER: Sign up for our Tuesday afternoon newsletter that breaks down the latest developments in how technology is changing the auto sales landscape.
    WEEKLY NEWSLETTER: Sign up to start your week with the top stories from the latest issue of Automotive News, delivered to your inbox first thing each Monday morning.
    Get Free Newsletters

    Sign up and get the best of Automotive News delivered straight to your email inbox, free of charge. Choose your news – we will deliver.

    Subscribe Today

    Get 24/7 access to in-depth, authoritative coverage of the auto industry from a global team of reporters and editors covering the news that’s vital to your business.

    Subscribe Now
    Connect With Us
    • Facebook
    • Instagram
    • LinkedIn
    • Twitter

    Our mission

    The Automotive News mission is to be the primary source of industry news, data and understanding for the industry's decision-makers interested in North America.

    AN-LOGO-BLUE
    Contact Us

    1155 Gratiot Avenue
    Detroit, Michigan
    48207-2997

    (877) 812-1584

    Email us

    Automotive News
    ISSN 0005-1551 (print)
    ISSN 1557-7686 (online)

    Fixed Ops Journal
    ISSN 2576-1064 (print)
    ISSN 2576-1072 (online)

    Resources
    • About us
    • Contact Us
    • Media Kit
    • Subscribe
    • Manage your account
    • Reprints
    • Ad Choices Ad Choices
    • Sitemap
    Legal
    • Terms and Conditions
    • Privacy Policy
    • Privacy Request
    Automotive News
    Copyright © 1996-2021. Crain Communications, Inc. All Rights Reserved.
    • HOME
    • NEWS
      • Dealers
        • Access F&I
        • Fixed Ops Journal
        • Marketing
        • Used Cars
        • Retail Technology
        • Sales
        • Best Practices
        • Dealership Buy/Sell
        • NADA
        • NADA Show
      • Automakers & Suppliers
        • Automakers
        • Manufacturing
        • Suppliers
        • Regulations & Safety
        • Executives
        • Talk From The Top
        • Leading Women Network
        • Guide to Economic Development
        • PACE Awards
        • Management Briefing Seminars
        • World Congress
      • News by Brand
        • Aston Martin
        • BMW
          • Mini
          • Rolls-Royce
        • Daimler
          • Mercedes Benz
          • Smart
        • Ford
          • Lincoln
        • General Motors
          • Buick
          • Cadillac
          • Chevrolet
          • GMC
        • Honda
          • Acura
        • Hyundai
          • Genesis
          • Kia
        • Mazda
        • McLaren
        • Mitsubishi
        • Nissan
          • Infiniti
        • Stellantis
          • Alfa Romeo
          • Citroen
          • Chrysler
          • Dodge
          • Ferrari
          • Fiat
          • Jeep
          • Lancia
          • Maserati
          • Opel
          • Peugeot
          • Ram
          • Vauxhall
        • Renault
        • Subaru
        • Suzuki
        • Tata
          • Jaguar
          • Land Rover
        • Tesla
        • Toyota
          • Lexus
        • Volkswagen
          • Audi
          • Bentley
          • Bugatti
          • Lamborghini
          • Porsche
          • Seat
          • Skoda
        • Volvo
        • (Discontinued Brands)
      • Cars & Concepts
        • Virtual reveals (Sponsored)
          • MITSUBISHI: 2022 Outlander
          • NISSAN: 2022 Pathfinder and 2022 Frontier
          • GENESIS: 2021 GV80
          • KIA: 2021 K5
          • LEXUS: 2021 IS
          • NISSAN: 2021 Rogue
          • TOYOTA: 2021 Venza and 2021 Sienna
        • Auto Shows
        • Future Product Pipeline
        • Photo Galleries
        • Car Cutaways
        • Design
      • Final Assembly
      • China
      • Shift
      • Mobility Report
      • Special Reports
      • Digital Edition Archive
      • This Week's Issue
    • OPINION
      • Blogs
      • Cartoons
      • Keith Crain
      • Automotive Views with Jason Stein
      • Columnists
      • China Commentary
      • Editorials
      • Letters to the Editor
      • Send us a Letter
    • DATA CENTER
    • VIDEO
      • AutoNews Now
      • First Shift
      • Special Video Reports
      • Weekend Drive
    • EVENTS & AWARDS
      • Events
        • Congress Conversations
        • Retail Forum: NADA
        • Canada Congress
        • Europe Congress Conversations
        • Fixed Ops Journal Forum
        • Shift: Mobility at a Crossroads
      • Awards
        • 100 Leading Women
        • 40 Under 40 Retail
        • All-Stars
        • Best Dealerships To Work For
          • Register for the 2021 Best Dealership To Work For
        • PACE Program
        • Rising Stars
        • Europe Rising Stars
    • JOBS
    • AN Solutions
    • +MORE
      • Leading Women Network
      • Podcasts
        • Shift: A Podcast About Mobility
        • Special Reports Podcasts
        • Daily Drive Podcasts
      • Webinars
      • In the Driver's Seat
        • AAM
        • Cars.com
        • DealerPolicy
        • Gentex
        • IHS Markit
        • Remember Group
        • Reputation.com
        • Ricardo: Dave Shemmans
        • Ricardo: Marques McCammon
      • Publishing Partners
        • Allstate: Want more from your F&I?
        • Ally: Navigating the future of automotive retailing
        • Amazon Web Services: Any place, any time, any channel
        • Amazon Web Services: The power of the cloud
        • Amazon Web Services: Universal translator: Harnessing sensor data to build better automotive software
        • Capital One Auto: Dealerships remain vital to an increasingly digital car shopping journey
        • DealerSocket: 5 steps to modernizing the buyer's journey
        • Epic Games: Real-time tech is the next frontier of automotive
        • Epic Games: Transforming the auto industry with digital assets
        • FTI Consulting: Crisis as a catalyst for change
        • Google: 5 trends shaping the auto industry's approach to a new normal
        • Google: Google's dealer guidebook helps dealers navigate today's digital landscape
        • IHS Markit: Automotive loyalty in the wake of the COVID-19 recession
        • IHS Markit: COVID-19: The future mobility delusion
        • Level5: 2020 Automotive E-Commerce Report
        • Motormindz: Toward hyperconnectivity: 5 ways to position your business to profit from connected car
        • Naked Lime: Bring social reputation together as part of big-picture marketing
        • Wells Fargo Auto: Switching gears from LIBOR to SOFR
        • Ally: Do It Right
        • DealerSocket
        • Deloitte: Cyber everywhere: Preparing for automotive safety in the face of cyber threats
        • Facebook: The road to a zero-friction future
        • Guide To Economic Development
        • PayPal Credit: How consumer financing helps drive sales for online auto parts retailers
      • Classifieds
      • Companies on the Move
      • People on the Move
      • Newsletters
      • Contact Us
      • Media Kit
      • RSS Feeds