Distil Networks to fingerprint bots

Distil Networks has launched a hi-def fingerprinting solution to tackle the problem of bots. It intends to start: “actively pulling additional data from the browser to identify devices with precision.” This raises questions over Personally Identifiable Information (PII) especially when the EU GDPR comes into force.

The question over PII is a very real problem as companies try and improve their data capture. The European Court of Justice recently ruled that dynamic IP address are now classed as PII. It is not a big jump from there to seeing data that uniquely identifies a device in the same category. According to the Distil Networks press release it is going to target over 200 additional device attributes.

There are good reasons for using a wide range of data points to fingerprint a connection. It makes it harder for bot creators to evade detection. It also means that they cannot just change IP address or create another fake user account. These are easily spotted by the additional data.

There are risks to trying to extract data from devices. Distil Networks will need to work with security software vendors to ensure this new product is not seen as malware. It is also going to be flagged by threat intelligence software for exfiltrating increased amounts of data. The irony here is that Distil Networks uses the data it finds to improve its own threat intelligence.

Why is Distil Networks doing this?

There are good reasons for tracking bots. Back in August, Distil Networks announced that bots made up 46% of the traffic related to web sites. This volume of traffic is creating a lot of problems for webmasters and security teams. Many of the bots are screen scraping data. Some of that is then used by content aggregators to steal content and pass it off as their own. This helps them drive content and advertising to their sites rather than the content originators. This theft of intellectual property is ignored by law enforcement because it is too onerous to deal with.

Another risk posed by bots is security. The data they gather may contain sensitive data. Websites contain names, telephones numbers and addresses related to the company owning the website. It is not uncommon for sensitive customer data to end up on a webpage due to misconfiguration. There are also a lot of sites running older and insecure plug-ins for their websites. All of this data helps hackers create phishing attacks, create profiles of individuals and launch attacks on websites.

What will Distil Networks bot fingerprint technology do?

There are six things that Distil Networks are saying bot mitigation and detection software will do:

  1. Inspects traffic at the perimeter, identifies malicious devices, and intercepts bad bots before they can wreak havoc on a website
  2. Fingerprints stick to the bot even if it attempts to reconnect from random IP addresses or hide behind peer-to-peer networks or anonymous proxies
  3. Goes well beyond IP- and header-centric identification by analyzing over 200 additional device attributes
  4. Includes a tamper proofing layer, which detects manipulation of data values inside the fingerprint
  5. Gives users complete visibility into false positives
  6. Shares the Hi-def fingerprint across Distil customers in a globally-distributed, known violators database, and is made available for other security products like SIEM
Rami Essaid, co-founder and CEO of Distil Networks
Rami Essaid, co-founder and CEO of Distil Networks

According to Rami Essaid, CEO and co-founder of Distil Networks: “Combating bots is a science that requires accuracy. The process begins with gaining a clear understanding of web traffic and the devices that access your website, and for this reason, Hi-def device fingerprinting is the first step in successful bot detection and mitigation.

“Distil’s Hi-def fingerprinting identifies 60% more unique devices than an IP address alone, and 24% more unique devices than an IP address combined with header information. Instead of leaving those unaccounted for devices hidden from defenders, Distil’s Hi-def fingerprinting adds in over 200 additional attributes, increases accuracy, reduces false positives, and gives website defenders the confidence to not just throttle traffic, but to block it outright.”

Deciding which bots to allow will require tuning

The ability to identify the amount of bot traffic also allows companies to begin to block it. Not all bot traffic, however, is bad and companies will want to tune the traffic. How that is done is not clear on the Distil Networks site. It appears that this is not something for companies or even ISPs to install and manage themselves. The pitch appears to be as much around Distil Networks professional services as the product.

If companies do not tune the traffic it could cause them problems. Many sites rely on bots from search engines and comparison sites to funnel customers to them. Blocking those bots will reduce traffic and hurt the business. Conversely, stopping bots posting fake reviews and the constant scraping of content is a positive move.

Conclusion

This is a positive move by Distil Networks to try and reduce the impact of bot traffic. Anything that can reduce the damage caused by screen scraping and data theft is good news. However bots are also a positive thing for some companies. It is also hard to escape questions around the potential gathering of PII. How Distil Networks walks this fine line will be interesting to watch.

Perhaps the biggest surprise is that this does not appear to be targeted at ISPs. They have to deal with the traffic management of their customers and are always looking for new services. If Distil Networks created a white labelled solution that the ISP or cloud providers could sell as a service it would gather data on bots much more quickly.

LEAVE A REPLY

Please enter your comment!
Please enter your name here