26
MAR
2014

Web Scraping – Definition, Detection and Prevention

Web Scraping is the act of stealing data secretly from the web. It has become the most common threat to millions of online businesses today. The vast importance of data and the criticality to access it for multiple business purposes has inspired people to adopt various legal or...
22
JAN
2014
scraper-enemy

Know your enemy and learn how to prevent screen scraping

When protecting against screen scraping it is useful to understand whom you are up against.  This article will go through the common methods people and companies use to scrape data. Many scrapers are not aware that they are breaking the terms of use of a website.  A carefully...
17
JAN
2014

Why is data scraping difficult to counter?

Not all scraping is hard to counter. The simplest case is where normal web robots adhere to the Robot Exclusion Standard according to what you have entered into your robots.txt file for your website.  Generic robots not honoring the protocol that scrape indiscriminately are...
30
AUG
2013
scrape-website

Changing IP address in order to scrape website a violation against US law

Judge Charles Breyer of District Court in Northern California ruled that changing your IP address or using proxy servers in order to access public websites you have been forbidden to visit is a violation of the Computer Fraud and Abuse Act (CFAA). This was ruled in a case...
21
NOV
2011

Ryanair implements captchas to stop web scraping

Ryanair seems to have taken the drastic step of implementing captchas in the booking process in order to stop Online Travelagents using screen scraping techniques to book tickets on the site. It will be very interesting to se how this turns out, other companies that have tested...
27
JAN
2010

Is ScrapeSentry (ASSASSIN) the answer for screen scraping battle?

There are other ways besides to blocking screen scraping other than implementing Sentor’s ScrapeSentry service. But how useful are these other methods and what are they? Well, one of them is captcha, which will be familiar among many web users. When people encounter...
30
JUN
2009

Australian programmers ‘creating data scraping software’

Australian programmers have created software that uses data scraping to set up free electronic programme guides, it has been revealed. According to The Age, thousands of Australians use these guides and most of the aforementioned software scrapes data from websites such as...