What is scraping and why should you bother?
The growing phenomenon of systematic data theft known as “scraping” is definitely something to be taken seriously by online businesses who maintain large public databases on their web sites. As you read this article, you will learn more about why scraping can be a serious threat to your company.
Scraping (also web scraping, screen scraping or data scraping) is what you do when you copy large amounts of data from a web site – manually or with a script or program.
Scraping can sometimes be benevolent and totally acceptable like for example the search engine robots that index the web. You definitely want those to spider your site so that your customers can find you! On this web site however we will focus on malicious scraping which is carried out for commercial gain – and how to prevent it.
/ Martin Zetterlund, Anti Scraping Expert at Sentor Managed Security Services
What is malicious scraping?
Malicious scraping is systematic theft of intellectual property in the form of data accessible on a web site. This can be illustrated using an online directory as an example. They publish intellectual property online, names, addresses, and business information.
It is free for all to use the information as long as they comply with the term and conditions of the site. Unfortunately, scrapers do not care about terms and conditions, and will abuse the service by systematically downloading large amounts of data for personal gain.
The online directory looses control over its data which they have invested time and money to gather, maintain, and make available as a part of their service offering. In a worst case scenario, they wake up to a new competitor that is able to offer the same data as them.
Why it hurts your business
If a scrapers downloads all the data from your database and adds it to a competing site, you may suddenly have competition offering the same exact service.
Adding insult to injury, the company doing the scraping will not have the same overhead expenses as you, and can offer similar services at a significant discount to your target groups.
We see entire businesses threatened by out of control scraping every day. Many of these companies do not understand their exposure to the threat of scraping until it is too late.
Do you understand the impact of scraping on your business today?
Who is threatened by scraping?
All online businesses that share information on their websites are threatened by scraping. Examples of these include:
- Online directories
- Airlines
- Betting
- B2B-portals
- Online property
- Insurance