Exclusive interview with Oxylabs CEO: how ethical web data collection serves the public good?

Today, Oxylabs, the leading provider of premium proxies and public web data gathering solutions, has more than 1000 clients, with dozens of Global Fortune 500 companies. “We enable companies of all sizes to utilize the potential of big data. In short, we provide all the necessary tools and solutions for businesses looking to collect publicly available data on a large scale,” Julius Cerniauskas, CEO of Oxylabs, told The Baltic Times Magazine.

How has your product grown over the years?

Over the years, our product portfolio has grown to meet the ever-increasing demand for public web data. For instance, we've launched out-of-the-box scraping solutions, such as Scraper APIs, allowing our customers to concentrate on analyzing data instead of focusing on challenging web scraping processes. 

Oxylabs is 400 staff strong who show true dedication to delivering great products and experiences to our customers daily, and I'm grateful for each of them. 

Please explain what web scraping is?

To put it briefly, web scraping is an automated process of gathering large amounts of publicly available data from predefined websites. Businesses then use the collected data to get data-driven insights, fuel their operations, and support strategic business decisions.

For example, web scraping is an integral part of online shopping, as it allows businesses to access reliable real-time data and gives a better understanding of their competitors’ actions. As a result, companies can offer their customers the most desirable products and services at the most competitive prices. 

It is barely one web scraping business case, but there are many more. As the need for big data continues to grow at lightspeed, different use cases emerge, and new industries discover the benefits of external data. 

And how does a middle-aged man with mediocre internet savviness benefit from web scraping technology?

Automated web data gathering solutions benefit businesses but also bring a lot of convenience to the broader public. 

Imagine a situation where you want the best price for a specific product. Instead of opening multiple browser tabs of different retailer sites, filling notes with product information, and spending hours trying to identify the best deal, we simply visit price product comparison sites. All the comparison sites rely on web scraping technology to bring this information to us in seconds. 

Travel planning is another excellent example. Travel aggregator sites come in handy when organizing a family vacation or planning a short getaway in a city abroad. They offer the best plane ticket and accommodation prices and aid us in finding budget-friendly deals. Without web scraping, travelers would again have to spend hours or even days on best-deal research or rely on their local travel agents.

Of course, the most obvious example would be search engines, as they rely on web scraping technology to index all the websites on the internet. By doing so, search engines can effectively direct us to the sources online to answer our questions. Simply put, web scraping is a fascinating technology that makes our life much easier.

How does Oxylabs navigate vast data waters where the data protection laws are seemingly on the consumer's side? What are the "don'ts" you abide by while gathering public data? Are you an ethical provider in the data collection industry?

It's not the first time I've heard this question, and I'm sure it will not be the last. Nowadays, when most people hear the word "data", they instantly think it will affect their privacy. It's only natural due to so many data breaches worldwide.

At Oxylabs, we work on the B2B side of public web data. Regardless, it's still a tricky legal landscape. Web scraping is a relatively new technology that lacks widespread legal regulation. Therefore, companies must always be cautious about what kind of web data they are planning to collect and whether they won't breach any laws, for instance, copyright laws. Before any web scraping activity, we urge every company to consult with lawyers and evaluate the legal aspects.

At Oxylabs, we invest a lot of our time and effort in educating the industry and our clients on how to perform ethical web scraping. We have strict policies regarding public web data gathering and ensure that the highest standards of business ethics guide all our operations. 

Do you have success stories of how web scraping technology was beneficial for society at large? 

As part of our efforts to shine a positive light on ethical web scraping and show how beneficial it can be to the broader public, we launched "Project 4β", dedicated to helping universities, researchers, and organizations to solve critical questions and missions. 

Through "Project 4β" we had impactful partnerships with many universities, including the University of Michigan and CODE – University of Applied Sciences, with whom we shared our tools and expertise. 

Also, after winning the Govtech Lab Challenge, we fostered a wonderful pro bono partnership with The Communications Regulatory Authority of the Republic of Lithuania (CRA). One of their missions is safeguarding the Lithuanian internet space from illegal content. 

For the CRA’s challenge of automating unlawful content identification, our team created an AI-powered web scraping tool that scans the internet to detect illicit visual content related to child sexual abuse. Before, they primarily depended on the goodwill of internet users who reported illegal content to the hotline. As of 2022, the tool was fully employed in CRA's daily operations, bringing much-needed proactivity to take down these sites and ensure that bad actors face the consequences.

Oxylabs is always willing to be a sidekick of researchers and organizations that want to solve critical social questions and welcomes them to join "Project 4β". Just drop us a message via [email protected]. With the help of web data-gathering solutions, I'm sure that we can address more social problems.

About Oxylabs

Established in 2015, Oxylabs is a premium proxy and public web data acquisition solution provider, enabling companies of all sizes to utilize the power of big data. Constant innovation, a large patent portfolio, and a focus on ethics have allowed Oxylabs to become a global leader in the data acquisition industry and forge close ties with dozens of Fortune Global 500 companies. In 2022, Oxylabs was named the fastest-growing public data gathering solutions company in Europe in the Financial Times’ FT 1000 list.