GDPR and Web Scraping: What US Companies Need to Know
A US company can be bound by the GDPR when it scrapes personal data from EU residents, even with no office in Europe. When it applies, why "publicly available" is not an exemption, the legal-basis problem for scraped data, and how to shrink your exposure by extracting narrowly.
By the ClawEngine team
July 2026 · 10 min read
Hit Extract to turn this page into clean, LLM-ready data.
robots.txt respected · public data only
The short answer
A US company can be bound by the GDPR when it scrapes personal data about people in the EU, even with no office, server or employee in Europe. The regulation follows the data subject, not the company's address, so if your crawl collects names, emails or other personal data of EU residents while offering goods or services to them or monitoring their behavior, the GDPR can apply to you. "Publicly available" is not an exemption, and for scraped data you usually have no clean legal basis to rely on. This is general information, not legal advice. The practical defense is to extract narrowly and avoid personal data you do not need.
When does the GDPR apply to a US company?
The GDPR reaches outside the EU through its territorial scope rule, Article 3. A company with no EU establishment still falls under it when it processes the personal data of people who are in the EU, and it is either offering goods or services to them or monitoring their behavior. Scraping is squarely a form of processing. So the question is not where your servers sit; it is whose personal data you are collecting and what you do with it.
Two clarifications that trip people up. First, this is about the location of the data subject at the time, not their citizenship, so it covers EU residents generally, not only EU nationals. Second, incidental collection can still count. If you crawl a directory and it happens to include EU residents' contact details, you have processed their personal data, regardless of whether they were your target.
Is scraping publicly available data GDPR compliant?
No, not automatically. This is the single biggest misconception. The GDPR draws no exemption for personal data just because it is public. A name and job title on a company team page, a reviewer's handle, an email in a business listing: all of it is personal data whether or not the person chose to publish it, and the fact that it is visible to anyone does not remove the protections. European regulators have fined companies specifically for building databases by scraping publicly available personal data, so treating "public" as a green light is exactly the mistake that draws enforcement.
The legal-basis problem is the real trap
Every processing of personal data under the GDPR needs a lawful basis from Article 6, and scraped data is where this gets hard. You cannot realistically get consent from people you have never contacted. Contract does not fit, because you have no relationship with them. That leaves legitimate interests, which is a genuine basis but a demanding one: you have to run and document a balancing test weighing your interest against the person's rights and their reasonable expectations, and someone whose data was silently scraped from a third-party site generally did not expect to end up in your database. Legitimate interests can work for some narrow, low-risk cases, but it is not a rubber stamp, and it collapses fast for sensitive data or intrusive uses.
The obligations that follow you even after collection
Having a basis is only the start. If the GDPR applies, so do the duties around the data you now hold.
| Obligation | What it means for scraped data |
|---|---|
| Transparency (Art. 14) | You must inform people their data was collected, even when you got it from a third party, unless a narrow exception applies |
| Lawful basis (Art. 6) | Documented legitimate-interests assessment, in practice, for most scraping |
| Data minimization | Collect only fields you genuinely need, not everything on the page |
| Access and erasure | Honor subject requests to see or delete their data, which requires provenance |
| Special categories (Art. 9) | Health, political, religious and similar data is largely off-limits without a strong basis |
The transparency duty is the one people forget and the one regulators lean on. Collecting data quietly and never telling the person is itself a violation, separate from whether you had a basis to collect at all.
How do I reduce my GDPR exposure when scraping?
Shrink the personal-data surface to as close to zero as the job allows. Most commercial scraping does not actually need personal data, and the cleanest compliance posture is to not collect it in the first place. Concretely:
- Extract narrowly. Schema your extraction so you pull the price, SKU, spec or company fact you need and leave named contacts, handles and emails on the page.
- Prefer non-personal data. Product, pricing, market and firmographic data at the company level rarely identifies an individual. Aim there.
- Keep provenance. Store the source URL and timestamp per record so that if a subject request or objection arrives, you can find and delete exactly the right data.
- Respect the site's signals. Honor robots.txt and Terms of Service; a site's refusal is part of the reasonable-expectations picture the balancing test weighs.
- Get advice for anything involving EU individuals at scale. Building a contact database of EU residents is the high-risk case, not the exception.
Narrow extraction is both smaller compliance surface and cleaner data. A schema that returns price, currency and in_stock and nothing else is easier to defend and easier to use than a dump of the whole page. This is why extracting structured data to a schema is a compliance feature and not only a convenience, and teams that already map their regulatory obligations to controls will recognize minimization as one of the cheapest controls to build in at collection time.
What about the US side?
The GDPR is not the only privacy regime to watch. US state laws, led by the California Consumer Privacy Act as amended by the CPRA, create their own obligations around collecting and selling personal information about state residents, and more states pass comparable laws every year. The through-line is the same as under the GDPR: personal data carries duties wherever the person is, being public does not remove those duties, and collecting only what you need is the reliable way to stay out of trouble. For the broader legal picture beyond privacy, our guide to whether web scraping is legal covers the CFAA, Terms of Service and copyright, and our compliance policy spells out the boundaries we hold to.
The practical takeaway
US companies do not get a pass on the GDPR by virtue of being American, and they do not get one by virtue of the data being public. If your scraping touches EU residents' personal data, the regulation can reach you, the legal basis is genuinely hard to establish, and the transparency and erasure duties follow the data. The reliable answer is not a clever justification after the fact. It is to design the extraction so you collect non-personal, on-topic data on permitted sources, keep provenance, and bring in counsel before building anything that assembles personal data about individuals at scale.
See ClawEngine turn pages into clean data
Point ClawEngine at any public or permitted site and get back clean markdown, JSON, or typed structured fields in one call. Crawl at scale, render JavaScript, and feed your RAG pipelines and AI agents, robots.txt and Terms of Service respected.