ClawEngine.ai
All posts
Compliance

Is Scraping Competitor Prices Legal? A US Guide for Pricing Teams

Collecting public competitor prices is broadly lawful in the US, but the risk sits in Terms of Service, copyright and antitrust rather than the CFAA. What Van Buren, hiQ and Meta v. Bright Data actually mean for a pricing program.

By the ClawEngine team

July 2026 · 10 min read

Live Extraction
GET
try:

Hit Extract to turn this page into clean, LLM-ready data.

robots.txt respected · public data only

Markdown · JSON · structured fields, from one API call. Crawling, rendering and extracting ...

The short answer

Collecting publicly displayed competitor prices is broadly lawful in the United States. Courts have repeatedly declined to treat access to public web pages as unauthorized access under the Computer Fraud and Abuse Act, so a script reading a price anyone can see is treated much like an employee writing it down by hand. The risk is not the federal computer crime statute. It sits in three quieter places: the site's Terms of Service as a contract claim, copyright in the content you copy, and antitrust law governing what you do with the prices once you have them. This is general information and not legal advice.

Why the CFAA usually is not the problem

For years the Computer Fraud and Abuse Act was the thing every scraping conversation feared, because "exceeds authorized access" sounded broad enough to cover almost anything. Two decisions narrowed it substantially.

In Van Buren v. United States (2021), the Supreme Court held that the "exceeds authorized access" clause applies when someone obtains information from areas of a computer they are not entitled to reach at all, not when someone with legitimate access uses the data for an improper purpose. That rejected the reading under which violating a website's terms could turn ordinary browsing into a federal crime.

In hiQ Labs v. LinkedIn, the Ninth Circuit concluded that scraping publicly available profile data, information available to anyone with a browser and no login, did not amount to access "without authorization" under the CFAA. Worth knowing how that story finished, though: hiQ ultimately lost on LinkedIn's breach of contract claim, because it had agreed to LinkedIn's user agreement. The public data was not the problem. The contract was.

The pattern held in Meta v. Bright Data, decided in early 2024, where the court found Bright Data could only have breached Meta's terms if the scraping happened while logged into an account, and Meta had not shown that. Logged-out collection of public pages survived.

Read together, the line these cases draw is remarkably practical. Public and logged-out is defensible. Authenticated, paywalled or access-controlled is where the real exposure begins.

The three risks that actually apply to price data

1. Terms of Service, as a contract

This is the most common way price scraping goes wrong, and it has nothing to do with computer crime law. If you accepted a site's terms, by registering an account, clicking through, or in some cases by continuing to use a site after clear notice, those terms may bind you as a contract. Many retailer terms prohibit automated collection outright. Breaching them is a civil matter rather than a criminal one, but injunctions and damages are still expensive.

The practical consequence is a rule worth writing into your process: never scrape a site with credentials you agreed to terms for. Reading a competitor's public product page in a logged-out session is a very different posture from logging into their reseller portal.

2. Copyright in what you copy

A price is a fact, and facts are not copyrightable. Neither is a purely mechanical listing of them. Product descriptions, editorial copy, curated specification write-ups and photography are a different matter, and those are copyrighted by whoever produced them.

So the distinction that matters is collect versus republish. Extracting prices to inform your own pricing decisions is on solid ground. Copying a competitor's product descriptions and photos onto your own storefront is not, regardless of how you obtained them. Keep the internal analytical use and the public-facing use in separate buckets, and be careful about what leaves the first bucket.

3. Antitrust, which is the one people forget

Here is the risk unique to price data, and it is about what you do with the numbers rather than how you got them.

Unilaterally observing public prices and setting your own accordingly is normal competitive behavior. Agreeing with a competitor about what prices to set is price fixing under Section 1 of the Sherman Act, and it does not stop being price fixing because the agreement was implemented in software. Regulators have been increasingly attentive to algorithmic pricing where competitors feed non-public data into a shared system and adjust in coordinated ways.

Practical guardrails that keep a monitoring program clean: collect only publicly displayed prices, keep collection unilateral rather than pooled with competitors, do not exchange your own future pricing intentions with rivals, and be careful with third-party pricing tools that ingest confidential data from multiple competitors and hand back recommendations. If your program touches that last pattern, involve counsel before it ships, not after.

Where MAP monitoring sits

Monitoring your own resellers for minimum advertised price violations is a distinct case and generally a comfortable one. You are collecting public prices, and a MAP policy that a manufacturer sets and enforces unilaterally has been evaluated under the rule of reason since Leegin Creative Leather Products v. PSKS (2007) rather than being treated as automatically unlawful.

The nuance is the difference between a policy and an agreement. A unilateral policy that you announce and enforce by declining to do business with violators is one thing. Negotiating resale prices with resellers, or coordinating enforcement across competing manufacturers, is another. That distinction is your counsel's to draw. The data collection side, extracting advertised prices from public reseller pages on a schedule, is the straightforward part. We walk through the mechanics in MAP monitoring and detecting price violations.

What compliant price collection looks like in practice

Do Do not
Collect prices from public, logged-out pagesLog into a competitor or reseller portal to collect
Read and honor robots.txtCrawl paths a site has disallowed
Identify yourself in the User-AgentRotate identities to evade blocking
Rate limit and honor crawl-delayHammer a host until it degrades
Store prices and factsRepublish descriptions or photography
Keep source URL and timestamp per recordKeep a bare price with no provenance
Keep pricing decisions unilateralShare or coordinate pricing with competitors
Stop when a site clearly refuses automationEscalate against anti-bot systems

Provenance is a compliance feature, not a nice-to-have

One habit does more for your legal position than any other: record where every number came from and when. A row that carries the source URL, the exact fetch timestamp and the price as displayed lets you demonstrate that a price was public at the time, respond to a takedown or correction request precisely, and delete a specific source's data cleanly if you ever need to.

It also happens to be the thing that makes the data useful. A price with no timestamp is an assertion. A price with a URL and a UTC timestamp is evidence, which is exactly what a reseller enforcement conversation or an internal audit needs. Teams that already track their obligations and map them to controls will recognize the pattern: the audit trail is the control, and building it in at collection time costs almost nothing while retrofitting it costs a rewrite.

Anti-bot systems are an answer, not an obstacle

A tempting reading of the case law is that since public data is fair game, defeating whatever stands between you and public data must also be fine. That is the wrong lesson. When a site deploys aggressive bot detection, it is stating its position clearly, and escalating past it moves you from "reading public pages" toward conduct that supports trespass to chattels and contract claims, and gives a plaintiff a much more sympathetic story.

This is why we do not target large marketplaces, and why we say so plainly instead of selling coverage that erodes. There is far more durable value in the open retail web that permits crawling: direct-to-consumer brand sites, specialty and regional retailers, distributor catalogs and authorized reseller storefronts. That is where most US pricing decisions are actually made. Our compliance policy spells out the boundaries we hold to.

What about prices outside the United States?

The analysis above is US-specific. If you collect prices from EU or UK sites, the GDPR becomes relevant even for commercial data whenever anything personal rides along, a named seller on a marketplace listing, a reviewer's handle, a contact name in a business listing. The safest posture is to schema your extraction narrowly so you pull the price, currency, stock and SKU and nothing that identifies a person. Narrow extraction is both cleaner data and a smaller compliance surface.

A five-line policy your team can actually follow

Most legal exposure in price monitoring comes from drift rather than from a decision anyone made. Write the rules down once and the program stays inside them.

  • Public and logged-out only. No credentials, no paywalls, no portals.
  • robots.txt is binding. Disallowed paths are not collected, ever.
  • Facts in, content out. Prices, stock and identifiers are stored. Descriptions and images are not republished.
  • Every record carries its URL and timestamp. No provenance, no row.
  • No coordination. Pricing decisions are unilateral and pricing intentions are never shared with rivals.

If you want the collection half of that handled with the boundaries enforced by default, the price monitoring API reads robots.txt, stays on public pages, and returns price, promo, stock and seller as typed JSON with the source URL attached. For the broader legal picture beyond pricing, start with our guide to whether web scraping is legal.

See ClawEngine turn pages into clean data

Point ClawEngine at any public or permitted site and get back clean markdown, JSON, or typed structured fields in one call. Crawl at scale, render JavaScript, and feed your RAG pipelines and AI agents, robots.txt and Terms of Service respected.

Turn any site into LLM-ready data

ClawEngine crawls public and permitted sites, renders JavaScript, and returns clean markdown, JSON, or typed structured fields in one call, ready for your RAG pipelines and AI agents.

Clean markdown in one call · JavaScript rendered · robots.txt respected

Public and permitted data only · respects robots.txt & Terms of Service · you are responsible for what you crawl.