Bright Data
Search, Crawl and Scrape any site, at scale, without getting blocked
0.5.1The Bright Data toolkit connects Arcade to Bright Data's proxy and scraping infrastructure, enabling large-scale web scraping, search engine querying, and structured data extraction without being blocked.
Capabilities
- Web scraping: Fetch any public webpage and receive clean Markdown output, suitable for LLM ingestion or content pipelines.
- Multi-engine search: Query Google, Bing, or Yandex with control over result count, search type (web/images), and country targeting.
- Structured data feeds: Extract pre-structured JSON from major platforms — Amazon products and reviews, LinkedIn profiles and companies, Instagram, Facebook, X (Twitter), YouTube, Zillow, Booking.com, and ZoomInfo — without parsing raw HTML.
Secrets
This toolkit requires two secrets:
-
BRIGHTDATA_API_KEY— Your Bright Data API key, used to authenticate all requests. Obtain it from the Bright Data control panel under Account Settings → API Keys. A paid Bright Data account is required; free trials may have limited access. -
BRIGHTDATA_ZONE— The Bright Data zone (proxy zone or dataset zone) that determines routing, IP pool, and rate limits for requests. Create or manage zones in the Bright Data control panel under Proxies & Scraping Infrastructure. The zone name must match the type of traffic you intend to route (e.g., a Web Unlocker zone for scraping, or a SERP zone for search).
For instructions on configuring secrets in Arcade, see the Arcade secrets guide. You can manage your configured secrets at https://api.arcade.dev/dashboard/auth/secrets.
Available tools(3)
| Tool name | Description | Secrets | |
|---|---|---|---|
Scrape a webpage and return content in Markdown format using Bright Data.
Examples:
scrape_as_markdown("https://example.com") -> "# Example Page
Content..."
scrape_as_markdown("https://news.ycombinator.com") -> "# Hacker News
..."
| 2 | ||
Search using Google, Bing, or Yandex with advanced parameters using Bright Data.
Examples:
search_engine("climate change") -> "# Search Results
## Climate Change - Wikipedia
..."
search_engine("Python tutorials", engine="bing", num_results=5) -> "# Bing Results
..."
search_engine("cats", search_type="images", country_code="us") -> "# Image Results
..."
| 2 | ||
Extract structured data from various websites like LinkedIn, Amazon, Instagram, etc.
NEVER MADE UP LINKS - IF LINKS ARE NEEDED, EXECUTE search_engine FIRST.
Supported source types:
- amazon_product, amazon_product_reviews
- linkedin_person_profile, linkedin_company_profile
- zoominfo_company_profile
- instagram_profiles, instagram_posts, instagram_reels, instagram_comments
- facebook_posts, facebook_marketplace_listings, facebook_company_reviews
- x_posts
- zillow_properties_listing
- booking_hotel_listings
- youtube_videos
Examples:
web_data_feed("amazon_product", "https://amazon.com/dp/B08N5WRWNW")
-> "{"title": "Product Name", ...}"
web_data_feed("linkedin_person_profile", "https://linkedin.com/in/johndoe")
-> "{"name": "John Doe", ...}"
web_data_feed(
"facebook_company_reviews", "https://facebook.com/company", num_of_reviews=50
) -> "[{"review": "...", ...}]" | 2 |