1. Getting Started
Step 1: Register & Purchase Proxy Traffic
Sign up at the NodeMaven Dashboard.
Purchase proxy traffic to start using Scraping Browser.
Step 2: Generate Your Connection URL
Scraping Browser uses a WebSocket connection URL to integrate with web automation tools like Puppeteer, Playwright. (Selenium will be supported in a near future).
Connection URL Format:
wss://username-country-{country}-sid-{sid}-pid-{pid}:[email protected]Step 3: Understanding URL Parameters
Parameter | Description |
| Your NodeMaven username (same as your proxy username). |
| Your NodeMaven password (same as your proxy password). |
| Selects an IP from a specific country. |
| Further refine proxy targeting (e.g., |
| Flag that enables to use mobile IPs only. |
| The session ID (
Remove |
| Profile ID ( |
2. Making Your First Connection
Run these basic code examples to test Scraping Browser (replace example of connection string with your actual Browser Connection URL):
NodeJS - Puppeteer
const puppeteer = require("puppeteer");
(async () => {
console.log("Connecting to browser...");
const browser = await puppeteer.connect({
browserWSEndpoint:
"wss://username_country-{country}-sid-{sid}-pid-{pid}:[email protected]",
});
console.log("Browser connected! Opening a new page...");
const page = await browser.newPage();
await page.goto("https://example.com");
console.log("The page is open! The title is:" + (await page.title()));
await browser.close();
})();NodeJs - Playwright
const { chromium } = require("playwright");
(async () => {
console.log("Connecting to the browser...");
const browser = await chromium.connectOverCDP(
"wss://username_country-{country}-sid-{sid}-pid-{pid}:[email protected]"
);
console.log("Connected to the browser! Opening a new page...");
const page = await browser.newPage();
await page.goto("https://example.com");
console.log("The page is open! The title is:", await page.title());
await browser.close();
})();3. Viewing Live Browser Sessions in Chrome DevTools
Scraping Browser supports real-time debugging via Chrome DevTools to help monitor script execution and troubleshoot issues.
To open DevTools for every session, you can integrate this code snippet:
const { chromium } = require('playwright');
const { exec } = require('child_process');
const chromeExecutable = '/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome'; // Adjust for your OS (e.g., "chrome.exe" on Windows)
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const openDevtools = async (page, client) => {
// Get the main frame ID
const { frameTree } = await client.send('Page.getFrameTree', {});
const frameId = frameTree.frame.id;
// Get URL for DevTools from Scraping Browser
const { url: inspectUrl } = await client.send('Page.inspect', { frameId });
if (!inspectUrl) {
console.error("No DevTools URL received. DevTools could not be opened.");
return;
}
// Open DevTools URL in local Chrome
await exec(`"${chromeExecutable}" "${inspectUrl}"`, error => {
if (error) console.error('Unable to open DevTools:', error);
});
// Wait for DevTools Interface to load
await delay(10000);
};
(async () => {
const browser = await chromium.connectOverCDP('wss://your_browser_connection_url');
const page = await browser.newPage();
const client = await page.context().newCDPSession(page);
await openDevtools(page, client);
await page.goto('http://example.com');
console.log("Page loaded:", await page.title());
await delay(10000);
await browser.close();
})();
4. Important Limits & Usage Notes
Limit | Details |
Max Amount of One-Time sessions | Currently, we support an unlimited number of simultaneous one-time profiles. A one-time profile can be used only once and is then deleted. If you need persistent profiles with long-term sessions, contact us! |
Max Amount of real-time debugging via Chrome DevTools | We support opening Chrome DevTools for each browser tab.
|