Scraping Browser - Quick Start Guide

1. Getting Started

Step 1: Register & Purchase Proxy Traffic

Sign up at the NodeMaven Dashboard.
Purchase proxy traffic to start using Scraping Browser.

Step 2: Generate Your Connection URL

Scraping Browser uses a WebSocket connection URL to integrate with web automation tools like Puppeteer, Playwright. (Selenium will be supported in a near future).

Connection URL Format:

wss://username_country-{country}-sid-{sid}-pid-{pid}:[email protected]

Step 3: Understanding URL Parameters

Parameter	Description
`username`	Your NodeMaven username (same as your proxy username).
`password`	Your NodeMaven password (same as your proxy password).
`country` (Required)	Selects an IP from a specific country.
`region, city, isp` (Optional)	Further refine proxy targeting (e.g., `region-new_york`).
`type-mobile` (Optional)	Flag that enables to use mobile IPs only.
`sid` (Optional)	The session ID (`sid`) can be any random string of characters. It enables proxy session persistence by keeping the same IP for multiple requests. Remove `sid-{sid}` to get a new IP on each request. The session ID can be any random string of characters.
`pid` (Optional)	Profile ID (`pid`) determines which set of fingerprint settings the browser should use. Currently, our software automatically generates the most suitable fingerprint settings and supports customization.

Learn More About Connection Parameters

2. Making Your First Connection

Run these basic code examples to test Scraping Browser (replace example of connection string with your actual Browser Connection URL):

NodeJS - Puppeteer

const puppeteer = require("puppeteer");

(async () => {
console.log("Connecting to browser...");
const browser = await puppeteer.connect({
browserWSEndpoint:
"wss://username_country-{country}-sid-{sid}-pid-{pid}:[email protected]",
});
console.log("Browser connected! Opening a new page...");

const page = await browser.newPage();
await page.goto("https://example.com");
console.log("The page is open! The title is:" + (await page.title()));
await browser.close();
})();

NodeJs - Playwright

const { chromium } = require("playwright");

(async () => {
console.log("Connecting to the browser...");
const browser = await chromium.connectOverCDP(
"wss://username_country-{country}-sid-{sid}-pid-{pid}:[email protected]"
);
console.log("Connected to the browser! Opening a new page...");

const page = await browser.newPage();
await page.goto("https://example.com");
console.log("The page is open! The title is:", await page.title());
await browser.close();
})();

3. Viewing Live Browser Sessions in Chrome DevTools

Scraping Browser supports real-time debugging via Chrome DevTools to help monitor script execution and troubleshoot issues.

To open DevTools for every session, you can integrate this code snippet:

const { chromium } = require('playwright');
const { exec } = require('child_process');

const chromeExecutable = '/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome'; // Adjust for your OS (e.g., "chrome.exe" on Windows)
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

const openDevtools = async (page, client) => {
    // Get the main frame ID
    const { frameTree } = await client.send('Page.getFrameTree', {});
    const frameId = frameTree.frame.id;

    // Get URL for DevTools from Scraping Browser
    const { url: inspectUrl } = await client.send('Page.inspect', { frameId });

    if (!inspectUrl) {
        console.error("No DevTools URL received. DevTools could not be opened.");
        return;
    }

    // Open DevTools URL in local Chrome
    await exec(`"${chromeExecutable}" "${inspectUrl}"`, error => {
        if (error) console.error('Unable to open DevTools:', error);
    });

    // Wait for DevTools Interface to load
    await delay(10000);
};

(async () => {
    const browser = await chromium.connectOverCDP('wss://your_browser_connection_url');
    const page = await browser.newPage();
    const client = await page.context().newCDPSession(page);

    await openDevtools(page, client);
    await page.goto('http://example.com');

    console.log("Page loaded:", await page.title());
    await delay(10000);
    await browser.close();
})();

4. Important Limits & Usage Notes

Limit	Details
Max Amount of One-Time sessions	Currently, we support an unlimited number of simultaneous one-time profiles. A one-time profile can be used only once and is then deleted. If you need persistent profiles with long-term sessions, contact us!
Max Amount of real-time debugging via Chrome DevTools	We support opening Chrome DevTools for each browser tab.

Puppeteer

Playwright

Introduction to Scraping Browser