VisualBs4Scraper

Build a visual bs4 scraper for websites

Paste a URL, pick the elements you care about right on the live page, and get back working BeautifulSoup code — no manual selector hunting.

View on GitHub Quickstart Frontend docs Backend docs

How it works

01

Paste a URL

The backend fetches and bundles the page so it renders safely in a live preview, styling intact.

02

Pick elements

Click a header, a text block, a link list, a table, or one row of a repeating list — right on the rendered page.

03

Get Python back

A real, runnable requests + BeautifulSoup script, generated from what you picked.

Screenshots

VisualBs4Scraper landing page
Landing page
Picking an element on example.com and getting a generated BeautifulSoup function back
Pick an element, get a working Python function back
Picking one row of a jobs table selects and highlights every matching row as a group
Picking one row of a list selects the whole matching group

Quickstart

Terminal
# backend
cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python Controller/app.py

# frontend (in another terminal)
cd front
npm install
npm run dev

What it's built on

Frontend

React + TypeScript + Vite, styled with Tailwind. Click-to-pick element selection, a visual function builder, and a live BeautifulSoup code generator.

Backend

Flask + BeautifulSoup (html5lib). Fetches and bundles pages for the preview, proxies their assets, and keeps parsing consistent with what a browser renders.