codecov Language grade: Python Run tests view examples PyPI version


Robox is a simple library with a clean interface for exploring/scraping the web or testing a website you’re developing. Robox can fetch a page, click on links and buttons, and fill out and submit forms. Robox is built on top of two excelent libraries: httpx and beautifulsoup4.

Robox has all the standard features of httpx, including async, plus:

  • clean api
  • caching
  • downloading files
  • history
  • understands robots.txt


from robox import Robox

with Robox() as robox:
    page ="")
    form = page.get_form()
    form.fill_in("custname", value="foo")
    form.check("topping", values=["Onion"])
    form.choose("size", option="Medium")
    form.fill_in("comments", value="all good in the hood")
    form.fill_in("delivery", value="13:37")
    page = page.submit_form(form)
    assert page.url == ""

or use async version:

import asyncio

from pprint import pprint
from robox import AsyncRobox

async def main():
    async with AsyncRobox(follow_redirects=True) as robox:
        page = await"")
        form = page.get_form()
        form.fill_in("q", value="python")
        consent_page = await page.submit_form(form)
        form = consent_page.get_form()
        page = await consent_page.submit_form(form)

Caching can be easily configured via httpx-cache

from robox import Robox, DictCache, FileCache

with Robox(cache=DictCache()) as robox:
    p1 ="")
    assert not p1.from_cache
    p2 ="")
    assert p2.from_cache

See examples folder for more detailed examples.


Using pip:

pip install robox

Robox requires Python 3.8+. See Changelog for changes.


View Github