snscrape-jsonl-urls-extractor
extracts urls from jsonl produced by snscrape
Usage
Just python3 the one for the scraper you’re using
Yes, this uses input() on stderr. That’ll be fixed when I switch this to Rust
(because sys.argv is confusing in interpreted languages – sometimes the thing you want is arg 1, sometimes arg 0, sometimes arg 342e+332, I dont want to deal with that.);
until then, deal with it.
(It’ll also be unified into one project once I switch to rust.
Switching to Rust
Will happen when I have access to my desktop again, because F A S T
Dupes
This program seems to produce a LOT of duplicates. Use the sort command with the -u param to fix this; however this will disrupt the order of URLs.
