A framework for quick web archiving
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
JustAnotherArchivist 820384fe1e Stop deduping small responses 4 years ago
..
__init__.py Catch exceptions in Item.process and mark the items as errors instead of crashing 4 years ago
aiohttp.py Flush big responses to a temporary file instead of trying to keep everything in-memory 4 years ago
cli.py Support deep/multiple inheritance from Item 4 years ago
const.py Catch exceptions in Item.process and mark the items as errors instead of crashing 4 years ago
utils.py Write only successful retrievals (i.e. ones that don't cause an exception) to WARC 4 years ago
version.py Use setuptools_scm for versioning 4 years ago
warc.py Stop deduping small responses 4 years ago