qwarc

484 KiB

Aborescence: f025c4e9f3

Auteur	SHA1	Message	Date
JustAnotherArchivist	f025c4e9f3	Add extensive debug logging	il y a 3 ans
JustAnotherArchivist	b29db245fb	Configurable verbosity for log file and stderr	il y a 3 ans
JustAnotherArchivist	3c8b45b3a6	Refactor cleanup code - Run the cleanup code on exceptions (e.g. ^C). There were several effects of that not happening previously; most notably, the log file was not written to the meta WARC. - Cancel remaining tasks, which avoids a pile of asyncio warnings and errors on crashes. - Close the DB before the WARC, or rather, close the WARC last. This is mostly a semantic change to further ensure that the log written to the meta WARC is as complete as possible.	il y a 3 ans
JustAnotherArchivist	6bdcfe71f0	Refactor database creation and item generation: call `Item.generate()` on every qwarc run and dedupe its output, allowing the addition of further items by modifying the spec file	il y a 3 ans
JustAnotherArchivist	08d96b37c5	Support deep/multiple inheritance from Item	il y a 4 ans
JustAnotherArchivist	50d46ad51c	Use log filename in the target URI of the log resource record	il y a 4 ans
JustAnotherArchivist	a5dfd5c805	Write spec file + its dependencies and command line to meta WARC	il y a 4 ans
JustAnotherArchivist	e99e2304c9	Write meta WARC with log file	il y a 4 ans
JustAnotherArchivist	21cf784102	Use setuptools_scm for versioning	il y a 4 ans
JustAnotherArchivist	ab22966fef	Add to log which item a message is coming from	il y a 4 ans
JustAnotherArchivist	8647d6b396	Use f-strings instead of str.format	il y a 4 ans
JustAnotherArchivist	984d28ede0	Fix type of --memorylimit, --disklimit, and --warcsplit values	il y a 5 ans
JustAnotherArchivist	be5673cfbf	Add record deduplication within a process	il y a 5 ans
JustAnotherArchivist	e892a6b6a7	Initial commit	il y a 5 ans

14 Révisions (f025c4e9f36a661c44773fc60bba32f8ec1300e2)