qwarc

Commit Graph

Автор	SHA1	Съобщение	Дата
JustAnotherArchivist	26aab15605	urn:X-qwarc instead of urn:qwarc	преди 4 години
JustAnotherArchivist	50d46ad51c	Use log filename in the target URI of the log resource record	преди 4 години
JustAnotherArchivist	e093211496	Set content type for resource records	преди 4 години
JustAnotherArchivist	ae46b53401	Always write a WARC-Warcinfo-ID header	преди 4 години
JustAnotherArchivist	23fcdd4026	Write microsecond dates for request and response records	преди 4 години
JustAnotherArchivist	3030ad10ab	Mark private API accordingly	преди 4 години
JustAnotherArchivist	e0b4104d21	Remove log handler before writing log record since that requires closing the stream	преди 4 години
JustAnotherArchivist	6cfd352f68	Write WARC/1.1 files	преди 4 години
JustAnotherArchivist	e1ad5c232e	Write warcinfo and resource records in meta WARC on firing up qwarc rather than at the end	преди 4 години
JustAnotherArchivist	f038cf91db	Fix unfound distribution handling	преди 4 години
JustAnotherArchivist	a5dfd5c805	Write spec file + its dependencies and command line to meta WARC	преди 4 години
JustAnotherArchivist	e99e2304c9	Write meta WARC with log file	преди 5 години
JustAnotherArchivist	d751844626	Fix starting another item before stopping on STOP file or memory limit exceedance	преди 5 години
JustAnotherArchivist	2b0778f9b5	Remove leftovers from initial code rewrite	преди 5 години
JustAnotherArchivist	85d78cee13	Add warcinfo record with version information on Python, system, and dependencies	преди 5 години
JustAnotherArchivist	9cff6bd5c1	Only open a WARC file when necessary to avoid producing empty WARCs at the end	преди 5 години
JustAnotherArchivist	21cf784102	Use setuptools_scm for versioning	преди 5 години
JustAnotherArchivist	ab22966fef	Add to log which item a message is coming from	преди 5 години
JustAnotherArchivist	6fafd32685	Error when the retries are exceeded	преди 5 години
JustAnotherArchivist	8647d6b396	Use f-strings instead of str.format	преди 5 години
JustAnotherArchivist	5008e6e8cd	Deduplicate items	преди 5 години
JustAnotherArchivist	46c95e2157	Disable decoding the response content chardet can be very slow (https://github.com/chardet/chardet/issues/29 https://github.com/psf/requests/issues/2359) and the decoding may be unnecessary if it's binary content.	преди 5 години
JustAnotherArchivist	85f6f7bd82	Make qwarc.utils.handle_response_limit_error_retries more useful by passing the deferring handler as an argument	преди 5 години
JustAnotherArchivist	ad22a2327a	Support adding headers to individual requests	преди 5 години
JustAnotherArchivist	67076f964c	Add support for POST requests	преди 5 години
JustAnotherArchivist	2d52e78d85	Fix reference to aiohttp.CientError	преди 5 години
JustAnotherArchivist	c1574a06c9	Fix sleep task type	преди 5 години
JustAnotherArchivist	e0ca88c807	Fix reference to get_rss	преди 5 години
JustAnotherArchivist	984d28ede0	Fix type of --memorylimit, --disklimit, and --warcsplit values	преди 5 години
JustAnotherArchivist	8a8935810d	Fix references to memory and disk space check methods	преди 5 години
JustAnotherArchivist	be5673cfbf	Add record deduplication within a process	преди 5 години
JustAnotherArchivist	e892a6b6a7	Initial commit	преди 5 години

32 Ревизии (26aab156059b8740f88e6c81d85728abe509dceb)