JustAnotherArchivist
|
73f35f5591
|
Fix infinite loop when file ends with something that is not a WARC record
|
преди 2 години |
JustAnotherArchivist
|
06d60a798c
|
Bump read size
|
преди 2 години |
JustAnotherArchivist
|
74485c399b
|
Require decompressed WARCs with warc-tiny
|
преди 2 години |
JustAnotherArchivist
|
fe0b020352
|
Add support for reading from stdin
|
преди 2 години |
JustAnotherArchivist
|
5b731fbde1
|
Fix compatibility with wpull 2.x
|
преди 3 години |
JustAnotherArchivist
|
743e0582ba
|
Fix confusing error message when lxml is not installed
|
преди 3 години |
JustAnotherArchivist
|
491a80a04b
|
Add warc-tiny scrape command for parsing HTTP responses using wpull and extracting links
|
преди 3 години |
JustAnotherArchivist
|
01274e461a
|
Prevent constantly moving bytes around for better performance on large chunked records
|
преди 3 години |
JustAnotherArchivist
|
4c90bacaed
|
Shield values in colons with angled brackets
|
преди 4 години |
JustAnotherArchivist
|
f51adccd3f
|
Add --meta mode for dump-responses which prefixes each line with information about the file and record
|
преди 4 години |
JustAnotherArchivist
|
9cc1f41917
|
Pass the filename in NewFile events
|
преди 4 години |
JustAnotherArchivist
|
a38efc31b6
|
Introduce a way to provide additional arguments to processors
|
преди 4 години |
JustAnotherArchivist
|
49376db51b
|
Decode HTTP request bodies
|
преди 4 години |
JustAnotherArchivist
|
34c1a58034
|
Fix detection of multiple transfer encodings
|
преди 4 години |
JustAnotherArchivist
|
5982e131a4
|
Stop gracefully when encountering a SIGPIPE
|
преди 4 години |
JustAnotherArchivist
|
c13a1150df
|
Add support for WARC/1.1
|
преди 4 години |
JustAnotherArchivist
|
376cde7b8c
|
Fix broken block digest calculation on malformed HTTP responses
|
преди 4 години |
JustAnotherArchivist
|
b121cbd958
|
Write all log messages to stderr
|
преди 4 години |
JustAnotherArchivist
|
ed1270d988
|
Add support for upper-cased chunk lengths
|
преди 4 години |
JustAnotherArchivist
|
d4826abde2
|
Add record ID to log messages
|
преди 4 години |
JustAnotherArchivist
|
552a4147c2
|
Fix not returning complete body for non-chunked responses
Leftover from debugging
|
преди 5 години |
JustAnotherArchivist
|
f2e836d2e9
|
Add support for differently formatted digests
|
преди 5 години |
JustAnotherArchivist
|
94c4f76570
|
Fix crash when a digest is missing from a record
|
преди 5 години |
JustAnotherArchivist
|
ef78a3318c
|
Colour only the header field names but not the values
|
преди 5 години |
JustAnotherArchivist
|
9ce4653094
|
Document colouring and usage
|
преди 5 години |
JustAnotherArchivist
|
e7c5d82254
|
Coloured WARCs?!
|
преди 5 години |
JustAnotherArchivist
|
70b413f5c1
|
Better events: include raw WARC header data and separate HTTP requests into headers and body
|
преди 5 години |
JustAnotherArchivist
|
641bc7a207
|
Fix infinite loop at end of WARC
|
преди 5 години |
JustAnotherArchivist
|
859c75a591
|
Add tool for WARC verification and extraction
|
преди 5 години |