229 Commits (master)
 

Author SHA1 Message Date
  JustAnotherArchivist e3b77bf322 Ignore TLS issues 1 week ago
  JustAnotherArchivist d915a18a40 Pass through datetime, math, re, and time to --pyfilter 1 week ago
  JustAnotherArchivist 5486b6a224 Add --grafana-eta 1 week ago
  JustAnotherArchivist 885001aab6 Fix --no-table help text (leftover from pre-pure-Python) 1 week ago
  JustAnotherArchivist fd8d502266 Nit: replace double quotes with single quotes 1 week ago
  JustAnotherArchivist ff096bf8dd Fix display of zero timestamps 1 week ago
  JustAnotherArchivist db3e79b9c9 Print repository URLs instead of names 1 week ago
  JustAnotherArchivist 22744fe908 Add script for listing repos of a user or group on GitLab.com 1 week ago
  JustAnotherArchivist 5678b586ac Add script for requeueing skipped URLs due to too many failed attempts on wpull crawls 2 weeks ago
  JustAnotherArchivist f05a8a79bc Clean up wpull DB commands 2 weeks ago
  JustAnotherArchivist cbebafe588 Colourise sha1sum output 1 month ago
  JustAnotherArchivist 18a3305e79 Fix handling of filenames with spaces and ampersands 1 month ago
  JustAnotherArchivist 788b25707d Handle more domains and case variations 1 month ago
  JustAnotherArchivist 36aa2e8259 Add archivebot-log-extract-ignores 1 month ago
  JustAnotherArchivist 5b731fbde1 Fix compatibility with wpull 2.x 2 months ago
  JustAnotherArchivist 743e0582ba Fix confusing error message when lxml is not installed 2 months ago
  JustAnotherArchivist 491a80a04b Add warc-tiny scrape command for parsing HTTP responses using wpull and extracting links 2 months ago
  JustAnotherArchivist fd2728f1b8 Add archivebot-irccloud-paste 2 months ago
  JustAnotherArchivist 4eff3c3eb3 Refactor, strip query/fragment 2 months ago
  JustAnotherArchivist 821cacf626 Add --help 2 months ago
  JustAnotherArchivist caffebab2e Add parent-urls 2 months ago
  JustAnotherArchivist 77ec76bc04 Add --urls and --nodl options 3 months ago
  JustAnotherArchivist 06cf71f73d Fix gofile.io download: getServer is not used by the website anymore, and getUpload no longer returns the MD5 3 months ago
  JustAnotherArchivist bff1490871 Add github-list-repos 3 months ago
  JustAnotherArchivist bf695d63a3 Fix channel URLs 3 months ago
  JustAnotherArchivist dde4464555 Cover two more rare URLs 4 months ago
  JustAnotherArchivist bbf2d2c315 Be more lenient regarding slashes to catch things with collapsed URLs in paths etc. 4 months ago
  JustAnotherArchivist 362f66eb26 Handle youtube-nocookie.com and fix removenonyt mode not recognising CC domains 4 months ago
  JustAnotherArchivist 81e2b4b999 Refine patterns 4 months ago
  JustAnotherArchivist 9974d4613c Stop trying to rewrite patterns for percent encoding 4 months ago
  JustAnotherArchivist 0ee83bc0f2 Refactor 4 months ago
  JustAnotherArchivist b66260ca94 Add youtube-extract 4 months ago
  JustAnotherArchivist d82dff8b71 Add ETA column 4 months ago
  JustAnotherArchivist 01274e461a Prevent constantly moving bytes around for better performance on large chunked records 5 months ago
  JustAnotherArchivist 77d9f61de0 Colourise output 5 months ago
  JustAnotherArchivist 6512669cfd Refactor and compare file list as well 5 months ago
  JustAnotherArchivist 8e0cb30d0a Add atdash mode 5 months ago
  JustAnotherArchivist 5fe595d71c Record wrapper script in meta WARC as well 6 months ago
  JustAnotherArchivist c1def0e7a8 Fix S3_WITH_LIST_URLS being defined (but empty) when --with-list-urls is not used 6 months ago
  JustAnotherArchivist 398cbfdcda Add s3-bucket-list-qwarc, rewritten s3-bucket-list on top of qwarc 6 months ago
  JustAnotherArchivist 80084e0d35 Another alternative and performance/memory comparison 6 months ago
  JustAnotherArchivist 6a288a6338 Use grep instead, which is faster but uses more memory 6 months ago
  JustAnotherArchivist 4d274e64e0 Add dedupe 6 months ago
  JustAnotherArchivist a4af8e6ca6 Add IE6 UA 6 months ago
  JustAnotherArchivist ac277437a3 Add Googlebot UA 6 months ago
  JustAnotherArchivist 0181e53f01 Treat NXDOMAIN and no A/AAAA record errors as ok 7 months ago
  JustAnotherArchivist 41c2a9d2d4 Add support for alternative xmlns 7 months ago
  JustAnotherArchivist 830e9dbc43 Treat redirects as successful retrievals 7 months ago
  JustAnotherArchivist 7a999c9b0a Ignore redirects 7 months ago
  JustAnotherArchivist 579d589853 Add a script to extract errors from wpull 2.x logs 7 months ago