The little things give you away... A collection of various small helper stuff
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 

17 lines
862 B

  1. #!/bin/bash
  2. # Check how much of an upload made it into the item yet (vs. how much is stuck in the S3 queue if you also watch the upload process)
  3. # Usage: ia-upload-progress IA_IDENTIFIER [FILES]
  4. # If no files are specified, all WARCs in the current directory are used.
  5. identifier=$1
  6. shift
  7. files=("$@")
  8. if [[ ${#files[@]} -eq 0 ]]
  9. then
  10. files=(*.warc.gz)
  11. fi
  12. scriptpath="$(cd "$(dirname "$0")"; pwd -P)"
  13. uploaded=$(ia metadata "${identifier}" | grep -Po '("source"\s*:\s*"original",[^}]*"size"\s*:\s*"\K\d+|"size"\s*:\s*"\K\d+(?="\s*,[^}]*"source"\s*:\s*"original"))' | awk '{sum+=$1} END {printf "%.0f\n", sum}')
  14. localSize=$(du -bc "${files[@]}" | tail -1 | cut -f1)
  15. echo "Uploaded $("${scriptpath}/format-size" <<<"${uploaded}") of $("${scriptpath}/format-size" <<<"${localSize}") ($(awk '{ printf "%.1f", 100 * ($1 / $2); }' <<<"${uploaded} ${localSize}") %)"