JustAnotherArchivist
|
42e420ad0d
|
Disable prompts on password-protected repos
|
1 vuosi sitten |
JustAnotherArchivist
|
a9e838adde
|
Raise exception if file already exists in DirectoryStorage target
|
1 vuosi sitten |
JustAnotherArchivist
|
6af07cb51c
|
Raise exceptions on fatal errors
|
1 vuosi sitten |
JustAnotherArchivist
|
2257305872
|
Disallow underscores in module names
Using the preferred file naming scheme of {moduleName}_{someInputURLDerivative}_{date}*, this allows mapping files to modules without ambiguity.
|
1 vuosi sitten |
JustAnotherArchivist
|
4dcac08585
|
Fix import order
|
1 vuosi sitten |
JustAnotherArchivist
|
0f1f5abc64
|
Add indices for files
|
1 vuosi sitten |
JustAnotherArchivist
|
e3da8c7736
|
Use generic alias types
This requires at least Python 3.9.
|
1 vuosi sitten |
JustAnotherArchivist
|
f2d2df9428
|
Simplify storage design; there is no need for the queue
|
1 vuosi sitten |
JustAnotherArchivist
|
550afa8644
|
Add storage abstraction
|
1 vuosi sitten |
JustAnotherArchivist
|
06daea162f
|
Remove GitHub module as it is not ready for use yet
|
1 vuosi sitten |
JustAnotherArchivist
|
240dcceb10
|
Add subprocess wrapper for logging stderr
|
1 vuosi sitten |
JustAnotherArchivist
|
6fb0ac4e5e
|
Initial GitHub module only retrieving the actual repository
|
4 vuotta sitten |
JustAnotherArchivist
|
2a2c9373d0
|
Documentation of the core
|
4 vuotta sitten |
JustAnotherArchivist
|
715420e298
|
Fix imports in CLI: core and modules aren't needed in the argument parser
|
4 vuotta sitten |
JustAnotherArchivist
|
1b73693b37
|
Keep track of and handle errors in modules via metaclass
|
4 vuotta sitten |
JustAnotherArchivist
|
922900ac4e
|
Add support for selecting a module explicitly using `name+` URL prefix
E.g. `git+https://example.org/`
|
4 vuotta sitten |
JustAnotherArchivist
|
22c707c04f
|
Add Module.name attribute
|
4 vuotta sitten |
JustAnotherArchivist
|
90e0af88b9
|
Fix return type of get_module_{class,instance}
No need to quote the class name since the methods are not inside the class (anymore)
|
4 vuotta sitten |
JustAnotherArchivist
|
5f9547d600
|
Get rid of inheritance-level-based module selection and instead raise an exception if there are no or multiple matching modules
|
4 vuotta sitten |
JustAnotherArchivist
|
7e8958b063
|
Allow overriding the archive ID
|
4 vuotta sitten |
JustAnotherArchivist
|
90f80e41a9
|
Add __repr__ methods
|
4 vuotta sitten |
JustAnotherArchivist
|
9f6e5a9f48
|
Move InputURL handling to base Module.__init__ and extract URL string for convenience
|
4 vuotta sitten |
JustAnotherArchivist
|
ca68893a59
|
Run submodules directly within the modules and return results from there instead of processing that externally
|
4 vuotta sitten |
JustAnotherArchivist
|
74a6fc7641
|
Use dataclass instead of namedtuple for module results
|
4 vuotta sitten |
JustAnotherArchivist
|
07dc1927cf
|
Initial commit
A significant part of this code (e.g. the module loading, HTTP retrieval, CLI) was mostly or entirely copied from snscrape.
|
4 vuotta sitten |