Discover all files, respecting gitignore files along the way
Register all directories traversed to the database and build a directory graph
Then asynchronously using IAsyncEnumerator, with batching and other stuff, for every file discovered:
Hash file, the hash will be used to ensure file contents of identical files are only written to TAR file once, a unique entry of file contents is refered to as a BLOB
Check database if a BLOB has already been registered in the database, if not then write the blob entry to the database
Register the file record with relation to the blob record in the database and set its relation to the directory hierarchy
**Only if** the blob record was inserted into the database, write the file to the tar file with a filename that is the hash of its content (the blob hash)
Discover all files, respecting gitignore files along the way
Register all directories traversed to the database and build a directory graph
Then asynchronously using IAsyncEnumerator, with batching and other stuff, for every file discovered:
Hash file, the hash will be used to ensure file contents of identical files are only written to TAR file once, a unique entry of file contents is refered to as a BLOB
Check database if a BLOB has already been registered in the database, if not then write the blob entry to the database
Register the file record with relation to the blob record in the database and set its relation to the directory hierarchy
**Only if** the blob record was inserted into the database, write the file to the tar file with a filename that is the hash of its content (the blob hash)
tar writer has to wait for DB job before doing its thing because if a BLOB with a matching hash has already been written to the TAR file then there is no use writing it again
