aboutsummaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
2017-10-21Revert most of "fusefrontend: clamp oversized reads"Jakob Unterwurzacher
We cannot return less data than requested to the kernel! From https://libfuse.github.io/doxygen/structfuse__operations.html: Read should return exactly the number of bytes requested except on EOF or error, otherwise the rest of the data will be substituted with zeroes. Reverts commit 3009ec9852316c3c696f77f476390ab5a6d8d6d7 minus the formatting improvements we want to keep. Fixes https://github.com/rfjakob/gocryptfs/issues/147 Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-19contentenc: reserve one additional block in CReqPoolJakob Unterwurzacher
...to account for unaligned reads. I have not seen this happen in the wild because the kernel always seems to issue 4k-aligned requests. But the cost of the additional block in the pool is low and prevents a buffer overrun panic when an unaligned read does happen.
2017-10-17fusefrontend: clamp oversized readsJakob Unterwurzacher
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the running kernel may have a higher limit set. Clamp to what we can handle. Fixes a panic on a Synology NAS reported at https://github.com/rfjakob/gocryptfs/issues/145
2017-10-01fusefrontend_reverse: fix 176-byte namesJakob Unterwurzacher
A file with a name of exactly 176 bytes length caused this error: ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY: No such file or directory ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name: No such file or directory -????????? ? ? ? ? ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY -????????? ? ? ? ? ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name Root cause was a wrong shortNameMax constant that failed to account for the obligatory padding byte. Fix the constant and also expand the TestLongnameStat test case to test ALL file name lengths from 1-255 bytes. Fixes https://github.com/rfjakob/gocryptfs/issues/143 .
2017-09-17siv_aead: fix trivial typo in commentJakob Unterwurzacher
2017-09-17contentenc: deduplicate AD packing into new concatAD() funcJakob Unterwurzacher
The encrypt and decrypt path both had a copy that were equivalent but ordered differently, which was confusing. Consolidate it in a new dedicated function.
2017-09-17contentenc: DecryptBlocks: give block number counter a clearer nameJakob Unterwurzacher
Using firstBlockNo as the counter is confusing, create a copy named "blockNo" and use that.
2017-09-05macos: automatically remove .DS_Store on RmdirJakob Unterwurzacher
MacOS sprinkles .DS_Store files everywhere. This is hard to avoid for users, so handle it transparently in Rmdir(). Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05fusefrontend: reorder logic in Rmdir to get rid of one indentation levelJakob Unterwurzacher
Handle the errors first so that the normal code path is not indented. This should not cause any behavoir changes.
2017-09-05macos: don't throw IO errors because of .DS_Store filesJakob Unterwurzacher
MacOS creates lots of these files, and if the directory is otherwise empty, we would throw an IO error to the unsuspecting user. With this patch, we log a warning, but otherwise pretend we did not see it. Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-03syscallcompat: Getdents: warn once if we get DT_UNKNOWNJakob Unterwurzacher
...and if Getdents is not available at all. Due to this warning I now know that SSHFS always returns DT_UNKNOWN: gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat This behavoir is confirmed at http://ahefner.livejournal.com/16875.html: "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03fusefrontend: use DirIVCache in OpenDir()Jakob Unterwurzacher
Previously, OpenDir() did not use the cache at all, missing an opportunity to speed up repeated directory reads.
2017-09-03dirivcache: add better function comments + a sanity check on Store()Jakob Unterwurzacher
The comments were unclear on whether relative or absolute paths have to be passed.
2017-08-21Fix misspellings reported by goreportcard.comJakob Unterwurzacher
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-16cryptocore: add urandom + randprefetch benchmarksJakob Unterwurzacher
The benchmark that supported the decision for 512-byte prefetching previously lived outside the repo. Let's add it where it belongs so it cannot get lost.
2017-08-15fusefrontend: use Getdents if availableJakob Unterwurzacher
Getdents avoids calling Lstat on each file.
2017-08-15syscallcompat: implement Getdents()Jakob Unterwurzacher
The Readdir function provided by os is inherently slow because it calls Lstat on all files. Getdents gives us all the information we need, but does not have a proper wrapper in the stdlib. Implement the "Getdents()" wrapper function that calls syscall.Getdents() and parses the returned byte blob to a fuse.DirEntry slice.
2017-08-11main: purge masterkey from memory as soon as possibleJakob Unterwurzacher
Remove the "Masterkey" field from fusefrontend.Args because it should not be stored longer than neccessary. Instead pass the masterkey as a separate argument to the filesystem initializers. Then overwrite it with zeros immediately so we don't have to wait for garbage collection. Note that the crypto implementation still stores at least a masterkey-derived value, so this change makes it harder, but not impossible, to extract the encryption keys from memory. Suggested at https://github.com/rfjakob/gocryptfs/issues/137
2017-08-09nametransform: extend diriv cache to 100 entriesJakob Unterwurzacher
* extend the diriv cache to 100 entries * add special handling for the immutable root diriv The better cache allows to shed some complexity from the path encryption logic (parent-of-parent check). Mitigates https://github.com/rfjakob/gocryptfs/issues/127
2017-08-06nametransform: add Dir() functionJakob Unterwurzacher
Dir is like filepath.Dir but returns "" instead of ".". This was already implemented in fusefrontend_reverse as saneDir(). We will need it in nametransform for the improved diriv caching.
2017-08-06nametransform: move diriv cache into it's own packageJakob Unterwurzacher
Needs some space to grow. renamed: internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
2017-08-06nametransform: deduplicate code to encryptAndHashName()Jakob Unterwurzacher
This operation has been done three time by identical sections of code. Create a function for it.
2017-07-29fusefronted_reverse: fix ino collision between .name and .diriv filesJakob Unterwurzacher
A directory with a long name has two associated virtual files: the .name file and the .diriv files. These used to get the same inode number: $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name With this change we use another prefix (2 instead of 1) for .name files. $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 2000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
2017-07-27fusefrontend_reverse: return ENOENT for undecryptable namesJakob Unterwurzacher
This was working until DecryptName switched to returning EBADMSG instead of EINVAL. Add a test to catch the regression next time.
2017-07-14macos: make testing without openssl work properlyJakob Unterwurzacher
On MacOS, building and testing without openssl is much easier. The tests should skip tests that fail because of missing openssl instead of aborting. Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14stupidgcm: fix openssl 1.1 build failureJakob Unterwurzacher
Fixed by including the correct header. Should work on older openssl versions as well. Error was: locking.go:21: undefined reference to `CRYPTO_set_locking_callback'
2017-07-11fusefronted: enable writing to write-only filesJakob Unterwurzacher
Due to RMW, we always need read permissions on the backing file. This is a problem if the file permissions do not allow reading (i.e. 0200 permissions). This patch works around that problem by chmod'ing the file, obtaining a fd, and chmod'ing it back. Test included. Issue reported at: https://github.com/rfjakob/gocryptfs/issues/125
2017-07-02contentenc: MergeBlocks: short-circuit the trivial caseJakob Unterwurzacher
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-07-02fusefrontend: doRead: skip decryption for an empty readJakob Unterwurzacher
Previously we ran through the decryption steps even for an empty ciphertext slice. The functions handle it correctly, but returning early skips all the extra calls. Speeds up the tar extract benchmark by about 4%.
2017-07-01stupidgcm: add test for in-place OpenJakob Unterwurzacher
Adds a test for the optimization introduced in: stupidgcm: Open: if "dst" is big enough, use it as the output buffer
2017-06-30contentenc: add PReqPool and use it in DecryptBlocksJakob Unterwurzacher
This gets us a massive speed boost in streaming reads.
2017-06-30stupidgcm: Open: if "dst" is big enough, use it as the output bufferJakob Unterwurzacher
This means we won't need any allocation for the plaintext.
2017-06-30fusefrontend: doRead: use CReqPool for ciphertext bufferJakob Unterwurzacher
Easily saves lots of allocations.
2017-06-30fusefrontend: Read: use provided bufferJakob Unterwurzacher
This will allow us to return internal buffers to a pool.
2017-06-29contentenc: add safer "bPool" pool variant; add pBlockPoolJakob Unterwurzacher
bPool verifies the lengths of slices going in and out. Also, add a plaintext block pool - pBlockPool - and use it for decryption.
2017-06-29stupidgcm: use "dst" as the output buffer it is big enoughJakob Unterwurzacher
This saves an allocation of the ciphertext block.
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-18main, syscallcompat: use Dup3 instead of Dup2Jakob Unterwurzacher
Dup2 is not implemented on linux/arm64. Fixes https://github.com/rfjakob/gocryptfs/issues/121 . Also adds cross-compilation to CI.
2017-06-11contentenc: parallelize encryption for 128kiB writesJakob Unterwurzacher
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting up smaller writes is probably not worth it. Parallelism is limited to two for now.
2017-06-11cryptocore: prefetch nonces in the backgroundJakob Unterwurzacher
Spawn a worker goroutine that reads the next 512-byte block while the current one is being drained. This should help reduce waiting times when /dev/urandom is very slow (like on Linux 3.16 kernels).
2017-06-09cryptocore: prefetch nonces in 512-byte blocksJakob Unterwurzacher
On my machine, reading 512-byte blocks from /dev/urandom (same via getentropy syscall) is a lot faster in terms of throughput: Blocksize Throughput 16 28.18 MB/s 512 83.75 MB/s For a single-threaded streaming write, this drops the CPU usage of nonceGenerator.Get to almost 1/3: flat flat% sum% cum cum% Before 0 0% 95.08% 0.35s 2.92% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get After 0.01s 0.092% 92.34% 0.13s 1.20% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get This change makes the nonce reading single-threaded, which may hurt massively-parallel writes.
2017-06-09Fix missing Owner coercion for already-open files (#117)Charles Duffy
2017-06-07cryptocore: remove lastNonce checkJakob Unterwurzacher
This check would need locking to be multithreading-safe. But as it is in the fastpath, just remove it. rand.Read() already guarantees that the value is random.
2017-06-07contentenc: move EncryptBlocks() loop into its own functionsJakob Unterwurzacher
This allows easy parallelization in the future.
2017-06-07Add "-trace" flag (record execution trace)Jakob Unterwurzacher
Uses the runtime/trace functionality. TODO: add to man page.
2017-06-01fusefrontend: write: consolidate and move encryption to contentencJakob Unterwurzacher
Collect all the plaintext and pass everything to contentenc in one call. This will allow easier parallization of the encryption. https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01Fix two commentsJakob Unterwurzacher
One out-of-date and the other with a typo.
2017-06-01Implement force_owner option to display ownership as a specific user.Charles Duffy
2017-05-31pathiv: fix test failure on Go 1.6Jakob Unterwurzacher
Travis failed on Go 1.6.3 with this error: internal/pathiv/pathiv_test.go:20: no args in Error call This change should solve the problem and provides a better error message on (real) test failure.
2017-05-30pathiv: move block IV algorithm into this packageJakob Unterwurzacher
This was implemented in fusefrontend_reverse, but we need it in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-02 22:09:15 +0000