summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
2017-08-09nametransform: extend diriv cache to 100 entriesJakob Unterwurzacher
* extend the diriv cache to 100 entries * add special handling for the immutable root diriv The better cache allows to shed some complexity from the path encryption logic (parent-of-parent check). Mitigates https://github.com/rfjakob/gocryptfs/issues/127
2017-08-06nametransform: add Dir() functionJakob Unterwurzacher
Dir is like filepath.Dir but returns "" instead of ".". This was already implemented in fusefrontend_reverse as saneDir(). We will need it in nametransform for the improved diriv caching.
2017-08-06nametransform: move diriv cache into it's own packageJakob Unterwurzacher
Needs some space to grow. renamed: internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
2017-08-06nametransform: deduplicate code to encryptAndHashName()Jakob Unterwurzacher
This operation has been done three time by identical sections of code. Create a function for it.
2017-07-29fusefronted_reverse: fix ino collision between .name and .diriv filesJakob Unterwurzacher
A directory with a long name has two associated virtual files: the .name file and the .diriv files. These used to get the same inode number: $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name With this change we use another prefix (2 instead of 1) for .name files. $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 2000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
2017-07-27fusefrontend_reverse: return ENOENT for undecryptable namesJakob Unterwurzacher
This was working until DecryptName switched to returning EBADMSG instead of EINVAL. Add a test to catch the regression next time.
2017-07-14macos: make testing without openssl work properlyJakob Unterwurzacher
On MacOS, building and testing without openssl is much easier. The tests should skip tests that fail because of missing openssl instead of aborting. Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14stupidgcm: fix openssl 1.1 build failureJakob Unterwurzacher
Fixed by including the correct header. Should work on older openssl versions as well. Error was: locking.go:21: undefined reference to `CRYPTO_set_locking_callback'
2017-07-11fusefronted: enable writing to write-only filesJakob Unterwurzacher
Due to RMW, we always need read permissions on the backing file. This is a problem if the file permissions do not allow reading (i.e. 0200 permissions). This patch works around that problem by chmod'ing the file, obtaining a fd, and chmod'ing it back. Test included. Issue reported at: https://github.com/rfjakob/gocryptfs/issues/125
2017-07-02contentenc: MergeBlocks: short-circuit the trivial caseJakob Unterwurzacher
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-07-02fusefrontend: doRead: skip decryption for an empty readJakob Unterwurzacher
Previously we ran through the decryption steps even for an empty ciphertext slice. The functions handle it correctly, but returning early skips all the extra calls. Speeds up the tar extract benchmark by about 4%.
2017-07-01stupidgcm: add test for in-place OpenJakob Unterwurzacher
Adds a test for the optimization introduced in: stupidgcm: Open: if "dst" is big enough, use it as the output buffer
2017-06-30contentenc: add PReqPool and use it in DecryptBlocksJakob Unterwurzacher
This gets us a massive speed boost in streaming reads.
2017-06-30stupidgcm: Open: if "dst" is big enough, use it as the output bufferJakob Unterwurzacher
This means we won't need any allocation for the plaintext.
2017-06-30fusefrontend: doRead: use CReqPool for ciphertext bufferJakob Unterwurzacher
Easily saves lots of allocations.
2017-06-30fusefrontend: Read: use provided bufferJakob Unterwurzacher
This will allow us to return internal buffers to a pool.
2017-06-29contentenc: add safer "bPool" pool variant; add pBlockPoolJakob Unterwurzacher
bPool verifies the lengths of slices going in and out. Also, add a plaintext block pool - pBlockPool - and use it for decryption.
2017-06-29stupidgcm: use "dst" as the output buffer it is big enoughJakob Unterwurzacher
This saves an allocation of the ciphertext block.
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-18main, syscallcompat: use Dup3 instead of Dup2Jakob Unterwurzacher
Dup2 is not implemented on linux/arm64. Fixes https://github.com/rfjakob/gocryptfs/issues/121 . Also adds cross-compilation to CI.
2017-06-11contentenc: parallelize encryption for 128kiB writesJakob Unterwurzacher
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting up smaller writes is probably not worth it. Parallelism is limited to two for now.
2017-06-11cryptocore: prefetch nonces in the backgroundJakob Unterwurzacher
Spawn a worker goroutine that reads the next 512-byte block while the current one is being drained. This should help reduce waiting times when /dev/urandom is very slow (like on Linux 3.16 kernels).
2017-06-09cryptocore: prefetch nonces in 512-byte blocksJakob Unterwurzacher
On my machine, reading 512-byte blocks from /dev/urandom (same via getentropy syscall) is a lot faster in terms of throughput: Blocksize Throughput 16 28.18 MB/s 512 83.75 MB/s For a single-threaded streaming write, this drops the CPU usage of nonceGenerator.Get to almost 1/3: flat flat% sum% cum cum% Before 0 0% 95.08% 0.35s 2.92% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get After 0.01s 0.092% 92.34% 0.13s 1.20% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get This change makes the nonce reading single-threaded, which may hurt massively-parallel writes.
2017-06-09Fix missing Owner coercion for already-open files (#117)Charles Duffy
2017-06-07cryptocore: remove lastNonce checkJakob Unterwurzacher
This check would need locking to be multithreading-safe. But as it is in the fastpath, just remove it. rand.Read() already guarantees that the value is random.
2017-06-07contentenc: move EncryptBlocks() loop into its own functionsJakob Unterwurzacher
This allows easy parallelization in the future.
2017-06-07Add "-trace" flag (record execution trace)Jakob Unterwurzacher
Uses the runtime/trace functionality. TODO: add to man page.
2017-06-01fusefrontend: write: consolidate and move encryption to contentencJakob Unterwurzacher
Collect all the plaintext and pass everything to contentenc in one call. This will allow easier parallization of the encryption. https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01Fix two commentsJakob Unterwurzacher
One out-of-date and the other with a typo.
2017-06-01Implement force_owner option to display ownership as a specific user.Charles Duffy
2017-05-31pathiv: fix test failure on Go 1.6Jakob Unterwurzacher
Travis failed on Go 1.6.3 with this error: internal/pathiv/pathiv_test.go:20: no args in Error call This change should solve the problem and provides a better error message on (real) test failure.
2017-05-30pathiv: move block IV algorithm into this packageJakob Unterwurzacher
This was implemented in fusefrontend_reverse, but we need it in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
2017-05-30pathiv: move derivedIVContainer into the packageJakob Unterwurzacher
...under the new name "FileIVs". This will also be used by forward mode.
2017-05-30fusefrontend_reverse: move pathiv to its own packageJakob Unterwurzacher
We will also need it in forward mode.
2017-05-27cryptocore: improve comments and add tests for hkdfDeriveJakob Unterwurzacher
These should make it easier to re-implement the key derivation that was enabled with the "HKDF" feature flag.
2017-05-25fusefrontend_reverse: store derived values for hard-linked filesJakob Unterwurzacher
With hard links, the path to a file is not unique. This means that the ciphertext data depends on the path that is used to access the files. Fix that by storing the derived values when we encounter a hard-linked file. This means that the first path wins.
2017-05-25nametransform: reject all-zero dirIVJakob Unterwurzacher
This should never happen in normal operation and is a sign of data corruption. Catch it early.
2017-05-25contenenc: reject all-zero file IDJakob Unterwurzacher
This should never happen in normal operation and is a sign of data corruption. Catch it early.
2017-05-25contentenc: better error reporting in ParseHeaderJakob Unterwurzacher
Log the message ourselves and return EINVAL. Before: gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2 After: gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-23nametransform: harden name decryption against invalid inputJakob Unterwurzacher
This fixes a few issues I have found reviewing the code: 1) Limit the amount of data ReadLongName() will read. Previously, you could send gocryptfs into out-of-memory by symlinking gocryptfs.diriv to /dev/zero. 2) Handle the empty input case in unPad16() by returning an error. Previously, it would panic with an out-of-bounds array read. It is unclear to me if this could actually be triggered. 3) Reject empty names after base64-decoding in DecryptName(). An empty name crashes emeCipher.Decrypt(). It is unclear to me if B64.DecodeString() can actually return a non-error empty result, but let's guard against it anyway.
2017-05-23main: downgrade panic log create failure from fatal error to warningJakob Unterwurzacher
Exiting with a fatal error just pushes users to use "-nosyslog", which is even worse than not having a paniclog.
2017-05-22nametransform: diriv cache: fall back to the grandparentJakob Unterwurzacher
When a user calls into a deep directory hierarchy, we often get a sequence like this from the kernel: LOOKUP a LOOKUP a/b LOOKUP a/b/c LOOKUP a/b/c/d The diriv cache was not effective for this pattern, because it was designed for this: LOOKUP a/a LOOKUP a/b LOOKUP a/c LOOKUP a/d By also using the cached entry of the grandparent we can avoid lots of diriv reads. This benchmark is against a large encrypted directory hosted on NFS: Before: $ time ls -R nfs-backed-mount > /dev/null real 1m35.976s user 0m0.248s sys 0m0.281s After: $ time ls -R nfs-backed-mount > /dev/null real 1m3.670s user 0m0.217s sys 0m0.403s
2017-05-14exitcodes: specific codes for failure to read or write gocryptfs.confJakob Unterwurzacher
New codes: * OpenConf = 23 * WriteConf = 24
2017-05-14exitcodes: add code 22 for "password is empty"Jakob Unterwurzacher
Empty passwords are not allowed. Let's give the error it's own exit code.
2017-05-14exitcodes: get rid of generic "Mount" exit codeJakob Unterwurzacher
Instead, create three new specific exit codes: * FuseNewServer = 19 * CtlSock = 20 * PanicLogCreate = 21
2017-05-07exitcodes: pull all exit code definitions into the packageJakob Unterwurzacher
This commit defines all exit codes in one place in the exitcodes package. Also, it adds a test to verify the exit code on incorrect password, which is what SiriKali cares about the most. Fixes https://github.com/rfjakob/gocryptfs/issues/77 .
2017-05-07fusefrontend: implement path decryption via ctlsockJakob Unterwurzacher
Closes https://github.com/rfjakob/gocryptfs/issues/84 .
2017-05-07nametranform, fusefrontend: better errors on invalid namesJakob Unterwurzacher
nametransform.DecryptName() now always returns syscall.EBADMSG if the name was invalid. fusefrontend.OpenDir error messages have been normalized.
2017-05-07Fix typos found by MisspellJakob Unterwurzacher
Misspell Finds commonly misspelled English words gocryptfs/internal/configfile/scrypt.go Line 41: warning: "paramter" is a misspelling of "parameter" (misspell) gocryptfs/internal/ctlsock/ctlsock_serve.go Line 1: warning: "implementes" is a misspelling of "implements" (misspell) gocryptfs/tests/test_helpers/helpers.go Line 27: warning: "compatability" is a misspelling of "compatibility" (misspell)
2017-05-03fusefrontend: log "too many open files" errorsJakob Unterwurzacher
This usually indicates that the open file limit for gocryptfs is too low. We should report this to the user.