| Age | Commit message (Collapse) | Author |
|---|
|
MacOS creates lots of these files, and if the directory is otherwise
empty, we would throw an IO error to the unsuspecting user.
With this patch, we log a warning, but otherwise pretend we did not
see it.
Mitigates https://github.com/rfjakob/gocryptfs/issues/140
|
|
...and if Getdents is not available at all.
Due to this warning I now know that SSHFS always returns DT_UNKNOWN:
gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat
This behavoir is confirmed at http://ahefner.livejournal.com/16875.html:
"With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
|
|
Previously, OpenDir() did not use the cache at all, missing
an opportunity to speed up repeated directory reads.
|
|
The comments were unclear on whether relative or absolute paths
have to be passed.
|
|
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
|
|
The benchmark that supported the decision for 512-byte
prefetching previously lived outside the repo.
Let's add it where it belongs so it cannot get lost.
|
|
Getdents avoids calling Lstat on each file.
|
|
The Readdir function provided by os is inherently slow because
it calls Lstat on all files.
Getdents gives us all the information we need, but does not have
a proper wrapper in the stdlib.
Implement the "Getdents()" wrapper function that calls
syscall.Getdents() and parses the returned byte blob to a
fuse.DirEntry slice.
|
|
Remove the "Masterkey" field from fusefrontend.Args because it
should not be stored longer than neccessary. Instead pass the
masterkey as a separate argument to the filesystem initializers.
Then overwrite it with zeros immediately so we don't have
to wait for garbage collection.
Note that the crypto implementation still stores at least a
masterkey-derived value, so this change makes it harder, but not
impossible, to extract the encryption keys from memory.
Suggested at https://github.com/rfjakob/gocryptfs/issues/137
|
|
* extend the diriv cache to 100 entries
* add special handling for the immutable root diriv
The better cache allows to shed some complexity from the path
encryption logic (parent-of-parent check).
Mitigates https://github.com/rfjakob/gocryptfs/issues/127
|
|
Dir is like filepath.Dir but returns "" instead of ".".
This was already implemented in fusefrontend_reverse as saneDir().
We will need it in nametransform for the improved diriv caching.
|
|
Needs some space to grow.
renamed: internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
|
|
This operation has been done three time by identical
sections of code. Create a function for it.
|
|
A directory with a long name has two associated virtual files:
the .name file and the .diriv files.
These used to get the same inode number:
$ ls -di1 * */*
33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw
1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv
1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
With this change we use another prefix (2 instead of 1) for .name files.
$ ls -di1 * */*
33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw
1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv
2000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
|
|
This was working until DecryptName switched to returning
EBADMSG instead of EINVAL.
Add a test to catch the regression next time.
|
|
On MacOS, building and testing without openssl is much easier.
The tests should skip tests that fail because of missing openssl
instead of aborting.
Fixes https://github.com/rfjakob/gocryptfs/issues/123
|
|
Fixed by including the correct header. Should work on older openssl
versions as well.
Error was:
locking.go:21: undefined reference to `CRYPTO_set_locking_callback'
|
|
Due to RMW, we always need read permissions on the backing file. This is a
problem if the file permissions do not allow reading (i.e. 0200 permissions).
This patch works around that problem by chmod'ing the file, obtaining a fd,
and chmod'ing it back.
Test included.
Issue reported at: https://github.com/rfjakob/gocryptfs/issues/125
|
|
Saves 3% for the tar extract benchmark because we skip the allocation.
|
|
Previously we ran through the decryption steps even for an empty
ciphertext slice. The functions handle it correctly, but returning
early skips all the extra calls.
Speeds up the tar extract benchmark by about 4%.
|
|
Adds a test for the optimization introduced in:
stupidgcm: Open: if "dst" is big enough, use it as the output buffer
|
|
This gets us a massive speed boost in streaming reads.
|
|
This means we won't need any allocation for the plaintext.
|
|
Easily saves lots of allocations.
|
|
This will allow us to return internal buffers to a pool.
|
|
bPool verifies the lengths of slices going in and out.
Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
|
|
This saves an allocation of the ciphertext block.
|
|
We use two levels of buffers:
1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)
This commit adds a sync.Pool for both levels.
The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
|
|
Dup2 is not implemented on linux/arm64.
Fixes https://github.com/rfjakob/gocryptfs/issues/121 .
Also adds cross-compilation to CI.
|
|
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting
up smaller writes is probably not worth it.
Parallelism is limited to two for now.
|
|
Spawn a worker goroutine that reads the next 512-byte block
while the current one is being drained.
This should help reduce waiting times when /dev/urandom is very
slow (like on Linux 3.16 kernels).
|
|
On my machine, reading 512-byte blocks from /dev/urandom
(same via getentropy syscall) is a lot faster in terms of
throughput:
Blocksize Throughput
16 28.18 MB/s
512 83.75 MB/s
For a single-threaded streaming write, this drops the CPU usage of
nonceGenerator.Get to almost 1/3:
flat flat% sum% cum cum%
Before 0 0% 95.08% 0.35s 2.92% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get
After 0.01s 0.092% 92.34% 0.13s 1.20% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get
This change makes the nonce reading single-threaded, which may
hurt massively-parallel writes.
|
|
|
|
This check would need locking to be multithreading-safe.
But as it is in the fastpath, just remove it.
rand.Read() already guarantees that the value is random.
|
|
This allows easy parallelization in the future.
|
|
Uses the runtime/trace functionality.
TODO: add to man page.
|
|
Collect all the plaintext and pass everything to contentenc in
one call.
This will allow easier parallization of the encryption.
https://github.com/rfjakob/gocryptfs/issues/116
|
|
One out-of-date and the other with a typo.
|
|
|
|
Travis failed on Go 1.6.3 with this error:
internal/pathiv/pathiv_test.go:20: no args in Error call
This change should solve the problem and provides a better error
message on (real) test failure.
|
|
This was implemented in fusefrontend_reverse, but we need it
in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
|
|
...under the new name "FileIVs".
This will also be used by forward mode.
|
|
We will also need it in forward mode.
|
|
These should make it easier to re-implement the key derivation
that was enabled with the "HKDF" feature flag.
|
|
With hard links, the path to a file is not unique. This means
that the ciphertext data depends on the path that is used to access
the files.
Fix that by storing the derived values when we encounter a hard-linked
file. This means that the first path wins.
|
|
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
|
|
This should never happen in normal operation and is a sign of
data corruption. Catch it early.
|
|
Log the message ourselves and return EINVAL.
Before:
gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2
After:
gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
|
|
This fixes a few issues I have found reviewing the code:
1) Limit the amount of data ReadLongName() will read. Previously,
you could send gocryptfs into out-of-memory by symlinking
gocryptfs.diriv to /dev/zero.
2) Handle the empty input case in unPad16() by returning an
error. Previously, it would panic with an out-of-bounds array
read. It is unclear to me if this could actually be triggered.
3) Reject empty names after base64-decoding in DecryptName().
An empty name crashes emeCipher.Decrypt().
It is unclear to me if B64.DecodeString() can actually return
a non-error empty result, but let's guard against it anyway.
|
|
Exiting with a fatal error just pushes users to use "-nosyslog",
which is even worse than not having a paniclog.
|
