| Age | Commit message (Collapse) | Author |
|---|
|
bPool verifies the lengths of slices going in and out.
Also, add a plaintext block pool - pBlockPool - and use
it for decryption.
|
|
We use two levels of buffers:
1) 4kiB+overhead for each ciphertext block
2) 128kiB+overhead for each FUSE write (32 ciphertext blocks)
This commit adds a sync.Pool for both levels.
The memory-efficiency for small writes could be improved,
as we now always use a 128kiB buffer.
|
|
|
|
Collect all the plaintext and pass everything to contentenc in
one call.
This will allow easier parallization of the encryption.
https://github.com/rfjakob/gocryptfs/issues/116
|
|
...and IDLock to HeaderLock. This matches what the locks actually
protect.
|
|
Now that we embed nodefs.NewDefaultFile(), we can drop our own
no-ops.
|
|
We do not have to track the writeOnly status because the kernel
will not forward read requests on a write-only FD to us anyway.
I have verified this behavoir manually on a 4.10.8 kernel and also
added a testcase.
|
|
The open file table code needs some room to grow for the upcoming
FD multiplexing implementation.
|
|
This is a very old leftover.
|
|
...and fix a few golint issues and print a scary warning message on mount.
Also, force the fs to ro,noexec.
|
|
Force decode of encrypted files even if the integrity check fails, instead of
failing with an IO error. Warning messages are still printed to syslog if corrupted
files are encountered.
It can be useful to recover files from disks with bad sectors or other corrupted
media.
Closes https://github.com/rfjakob/gocryptfs/pull/102 .
|
|
go-fuse has added a new method to the nodefs.File interface that
caused this build error:
internal/fusefrontend/file.go:75: cannot use file literal (type *file) as type nodefs.File in return argument:
*file does not implement nodefs.File (missing Flock method)
Fixes https://github.com/rfjakob/gocryptfs/issues/104 and
prevents the problem from happening again.
|
|
|
|
Due to kernel readahead, we usually get multiple read requests
at the same time. These get submitted to the backing storage in
random order, which is a problem if seeking is very expensive.
Details: https://github.com/rfjakob/gocryptfs/issues/92
|
|
A header-only file will be considered empty (this is not supposed to happen).
This makes File ID poisoning more difficult.
|
|
Preallocation is very slow on hdds that run btrfs. Give the
user the option to disable it. This greatly speeds up small file
operations but reduces the robustness against out-of-space errors.
Also add the option to the man page.
More info: https://github.com/rfjakob/gocryptfs/issues/63
|
|
This improves performance on hdds running ext4, and improves
streaming write performance on hdds running btrfs. Tar extract
slows down on btrfs for some reason.
See https://github.com/rfjakob/gocryptfs/issues/63
Benchmarks:
encfs v1.9.1
============
$ ./benchmark.bash -encfs /mnt/hdd-ext4
Testing EncFS at /mnt/hdd-ext4/benchmark.bash.u0g
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,48354 s, 88,4 MB/s
UNTAR: 20.79
LS: 3.04
RM: 6.62
$ ./benchmark.bash -encfs /mnt/hdd-btrfs
Testing EncFS at /mnt/hdd-btrfs/benchmark.bash.h40
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,52552 s, 85,9 MB/s
UNTAR: 24.51
LS: 2.73
RM: 5.32
gocryptfs v1.1.1-26-g4a7f8ef
============================
$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.1KG
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,55782 s, 84,1 MB/s
UNTAR: 22.23
LS: 1.47
RM: 4.17
$ ./benchmark.bash /mnt/hdd-btrfs
Testing gocryptfs at /mnt/hdd-btrfs/benchmark.bash.2t8
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 6,87206 s, 19,1 MB/s
UNTAR: 69.87
LS: 1.52
RM: 5.33
gocryptfs v1.1.1-32
===================
$ ./benchmark.bash /mnt/hdd-ext4
Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.Qt3
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,22577 s, 107 MB/s
UNTAR: 23.46
LS: 1.46
RM: 4.67
$ ./benchmark.bash /mnt/hdd-btrfs/
Testing gocryptfs at /mnt/hdd-btrfs//benchmark.bash.XVk
WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 3,68735 s, 35,5 MB/s
UNTAR: 116.87
LS: 1.84
RM: 6.34
|
|
This fixes the problem that a truncate can reset the file
ID without the other open FDs noticing it.
|
|
If there are multiple filesystems backing the gocryptfs filesystems
inode numbers are not guaranteed to be unique.
|
|
The fix at https://github.com/hanwen/go-fuse/pull/131 has been merged.
Drop the workarounds and re-enable the tests.
|
|
Calculating the block offset is easy enough, even more now
that gocryptfs-xray exists.
|
|
Stat() calls are expensive on NFS as they need a full network
round-trip. We detect when a write immediately follows the
last one and skip the Stat in this case because the write
cannot create a file hole.
On my (slow) NAS, this takes the write speed from 24MB/s to
41MB/s.
|
|
The details of the hole handling don't have to be in
Write, so move it away.
|
|
...and add comments for what is happening.
|
|
Close https://github.com/rfjakob/gocryptfs/issues/54
|
|
Test that we get the right timestamp when extracting a tarball.
Also simplify the workaround in doTestUtimesNano() and fix the
fact that it was running no test at all.
|
|
|
|
Revert once https://github.com/hanwen/go-fuse/pull/131 is merged.
|
|
Crash is described at https://github.com/rfjakob/gocryptfs/issues/48 .
Revert this once https://github.com/hanwen/go-fuse/pull/131 is merged.
|
|
|
|
Also pull all the deterministic nonce code into fusefrontend_reverse
to greatly simplify the normal code path.
|
|
This will be used for strong symlink encryption in reverse mode.
|
|
Also add ReverseDummyNonce nonce generation.
|
|
Commit af5441dcd9033e81da43ab77887a7b5aac693ab6 has caused a
regression ( https://github.com/rfjakob/gocryptfs/issues/35 )
that is fixed by this commit.
The go-fuse library by now has all the syscall wrappers in
place to correctly handle Utimens, also for symlinks.
Instead of duplicating the effort here just call into go-fuse.
Closes #35
|
|
This fixes a build problem on 32-bit hosts:
internal/fusefrontend/file.go:400: cannot use a.Unix() (type int64) as
type int32 in assignment
internal/fusefrontend/file.go:406: cannot use m.Unix() (type int64) as
type int32 in assignment
It also enables full nanosecond timestamps for dates
after 1970.
|
|
...and convert all calls to syscall.{Fallocate,Openat}
to syscallcompat .
Both syscalls are not available on OSX. We emulate Openat and just
return EOPNOTSUPP for Fallocate.
|
|
We will get more of them as OSX also lacks support for openat.
|
|
Mode=0 (default) and mode=1 (keep size) are supported.
The patch includes test cases and the whole thing passed xfstests.
Fixes https://github.com/rfjakob/gocryptfs/issues/1 .
|
|
These are large complicated implementations that will share some
code.
|
|
The name could be misunderstood and actually caused a bug:
doWrite used to always preallocate 4128 instead of the actual
data length.
|
|
We were growing the file block-by-block which was pretty
inefficient. We now coalesce all the grows into a single
Ftruncate. Also simplifies the code!
Simplistic benchmark: Before:
$ time truncate -s 1000M foo
real 0m0.568s
After:
$ time truncate -s 1000M foo
real 0m0.205s
|
|
|
|
FUSE filesystems are mounted with "nosuid" by default. If we run as root,
we can use device files by passing the opposite mount option, "suid".
Also we have to use syscall.Chmod instead of os.Chmod because the
portability translation layer "syscallMode" messes up the sgid
and suid bits.
Fixes 70% of the failures in xfstests generic/193. The remaining are
related to truncate, but we err on the safe side:
$ diff -u tests/generic/193.out /home/jakob/src/fuse-xfstests/results//generic/193.out.bad
[...]
check that suid/sgid bits are cleared after successful truncate...
with no exec perm
before: -rwSr-Sr--
-after: -rw-r-Sr--
+after: -rw-r--r--
|
|
Support truncate(2) by opening the file and calling ftruncate(2)
While the glibc "truncate" wrapper seems to always use ftruncate, fsstress from
xfstests uses this a lot by calling "truncate64" directly.
|
|
tlog is used heavily everywhere and deserves a shorter name.
Renamed using sed magic, without any manual rework:
find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
|
|
Warnings were:
main.go:234: declaration of err shadows declaration at main.go:163:
internal/fusefrontend/file.go:401: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
internal/fusefrontend/file.go:419: declaration of err shadows declaration at internal/fusefrontend/file.go:379:
internal/fusefrontend/fs_dir.go:140: declaration of err shadows declaration at internal/fusefrontend/fs_dir.go:97:
|
|
If /proc/self/fd/X did not exist, the actual error is that the file
descriptor was invalid.
go-fuse's pathfs prefers using an open fd even for path-based operations
but does not take any locks to prevent the fd from being closed.
Instead, it retries the operation by path if it get EBADF. So this
change allows the retry logic to work correctly.
This fixes the error
rsync: failed to set times on "/tmp/ping.Kgw.mnt/linux-3.0/[...]/.dvb_demux.c.N7YlEM":
No such file or directory (2)
that was triggered by pingpong-rsync.bash.
|
|
... with the "released" boolean.
For some reason, the "f.fd.Fd() < 0" check did not work reliably,
leading to nil pointer panics on the following wlock.lock().
The problem was discovered during fsstress testing and is unlikely
to happen in normal operations.
With this change, we passed 1700+ fsstress iterations.
|
|
Using a simple boolean was racy (which was harmless
in this case) and non-idomatic.
|
|
The Fstat call should never fail, but still, if it does return an error
it should be handled properly.
|
