aboutsummaryrefslogtreecommitdiff
path: root/internal/fusefrontend/file.go
AgeCommit message (Collapse)Author
2017-06-30contentenc: add PReqPool and use it in DecryptBlocksJakob Unterwurzacher
This gets us a massive speed boost in streaming reads.
2017-06-30fusefrontend: doRead: use CReqPool for ciphertext bufferJakob Unterwurzacher
Easily saves lots of allocations.
2017-06-30fusefrontend: Read: use provided bufferJakob Unterwurzacher
This will allow us to return internal buffers to a pool.
2017-06-29contentenc: add safer "bPool" pool variant; add pBlockPoolJakob Unterwurzacher
bPool verifies the lengths of slices going in and out. Also, add a plaintext block pool - pBlockPool - and use it for decryption.
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-09Fix missing Owner coercion for already-open files (#117)Charles Duffy
2017-06-01fusefrontend: write: consolidate and move encryption to contentencJakob Unterwurzacher
Collect all the plaintext and pass everything to contentenc in one call. This will allow easier parallization of the encryption. https://github.com/rfjakob/gocryptfs/issues/116
2017-05-01openfiletable: rename WriteLock to ContentLockJakob Unterwurzacher
...and IDLock to HeaderLock. This matches what the locks actually protect.
2017-05-01fusefrontend: rely on nodefs.defaultFile for no-op functionsJakob Unterwurzacher
Now that we embed nodefs.NewDefaultFile(), we can drop our own no-ops.
2017-05-01fusefrontend: drop writeOnly flagJakob Unterwurzacher
We do not have to track the writeOnly status because the kernel will not forward read requests on a write-only FD to us anyway. I have verified this behavoir manually on a 4.10.8 kernel and also added a testcase.
2017-05-01fusefronted, openfiletable: move the open file table to its own packageJakob Unterwurzacher
The open file table code needs some room to grow for the upcoming FD multiplexing implementation.
2017-04-29fusefronted: drop unused file.String() functionJakob Unterwurzacher
This is a very old leftover.
2017-04-24forcedecode: tighten checksJakob Unterwurzacher
...and fix a few golint issues and print a scary warning message on mount. Also, force the fs to ro,noexec.
2017-04-23Add -forcedecodedanim7
Force decode of encrypted files even if the integrity check fails, instead of failing with an IO error. Warning messages are still printed to syslog if corrupted files are encountered. It can be useful to recover files from disks with bad sectors or other corrupted media. Closes https://github.com/rfjakob/gocryptfs/pull/102 .
2017-04-23Fix Flock build breakageJakob Unterwurzacher
go-fuse has added a new method to the nodefs.File interface that caused this build error: internal/fusefrontend/file.go:75: cannot use file literal (type *file) as type nodefs.File in return argument: *file does not implement nodefs.File (missing Flock method) Fixes https://github.com/rfjakob/gocryptfs/issues/104 and prevents the problem from happening again.
2017-03-18fusefrontend: get rid of leftover debug outputJakob Unterwurzacher
2017-03-18serialize_reads: add read serialization logicJakob Unterwurzacher
Due to kernel readahead, we usually get multiple read requests at the same time. These get submitted to the backing storage in random order, which is a problem if seeking is very expensive. Details: https://github.com/rfjakob/gocryptfs/issues/92
2017-03-12fusefrontend: readFileID: reject files that consist only of a headerJakob Unterwurzacher
A header-only file will be considered empty (this is not supposed to happen). This makes File ID poisoning more difficult.
2016-11-25main, fusefrontend: add "-noprealloc" optionJakob Unterwurzacher
Preallocation is very slow on hdds that run btrfs. Give the user the option to disable it. This greatly speeds up small file operations but reduces the robustness against out-of-space errors. Also add the option to the man page. More info: https://github.com/rfjakob/gocryptfs/issues/63
2016-11-25fusefrontend: coalesce 4kB writesJakob Unterwurzacher
This improves performance on hdds running ext4, and improves streaming write performance on hdds running btrfs. Tar extract slows down on btrfs for some reason. See https://github.com/rfjakob/gocryptfs/issues/63 Benchmarks: encfs v1.9.1 ============ $ ./benchmark.bash -encfs /mnt/hdd-ext4 Testing EncFS at /mnt/hdd-ext4/benchmark.bash.u0g WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,48354 s, 88,4 MB/s UNTAR: 20.79 LS: 3.04 RM: 6.62 $ ./benchmark.bash -encfs /mnt/hdd-btrfs Testing EncFS at /mnt/hdd-btrfs/benchmark.bash.h40 WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,52552 s, 85,9 MB/s UNTAR: 24.51 LS: 2.73 RM: 5.32 gocryptfs v1.1.1-26-g4a7f8ef ============================ $ ./benchmark.bash /mnt/hdd-ext4 Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.1KG WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,55782 s, 84,1 MB/s UNTAR: 22.23 LS: 1.47 RM: 4.17 $ ./benchmark.bash /mnt/hdd-btrfs Testing gocryptfs at /mnt/hdd-btrfs/benchmark.bash.2t8 WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 6,87206 s, 19,1 MB/s UNTAR: 69.87 LS: 1.52 RM: 5.33 gocryptfs v1.1.1-32 =================== $ ./benchmark.bash /mnt/hdd-ext4 Testing gocryptfs at /mnt/hdd-ext4/benchmark.bash.Qt3 WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 1,22577 s, 107 MB/s UNTAR: 23.46 LS: 1.46 RM: 4.67 $ ./benchmark.bash /mnt/hdd-btrfs/ Testing gocryptfs at /mnt/hdd-btrfs//benchmark.bash.XVk WRITE: 131072000 bytes (131 MB, 125 MiB) copied, 3,68735 s, 35,5 MB/s UNTAR: 116.87 LS: 1.84 RM: 6.34
2016-11-17fusefrontend: get the file ID from the open files tableJakob Unterwurzacher
This fixes the problem that a truncate can reset the file ID without the other open FDs noticing it.
2016-11-17fusefrontend: upgrade wlockMap to use device AND inode numberJakob Unterwurzacher
If there are multiple filesystems backing the gocryptfs filesystems inode numbers are not guaranteed to be unique.
2016-10-30fusefrontend: drop atime workaroundsJakob Unterwurzacher
The fix at https://github.com/hanwen/go-fuse/pull/131 has been merged. Drop the workarounds and re-enable the tests.
2016-10-28fusefronted: more concise corrupt block log messageJakob Unterwurzacher
Calculating the block offset is easy enough, even more now that gocryptfs-xray exists.
2016-10-28fusefronted: optimize NFS streaming writes by saving one Stat()Jakob Unterwurzacher
Stat() calls are expensive on NFS as they need a full network round-trip. We detect when a write immediately follows the last one and skip the Stat in this case because the write cannot create a file hole. On my (slow) NAS, this takes the write speed from 24MB/s to 41MB/s.
2016-10-25fusefrontend: move hole padding check out of Write()Jakob Unterwurzacher
The details of the hole handling don't have to be in Write, so move it away.
2016-10-25fusefrontend: rename "createsHole" to clearer "createsCiphertextHole"Jakob Unterwurzacher
...and add comments for what is happening.
2016-10-24Fix misspellingsJakob Unterwurzacher
Close https://github.com/rfjakob/gocryptfs/issues/54
2016-10-19tests: add 1980.tar.gz extract testJakob Unterwurzacher
Test that we get the right timestamp when extracting a tarball. Also simplify the workaround in doTestUtimesNano() and fix the fact that it was running no test at all.
2016-10-19lint fixesJakob Unterwurzacher
2016-10-16fusefrontend: Utimens: one more band-aidJakob Unterwurzacher
Revert once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-16fusefrontend: Utimens: ugly band-aid for nil pointer crash in go-fuseJakob Unterwurzacher
Crash is described at https://github.com/rfjakob/gocryptfs/issues/48 . Revert this once https://github.com/hanwen/go-fuse/pull/131 is merged.
2016-10-04lint fixesValient Gough
2016-09-29reverse: use per-purpose nonce generationJakob Unterwurzacher
Also pull all the deterministic nonce code into fusefrontend_reverse to greatly simplify the normal code path.
2016-09-25contentenc: add "ExternalNonce" modeJakob Unterwurzacher
This will be used for strong symlink encryption in reverse mode.
2016-09-25contentenc: add GCM-SIV supportJakob Unterwurzacher
Also add ReverseDummyNonce nonce generation.
2016-09-25fusefrontend: relay Utimens to go-fuseJakob Unterwurzacher
Commit af5441dcd9033e81da43ab77887a7b5aac693ab6 has caused a regression ( https://github.com/rfjakob/gocryptfs/issues/35 ) that is fixed by this commit. The go-fuse library by now has all the syscall wrappers in place to correctly handle Utimens, also for symlinks. Instead of duplicating the effort here just call into go-fuse. Closes #35
2016-08-09fusefrontend: use NsecToTimespec() for UtimensJakob Unterwurzacher
This fixes a build problem on 32-bit hosts: internal/fusefrontend/file.go:400: cannot use a.Unix() (type int64) as type int32 in assignment internal/fusefrontend/file.go:406: cannot use m.Unix() (type int64) as type int32 in assignment It also enables full nanosecond timestamps for dates after 1970.
2016-07-03syscallcompat: OSX: add Fallocate and Openat wrappersJakob Unterwurzacher
...and convert all calls to syscall.{Fallocate,Openat} to syscallcompat . Both syscalls are not available on OSX. We emulate Openat and just return EOPNOTSUPP for Fallocate.
2016-07-03syscallcompat: move syscall wrapper to their own packageJakob Unterwurzacher
We will get more of them as OSX also lacks support for openat.
2016-07-02fusefrontend: add fallocate supportJakob Unterwurzacher
Mode=0 (default) and mode=1 (keep size) are supported. The patch includes test cases and the whole thing passed xfstests. Fixes https://github.com/rfjakob/gocryptfs/issues/1 .
2016-07-02fusefronted: move Truncate() and Allocate() to their own fileJakob Unterwurzacher
These are large complicated implementations that will share some code.
2016-07-02contentenc: rename PlaintextRange and CiphertextRangeJakob Unterwurzacher
The name could be misunderstood and actually caused a bug: doWrite used to always preallocate 4128 instead of the actual data length.
2016-07-01fusefrontend: coalesce grows in Truncate()Jakob Unterwurzacher
We were growing the file block-by-block which was pretty inefficient. We now coalesce all the grows into a single Ftruncate. Also simplifies the code! Simplistic benchmark: Before: $ time truncate -s 1000M foo real 0m0.568s After: $ time truncate -s 1000M foo real 0m0.205s
2016-07-01fusefrontend: better comments for TruncateJakob Unterwurzacher
2016-06-26main, fusefrontend: enable suid functionalityJakob Unterwurzacher
FUSE filesystems are mounted with "nosuid" by default. If we run as root, we can use device files by passing the opposite mount option, "suid". Also we have to use syscall.Chmod instead of os.Chmod because the portability translation layer "syscallMode" messes up the sgid and suid bits. Fixes 70% of the failures in xfstests generic/193. The remaining are related to truncate, but we err on the safe side: $ diff -u tests/generic/193.out /home/jakob/src/fuse-xfstests/results//generic/193.out.bad [...] check that suid/sgid bits are cleared after successful truncate... with no exec perm before: -rwSr-Sr-- -after: -rw-r-Sr-- +after: -rw-r--r--
2016-06-26fuserfrontend: support truncate(2) by wrapping ftruncate(2)Jakob Unterwurzacher
Support truncate(2) by opening the file and calling ftruncate(2) While the glibc "truncate" wrapper seems to always use ftruncate, fsstress from xfstests uses this a lot by calling "truncate64" directly.
2016-06-15Rename internal "toggledlog" package to "tlog"Jakob Unterwurzacher
tlog is used heavily everywhere and deserves a shorter name. Renamed using sed magic, without any manual rework: find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-06-14Fix warnings reported by Go 1.6 "go tool vet -shadow=true"Jakob Unterwurzacher
Warnings were: main.go:234: declaration of err shadows declaration at main.go:163: internal/fusefrontend/file.go:401: declaration of err shadows declaration at internal/fusefrontend/file.go:379: internal/fusefrontend/file.go:419: declaration of err shadows declaration at internal/fusefrontend/file.go:379: internal/fusefrontend/fs_dir.go:140: declaration of err shadows declaration at internal/fusefrontend/fs_dir.go:97:
2016-06-09fusefrontend: Utimens: convert ENOENT to EBADFJakob Unterwurzacher
If /proc/self/fd/X did not exist, the actual error is that the file descriptor was invalid. go-fuse's pathfs prefers using an open fd even for path-based operations but does not take any locks to prevent the fd from being closed. Instead, it retries the operation by path if it get EBADF. So this change allows the retry logic to work correctly. This fixes the error rsync: failed to set times on "/tmp/ping.Kgw.mnt/linux-3.0/[...]/.dvb_demux.c.N7YlEM": No such file or directory (2) that was triggered by pingpong-rsync.bash.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-17 06:35:31 +0000