summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-10-21Revert most of "fusefrontend: clamp oversized reads"Jakob Unterwurzacher
We cannot return less data than requested to the kernel! From https://libfuse.github.io/doxygen/structfuse__operations.html: Read should return exactly the number of bytes requested except on EOF or error, otherwise the rest of the data will be substituted with zeroes. Reverts commit 3009ec9852316c3c696f77f476390ab5a6d8d6d7 minus the formatting improvements we want to keep. Fixes https://github.com/rfjakob/gocryptfs/issues/147 Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-19MANPAGE: explain that you may have to pass -aessiv with -masterkeyJakob Unterwurzacher
...if the filesystem was created with that option (or reverse mode). Mitigates https://github.com/rfjakob/gocryptfs/issues/148
2017-10-19contentenc: reserve one additional block in CReqPoolJakob Unterwurzacher
...to account for unaligned reads. I have not seen this happen in the wild because the kernel always seems to issue 4k-aligned requests. But the cost of the additional block in the pool is low and prevents a buffer overrun panic when an unaligned read does happen.
2017-10-18main: call logger with full pathJakob Unterwurzacher
If $PATH contains the mountpoint, searching through it will lock us up. Use an absolute path to avoid looking at $PATH. Fixes https://github.com/rfjakob/gocryptfs/issues/146
2017-10-17fusefrontend: clamp oversized readsJakob Unterwurzacher
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the running kernel may have a higher limit set. Clamp to what we can handle. Fixes a panic on a Synology NAS reported at https://github.com/rfjakob/gocryptfs/issues/145
2017-10-03fusefrontend_reverse: workaround ext4 test failureJakob Unterwurzacher
The extended TestLongnameStat() exposes a pathological case when run on ext4, as ext4 reuses inode numbers immediately. This change modifies the test to not delete the files immediately, so the inode numbers cannot be reused immediately. Fix for the underlying issue is a TODO.
2017-10-01fusefrontend_reverse: fix 176-byte namesJakob Unterwurzacher
A file with a name of exactly 176 bytes length caused this error: ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY: No such file or directory ls: cannot access ./tmp/dsg/sXSGJLTuZuW1FarwIkJs0w/b6mGjdxIRpaeanTo0rbh0A/QjMRrQZC_4WLhmHI1UOBcA/gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name: No such file or directory -????????? ? ? ? ? ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY -????????? ? ? ? ? ? gocryptfs.longname.QV-UipdDXeUVdl05WruoEzBNPrQCfpu6OzJL0_QnDKY.name Root cause was a wrong shortNameMax constant that failed to account for the obligatory padding byte. Fix the constant and also expand the TestLongnameStat test case to test ALL file name lengths from 1-255 bytes. Fixes https://github.com/rfjakob/gocryptfs/issues/143 .
2017-09-17siv_aead: fix trivial typo in commentJakob Unterwurzacher
2017-09-17contentenc: deduplicate AD packing into new concatAD() funcJakob Unterwurzacher
The encrypt and decrypt path both had a copy that were equivalent but ordered differently, which was confusing. Consolidate it in a new dedicated function.
2017-09-17contentenc: DecryptBlocks: give block number counter a clearer nameJakob Unterwurzacher
Using firstBlockNo as the counter is confusing, create a copy named "blockNo" and use that.
2017-09-10performance.txt: specify READ testsJakob Unterwurzacher
2017-09-06build.bash: make reproduceable builds easierJakob Unterwurzacher
* Reduce the build time precision from seconds to days * Allow to specify an arbitrary build date through an env variable
2017-09-06Add "dep" files: Gopkg.toml and Gopkg.lockJakob Unterwurzacher
Allows users to get a reproduceable build. Still needs to be integrated into build.bash. Suggested at https://github.com/rfjakob/gocryptfs/issues/142
2017-09-05macos: automatically remove .DS_Store on RmdirJakob Unterwurzacher
MacOS sprinkles .DS_Store files everywhere. This is hard to avoid for users, so handle it transparently in Rmdir(). Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-05fusefrontend: reorder logic in Rmdir to get rid of one indentation levelJakob Unterwurzacher
Handle the errors first so that the normal code path is not indented. This should not cause any behavoir changes.
2017-09-05macos: don't throw IO errors because of .DS_Store filesJakob Unterwurzacher
MacOS creates lots of these files, and if the directory is otherwise empty, we would throw an IO error to the unsuspecting user. With this patch, we log a warning, but otherwise pretend we did not see it. Mitigates https://github.com/rfjakob/gocryptfs/issues/140
2017-09-03syscallcompat: Getdents: warn once if we get DT_UNKNOWNJakob Unterwurzacher
...and if Getdents is not available at all. Due to this warning I now know that SSHFS always returns DT_UNKNOWN: gocryptfs[8129]: Getdents: convertDType: received DT_UNKNOWN, falling back to Lstat This behavoir is confirmed at http://ahefner.livejournal.com/16875.html: "With sshfs, I finally found that obscure case. The dtype is always set to DT_UNKNOWN [...]"
2017-09-03performance.txt: update for v1.4.1 latest commits, and kernel updateJakob Unterwurzacher
$ uname -a Linux brikett 4.12.5-300.fc26.x86_64 #1 SMP Mon Aug 7 15:27:25 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
2017-09-03fusefrontend: use DirIVCache in OpenDir()Jakob Unterwurzacher
Previously, OpenDir() did not use the cache at all, missing an opportunity to speed up repeated directory reads.
2017-09-03dirivcache: add better function comments + a sanity check on Store()Jakob Unterwurzacher
The comments were unclear on whether relative or absolute paths have to be passed.
2017-09-03profiling: add write-trace.bashJakob Unterwurzacher
Save an execution trace of writing 100MB of data to a new gocryptfs mount on /tmp
2017-09-02travis ci: also test Go 1.9Jakob Unterwurzacher
2017-08-21Fix misspellings reported by goreportcard.comJakob Unterwurzacher
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-08-21MANPAGE: update exit codesJakob Unterwurzacher
The exit codes have been documented in CLI_ABI.md for a while, but they should also be listed in the man page. Also fix the rendering of "[-o COMMA-SEPARATED-OPTIONS]", where the square brackets where interpreted as something. Escape all square brackets to be safe.
2017-08-21README: update changelog for v1.4.1 releasev1.4.1Jakob Unterwurzacher
2017-08-21package[-static].bash: stop leaking the local user id in the tarballJakob Unterwurzacher
The local user id of the packager is not interesting for users who download the tarball. Also it will cause the gocryptfs binary to have an unintended owner when the tarball is extraced as root. Fix the issue by using "tar --owner=root --group=root" which overwrites user and group id with zero.
2017-08-16cryptocore: add urandom + randprefetch benchmarksJakob Unterwurzacher
The benchmark that supported the decision for 512-byte prefetching previously lived outside the repo. Let's add it where it belongs so it cannot get lost.
2017-08-15Update performance.txt for Linux kernel upgradeJakob Unterwurzacher
For some reason, writing became a lot faster in Linux 4.11 (scheduler improvements?).
2017-08-15Update performance.txt for to Getdents changeJakob Unterwurzacher
2017-08-15docs: label "ENV CHANGE" columnJakob Unterwurzacher
2017-08-15fusefrontend: use Getdents if availableJakob Unterwurzacher
Getdents avoids calling Lstat on each file.
2017-08-15syscallcompat: implement Getdents()Jakob Unterwurzacher
The Readdir function provided by os is inherently slow because it calls Lstat on all files. Getdents gives us all the information we need, but does not have a proper wrapper in the stdlib. Implement the "Getdents()" wrapper function that calls syscall.Getdents() and parses the returned byte blob to a fuse.DirEntry slice.
2017-08-15main: rework "you need Go 1.5" lockout to make "go vet" happyJakob Unterwurzacher
When you run "go vet" explicitely against go1.4.go, it ignores the "+build !go1.5" tag and, of course, throws a syntax error: $ go vet go1.4.go can't load package: package main: go1.4.go:5:1: expected 'package', found 'STRING' "You need Go 1.5 or higher to compile gocryptfs!" Unfortunatey, this is how https://goreportcard.com/ seems to call "go vet", and means we get 0% on the "go vet" test and see this error: An error occurred while running this test (strconv.Atoi: parsing " go1.4.go": invalid syntax) By reworking the logic to use a non-existant package we get an uglier error $ GOROOT=/opt/go1.4.3 /opt/go1.4.3/bin/go build go1.4.go:7:8: cannot find package "You_need_Go_1.5_or_higher_to_compile_gocryptfs" in any of: /opt/go1.4.3/src/You_need_Go_1.5_or_higher_to_compile_gocryptfs (from $GOROOT) /home/jakob/go/src/You_need_Go_1.5_or_higher_to_compile_gocryptfs (from $GOPATH) profiling.go:6:2: cannot find package "runtime/trace" in any of: /opt/go1.4.3/src/runtime/trace (from $GOROOT) /home/jakob/go/src/runtime/trace (from $GOPATH) but make "go vet" happy.
2017-08-11main: purge masterkey from memory as soon as possibleJakob Unterwurzacher
Remove the "Masterkey" field from fusefrontend.Args because it should not be stored longer than neccessary. Instead pass the masterkey as a separate argument to the filesystem initializers. Then overwrite it with zeros immediately so we don't have to wait for garbage collection. Note that the crypto implementation still stores at least a masterkey-derived value, so this change makes it harder, but not impossible, to extract the encryption keys from memory. Suggested at https://github.com/rfjakob/gocryptfs/issues/137
2017-08-10tests: add diriv cache race testJakob Unterwurzacher
Passes.
2017-08-09nametransform: extend diriv cache to 100 entriesJakob Unterwurzacher
* extend the diriv cache to 100 entries * add special handling for the immutable root diriv The better cache allows to shed some complexity from the path encryption logic (parent-of-parent check). Mitigates https://github.com/rfjakob/gocryptfs/issues/127
2017-08-06nametransform: add Dir() functionJakob Unterwurzacher
Dir is like filepath.Dir but returns "" instead of ".". This was already implemented in fusefrontend_reverse as saneDir(). We will need it in nametransform for the improved diriv caching.
2017-08-06nametransform: move diriv cache into it's own packageJakob Unterwurzacher
Needs some space to grow. renamed: internal/nametransform/diriv_cache.go -> internal/nametransform/dirivcache/dirivcache.go
2017-08-06nametransform: deduplicate code to encryptAndHashName()Jakob Unterwurzacher
This operation has been done three time by identical sections of code. Create a function for it.
2017-08-02build.bash: implement "you need Go 1.5" lockout in pure GoJakob Unterwurzacher
As noticed by @riking, the logic in the bash script will break when Go 1 version numbers reach double-digits. Instead, use a build tag "!go1.5" to cause a syntax error: $ /opt/go1.4.3/bin/go build can't load package: package github.com/rfjakob/gocryptfs: go1.4.go:5:1: expected 'package', found 'STRING' "You need Go 1.5 or higher to compile gocryptfs!" Fixes https://github.com/rfjakob/gocryptfs/issues/133
2017-07-30main: save memory profile every 60 secondsJakob Unterwurzacher
...and move all profiling functionality to its own file, as the main function is already long enough. Periodically saving the memory profile allows capturing the used memory during normal operation, as opposed to on exit, where the kernel has already issued FORGETs for all inodes. This functionality has been used to create the memory profile shown in https://github.com/rfjakob/gocryptfs/issues/132 .
2017-07-29main: doMount: call FreeOSMemory() before jumping into server loopJakob Unterwurzacher
scrypt (used during masterkey decryption) allocates a lot of memory. Go only returns memory to the OS after 5 minutes, which looks like a waste. Call FreeOSMemory() to return it immediately. Looking a fresh mount: before: VmRSS: 73556 kB after: VmRSS: 8568 kB
2017-07-29fusefronted_reverse: fix ino collision between .name and .diriv filesJakob Unterwurzacher
A directory with a long name has two associated virtual files: the .name file and the .diriv files. These used to get the same inode number: $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name With this change we use another prefix (2 instead of 1) for .name files. $ ls -di1 * */* 33313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw 1000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw/gocryptfs.diriv 2000000000033313535 gocryptfs.longname.2togDFouca9mrTwtfF1RNW5DZRAQY8alaR7wO_Xd5Zw.name
2017-07-27fusefrontend_reverse: return ENOENT for undecryptable namesJakob Unterwurzacher
This was working until DecryptName switched to returning EBADMSG instead of EINVAL. Add a test to catch the regression next time.
2017-07-24main: redirectStdFds: keep logger from holding stdout openJakob Unterwurzacher
We passed our stdout and stderr to the new logger instance, which makes sense to see any error message, but also means that the fd is kept open even when we close it. Fixes the new TestMountBackground test and https://github.com/rfjakob/gocryptfs/issues/130 .
2017-07-24tests: check if we close stderr and stdout correctly on mountJakob Unterwurzacher
Currently fails, as reported at https://github.com/rfjakob/gocryptfs/issues/130 .
2017-07-23main: move redirectStdFds() to daemonize.goJakob Unterwurzacher
This really is a part of daemonization. No code changes.
2017-07-21tests: fsstress-gocryptfs.bash: sync up with EncFSJakob Unterwurzacher
I have added a subset of fsstress-gocryptfs.bash to EncFS as fsstress-encfs.sh, improving the code a bit. This change forward-ports these improvements to fsstress-gocryptfs.bash.
2017-07-14macos: make testing without openssl work properlyJakob Unterwurzacher
On MacOS, building and testing without openssl is much easier. The tests should skip tests that fail because of missing openssl instead of aborting. Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-14stupidgcm: fix openssl 1.1 build failureJakob Unterwurzacher
Fixed by including the correct header. Should work on older openssl versions as well. Error was: locking.go:21: undefined reference to `CRYPTO_set_locking_callback'
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-05 05:42:10 +0000