aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-06-19fusefrontend: sharedstorage: add warnings for lock failureLockSharedStorageJakob Unterwurzacher
Complain loudly when the underlying storage does not support byte-range locks. https://github.com/rfjakob/gocryptfs/issues/754
2023-06-19fusefrontend: sharedstorage: retry read-path on EIO errorJakob Unterwurzacher
With -sharedstorage, when we get a decryption error, we lock the byte range and try again. This makes concurrent R/W safe agains torn writes. https://github.com/rfjakob/gocryptfs/issues/754
2023-06-09tests/cluster: enable TestClusterConcurrentRW per defaultJakob Unterwurzacher
Add the -sharedstorage flag to the mounts and make it more severe by adjusting the I/O and file sizes to arbitrary unaligned values. It passes *most of the time*. If it fails, it's like this, and the reason is that there is no fcntl locks in the READ path yet. gocryptfs/tests/cluster$ go test -run TestClusterConcurrentRW -v === RUN TestClusterConcurrentRW doRead 2895433: corrupt block #0: cipher: message authentication failed doRead 2895433: corrupt block #0: cipher: message authentication failed doRead 2895433: corrupt block #0: cipher: message authentication failed cluster_test.go:98: readThread iteration 9165: ReadAt failed: read /var/tmp/gocryptfs-test-parent-1026/3021427391/TestClusterConcurrentRW.366469887.mnt1/foo: input/output error --- FAIL: TestClusterConcurrentRW (0.40s) FAIL exit status 1 FAIL github.com/rfjakob/gocryptfs/v2/tests/cluster 0.406s
2023-06-09fusefrontend: sharedstorage: add file content byte-range locksJakob Unterwurzacher
As we must write complete ciphertext blocks (except at EOF), non-overlapping plaintext writes can overlap in the ciphertext. And because overlapping writes can turn the data into data soup (see TestPoCTornWrite) we serialize them using fcntl locking.
2023-06-09tests/cluster: add TestPoCTornWriteLockedJakob Unterwurzacher
2023-06-09tests/cluster: note that ext4 provides no atomicityJakob Unterwurzacher
2023-06-09tests/cluster: simplify TestPoCTornWriteJakob Unterwurzacher
Turns out we can get the same error with a simpler flow.
2023-06-09tests/cluster: add TestPoCTornWriteJakob Unterwurzacher
Scary. But explains why TestConcurrentCreate fails. gocryptfs/tests/cluster$ go test -run TestPoCTornWrite --- FAIL: TestPoCTornWrite (0.00s) poc_test.go:210: iteration 214: inconsistent block: d6d6d6d6d6d6d6d6d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1 FAIL
2023-06-09tests/cluster: TestConcurrentCreate: make sure buf2 is zero'edJakob Unterwurzacher
2023-06-06fusefrontend: sharedstorage: keep file header on truncateJakob Unterwurzacher
With -sharedstorage, we keep the on-disk file header. Other mounts may have the file ID cached so we cannot mess with it. This makes TestOpenTruncate pass.
2023-06-06tests/cluster: add TestOpenTruncateJakob Unterwurzacher
This fails right now: 1 jakob@brikett:~/go/src/github.com/rfjakob/gocryptfs/tests/cluster$ go test -run TestOpenTruncate -v === RUN TestOpenTruncate cluster_test.go:235: POSIX compliance issue: non-exlusive create failed with err=file exists doWrite 2898550: corrupt header: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000 cluster_test.go:240: iteration 1: WriteAt: write /var/tmp/gocryptfs-test-parent-1026/1896094179/TestOpenTruncate.4202105280.mnt2/foo: input/output error --- FAIL: TestOpenTruncate (0.10s) FAIL exit status 1 FAIL github.com/rfjakob/gocryptfs/v2/tests/cluster 0.099s 1 jakob@brikett:~/go/src/github.com/rfjakob/gocryptfs/tests/cluster$ go test -run TestOpenTruncate -v === RUN TestOpenTruncate cluster_test.go:235: POSIX compliance issue: non-exlusive create failed with err=file exists doRead 2898565: corrupt block #0: cipher: message authentication failed ino2898565 fh9: RMW read failed: errno=5 cluster_test.go:240: iteration 8: WriteAt: write /var/tmp/gocryptfs-test-parent-1026/652691834/TestOpenTruncate.281532388.mnt1/foo: input/output error --- FAIL: TestOpenTruncate (0.09s) FAIL exit status 1 FAIL github.com/rfjakob/gocryptfs/v2/tests/cluster 0.095s
2023-06-06tests/cluster: add TestPoCHeaderCreationJakob Unterwurzacher
2023-06-06tests/cluster: move PoC tests to poc_test.goJakob Unterwurzacher
2023-06-06tests/cluster: add TestConcurrentCreateJakob Unterwurzacher
This exercises the byte-range locks we just added.
2023-06-05fusefrontend: sharedstorage: use byte-range lock on file header creationJakob Unterwurzacher
Multiple hosts creating the same file at the same time could have overwritten each other's file header, leading to data corruption. Fix the race by placing a byte-range lock on the file when creating the file header.
2023-05-30tests: add cluster testJakob Unterwurzacher
finds out what happens if multiple gocryptfs mounts write to one file concurrently (usually, nothing good). This use case is relevant for HPC clusters.
2023-05-19fsstress-gocryptfs.bash: improve header commentJakob Unterwurzacher
I maybe should have noted that this is xfstests generic/013.
2023-05-18tests: TestDirectMount: check for default_permissionsJakob Unterwurzacher
2023-05-17fusefrontend: implement our own Access()Jakob Unterwurzacher
Not having Access() means go-fuse emulates it by looking at Getattr(). This works fine most of the time, but breaks down on sshfs, where sshfs-benchmark.bash shows this: gocryptfs/tests$ ./sshfs-benchmark.bash nuetzlich.net working directory: /tmp/sshfs-benchmark.bash.JQC sshfs mounted: nuetzlich.net:/tmp -> sshfs.mnt gocryptfs mounted: sshfs.mnt/sshfs-benchmark.bash.Wrz/gocryptfs.crypt -> gocryptfs.mnt sshfs-benchmark.bash: sshfs gocryptfs-on-sshfs git init 3.98 6.80 rsync 7.71 10.84 rm -R 4.30rm: descend into write-protected directory 'gocryptfs.mnt/git1'? The go-fuse emulation gets it wrong here because sshfs reports permissions but does not enforce them. Implement it ourselves properly.
2023-05-17go.mod: add test dependency github.com/moby/sys/mountinfoJakob Unterwurzacher
2023-05-17mount: set DirectMount: trueJakob Unterwurzacher
Attempt to directly call mount(2) before trying fusermount. This means we can do without fusermount if running as root. https://github.com/rfjakob/gocryptfs/issues/697
2023-05-17tests: TestDirectMount: also check dev, suidJakob Unterwurzacher
2023-05-17mount: drop "max_read="Jakob Unterwurzacher
go-fuse now sets this internally. Regression-tested in TestDirectMount.
2023-05-17mount: set FsName via go-fuseJakob Unterwurzacher
go-fuse now handles setting FsName, including DirectMount, so use that instead of our own solution. Regression-tested in TestDirectMount.
2023-05-17tests: add TestDirectMountJakob Unterwurzacher
This is in preparation of adding directmount capability. It also check that FsName is set correctly, which is in preparation for the next patch.
2023-05-17tests: root_test: use TMPDIR=/var/tmpJakob Unterwurzacher
Otherwise we fail like this on my Fedora 38 box: === RUN TestOverlay DetectQuirks: tmpfs detected, no extended attributes except acls will work. root_test.go:379: No user xattrs! overlay mount will likely fail. 15:15:57.957960 Unimplemented opcode OPCODE-51 root_test.go:398: mount: /tmp/gocryptfs-test-parent-0/3652394902/TestOverlay.2374697046.mnt/merged: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. root_test.go:399: exit status 32 --- FAIL: TestOverlay (0.04s) FAIL Also fix the messed-up DetectQuirks bit test.
2023-05-12canonical-benchmarks.bash: drop page cache of "zero" fileJakob Unterwurzacher
For the streaming read benchmark, we don't want to benchmark the page cache.
2023-05-03cli: don't split multiple-strings flags on commaJakob Unterwurzacher
Looks like I used StringSliceVar (which splits on comma) where I should have always used StringArrayVar (which does not). Bug report contains this example of misbehavoir: #gocryptfs -extpass 'echo abc,123' -init testdir Reading password from extpass program "echo abc", arguments: ["123"] extpass cmd start failed: exec: "echo abc": executable file not found in $PATH Fixes https://github.com/rfjakob/gocryptfs/issues/730
2023-05-03tests: TestParseCliOpts: de-uglify testcase listJakob Unterwurzacher
2023-04-29Update changelog for v2.3.2v2.3.2Jakob Unterwurzacher
2023-04-29Update go-fuse to v2.3.0Jakob Unterwurzacher
go get github.com/hanwen/go-fuse/v2
2023-03-29fusefrontent: report correct size on hard link creationJakob Unterwurzacher
And add a test for it. Fixes https://github.com/rfjakob/gocryptfs/issues/724
2023-03-08speed: GoGCM: start at block size 16Jakob Unterwurzacher
BenchmarkGoGCMBlockSize/16-4 5499200 219.7 ns/op 72.83 MB/s BenchmarkGoGCMBlockSize/32-4 4497284 266.2 ns/op 120.22 MB/s BenchmarkGoGCMBlockSize/64-4 3296336 363.4 ns/op 176.10 MB/s BenchmarkGoGCMBlockSize/128-4 4204794 285.5 ns/op 448.36 MB/s BenchmarkGoGCMBlockSize/256-4 2928472 409.7 ns/op 624.83 MB/s BenchmarkGoGCMBlockSize/512-4 1825164 658.0 ns/op 778.09 MB/s BenchmarkGoGCMBlockSize/1024-4 1000000 1151 ns/op 889.98 MB/s BenchmarkGoGCMBlockSize/2048-4 560275 2135 ns/op 959.47 MB/s BenchmarkGoGCMBlockSize/4096-4 291906 4099 ns/op 999.28 MB/s BenchmarkGoGCMBlockSize/8192-4 148916 8033 ns/op 1019.83 MB/s BenchmarkGoGCMBlockSize/16384-4 75337 15911 ns/op 1029.75 MB/s BenchmarkGoGCMBlockSize/32768-4 37912 31651 ns/op 1035.30 MB/s BenchmarkGoGCMBlockSize/65536-4 19000 64287 ns/op 1019.43 MB/s BenchmarkGoGCMBlockSize/131072-4 9225 127636 ns/op 1026.92 MB/s BenchmarkGoGCMBlockSize/262144-4 4752 252300 ns/op 1039.02 MB/s BenchmarkGoGCMBlockSize/524288-4 2377 504612 ns/op 1038.99 MB/s BenchmarkGoGCMBlockSize/1048576-4 1183 1011637 ns/op 1036.51 MB/s
2023-03-08speed: add per-blocksize GoGCM benchmarksJakob Unterwurzacher
Only visible when you run "go test -bench" like this: $ cd gocryptfs/internal/speed $ go test -bench . goos: linux goarch: amd64 pkg: github.com/rfjakob/gocryptfs/v2/internal/speed cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz BenchmarkStupidGCM-4 202352 5937 ns/op 689.96 MB/s BenchmarkStupidGCMDecrypt-4 206023 5782 ns/op 708.38 MB/s BenchmarkGoGCM-4 291878 4098 ns/op 999.45 MB/s BenchmarkGoGCMBlockSize/1024-4 1000000 1151 ns/op 889.88 MB/s BenchmarkGoGCMBlockSize/2048-4 561182 2134 ns/op 959.60 MB/s BenchmarkGoGCMBlockSize/4096-4 292057 4101 ns/op 998.87 MB/s BenchmarkGoGCMBlockSize/8192-4 149216 8031 ns/op 1020.09 MB/s BenchmarkGoGCMBlockSize/16384-4 75361 15917 ns/op 1029.34 MB/s BenchmarkGoGCMBlockSize/32768-4 37916 31649 ns/op 1035.35 MB/s BenchmarkGoGCMBlockSize/65536-4 19005 63117 ns/op 1038.33 MB/s BenchmarkGoGCMBlockSize/131072-4 9498 126166 ns/op 1038.89 MB/s BenchmarkGoGCMBlockSize/262144-4 4755 252149 ns/op 1039.64 MB/s BenchmarkGoGCMBlockSize/524288-4 2377 504108 ns/op 1040.03 MB/s BenchmarkGoGCMBlockSize/1048576-4 1188 1008675 ns/op 1039.56 MB/s BenchmarkGoGCMDecrypt-4 294664 4059 ns/op 1009.02 MB/s BenchmarkAESSIV-4 46498 25432 ns/op 161.05 MB/s BenchmarkAESSIVDecrypt-4 46908 25509 ns/op 160.57 MB/s BenchmarkXchacha-4 244473 4894 ns/op 836.97 MB/s BenchmarkXchachaDecrypt-4 249710 4798 ns/op 853.75 MB/s BenchmarkStupidXchacha-4 166988 7101 ns/op 576.79 MB/s BenchmarkStupidXchachaDecrypt-4 163093 7240 ns/op 565.72 MB/s BenchmarkStupidChacha-4 184172 6527 ns/op 627.58 MB/s BenchmarkStupidChachaDecrypt-4 179796 6659 ns/op 615.11 MB/s PASS ok github.com/rfjakob/gocryptfs/v2/internal/speed 30.068s
2023-03-04README: update changelog for v2.3.1v2.3.1Jakob Unterwurzacher
2023-02-25Update README.mdrfjakob
Package has been removed from Fedora. https://github.com/rfjakob/gocryptfs/issues/659
2023-02-21fusefrontend: unbreak isConsecutiveWrite streaming write optimizationJakob Unterwurzacher
Commit 6196a5b5 got the logic inverted, hence we never set the last position markers. Fixes https://github.com/rfjakob/gocryptfs/issues/712
2023-02-21fusefrontend: doWrite: report readFileID errors as I/O errorJakob Unterwurzacher
It used to be reported as "function not implemented", accompanied with this log output: go-fuse: can't convert error type: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000 Now we report EIO and log this: doWrite 1372183: corrupt header: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000
2023-02-21contentenc: simplify testRange tablesJakob Unterwurzacher
Get rid of this eyesore.
2023-02-01MANPAGE: add a note on enabling Trash on macOSEvgeny
2023-01-08MANPAGE: scryptn: list how much memory is neededJakob Unterwurzacher
Calculated acc. to https://words.filippo.io/the-scrypt-parameters/ , and add benchmarks to double-check the numbers. They match.
2023-01-07Update MANPAGE.mdGisi0
added which package on linux is needed to use fido2 stick
2023-01-01MANPAGE.md: use correct indefinite article a->ana1346054
2022-12-29github ci: bump actions ; add "stable" and "oldstable" Go versionssetup-go-v3Jakob Unterwurzacher
2022-12-29main: BuildInfo: fix build with Go 1.17 and olderJakob Unterwurzacher
On Go 1.17 and older we get this: Error: ./version.go:67:24: info.Settings undefined (type *debug.BuildInfo has no field or method Settings) Fix the build error by shedding some nice-to-have features.
2022-12-29make formatJakob Unterwurzacher
Run "make format" using go version go1.19.4 linux/amd64
2022-12-29main: refactor BuildInfo codeJakob Unterwurzacher
Simplify and move it into a new file version.go.
2022-12-29Use existing build information for version if not embedded with build scriptDaniel Theophanes
Go1.12 introduced BuildInfo which embeds build information. It does not embed build date to facilitate reproducable builds by default. If build information is embedded from build script, use the information provided by the Go build system.
2022-12-29main: doMount: call Setsid before starting loggerJakob Unterwurzacher
The logger should be in the new background session together with the gocryptfs process. Before: $ xfce4-terminal -x gocryptfs a b $ ps xao pid,ppid,pgid,sid,comm,args PID PPID PGID SID COMMAND COMMAND 192272 1371 192272 192272 gocryptfs /ssd2/jakob.donotbackup/go/bin/gocryptfs -fg -notifypid=192265 a b 192292 192272 192265 192265 logge <defunct> [logger] <defunct> After: $ xfce4-terminal -x gocryptfs a b $ ps xao pid,ppid,pgid,sid,comm,args PID PPID PGID SID COMMAND COMMAND 211714 1371 211714 211714 gocryptfs /ssd2/jakob.donotbackup/go/bin/gocryptfs -fg -notifypid=211708 a b 211776 211714 211714 211714 logger logger -t gocryptfs-211714-logger Fixes https://github.com/rfjakob/gocryptfs/issues/660
2022-12-21go.mod: fix jacobsa/crypto build on riscv64Christian Stewart
Replace dependency jacobsa/crypto with a fork with support for riscv64. Issue: https://github.com/rfjakob/gocryptfs/issues/666 Upstream PR: https://github.com/jacobsa/crypto/issues/13 Unaddressed on jacobsa/crypto: https://github.com/jacobsa/crypto/pull/14#issuecomment-1182744229 Signed-off-by: Christian Stewart <christian@paral.in>
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-18 17:23:42 +0000