aboutsummaryrefslogtreecommitdiff
path: root/internal/fusefrontend/file.go
AgeCommit message (Collapse)Author
2024-12-04fusefrontend: sharedstorage: add file content byte-range locksJakob Unterwurzacher
As we must write complete ciphertext blocks (except at EOF), non-overlapping plaintext writes can overlap in the ciphertext. And because overlapping writes can turn the data into data soup (see TestPoCTornWrite) we serialize them using fcntl locking.
2024-12-04fusefrontend: sharedstorage: use byte-range lock on file header creationJakob Unterwurzacher
Multiple host writing to the same empty file at the same time could have overwritten each other's newly created file header, leading to data corruption. Fix the race by placing a byte-range lock on the file when creating the file header.
2024-03-09fusefrontend: fix excessive file fragmentation on BTRFSAlex Shumsky
2023-02-21fusefrontend: unbreak isConsecutiveWrite streaming write optimizationJakob Unterwurzacher
Commit 6196a5b5 got the logic inverted, hence we never set the last position markers. Fixes https://github.com/rfjakob/gocryptfs/issues/712
2023-02-21fusefrontend: doWrite: report readFileID errors as I/O errorJakob Unterwurzacher
It used to be reported as "function not implemented", accompanied with this log output: go-fuse: can't convert error type: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000 Now we report EIO and log this: doWrite 1372183: corrupt header: ParseHeader: header is all-zero. Header hexdump: 000000000000000000000000000000000000
2022-06-26Fix typosYuta Hayashibe
2021-09-10cli: drop -forcedecode flagJakob Unterwurzacher
The rewritten openssl backend does not support this flag anymore, and it was inherently dangerour. Drop it (ignored for compatibility)
2021-08-30fusefrontend: remove leftover PrintfJakob Unterwurzacher
Commit b83ca9c921019fc3b790dabb6198bb77ef2f9a34 inadveredly added a leftover debug Printf. Delete it.
2021-08-30Remove serialize_reads packageJakob Unterwurzacher
Will be replaced by go-fuse's new SyncRead flag. More info: https://github.com/hanwen/go-fuse/issues/395 SyncRead commit: https://github.com/hanwen/go-fuse/commit/15a8bb029a4e1a51e10043c370970596b1fbb737
2021-08-23go mod: declare module version v2Jakob Unterwurzacher
Our git version is v2+ for some time now, but go.mod still declared v1. Hopefully making both match makes https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work. All the import paths have been fixed like this: find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/%
2021-08-18Fix issues found by ineffassignJakob Unterwurzacher
gocryptfs$ ineffassign ./... /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:243:2: ineffectual assignment to scryptHash /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:272:2: ineffectual assignment to scryptHash /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:285:3: ineffectual assignment to fileID /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node.go:367:3: ineffectual assignment to err /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/node_open_create.go:68:2: ineffectual assignment to fd /home/jakob/go/src/github.com/rfjakob/gocryptfs/mount.go:308:2: ineffectual assignment to masterkey /home/jakob/go/src/github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_main.go:156:13: ineffectual assignment to err /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/prepare_syscall_test.go:65:16: ineffectual assignment to errno /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/syscallcompat/open_nofollow_test.go:34:2: ineffectual assignment to fd /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:111:6: ineffectual assignment to err /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:181:2: ineffectual assignment to sz /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/acl_test.go:198:2: ineffectual assignment to sz /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/defaults/main_test.go:365:8: ineffectual assignment to err /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:30:6: ineffectual assignment to err /home/jakob/go/src/github.com/rfjakob/gocryptfs/tests/xattr/xattr_fd_test.go:66:6: ineffectual assignment to err
2021-08-11syscallcompat: move quirks logic here & fix darwinJakob Unterwurzacher
We need to look at f_fstypename acc. to https://stackoverflow.com/a/52299141/1380267 : > As filesystem type numbers are now assigned at runtime in > recent versions of MacOS, you must use f_fstypename to > determine the type. https://github.com/rfjakob/gocryptfs/issues/585
2021-08-02fusefrontend: add quirks for MacOS ExFATJakob Unterwurzacher
This also moves the quirks logic into fusefrontend. Fixes https://github.com/rfjakob/gocryptfs/issues/585
2021-07-29fusefrontend: implement fsync on directoriesJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/587
2021-06-02fusefrontend: catch ReadAt integer overflowJakob Unterwurzacher
Discovered by xfstests generic/564 . Failure was: generic/564 - output mismatch (see /opt/fuse-xfstests/results//generic/564.out.bad) --- tests/generic/564.out 2021-05-08 21:11:05.307395966 +0200 +++ /opt/fuse-xfstests/results//generic/564.out.bad 2021-05-19 19:01:16.912888879 +0200 @@ -31,7 +31,7 @@ source range beyond 8TiB returns 0 destination range beyond 8TiB returns EFBIG -copy_range: File too large +copy_range: Function not implemented
2021-03-21fusefronted: move Create() and Open() to new fileJakob Unterwurzacher
And deduplicate the code a little.
2020-10-14syscallcompat: don't retry Close()Jakob Unterwurzacher
After Close() returns, the fd is dead, even if we received EINTR. Don't retry, we could shoot down an unrelated fd that received the same fd number.
2020-10-14syscallcompat: retry ops on EINTRJakob Unterwurzacher
Retry operations that have been shown to throw EINTR errors on CIFS. Todo: Solution for this pain in the back: warning: unix.Getdents returned errno 2 in the middle of data rm: cannot remove 'linux-3.0.old3/Documentation/ABI/removed': Input/output error Progress towards fixing https://github.com/rfjakob/gocryptfs/issues/483 .
2020-08-16v2api: rename "File2" to just "File"Jakob Unterwurzacher
Rename the symbols and the files.
2020-07-26v2api: delete (most) fusefrontend v1 filesJakob Unterwurzacher
All the functionality in these files has been reimplemented for the v2 api. Drop the old files.
2020-05-17Update go-fuse import path to github.com/hanwen/go-fuse/v2Jakob Unterwurzacher
We need https://github.com/hanwen/go-fuse/commit/fd7328faf9fdf75709f7ba7df7072aaf4eeb18b3 to fix a crash reported in https://github.com/rfjakob/gocryptfs/issues/430 : 2019/10/30 17:14:16 Unknown opcode 2016 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x508d38] This patch is only in the v2.x.x branch. Upgrade to v2, as the old API is also supported there. Running git grep hanwen/go-fuse | grep -v hanwen/go-fuse/v2 to check for forgotten references comes back clean.
2020-04-13inomap: split into separate packageJakob Unterwurzacher
inomap will also be used by fusefrontend_reverse in the future. Split if off openfiletable to make it independent.
2019-11-16fusefrontend: use inummapJakob Unterwurzacher
translate inode numbers on different devices to fix collisions. Fixes https://github.com/rfjakob/gocryptfs/issues/435
2019-10-06fusefrontend: unregister from openfiletable before closing the fdJakob Unterwurzacher
Closing the fd means the inode number may be reused immediately by a new file, so we have to get the old fileID out of the table beforehand! Hopefully fixes https://github.com/rfjakob/gocryptfs/issues/363
2019-10-06fusefrontend: print file hexdump on header errorJakob Unterwurzacher
This should help debugging https://github.com/rfjakob/gocryptfs/issues/363 , but does no harm in normal operation as it only prints ciphertext to the log.
2019-01-16fusefrontend: Always use intFd() method instead of int(f.fd.Fd()).Sebastian Lackner
2019-01-16fusefrontend: Rework the Utimens handling on macOS.Sebastian Lackner
For Linux, everything effectively stays the same. For both path-based and fd-based Utimens() calls, we use unix.UtimesNanoAt(). To avoid introducing a separate syscall wrapper for futimens() (as done in go-fuse, for example), we instead use the /proc/self/fd - trick. On macOS, this changes quite a lot: * Path-based Utimens() calls were previously completely broken, since unix.UtimensNanoAt() ignores the passed file descriptor. Note that this cannot be fixed easily since there IS no appropriate syscall available on macOS prior to High Sierra (10.13). We emulate this case by using Fchdir() + setattrlist(). * Fd-based Utimens() calls were previously translated to f.GetAttr() (to fill any empty parameters) and syscall.Futimes(), which does not does support nanosecond precision. Both issues can be fixed by switching to fsetattrlist(). Fixes https://github.com/rfjakob/gocryptfs/issues/350
2019-01-01fusefrontend: Fix debug message in doWrite() method.Sebastian Lackner
2019-01-01fusefrontend: Fix order of arguments in debug message for Read() FUSE call.Sebastian Lackner
2018-12-27fusefrontend: Remove unnecessary check in doRead function.Sebastian Lackner
The same condition is already checked a few lines above, and 'err' is not changed inbetween.
2018-10-17fusefronted: log more details on WriteAt failuresJakob Unterwurzacher
Also log inode number, fd number, offset and length. Maybe help debugging https://github.com/rfjakob/gocryptfs/issues/269 .
2018-07-22fusefronted: disallow writes running concurrently with readsJakob Unterwurzacher
As uncovered by xfstests generic/465, concurrent reads and writes could lead to this, doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed, as the read could pick up a block that has not yet been completely written - write() is not atomic! Now writes take ContentLock exclusively, while reads take it shared, meaning that multiple reads can run in parallel with each other, but not with a write. This also simplifies the file header locking.
2018-07-15fusefrontend: doWrite: delete file header if first write failsJakob Unterwurzacher
xfstests generic/083 fills the filesystem almost completely while running fsstress in parallel. In fsck, these would show up: readFileID 2580: incomplete file, got 18 instead of 19 bytes This could happen when writing the file header works, but writing the actual data fails. Now we kill the header again by truncating the file to zero.
2018-07-15fusefrontend: doWrite: no need to take HeaderLock.RLock()Jakob Unterwurzacher
Other writers are blocked by ContentLock already.
2018-07-14fusefrontend: log prealloc failures at Info levelJakob Unterwurzacher
If the underlying filesystem is full, it is normal get ENOSPC here. Log at Info level instead of Warning. Fixes xfstests generic/015 and generic/027, which complained about the extra output.
2018-07-01Fix golint warningsJakob Unterwurzacher
2018-07-01fusefrontend: export "File" typeJakob Unterwurzacher
"gocryptfs -fsck" will need access to helper functions, and to get that, it will need to cast a gofuse.File to a fusefrontend.File. Make fusefrontend.File exported to make this work.
2018-07-01fsck: rename "CorruptItems" channel to "MitigatedCorruptions"Jakob Unterwurzacher
Make it clear that this channel is only used to report corruptions that are transparently mitigated and do not return an error to the user.
2018-04-03fsck: report skipped corrupt filesJakob Unterwurzacher
OpenDir and ListXAttr skip over corrupt entries, readFileID treats files the are too small as empty. This improves usability in the face of corruption, but hides the problem in a log message instead of putting it in the return code. Create a channel to report these corruptions to fsck so it can report them to the user. Also update the manpage and the changelog with the -fsck option. Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-02fsck: clean up log outputJakob Unterwurzacher
Make sure we get only 1 warning output per problem. Also, add new corruption types to broken_fs_v1.4.
2018-04-01fusefronted: reject oversized Read and Write requestsJakob Unterwurzacher
This should not happen via FUSE as the kernel caps the size, but with fsck we have the first user that calls Read directly. For symmetry, check it for Write as well.
2017-10-21Revert most of "fusefrontend: clamp oversized reads"Jakob Unterwurzacher
We cannot return less data than requested to the kernel! From https://libfuse.github.io/doxygen/structfuse__operations.html: Read should return exactly the number of bytes requested except on EOF or error, otherwise the rest of the data will be substituted with zeroes. Reverts commit 3009ec9852316c3c696f77f476390ab5a6d8d6d7 minus the formatting improvements we want to keep. Fixes https://github.com/rfjakob/gocryptfs/issues/147 Reopens https://github.com/rfjakob/gocryptfs/issues/145
2017-10-17fusefrontend: clamp oversized readsJakob Unterwurzacher
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the running kernel may have a higher limit set. Clamp to what we can handle. Fixes a panic on a Synology NAS reported at https://github.com/rfjakob/gocryptfs/issues/145
2017-07-02fusefrontend: doRead: skip decryption for an empty readJakob Unterwurzacher
Previously we ran through the decryption steps even for an empty ciphertext slice. The functions handle it correctly, but returning early skips all the extra calls. Speeds up the tar extract benchmark by about 4%.
2017-06-30contentenc: add PReqPool and use it in DecryptBlocksJakob Unterwurzacher
This gets us a massive speed boost in streaming reads.
2017-06-30fusefrontend: doRead: use CReqPool for ciphertext bufferJakob Unterwurzacher
Easily saves lots of allocations.
2017-06-30fusefrontend: Read: use provided bufferJakob Unterwurzacher
This will allow us to return internal buffers to a pool.
2017-06-29contentenc: add safer "bPool" pool variant; add pBlockPoolJakob Unterwurzacher
bPool verifies the lengths of slices going in and out. Also, add a plaintext block pool - pBlockPool - and use it for decryption.
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-09Fix missing Owner coercion for already-open files (#117)Charles Duffy
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-23 15:57:33 +0000