aboutsummaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
2018-09-23tlog: always trim trailing newlinesJakob Unterwurzacher
The messages we print through tlog sometimes do, sometimes do not contain a trailing newline. The stdlib logger usually drops trailing newlines automatically, but tlog postfixes ColorReset to the caller's message, so the logger logic does not work when we print colored output. Drop the newlines on our own, and add a test. Fixes the blank lines in fsck output: ~/go/src/github.com/rfjakob/gocryptfs/tests/fsck$ ./run_fsck.bash Reading password from extpass program Decrypting master key OpenDir "": invalid entry "invalid_file_name.3": illegal base64 data at input byte 17 OpenDir "": invalid entry "invalid_file_name_2": bad message fsck: corrupt entry in dir "": "invalid_file_name.3" fsck: corrupt entry in dir "": "invalid_file_name_2" OpenDir "": invalid entry "invalid_file_name____1": bad message fsck: corrupt entry in dir "": "invalid_file_name____1" doRead 4327225: corrupt block #0: stupidgcm: message authentication failed fsck: error reading file "corrupt_file" (inum 4327225): 5=input/output error cipherSize 40 < overhead 50: corrupt file doRead 4327074: corrupt header: ParseHeader: invalid version, want=2 have=22616 cipherSize 40 < overhead 50: corrupt file fsck: error reading file "corrupt_file_2" (inum 4327074): 5=input/output error Readlink "s-P7PcQDUcVkoeMDnC3EYA": decrypting target failed: stupidgcm: message authentication failed fsck: error reading symlink "corrupt_symlink": 5=input/output error Readlink "iI0MtUdzELPeOAZYwYZFee169hpGgd3l2PXQBcc9sl4": decrypting target failed: illegal base64 data at input byte 0 fsck: error reading symlink "corrupt_symlink_2": 5=input/output error OpenDir "yrwcjj2qoC4IYvhw9sbfRg": could not read gocryptfs.diriv: wanted 16 bytes, got 17 fsck: error opening dir "diriv_too_long": 5=input/output error OpenDir "trqecbMNXdzLqzpk7fSfKw": could not read gocryptfs.diriv: wanted 16 bytes, got 3 fsck: error opening dir "diriv_too_short": 5=input/output error cipherSize 8 < header size 18: corrupt file readFileID 4327049: incomplete file, got 8 instead of 19 bytes fsck: corrupt file "incomplete_file_1" (inode 4327049) readFileID 4327038: incomplete file, got 18 instead of 19 bytes fsck: corrupt file "incomplete_file_2" (inode 4327038) cipherSize 1 < header size 18: corrupt file readFileID 4327063: incomplete file, got 1 instead of 19 bytes fsck: corrupt file "incomplete_file_3" (inode 4327063) fsck: error opening dir "missing_diriv": 2=no such file or directory ListXAttr: invalid xattr name "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0": bad message fsck: corrupt xattr name on file "xattr_corrupt_name": "user.gocryptfs.0a5e7yWl0SGUGeWB0Sy2K0" GetXAttr: stupidgcm: message authentication failed fsck: error reading xattr "user.foo" from "xattr_corrupt_value": 5=input/output error fsck summary: 15 corrupt files
2018-09-22syscallcompat: untangle Openat flag checkJakob Unterwurzacher
Check for O_NWFOLLOW and O_EXCL separately to make the logic clearer.
2018-09-08fusefrontend: use OpenDirNofollow in openBackingDirJakob Unterwurzacher
Rename openBackingPath to openBackingDir and use OpenDirNofollow to be safe against symlink races. Note that openBackingDir is not used in several important code paths like Create(). But it is used in Unlink, and the performance impact in the RM benchmark to be acceptable: Before $ ./benchmark.bash Testing gocryptfs at /tmp/benchmark.bash.bYO: gocryptfs v1.6-12-g930c37e-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3 WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.07979 s, 243 MB/s READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.882413 s, 297 MB/s UNTAR: 16.703 MD5: 7.606 LS: 1.349 RM: 3.237 After $ ./benchmark.bash Testing gocryptfs at /tmp/benchmark.bash.jK3: gocryptfs v1.6-13-g84d6faf-dirty; go-fuse v20170619-49-gb11e293; 2018-09-08 go1.10.3 WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.06261 s, 247 MB/s READ: 262144000 bytes (262 MB, 250 MiB) copied, 0.947228 s, 277 MB/s UNTAR: 17.197 MD5: 7.540 LS: 1.364 RM: 3.410
2018-09-08syscallcompat: use O_PATH in OpenDirNofollowJakob Unterwurzacher
This fixes the "0100 directory" problem in reverse mode, and should be slightly faster.
2018-09-08syscallcompat: untangle OpenNofollow and rename to OpenDirNofollowJakob Unterwurzacher
The function used to do two things: 1) Walk the directory tree in a manner safe from symlink attacks 2) Open the final component in the mode requested by the caller This change drops (2), which was only used once, and lets the caller handle it. This simplifies the function and makes it fit for reuse in forward mode in openBackingPath(), and for using O_PATH on Linux.
2018-09-08configfile: add LoadAndDecrypt wrapperJakob Unterwurzacher
Callers that do not want to decrypt the masterkey should call plain Load(). https://github.com/rfjakob/gocryptfs/issues/258
2018-09-08configfile: split off masterkey decryptionJakob Unterwurzacher
Preparation for fixing https://github.com/rfjakob/gocryptfs/issues/258
2018-09-08configfile: return specific error on empty inputJakob Unterwurzacher
Report the actual problem instead of a generic "unexpected end of JSON input". https://github.com/rfjakob/gocryptfs/issues/258
2018-08-26syscallcompat: drop Fchmodat flagsv1.6.1Jakob Unterwurzacher
These were silently ignored until now (!) but are rejected by Go 1.11 stdlib. Drop the flags so the tests work again, until we figure out a better solution. https://github.com/golang/go/issues/20130
2018-08-15trezor: show support in version stringJakob Unterwurzacher
Show enable_trezor in the version string if we were compiled with `-tags enable_trezor`. And hide the `-trezor` flag from the help output if we were not.
2018-08-15fusefrontend: truncateGrowFile: pass zeroPad error to callerJakob Unterwurzacher
Errors from zeroPad were ignored until now, as discovered using xfstests generic/083.
2018-08-15fusefrontend_reverse: reject excludes for the root directory ""Jakob Unterwurzacher
This is most likely a mistake by the user. Reject it.
2018-08-11reverse mode: add --exclude optionJakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/235
2018-07-23configfile: drop superflous PrintfJakob Unterwurzacher
Before: $ gocryptfs -fsck . LoadConfFile: ReadFile: &os.PathError{Op:"open", Path:"/var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf", Err:0xd} Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied After: $ gocryptfs -fsck . Cannot open config file: open /var/tmp/check-gocryptfs/scratchdev/gocryptfs.conf: permission denied
2018-07-22fusefronted: disallow writes running concurrently with readsJakob Unterwurzacher
As uncovered by xfstests generic/465, concurrent reads and writes could lead to this, doRead 3015532: corrupt block #1039: stupidgcm: message authentication failed, as the read could pick up a block that has not yet been completely written - write() is not atomic! Now writes take ContentLock exclusively, while reads take it shared, meaning that multiple reads can run in parallel with each other, but not with a write. This also simplifies the file header locking.
2018-07-15fusefrontend: doWrite: delete file header if first write failsJakob Unterwurzacher
xfstests generic/083 fills the filesystem almost completely while running fsstress in parallel. In fsck, these would show up: readFileID 2580: incomplete file, got 18 instead of 19 bytes This could happen when writing the file header works, but writing the actual data fails. Now we kill the header again by truncating the file to zero.
2018-07-15fusefrontend: doWrite: no need to take HeaderLock.RLock()Jakob Unterwurzacher
Other writers are blocked by ContentLock already.
2018-07-15WriteDirIV: delete incomplete gocryptfs.diriv file if write failsJakob Unterwurzacher
If the underlying filesystem is full, writing to gocryptfs.diriv may fail, and later fsck show this: OpenDir "xyz": could not read gocryptfs.diriv: wanted 16 bytes, got 0 Uncovered by xfstests generic/083. Also fixes a fd leak in the error path.
2018-07-15contentenc: reserve one extra block in pool plaintext buffersJakob Unterwurzacher
File holes and -fsck can cause unaligned read accesses, which means we have to decrypt one extra plaintext block. xfstests generic/083 manage to crash -fsck like this: generic/083 2018/07/14 15:25:21 wrong len=266240, want=131072 panic: wrong len=266240, want=131072 goroutine 1 [running]: log.Panicf(0x67fc00, 0x15, 0xc4204fec90, 0x2, 0x2) /usr/local/go/src/log/log.go:333 +0xda github.com/rfjakob/gocryptfs/internal/contentenc.(*bPool).Put(0xc4200d4800, 0xc4202f2000, 0x21000, 0x41000) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/contentenc/bpool.go:27 +0x15d github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc4200b4500, 0xc42019e000, 0x0, 0x20000, 0x28400, 0x20000, 0xc42019e000, 0xc4204ff008, 0x435164, 0xc420000180) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:227 +0xba9 github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Read(0xc4200b4500, 0xc42019e000, 0x20000, 0x20000, 0x28400, 0x0, 0x0, 0x0) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:246 +0x23e main.(*fsckObj).file(0xc420069320, 0xc42001a630, 0x21) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:126 +0x21f main.(*fsckObj).dir(0xc420069320, 0xc420014dc0, 0x1d) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:76 +0x387 main.(*fsckObj).dir(0xc420069320, 0xc42021dae0, 0x19) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:74 +0x347
2018-07-14fusefrontend: log prealloc failures at Info levelJakob Unterwurzacher
If the underlying filesystem is full, it is normal get ENOSPC here. Log at Info level instead of Warning. Fixes xfstests generic/015 and generic/027, which complained about the extra output.
2018-07-04macos: fix O_DIRECT build failureJakob Unterwurzacher
O_DIRECT has no direct equivalent on MacOS (check out https://github.com/libuv/libuv/issues/1600 for details). Just define it to zero there.
2018-07-02fusefrontend: disallow O_DIRECT and fall back to buffered IOJakob Unterwurzacher
O_DIRECT accesses must be aligned in both offset and length. Due to our crypto header, alignment will be off, even if userspace makes aligned accesses. Running xfstests generic/013 on ext4 used to trigger lots of EINVAL errors due to missing alignment. Just fall back to buffered IO.
2018-07-02fusefronted: downgrade fallocate message severityJakob Unterwurzacher
The message causes output mismatches in xfstests generic/112. Downgrade the severity to Info so it gets disabled when using "-q".
2018-07-01Fix golint warningsJakob Unterwurzacher
2018-07-01trezor: hide behind compile tagJakob Unterwurzacher
The trezor libraries are not yet stable enough to build gocryptfs with trezor support by default. It does not even compile at the moment: $ ./build.bash -tags enable_trezor # github.com/conejoninja/tesoro/vendor/github.com/trezor/usbhid ../../conejoninja/tesoro/vendor/github.com/trezor/usbhid/hid.go:32:11: fatal error: os/threads_posix.c: No such file or directory #include "os/threads_posix.c" ^~~~~~~~~~~~~~~~~~~~ compilation terminated. https://github.com/conejoninja/tesoro/issues/9
2018-07-01fusefrontend: add File.SeekData() functionJakob Unterwurzacher
This function will enable "gocryptfs -fsck" to handle sparse files efficiently.
2018-07-01fusefrontend: export "File" typeJakob Unterwurzacher
"gocryptfs -fsck" will need access to helper functions, and to get that, it will need to cast a gofuse.File to a fusefrontend.File. Make fusefrontend.File exported to make this work.
2018-07-01fsck: rename "CorruptItems" channel to "MitigatedCorruptions"Jakob Unterwurzacher
Make it clear that this channel is only used to report corruptions that are transparently mitigated and do not return an error to the user.
2018-07-01Only print masterkey once on -initJakob Unterwurzacher
It is no longer printed at all when mounting a filesystem, printing on -init can be disabled with -q. https://github.com/rfjakob/gocryptfs/issues/76
2018-07-01trezor: add sanity checks for decrypted valueJakob Unterwurzacher
Check that the value has changed, is not all-zero and has the right length.
2018-07-01Implemented the support of Trezor devices.Dmitry Yu Okunev
2018-07-01trezor: add TrezorPayloadJakob Unterwurzacher
TrezorPayload stores 32 random bytes used for unlocking the master key using a Trezor security module. The randomness makes sure that a unique unlock value is used for each gocryptfs filesystem.
2018-07-01configfile: reduce function name stutterJakob Unterwurzacher
configfile.LoadConfFile() -> configfile.Load() configfile.CreateConfFile() -> configfile.Create()
2018-07-01trezor: add skeleton for Trezor supportJakob Unterwurzacher
readpassword.Trezor() is not implemented yet and returns a hardcoded dummy key.
2018-06-24configfile: use tlog.ColorYellow instead of hardcoded color codeJakob Unterwurzacher
2018-06-19Fix three golint warningsJakob Unterwurzacher
We are clean again. Warnings were: internal/fusefrontend/fs.go:443:14: should omit type string from declaration of var cTarget; it will be inferred from the right-hand side internal/fusefrontend/xattr.go:26:1: comment on exported method FS.GetXAttr should be of the form "GetXAttr ..." internal/syscallcompat/sys_common.go:9:7: exported const PATH_MAX should have comment or be unexported
2018-06-12xattr: return EOPNOTSUPP instead of ENODATA in GetXattrJakob Unterwurzacher
Reading system.posix_acl_access and system.posix_acl_default should return EOPNOTSUPP to inform user-space that we do not support ACLs. xftestest essientially does chacl -l | grep "Operation not supported" to determine if the filesystem supports ACLs, and used to wrongly believe that gocryptfs does.
2018-06-08darwin does not have PATH_MAXJakob Unterwurzacher
Define our own, with the value from Linux.
2018-05-27xattr: use LGet/LSet etcJakob Unterwurzacher
Support has been merged into the xattr package ( https://github.com/pkg/xattr/pull/29 ), use it.
2018-05-15xattr: return EOPNOTSUPP for unsupported attributesJakob Unterwurzacher
mv is unhappy when we return EPERM when it tries to set system.posix_acl_access: mv: preserving permissions for ‘b/x’: Operation not permitted Now we return EOPNOTSUPP like tmpfs does and mv seems happy.
2018-05-10xattr: optimize storage, store as binary instead of bae64Bolshevik
Values a binary-safe, there is no need to base64-encode them. Old, base64-encoded values are supported transparently on reading. Writing xattr values now always writes them binary.
2018-05-10stupidgcm: return error on too short input instead of panicingJakob Unterwurzacher
This is what Go GCM does as well.
2018-05-07xattr: added passing of a "flags" parameterBolshevik
Pass the "flags" parameter to the lower layer syscall. This makes Apple applications being able to successfully save data.
2018-05-01fusefrontend: xattr: return ENOSYS on unsupported flagsJakob Unterwurzacher
We previously returned EPERM to prevent the kernel from blacklisting our xattr support once we get an unsupported flag, but this causes lots of trouble on MacOS: Cannot save files from GUI apps, see https://github.com/rfjakob/gocryptfs/issues/229 Returning ENOSYS triggers the dotfiles fallback on MacOS and fixes the issue.
2018-04-17Improved xattr handling on non-linux systems (#227)bolshevik
* Fixed xattr filtering for MacOS. "system." and "user." prefixes are only relevant for Linux. * Small cleanup and additional tests.
2018-04-08Fix the easy golint warningsJakob Unterwurzacher
Reported by https://goreportcard.com/report/github.com/rfjakob/gocryptfs
2018-04-03fsck: report skipped corrupt filesJakob Unterwurzacher
OpenDir and ListXAttr skip over corrupt entries, readFileID treats files the are too small as empty. This improves usability in the face of corruption, but hides the problem in a log message instead of putting it in the return code. Create a channel to report these corruptions to fsck so it can report them to the user. Also update the manpage and the changelog with the -fsck option. Closes https://github.com/rfjakob/gocryptfs/issues/191
2018-04-02fusefrontend: xattr: return ENODATA for security.* and system.*Jakob Unterwurzacher
"ls -l" queries security.selinux, system.posix_acl_access, system.posix_acl_default and throws error messages if it gets something else than ENODATA.
2018-04-02fsck: test against example_filesystemsJakob Unterwurzacher
2018-04-02fsck: clean up log outputJakob Unterwurzacher
Make sure we get only 1 warning output per problem. Also, add new corruption types to broken_fs_v1.4.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-05 13:40:29 +0000