aboutsummaryrefslogtreecommitdiff
path: root/internal/contentenc
AgeCommit message (Collapse)Author
2018-07-15contentenc: reserve one extra block in pool plaintext buffersJakob Unterwurzacher
File holes and -fsck can cause unaligned read accesses, which means we have to decrypt one extra plaintext block. xfstests generic/083 manage to crash -fsck like this: generic/083 2018/07/14 15:25:21 wrong len=266240, want=131072 panic: wrong len=266240, want=131072 goroutine 1 [running]: log.Panicf(0x67fc00, 0x15, 0xc4204fec90, 0x2, 0x2) /usr/local/go/src/log/log.go:333 +0xda github.com/rfjakob/gocryptfs/internal/contentenc.(*bPool).Put(0xc4200d4800, 0xc4202f2000, 0x21000, 0x41000) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/contentenc/bpool.go:27 +0x15d github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).doRead(0xc4200b4500, 0xc42019e000, 0x0, 0x20000, 0x28400, 0x20000, 0xc42019e000, 0xc4204ff008, 0x435164, 0xc420000180) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:227 +0xba9 github.com/rfjakob/gocryptfs/internal/fusefrontend.(*File).Read(0xc4200b4500, 0xc42019e000, 0x20000, 0x20000, 0x28400, 0x0, 0x0, 0x0) /home/jakob/go/src/github.com/rfjakob/gocryptfs/internal/fusefrontend/file.go:246 +0x23e main.(*fsckObj).file(0xc420069320, 0xc42001a630, 0x21) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:126 +0x21f main.(*fsckObj).dir(0xc420069320, 0xc420014dc0, 0x1d) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:76 +0x387 main.(*fsckObj).dir(0xc420069320, 0xc42021dae0, 0x19) /home/jakob/go/src/github.com/rfjakob/gocryptfs/fsck.go:74 +0x347
2018-04-02fsck: clean up log outputJakob Unterwurzacher
Make sure we get only 1 warning output per problem. Also, add new corruption types to broken_fs_v1.4.
2018-03-25fusefrontend: add xattr supportJakob Unterwurzacher
At the moment, only for reverse mode. https://github.com/rfjakob/gocryptfs/issues/217
2017-10-19contentenc: reserve one additional block in CReqPoolJakob Unterwurzacher
...to account for unaligned reads. I have not seen this happen in the wild because the kernel always seems to issue 4k-aligned requests. But the cost of the additional block in the pool is low and prevents a buffer overrun panic when an unaligned read does happen.
2017-10-17fusefrontend: clamp oversized readsJakob Unterwurzacher
Our byte cache pools are sized acc. to MAX_KERNEL_WRITE, but the running kernel may have a higher limit set. Clamp to what we can handle. Fixes a panic on a Synology NAS reported at https://github.com/rfjakob/gocryptfs/issues/145
2017-09-17contentenc: deduplicate AD packing into new concatAD() funcJakob Unterwurzacher
The encrypt and decrypt path both had a copy that were equivalent but ordered differently, which was confusing. Consolidate it in a new dedicated function.
2017-09-17contentenc: DecryptBlocks: give block number counter a clearer nameJakob Unterwurzacher
Using firstBlockNo as the counter is confusing, create a copy named "blockNo" and use that.
2017-08-21Fix misspellings reported by goreportcard.comJakob Unterwurzacher
https://goreportcard.com/report/github.com/rfjakob/gocryptfs#misspell
2017-07-14macos: make testing without openssl work properlyJakob Unterwurzacher
On MacOS, building and testing without openssl is much easier. The tests should skip tests that fail because of missing openssl instead of aborting. Fixes https://github.com/rfjakob/gocryptfs/issues/123
2017-07-02contentenc: MergeBlocks: short-circuit the trivial caseJakob Unterwurzacher
Saves 3% for the tar extract benchmark because we skip the allocation.
2017-06-30contentenc: add PReqPool and use it in DecryptBlocksJakob Unterwurzacher
This gets us a massive speed boost in streaming reads.
2017-06-29contentenc: add safer "bPool" pool variant; add pBlockPoolJakob Unterwurzacher
bPool verifies the lengths of slices going in and out. Also, add a plaintext block pool - pBlockPool - and use it for decryption.
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-11contentenc: parallelize encryption for 128kiB writesJakob Unterwurzacher
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting up smaller writes is probably not worth it. Parallelism is limited to two for now.
2017-06-07contentenc: move EncryptBlocks() loop into its own functionsJakob Unterwurzacher
This allows easy parallelization in the future.
2017-06-01fusefrontend: write: consolidate and move encryption to contentencJakob Unterwurzacher
Collect all the plaintext and pass everything to contentenc in one call. This will allow easier parallization of the encryption. https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01Fix two commentsJakob Unterwurzacher
One out-of-date and the other with a typo.
2017-05-25contenenc: reject all-zero file IDJakob Unterwurzacher
This should never happen in normal operation and is a sign of data corruption. Catch it early.
2017-05-25contentenc: better error reporting in ParseHeaderJakob Unterwurzacher
Log the message ourselves and return EINVAL. Before: gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2 After: gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-01contentenc: downgrade "interrupted write?" warning to debugJakob Unterwurzacher
This can happen during normal operation, and is harmless since 14038a1644f17f50b113a05d09a2a0a3b3e973b2 "fusefrontend: readFileID: reject files that consist only of a header" causes dormant header-only files to be rewritten on the next write.
2017-04-24forcedecode: tighten checksJakob Unterwurzacher
...and fix a few golint issues and print a scary warning message on mount. Also, force the fs to ro,noexec.
2017-04-23Add -forcedecodedanim7
Force decode of encrypted files even if the integrity check fails, instead of failing with an IO error. Warning messages are still printed to syslog if corrupted files are encountered. It can be useful to recover files from disks with bad sectors or other corrupted media. Closes https://github.com/rfjakob/gocryptfs/pull/102 .
2017-03-07contentenc: catch integer underflow in file size calculationJakob Unterwurzacher
If you truncate a ciphertext file to 19 bytes, you could get the impression that the plaintext is 18446744073709551585 bytes long, as reported by "ls -l". Fix it by clamping the value to zero.
2017-03-05full stack: implement HKDF supportJakob Unterwurzacher
...but keep it disabled by default for new filesystems. We are still missing an example filesystem and CLI arguments to explicitely enable and disable it.
2017-03-05configfile: switch to 128-bit IVs for master key encryptionJakob Unterwurzacher
There is no security reason for doing this, but it will allow to consolidate the code once we drop compatibility with gocryptfs v1.2 (and earlier) filesystems.
2016-12-10Replace all calls to naked panic() with log.Panic()Jakob Unterwurzacher
We want all panics to show up in the syslog.
2016-10-28fusefrontend: I/O error instead of panic on all-zero nonceJakob Unterwurzacher
Running xfstests generic/075 on tmpfs often triggered a panic for what seems to be a tmpfs bug. Quoting from the email to lkml, http://www.spinics.net/lists/kernel/msg2370127.html : tmpfs seems to be incorrectly returning 0-bytes when reading from a file that is concurrently being truncated.
2016-10-04lint fixesValient Gough
2016-09-29reverse: use per-purpose nonce generationJakob Unterwurzacher
Also pull all the deterministic nonce code into fusefrontend_reverse to greatly simplify the normal code path.
2016-09-26reverse: switch from GCM-SIV to AES-SIVv1.1-beta1Jakob Unterwurzacher
GCM-SIV is not yet finalized, and the reference implemenation is painfully slow at about 2 MB/s. Switch to AES-SIV.
2016-09-25contentenc: rename constant "IVBitLen" to "DefaultIVBits" and clarify commentJakob Unterwurzacher
128-bit IVs are NOT used everywhere.
2016-09-25contentenc: add "ExternalNonce" modeJakob Unterwurzacher
This will be used for strong symlink encryption in reverse mode.
2016-09-25reverse: derive file ID and block IVs from file pathsJakob Unterwurzacher
2016-09-25contentenc: add GCM-SIV supportJakob Unterwurzacher
Also add ReverseDummyNonce nonce generation.
2016-09-25cryptocore: add support for GCM-SIVJakob Unterwurzacher
2016-09-25reverse: print value in file offset panicJakob Unterwurzacher
2016-09-25contentenc: add helpers for reverse modeJakob Unterwurzacher
Add the reverse variant of DecryptBlocks etc: * EncryptBlocks * JointPlaintextRange * ExplodeCipherRange
2016-09-25conentenc: handle zero-sized files in PlainSizeToCipherSizeJakob Unterwurzacher
Previously caused an integer underflow.
2016-07-06Add godoc comments to all internal packagesJakob Unterwurzacher
2016-07-02fusefronted: move Truncate() and Allocate() to their own fileJakob Unterwurzacher
These are large complicated implementations that will share some code.
2016-07-02contentenc: rename PlaintextRange and CiphertextRangeJakob Unterwurzacher
The name could be misunderstood and actually caused a bug: doWrite used to always preallocate 4128 instead of the actual data length.
2016-07-01fusefrontend: coalesce grows in Truncate()Jakob Unterwurzacher
We were growing the file block-by-block which was pretty inefficient. We now coalesce all the grows into a single Ftruncate. Also simplifies the code! Simplistic benchmark: Before: $ time truncate -s 1000M foo real 0m0.568s After: $ time truncate -s 1000M foo real 0m0.205s
2016-06-16Rename nametransform, contentenc source filesJakob Unterwurzacher
Let's have shorter names, and merge *_api.go into the "main" file. No code changes.
2016-06-15Rename internal "toggledlog" package to "tlog"Jakob Unterwurzacher
tlog is used heavily everywhere and deserves a shorter name. Renamed using sed magic, without any manual rework: find * -type f -exec sed -i 's/toggledlog/tlog/g' {} +
2016-03-03Enable openssl in tests to support old Go versionsJakob Unterwurzacher
Go 1.4 and older do not support 128-bit IVs which caused the tests to panic.
2016-02-07longnames part II: Rename, Unlink, Rmdir, Mknod, Mkdir + testsv0.9-rc1Jakob Unterwurzacher
2016-02-06Run go fmtJakob Unterwurzacher
2016-02-06Fix tests - were broken by the refactoringJakob Unterwurzacher
2016-02-06Major refactoring: Split up "cryptfs" into several internal packagesJakob Unterwurzacher
"git status" for reference: deleted: cryptfs/cryptfs.go deleted: cryptfs/names_core.go modified: integration_tests/cli_test.go modified: integration_tests/helpers.go renamed: cryptfs/config_file.go -> internal/configfile/config_file.go renamed: cryptfs/config_test.go -> internal/configfile/config_test.go renamed: cryptfs/config_test/.gitignore -> internal/configfile/config_test/.gitignore renamed: cryptfs/config_test/PlaintextNames.conf -> internal/configfile/config_test/PlaintextNames.conf renamed: cryptfs/config_test/StrangeFeature.conf -> internal/configfile/config_test/StrangeFeature.conf renamed: cryptfs/config_test/v1.conf -> internal/configfile/config_test/v1.conf renamed: cryptfs/config_test/v2.conf -> internal/configfile/config_test/v2.conf renamed: cryptfs/kdf.go -> internal/configfile/kdf.go renamed: cryptfs/kdf_test.go -> internal/configfile/kdf_test.go renamed: cryptfs/cryptfs_content.go -> internal/contentenc/content.go new file: internal/contentenc/content_api.go renamed: cryptfs/content_test.go -> internal/contentenc/content_test.go renamed: cryptfs/file_header.go -> internal/contentenc/file_header.go renamed: cryptfs/intrablock.go -> internal/contentenc/intrablock.go renamed: cryptfs/address_translation.go -> internal/contentenc/offsets.go new file: internal/cryptocore/crypto_api.go renamed: cryptfs/gcm_go1.4.go -> internal/cryptocore/gcm_go1.4.go renamed: cryptfs/gcm_go1.5.go -> internal/cryptocore/gcm_go1.5.go renamed: cryptfs/nonce.go -> internal/cryptocore/nonce.go renamed: cryptfs/openssl_aead.go -> internal/cryptocore/openssl_aead.go renamed: cryptfs/openssl_benchmark.bash -> internal/cryptocore/openssl_benchmark.bash renamed: cryptfs/openssl_test.go -> internal/cryptocore/openssl_test.go new file: internal/nametransform/name_api.go new file: internal/nametransform/names_core.go renamed: cryptfs/names_diriv.go -> internal/nametransform/names_diriv.go renamed: cryptfs/names_noiv.go -> internal/nametransform/names_noiv.go renamed: cryptfs/names_test.go -> internal/nametransform/names_test.go new file: internal/nametransform/pad16.go renamed: cryptfs/log.go -> internal/toggledlog/log.go renamed: cryptfs/log_go1.4.go -> internal/toggledlog/log_go1.4.go renamed: cryptfs/log_go1.5.go -> internal/toggledlog/log_go1.5.go modified: main.go modified: masterkey.go modified: pathfs_frontend/file.go modified: pathfs_frontend/file_holes.go modified: pathfs_frontend/fs.go modified: pathfs_frontend/fs_dir.go modified: pathfs_frontend/names.go modified: test.bash
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-02 17:13:16 +0000