aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-06-20Update performance numbersJakob Unterwurzacher
2017-06-20contentenc: use sync.Pool memory pools for encryptionJakob Unterwurzacher
We use two levels of buffers: 1) 4kiB+overhead for each ciphertext block 2) 128kiB+overhead for each FUSE write (32 ciphertext blocks) This commit adds a sync.Pool for both levels. The memory-efficiency for small writes could be improved, as we now always use a 128kiB buffer.
2017-06-20README: update changelog for v1.4v1.4Jakob Unterwurzacher
2017-06-20MANPAGE: reorder options to match "-hh" output; add "-hkdf", "-trace"Jakob Unterwurzacher
2017-06-20build.bash: use plain "git describe" for go-fuseJakob Unterwurzacher
go-fuse recently added a git tag - let's use it.
2017-06-18profiling: add streaming-write profiling helperJakob Unterwurzacher
Writes 1GB of zeros to a gocryptfs mount while collecting cpu and memory profiles.
2017-06-18main, syscallcompat: use Dup3 instead of Dup2Jakob Unterwurzacher
Dup2 is not implemented on linux/arm64. Fixes https://github.com/rfjakob/gocryptfs/issues/121 . Also adds cross-compilation to CI.
2017-06-11Add performance numbers for v1.3-69-ge52594dJakob Unterwurzacher
2017-06-11contentenc: parallelize encryption for 128kiB writesJakob Unterwurzacher
128kiB = 32 x 4kiB pages is the maximum we get from the kernel. Splitting up smaller writes is probably not worth it. Parallelism is limited to two for now.
2017-06-11Add performance numbers for last changeJakob Unterwurzacher
Slight streaming write improvement.
2017-06-11cryptocore: prefetch nonces in the backgroundJakob Unterwurzacher
Spawn a worker goroutine that reads the next 512-byte block while the current one is being drained. This should help reduce waiting times when /dev/urandom is very slow (like on Linux 3.16 kernels).
2017-06-11benchmark.bash: add dd-only mode, enable via "-dd"Jakob Unterwurzacher
Allows for quickly testing the streaming write throughput.
2017-06-09performance.txt: add numbers for latest changeJakob Unterwurzacher
Also, get rid of the half-empty line.
2017-06-09cryptocore: prefetch nonces in 512-byte blocksJakob Unterwurzacher
On my machine, reading 512-byte blocks from /dev/urandom (same via getentropy syscall) is a lot faster in terms of throughput: Blocksize Throughput 16 28.18 MB/s 512 83.75 MB/s For a single-threaded streaming write, this drops the CPU usage of nonceGenerator.Get to almost 1/3: flat flat% sum% cum cum% Before 0 0% 95.08% 0.35s 2.92% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get After 0.01s 0.092% 92.34% 0.13s 1.20% github.com/rfjakob/gocryptfs/internal/cryptocore.(*nonceGenerator).Get This change makes the nonce reading single-threaded, which may hurt massively-parallel writes.
2017-06-09Fix missing Owner coercion for already-open files (#117)Charles Duffy
2017-06-07cryptocore: remove lastNonce checkJakob Unterwurzacher
This check would need locking to be multithreading-safe. But as it is in the fastpath, just remove it. rand.Read() already guarantees that the value is random.
2017-06-07contentenc: move EncryptBlocks() loop into its own functionsJakob Unterwurzacher
This allows easy parallelization in the future.
2017-06-07Add "-trace" flag (record execution trace)Jakob Unterwurzacher
Uses the runtime/trace functionality. TODO: add to man page.
2017-06-07main: reorder force_owner flag parsingJakob Unterwurzacher
No functional changes, just keeping the profiling-related flags together.
2017-06-04Add "package-static.bash" helper scriptJakob Unterwurzacher
Creates a tar.gz with a static build of gocryptfs and the man page.
2017-06-04Remove obsolete TODO.mdJakob Unterwurzacher
This file has been obsolete for some time, TODOs are no longer tracked here but on github.
2017-06-01golint: get rid of underscore warnings for good.Jakob Unterwurzacher
We got another warning for force_other: cli_args.go:26:45: don't use underscores in Go names; struct field force_owner should be forceOwner Use a broader grep.
2017-06-01fusefrontend: write: consolidate and move encryption to contentencJakob Unterwurzacher
Collect all the plaintext and pass everything to contentenc in one call. This will allow easier parallization of the encryption. https://github.com/rfjakob/gocryptfs/issues/116
2017-06-01Don't cap GOMAXPROCS at 4.Jakob Unterwurzacher
Before Go 1.5, GOMAXPROCS defaulted to 1, hence it made sense to unconditionally increase it to 4. But since Go 1.5, GOMAXPROCS defaults to the number of cores, so don't keep it from increasing above 4. Also, update the performance numbers.
2017-06-01Fix two commentsJakob Unterwurzacher
One out-of-date and the other with a typo.
2017-06-01main: increase max write size to maximumJakob Unterwurzacher
Previously, it was at the go-fuse default of 64KiB. Getting bigger writes should increase throughput somewhat. Testing on tmpfs shows an improvement from 112MiB/s to 120MiB/s.
2017-06-01Implement force_owner option to display ownership as a specific user.Charles Duffy
2017-05-31pathiv: fix test failure on Go 1.6Jakob Unterwurzacher
Travis failed on Go 1.6.3 with this error: internal/pathiv/pathiv_test.go:20: no args in Error call This change should solve the problem and provides a better error message on (real) test failure.
2017-05-30main: add "-info" optionJakob Unterwurzacher
Pretty-prints the config while stripping out sensitive (and uninteresting) data https://github.com/rfjakob/gocryptfs/issues/111
2017-05-30README: update changelog with short help textJakob Unterwurzacher
2017-05-30main: add short help textJakob Unterwurzacher
We have accumulated so many options over time that they no longer fit on the screen. Display only a useful subset of options to the user unless they pass "-hh".
2017-05-30pathiv: move block IV algorithm into this packageJakob Unterwurzacher
This was implemented in fusefrontend_reverse, but we need it in fusefrontend as well. Move the algorithm into pathiv.BlockIV().
2017-05-30tests: add v1.3-reverse example filesystemJakob Unterwurzacher
We check the md5 sum of the encrypted version of a file to make sure we don't accidentially change the ciphertext generation.
2017-05-30pathiv: move derivedIVContainer into the packageJakob Unterwurzacher
...under the new name "FileIVs". This will also be used by forward mode.
2017-05-30fusefrontend_reverse: move pathiv to its own packageJakob Unterwurzacher
We will also need it in forward mode.
2017-05-29package.bash: RHEL: strip bracketsJakob Unterwurzacher
We now convert "CentOS release 5.11 (Final)" to "CentOS_release_5.11_Final" https://github.com/rfjakob/gocryptfs/issues/113
2017-05-28gocryptfs-xray: dumpmasterkey: disable "Reading password from stdin"Jakob Unterwurzacher
...and also exit with the proper exit code when we get an error.
2017-05-28gocryptfs-xray: add function to dump the master keyJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/83
2017-05-27package.bash: make it work somewhat on RHELJakob Unterwurzacher
RHEL and CentOS do not have /etc/os-release yet. Read from /etc/redhat-release instead. Fixes https://github.com/rfjakob/gocryptfs/issues/113
2017-05-27cryptocore: improve comments and add tests for hkdfDeriveJakob Unterwurzacher
These should make it easier to re-implement the key derivation that was enabled with the "HKDF" feature flag.
2017-05-25README: update changelogJakob Unterwurzacher
2017-05-25fusefrontend_reverse: store derived values for hard-linked filesJakob Unterwurzacher
With hard links, the path to a file is not unique. This means that the ciphertext data depends on the path that is used to access the files. Fix that by storing the derived values when we encounter a hard-linked file. This means that the first path wins.
2017-05-25doc: add performance numbers for 1.3-27Jakob Unterwurzacher
2017-05-25nametransform: reject all-zero dirIVJakob Unterwurzacher
This should never happen in normal operation and is a sign of data corruption. Catch it early.
2017-05-25contenenc: reject all-zero file IDJakob Unterwurzacher
This should never happen in normal operation and is a sign of data corruption. Catch it early.
2017-05-25contentenc: better error reporting in ParseHeaderJakob Unterwurzacher
Log the message ourselves and return EINVAL. Before: gocryptfs[26962]: go-fuse: can't convert error type: ParseHeader: invalid version: got 0, want 2 After: gocryptfs[617]: ParseHeader: invalid version: want 2, got 0. Returning EINVAL.
2017-05-24main: replace paniclog with logger(1)Jakob Unterwurzacher
Instead of redirecting stdout and stderr to /tmp/gocryptfs_paniclog, where it is hard to find, redirect them to a newly spawned logger(1) instance that forwards the messages to syslog. See https://github.com/rfjakob/gocryptfs/issues/109 for an example where the paniclog was lost due to a reboot. Also, instead of closing stdin, redirect it to /dev/null, like most daemons seem to do.
2017-05-23nametransform: harden name decryption against invalid inputJakob Unterwurzacher
This fixes a few issues I have found reviewing the code: 1) Limit the amount of data ReadLongName() will read. Previously, you could send gocryptfs into out-of-memory by symlinking gocryptfs.diriv to /dev/zero. 2) Handle the empty input case in unPad16() by returning an error. Previously, it would panic with an out-of-bounds array read. It is unclear to me if this could actually be triggered. 3) Reject empty names after base64-decoding in DecryptName(). An empty name crashes emeCipher.Decrypt(). It is unclear to me if B64.DecodeString() can actually return a non-error empty result, but let's guard against it anyway.
2017-05-23main: downgrade panic log create failure from fatal error to warningJakob Unterwurzacher
Exiting with a fatal error just pushes users to use "-nosyslog", which is even worse than not having a paniclog.
2017-05-22nametransform: diriv cache: fall back to the grandparentJakob Unterwurzacher
When a user calls into a deep directory hierarchy, we often get a sequence like this from the kernel: LOOKUP a LOOKUP a/b LOOKUP a/b/c LOOKUP a/b/c/d The diriv cache was not effective for this pattern, because it was designed for this: LOOKUP a/a LOOKUP a/b LOOKUP a/c LOOKUP a/d By also using the cached entry of the grandparent we can avoid lots of diriv reads. This benchmark is against a large encrypted directory hosted on NFS: Before: $ time ls -R nfs-backed-mount > /dev/null real 1m35.976s user 0m0.248s sys 0m0.281s After: $ time ls -R nfs-backed-mount > /dev/null real 1m3.670s user 0m0.217s sys 0m0.403s