Mirror of gocryptfs source code on Github http://nuetzlich.net/cgit/gocryptfs/atom?h=xattr_user_buffer 2021-08-23T13:05:15+00:00 2021-08-23T13:05:15+00:00 Jakob Unterwurzacher 2021-08-23T13:05:15+00:00 urn:sha1:69d88505fd7f4cb0d9e4f1918de296342fe05858 Our git version is v2+ for some time now, but go.mod still declared v1. Hopefully making both match makes https://pkg.go.dev/github.com/rfjakob/gocryptfs/v2 work. All the import paths have been fixed like this: find . -name \*.go | xargs sed -i s%github.com/rfjakob/gocryptfs/%github.com/rfjakob/gocryptfs/v2/% 2021-06-25T11:56:53+00:00 Jakob Unterwurzacher 2021-06-25T09:33:18+00:00 urn:sha1:84e702126ac4f017e12150532bfaed675dee2927 The new contrib/maxlen.bash showed that we have exponential runtime with respect to directory depth. The new recursive diriv caching is a lot smarter as it caches intermediate lookups. maxlen.bash now completes in a few seconds. xfstests results same as https://github.com/rfjakob/fuse-xfstests/blob/2d158e4c82be85c15269af77498e353f928f4fab/screenlog.0 : Failures: generic/035 generic/062 generic/080 generic/093 generic/099 generic/215 generic/285 generic/319 generic/426 generic/444 generic/467 generic/477 generic/523 Failed 13 of 580 tests benchmark.bash results are identical: $ ./benchmark.bash Testing gocryptfs at /tmp/benchmark.bash.BdQ: gocryptfs v2.0.1-17-g6b09bc0; go-fuse v2.1.1-0.20210611132105-24a1dfe6b4f8; 2021-06-25 go1.16.5 linux/amd64 /tmp/benchmark.bash.BdQ.mnt is a mountpoint WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 0,4821 s, 544 MB/s READ: 262144000 bytes (262 MB, 250 MiB) copied, 0,266061 s, 985 MB/s UNTAR: 8,280 MD5: 4,564 LS: 1,745 RM: 2,244 2021-04-05T16:20:17+00:00 Jakob Unterwurzacher 2021-04-05T16:20:17+00:00 urn:sha1:f73aee72f87ba6cd3e46184ae75824fd38250f04 2021-04-04T11:05:47+00:00 Jakob Unterwurzacher 2021-04-04T11:05:47+00:00 urn:sha1:043f81dd0129b78ec7d4553843f90203535fa406 2021-04-03T11:08:28+00:00 Jakob Unterwurzacher 2021-04-03T11:08:28+00:00 urn:sha1:24d5d39300629de48bb5a3700b0c219f741da028 dirfd caching was temporarily removed when moving to the v2api. Add it back to gain back some lost speed. 2020-05-17T19:37:36+00:00 Jakob Unterwurzacher 2020-05-17T19:37:36+00:00 urn:sha1:f8ad2ac3e252108ccfedd115eb3009a5a7d77106 Looking at the dircache debug output, we see that a "git status" workload has a very bad cache hit rate because the entries expire or get evicted before they can be reused. Increase both cache size and lifetime for a 4x speedup: Before: 75s After: 17s https://github.com/rfjakob/gocryptfs/issues/410 2020-05-17T19:26:56+00:00 Jakob Unterwurzacher 2020-05-17T19:26:56+00:00 urn:sha1:f6088e5008606ba8ca8f7be2b95b5087b59e5cab Before: Lookup "errno.html/1/2/3/4/5": miss Store: "errno.html/1/2/3/4/5" fd=26 iv=21be6e083d60dcabfe7368264d5082b7 Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7 Lookup "errno.html/1": hit 25 932a464c299b3430c5e55c924f98ac4d Lookup "errno.html/1/2": hit 25 7d53348b1692d537f017bf86b3cf5feb Lookup "errno.html/1/2/3": hit 25 2aef1c9d1ab2b55b163215053fefe703 Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6 Lookup "errno.html/1/2/3/4": hit 25 cb802be53721c46a46247c5e4e0f4ce6 Lookup "errno.html": hit 25 6d68a16d217978915036a3bd55428ae7 After: Lookup "earlyoom/.git/refs" hit fd=10 dup=17 iv=6ae2cecd269a25e8d946aff6afe9b8b8 Lookup "earlyoom/.git/refs/remotes" hit fd=19 dup=17 iv=f04c2d2a5bcc33ebdeaca664859c980d Lookup "earlyoom/.git/refs/remotes/origin" miss Store "earlyoom/.git/refs/remotes/origin" fd=17 iv=834a64a1697c9f5705455ba6dbed22b5 Lookup "earlyoom" hit fd=7 dup=25 iv=2303a892d6e2357c483574a8070b7679 Lookup "earlyoom/.git" hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00 Lookup "earlyoom/.git" hit fd=11 dup=25 iv=d43ca4aff23720c57789c9f62f0aee00 Lookup "earlyoom/.git/refs" hit fd=10 dup=25 iv=6ae2cecd269a25e8d946aff6afe9b8b8 Lookup "earlyoom/.git/refs/heads" hit fd=13 dup=25 iv=f9245e7c066b9adc768a1a666da9fbc8 2019-01-05T15:12:16+00:00 Sebastian Lackner 2019-01-05T14:44:32+00:00 urn:sha1:7e05e809b7579ad0473fff6ce466c19d417d4e93 The files are apparently processed in alphabetic order, so cli_args.go is processed before main.go. In order to run before the go-fuse imports, put the 'ensure fds' code in a separate package. Debug messages are omitted to avoid additional imports (that might contain other code messing up our file descriptors). 2019-01-05T11:27:55+00:00 Sebastian Lackner 2019-01-05T01:10:03+00:00 urn:sha1:f17721c364a522f87b3351a73676405d44b955f6 2019-01-04T22:50:01+00:00 Jakob Unterwurzacher 2019-01-04T22:50:01+00:00 urn:sha1:65eded4a229c52258054f7d4e710beb02aedfe93 As the dirCache now has 3 entries, the tests should accept up to 3 extra fds without declaring an fd leak.