aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-01-06fusefrontend: Properly convert plaintext <-> ciphertext offsets in SeekData().Sebastian Lackner
Fixes https://github.com/rfjakob/gocryptfs/issues/304
2019-01-05Update changelogJakob Unterwurzacher
2019-01-05tests: add parallel_cp stress testJakob Unterwurzacher
Modeled after xfstests generic/273 https://github.com/rfjakob/gocryptfs/issues/322
2019-01-05main: Run 'ensure fds' code early during the program startup.Sebastian Lackner
The files are apparently processed in alphabetic order, so cli_args.go is processed before main.go. In order to run before the go-fuse imports, put the 'ensure fds' code in a separate package. Debug messages are omitted to avoid additional imports (that might contain other code messing up our file descriptors).
2019-01-05main: ensure fds 0,1,2 are always openJakob Unterwurzacher
The Go stdlib, as well as the gocryptfs code, relies on the fact that fds 0,1,2 are always open. See https://github.com/rfjakob/gocryptfs/issues/320 for details.
2019-01-05fusefrontend: Allow to set/remove xattr on directory without read permission.Sebastian Lackner
Setting/removing extended attributes on directories was partially fixed with commit eff35e60b63331e3e10f921792baa10b236a721d. However, on most file systems it is also possible to do these operations without read access (see tests). Since we cannot open a write-access fd to a directory, we have to use the /proc/self/fd trick (already used for ListXAttr) for the other operations aswell. For simplicity, let's separate the Linux and Darwin code again (basically revert commit f320b76fd189a363a34bffe981aa67ab97df3362), and always use the /proc/self/fd trick on Linux. On Darwin we use the best-effort approach with openBackingFile() as a fallback. More discussion about the available options is available in https://github.com/rfjakob/gocryptfs/issues/308.
2019-01-05A few more spelling fixes.Sebastian Lackner
2019-01-05Add MakefileJakob Unterwurzacher
We are going to use the Makefile to save useful commands without creating too many shell scripts in the top dir.
2019-01-05doc: update extractloop resultsJakob Unterwurzacher
Update output and add a nice plot.
2019-01-05doc: update XFSTESTS resultsJakob Unterwurzacher
2019-01-04tests: bump maxCacheFds to 3Jakob Unterwurzacher
As the dirCache now has 3 entries, the tests should accept up to 3 extra fds without declaring an fd leak.
2019-01-04fusefrontend: fix fd leak in dirCacheJakob Unterwurzacher
The missing break meant that we may find a second hit in the cache, Dup() a second fd, and leak the first one. Thanks @slackner for finding this.
2019-01-04nametransform: ReadDirIVAt: return raw syscall errorJakob Unterwurzacher
Otherwise this can happen, as triggered by xfstests generic/011: go-fuse: can't convert error type: openat failed: too many open files The app then gets a misleading "Function not implemented" error.
2019-01-04fusefrontend: print warning when Create() runs out of file descriptorsJakob Unterwurzacher
We alread have this warning in Open(), but xfstests generic/488 causes "too many open files" via Create. Add the same message so the user sees what is going on.
2019-01-04fusefrontend: fix setting xattrs on directoriesJakob Unterwurzacher
Directories cannot be opened read-write. Retry with RDONLY.
2019-01-04fusefrontend: disable dirCache stats printingJakob Unterwurzacher
This was inadvertedly kept enabled after benchmarking.
2019-01-04Documentation: Fix MANPAGE-render.bash script.Sebastian Lackner
2019-01-04Update performance.txtJakob Unterwurzacher
2019-01-04fusefrontend: upgrade to three-entry dirCachev1.7-rc1Jakob Unterwurzacher
3 entries should work well for up to three parallel users. It works well for extractloop.bash (two parallel tar extracts).
2019-01-04fsck: don't misdetect a file hole at EOFJakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/304 : A second minor issue is that `if bytes.Equal(buf, allZero) {` compares the whole buffer, although the last read could have been shorter. This could trigger the "skip file hole" code at the end of a file although there isn't any hole to skip.
2019-01-04Update gocryptfs-xray manpage, have MANPAGE-render.bash generate bothJakob Unterwurzacher
have MANPAGE-render.bash generate both gocryptfs.1 and gocryptfs-xray.1
2019-01-04Update changelog and gocryptfs-xray manpageJakob Unterwurzacher
2019-01-04fusefrontend_reverse: Fix redeclaration of 'entries' variable.Sebastian Lackner
Go version go1.10.7 linux/amd64 complains with: internal/fusefrontend_reverse/rfs.go:333: declaration of "entries" shadows declaration at internal/fusefrontend_reverse/rfs.go:327
2019-01-04xray: print "assuming AES-GCM mode" unless -aessiv is passedJakob Unterwurzacher
To alert the user that they can and should choose the right mode.
2019-01-04file-format.md: document AES-SIV block layoutJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/299
2019-01-04xray: recreate test filesytems with -scrypt 10Jakob Unterwurzacher
Speeds up the dumpmasterkey test *a lot*: Before: ok github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_tests 0.398s After: ok github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_tests 0.023s
2019-01-04xray: add dumpmasterkey testJakob Unterwurzacher
2019-01-04xray: add support for inspecting AES-SIV files (-aessiv flag)Jakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/299 : In GCM mode the auth tags are at the end of each block, but in SIV mode the auth tags follow immediately after the nonce. As a result, in AES-SIV mode the output of gocryptfs-xray is misleading and does not actually print the auth tag, but just the last 16-byte of the ciphertext. diff --git a/gocryptfs-xray/xray_main.go b/gocryptfs-xray/xray_main.go index 74c9fb3..5a81caf 100644 --- a/gocryptfs-xray/xray_main.go +++ b/gocryptfs-xray/xray_main.go @@ -16,9 +16,10 @@ import ( ) const ( - ivLen = contentenc.DefaultIVBits / 8 - blockSize = contentenc.DefaultBS + ivLen + cryptocore.AuthTagLen - myName = "gocryptfs-xray" + ivLen = contentenc.DefaultIVBits / 8 + authTagLen = cryptocore.AuthTagLen + blockSize = contentenc.DefaultBS + ivLen + cryptocore.AuthTagLen + myName = "gocryptfs-xray" ) func errExit(err error) { @@ -26,13 +27,18 @@ func errExit(err error) { os.Exit(1) } -func prettyPrintHeader(h *contentenc.FileHeader) { +func prettyPrintHeader(h *contentenc.FileHeader, aessiv bool) { id := hex.EncodeToString(h.ID) - fmt.Printf("Header: Version: %d, Id: %s\n", h.Version, id) + msg := "Header: Version: %d, Id: %s" + if aessiv { + msg += ", assuming AES-SIV mode" + } + fmt.Printf(msg+"\n", h.Version, id) } func main() { dumpmasterkey := flag.Bool("dumpmasterkey", false, "Decrypt and dump the master key") + aessiv := flag.Bool("aessiv", false, "Assume AES-SIV mode instead of AES-GCM") flag.Parse() if flag.NArg() != 1 { fmt.Fprintf(os.Stderr, "Usage: %s [OPTIONS] FILE\n"+ @@ -54,7 +60,7 @@ func main() { if *dumpmasterkey { dumpMasterKey(fn) } else { - inspectCiphertext(fd) + inspectCiphertext(fd, *aessiv) } } @@ -72,7 +78,7 @@ func dumpMasterKey(fn string) { } } -func inspectCiphertext(fd *os.File) { +func inspectCiphertext(fd *os.File, aessiv bool) { headerBytes := make([]byte, contentenc.HeaderLen) n, err := fd.ReadAt(headerBytes, 0) if err == io.EOF && n == 0 { @@ -88,34 +94,30 @@ func inspectCiphertext(fd *os.File) { if err != nil { errExit(err) } - prettyPrintHeader(header) + prettyPrintHeader(header, aessiv) var i int64 + buf := make([]byte, blockSize) for i = 0; ; i++ { - blockLen := int64(blockSize) off := contentenc.HeaderLen + i*blockSize - iv := make([]byte, ivLen) - _, err := fd.ReadAt(iv, off) - if err == io.EOF { - break - } else if err != nil { + n, err := fd.ReadAt(buf, off) + if err != nil && err != io.EOF { errExit(err) } - tag := make([]byte, cryptocore.AuthTagLen) - _, err = fd.ReadAt(tag, off+blockSize-cryptocore.AuthTagLen) - if err == io.EOF { - fi, err2 := fd.Stat() - if err2 != nil { - errExit(err2) - } - _, err2 = fd.ReadAt(tag, fi.Size()-cryptocore.AuthTagLen) - if err2 != nil { - errExit(err2) - } - blockLen = (fi.Size() - contentenc.HeaderLen) % blockSize - } else if err != nil { - errExit(err) + if n == 0 && err == io.EOF { + break + } + // A block contains at least the IV, the Auth Tag and 1 data byte + if n < ivLen+authTagLen+1 { + errExit(fmt.Errorf("corrupt block: truncated data, len=%d", n)) + } + data := buf[:n] + // Parse block data + iv := data[:ivLen] + tag := data[len(data)-authTagLen:] + if aessiv { + tag = data[ivLen : ivLen+authTagLen] } fmt.Printf("Block %2d: IV: %s, Tag: %s, Offset: %5d Len: %d\n", - i, hex.EncodeToString(iv), hex.EncodeToString(tag), off, blockLen) + i, hex.EncodeToString(iv), hex.EncodeToString(tag), off, len(data)) } } diff --git a/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt b/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt new file mode 100644 index 0000000..70835ac --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt @@ -0,0 +1,5 @@ +Your master key is: + + 29dd219d-e227ff20-8474469d-9fc9fdc6- + b434ab35-404e808c-489d441e-2c1003f2 + diff --git a/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt b/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt new file mode 100644 index 0000000..6a48079 --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt @@ -0,0 +1,3 @@ +Header: Version: 2, Id: c2f21142e108952a47edfe16053d2bb9, assuming AES-SIV mode +Block 0: IV: 7621fdc35be7671ac6f369214436e8ff, Tag: e8108c158b22cad6bb3296645357eb75, Offset: 18 Len: 4128 +Block 1: IV: f096d86a4dc3461ef17655cfcf865b13, Tag: 925f23d647e4ab7add2c8d36362cc5a9, Offset: 4146 Len: 936 diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ b/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ new file mode 100644 index 0000000..bfd4dfe Binary files /dev/null and b/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ differ diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf new file mode 100644 index 0000000..9b8b95f --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf @@ -0,0 +1,21 @@ +{ + "Creator": "gocryptfs v1.7-beta1-7-g6b94f5e", + "EncryptedKey": "D0kHfg/pryMO9Ydo15EwpYjNHf3iWKq2GJyNocbjwJt9blEeMoLD5DnoARuDzQs54hblw+9MHwFjCSHYmJrFbA==", + "ScryptObject": { + "Salt": "ehn0LM/Hy/4QkXAMCZq3c3p0O9G7gu5e3OQSR8MiJ6c=", + "N": 65536, + "R": 8, + "P": 1, + "KeyLen": 32 + }, + "Version": 2, + "FeatureFlags": [ + "GCMIV128", + "HKDF", + "DirIV", + "EMENames", + "LongNames", + "Raw64", + "AESSIV" + ] +} diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv new file mode 100644 index 0000000..dd57ce1 --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv @@ -0,0 +1 @@ +.¨Í1Aiõ&Á4—öÉ \ No newline at end of file diff --git a/gocryptfs-xray/xray_tests/xray_test.go b/gocryptfs-xray/xray_tests/xray_test.go index a3374b0..8e5fc0c 100644 --- a/gocryptfs-xray/xray_tests/xray_test.go +++ b/gocryptfs-xray/xray_tests/xray_test.go @@ -24,3 +24,20 @@ func TestAesgcmXray(t *testing.T) { fmt.Printf("have:\n%s", string(out)) } } + +func TestAessivXray(t *testing.T) { + expected, err := ioutil.ReadFile("aessiv_fs.xray.txt") + if err != nil { + t.Fatal(err) + } + cmd := exec.Command("../gocryptfs-xray", "-aessiv", "aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ") + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatal(err) + } + if bytes.Compare(out, expected) != 0 { + t.Errorf("Unexpected output") + fmt.Printf("expected:\n%s", string(expected)) + fmt.Printf("have:\n%s", string(out)) + } +}
2019-01-04xray: add tests and example aes-gcm fsJakob Unterwurzacher
The single test compares the gocryptfs-xray output with the expected output.
2019-01-04Update changelogJakob Unterwurzacher
2019-01-04reverse mode: -exclude: filter out excluded .name filesJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/286 : While the actual file is properly excluded, the * .name file is still leaked in the directory listing: ``` drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:58 . drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 .. -r-------- 1 sebastian sebastian 408 Dez 17 14:56 gocryptfs.conf -r--r--r-- 1 sebastian sebastian 16 Dez 17 14:58 gocryptfs.diriv -r--r--r-- 1 sebastian sebastian 320 Dez 17 14:58 gocryptfs.longname.3vZ_r3eDPb1_fL3j5VA4rd_bcKWLKT9eaxOVIGK5HFA.name ```
2019-01-04tests: reverse: verify that longname .name files are exluded as wellJakob Unterwurzacher
Currently fails, will be fixed in the next commit. https://github.com/rfjakob/gocryptfs/issues/286
2019-01-04reverse mode: fix "-exclude" in "-plaintextnames" dir listingsJakob Unterwurzacher
Excluded files showed up in directory listing like this: drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:48 . drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 .. -????????? ? ? ? ? ? abcd -r-------- 1 sebastian sebastian 366 Dez 17 14:45 gocryptfs.conf Fixes https://github.com/rfjakob/gocryptfs/issues/285
2019-01-04tests: fix VerifyExistence() helper, it missed unstat()able filesJakob Unterwurzacher
VerifyExistence missed unstat()able files in the directory listing because ioutil.ReadDir() filtered them out. https://github.com/rfjakob/gocryptfs/issues/285
2019-01-04fsck: Do not show files/directories without access permissions as corrupt.Sebastian Lackner
When running as a regular user, error EACCES does not necessarily mean that the file/directory/xattr is corrupt, but just that we do not have sufficient access permissions. Add a hint that running as root can be used to check everything. Fixes: https://github.com/rfjakob/gocryptfs/issues/309
2019-01-04fusefrontend: Allow to create sparse file of size 4096.Sebastian Lackner
When the old size is zero, there are no existing blocks to merge the new data with. Directly use Ftruncate if the size is block-aligned. Fixes https://github.com/rfjakob/gocryptfs/issues/305
2019-01-04A few more spelling fixes.Sebastian Lackner
Found with the 'codespell' utility.
2019-01-03Update README for v1.7-beta1v1.7-beta1Jakob Unterwurzacher
2019-01-03travis: bump clone depth to 300Jakob Unterwurzacher
We are hitting the fatal: No names found, cannot describe anything. problem in the symlink_race branch.
2019-01-03tests: overengineer extractloop_plot_csv.m a littleJakob Unterwurzacher
Plot iteration time on second y axis, adjust line styles
2019-01-03Omit syscall.O_RDONLY flag when passing O_PATH.Sebastian Lackner
When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03fusefrontend: Use appropriate flags in decryptPathAt.Sebastian Lackner
2019-01-03fusefrontend: Open directory with syscall.O_DIRECTORY in OpenDir.Sebastian Lackner
2019-01-03fusefrontend: Open directory with syscall.O_DIRECTORY in Rmdir.Sebastian Lackner
2019-01-03fusefrontend: Do not Clear cache at end of Rmdir function.Sebastian Lackner
We already do 'defer fs.dirCache.Clear()', so this is no longer required.
2019-01-03nametransform: Add implicit syscall.O_RDONLY flag.Sebastian Lackner
2019-01-03syscallcompat: Use O_PATH to open base directory.Sebastian Lackner
Also remove some unnecessary flags: When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03fusefrontend: Remove debug code.Sebastian Lackner
This code was accidentially added in 4f66d66755da63c78b09201c6c72353009251cf2.
2019-01-03tests: extractloop: renice to level 19Jakob Unterwurzacher
Run at low priority to not annoy the user too much.
2019-01-03fusefronted: dirCache: fix bug handling ""Jakob Unterwurzacher
Bug looked like this: $ ls -l . total 0 drwxrwxr-x. 2 jakob jakob 60 Jan 3 15:42 foo -rw-rw-r--. 1 jakob jakob 0 Jan 3 15:46 x $ ls -l . ls: cannot access '.': No such file or directory (only happened when "" was in the dirCache)
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-05 22:48:56 +0000