summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-01-04fusefrontend: fix setting xattrs on directoriesJakob Unterwurzacher
Directories cannot be opened read-write. Retry with RDONLY.
2019-01-04fusefrontend: disable dirCache stats printingJakob Unterwurzacher
This was inadvertedly kept enabled after benchmarking.
2019-01-04Documentation: Fix MANPAGE-render.bash script.Sebastian Lackner
2019-01-04Update performance.txtJakob Unterwurzacher
2019-01-04fusefrontend: upgrade to three-entry dirCachev1.7-rc1Jakob Unterwurzacher
3 entries should work well for up to three parallel users. It works well for extractloop.bash (two parallel tar extracts).
2019-01-04fsck: don't misdetect a file hole at EOFJakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/304 : A second minor issue is that `if bytes.Equal(buf, allZero) {` compares the whole buffer, although the last read could have been shorter. This could trigger the "skip file hole" code at the end of a file although there isn't any hole to skip.
2019-01-04Update gocryptfs-xray manpage, have MANPAGE-render.bash generate bothJakob Unterwurzacher
have MANPAGE-render.bash generate both gocryptfs.1 and gocryptfs-xray.1
2019-01-04Update changelog and gocryptfs-xray manpageJakob Unterwurzacher
2019-01-04fusefrontend_reverse: Fix redeclaration of 'entries' variable.Sebastian Lackner
Go version go1.10.7 linux/amd64 complains with: internal/fusefrontend_reverse/rfs.go:333: declaration of "entries" shadows declaration at internal/fusefrontend_reverse/rfs.go:327
2019-01-04xray: print "assuming AES-GCM mode" unless -aessiv is passedJakob Unterwurzacher
To alert the user that they can and should choose the right mode.
2019-01-04file-format.md: document AES-SIV block layoutJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/299
2019-01-04xray: recreate test filesytems with -scrypt 10Jakob Unterwurzacher
Speeds up the dumpmasterkey test *a lot*: Before: ok github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_tests 0.398s After: ok github.com/rfjakob/gocryptfs/gocryptfs-xray/xray_tests 0.023s
2019-01-04xray: add dumpmasterkey testJakob Unterwurzacher
2019-01-04xray: add support for inspecting AES-SIV files (-aessiv flag)Jakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/299 : In GCM mode the auth tags are at the end of each block, but in SIV mode the auth tags follow immediately after the nonce. As a result, in AES-SIV mode the output of gocryptfs-xray is misleading and does not actually print the auth tag, but just the last 16-byte of the ciphertext. diff --git a/gocryptfs-xray/xray_main.go b/gocryptfs-xray/xray_main.go index 74c9fb3..5a81caf 100644 --- a/gocryptfs-xray/xray_main.go +++ b/gocryptfs-xray/xray_main.go @@ -16,9 +16,10 @@ import ( ) const ( - ivLen = contentenc.DefaultIVBits / 8 - blockSize = contentenc.DefaultBS + ivLen + cryptocore.AuthTagLen - myName = "gocryptfs-xray" + ivLen = contentenc.DefaultIVBits / 8 + authTagLen = cryptocore.AuthTagLen + blockSize = contentenc.DefaultBS + ivLen + cryptocore.AuthTagLen + myName = "gocryptfs-xray" ) func errExit(err error) { @@ -26,13 +27,18 @@ func errExit(err error) { os.Exit(1) } -func prettyPrintHeader(h *contentenc.FileHeader) { +func prettyPrintHeader(h *contentenc.FileHeader, aessiv bool) { id := hex.EncodeToString(h.ID) - fmt.Printf("Header: Version: %d, Id: %s\n", h.Version, id) + msg := "Header: Version: %d, Id: %s" + if aessiv { + msg += ", assuming AES-SIV mode" + } + fmt.Printf(msg+"\n", h.Version, id) } func main() { dumpmasterkey := flag.Bool("dumpmasterkey", false, "Decrypt and dump the master key") + aessiv := flag.Bool("aessiv", false, "Assume AES-SIV mode instead of AES-GCM") flag.Parse() if flag.NArg() != 1 { fmt.Fprintf(os.Stderr, "Usage: %s [OPTIONS] FILE\n"+ @@ -54,7 +60,7 @@ func main() { if *dumpmasterkey { dumpMasterKey(fn) } else { - inspectCiphertext(fd) + inspectCiphertext(fd, *aessiv) } } @@ -72,7 +78,7 @@ func dumpMasterKey(fn string) { } } -func inspectCiphertext(fd *os.File) { +func inspectCiphertext(fd *os.File, aessiv bool) { headerBytes := make([]byte, contentenc.HeaderLen) n, err := fd.ReadAt(headerBytes, 0) if err == io.EOF && n == 0 { @@ -88,34 +94,30 @@ func inspectCiphertext(fd *os.File) { if err != nil { errExit(err) } - prettyPrintHeader(header) + prettyPrintHeader(header, aessiv) var i int64 + buf := make([]byte, blockSize) for i = 0; ; i++ { - blockLen := int64(blockSize) off := contentenc.HeaderLen + i*blockSize - iv := make([]byte, ivLen) - _, err := fd.ReadAt(iv, off) - if err == io.EOF { - break - } else if err != nil { + n, err := fd.ReadAt(buf, off) + if err != nil && err != io.EOF { errExit(err) } - tag := make([]byte, cryptocore.AuthTagLen) - _, err = fd.ReadAt(tag, off+blockSize-cryptocore.AuthTagLen) - if err == io.EOF { - fi, err2 := fd.Stat() - if err2 != nil { - errExit(err2) - } - _, err2 = fd.ReadAt(tag, fi.Size()-cryptocore.AuthTagLen) - if err2 != nil { - errExit(err2) - } - blockLen = (fi.Size() - contentenc.HeaderLen) % blockSize - } else if err != nil { - errExit(err) + if n == 0 && err == io.EOF { + break + } + // A block contains at least the IV, the Auth Tag and 1 data byte + if n < ivLen+authTagLen+1 { + errExit(fmt.Errorf("corrupt block: truncated data, len=%d", n)) + } + data := buf[:n] + // Parse block data + iv := data[:ivLen] + tag := data[len(data)-authTagLen:] + if aessiv { + tag = data[ivLen : ivLen+authTagLen] } fmt.Printf("Block %2d: IV: %s, Tag: %s, Offset: %5d Len: %d\n", - i, hex.EncodeToString(iv), hex.EncodeToString(tag), off, blockLen) + i, hex.EncodeToString(iv), hex.EncodeToString(tag), off, len(data)) } } diff --git a/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt b/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt new file mode 100644 index 0000000..70835ac --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs.masterkey.txt @@ -0,0 +1,5 @@ +Your master key is: + + 29dd219d-e227ff20-8474469d-9fc9fdc6- + b434ab35-404e808c-489d441e-2c1003f2 + diff --git a/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt b/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt new file mode 100644 index 0000000..6a48079 --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs.xray.txt @@ -0,0 +1,3 @@ +Header: Version: 2, Id: c2f21142e108952a47edfe16053d2bb9, assuming AES-SIV mode +Block 0: IV: 7621fdc35be7671ac6f369214436e8ff, Tag: e8108c158b22cad6bb3296645357eb75, Offset: 18 Len: 4128 +Block 1: IV: f096d86a4dc3461ef17655cfcf865b13, Tag: 925f23d647e4ab7add2c8d36362cc5a9, Offset: 4146 Len: 936 diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ b/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ new file mode 100644 index 0000000..bfd4dfe Binary files /dev/null and b/gocryptfs-xray/xray_tests/aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ differ diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf new file mode 100644 index 0000000..9b8b95f --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.conf @@ -0,0 +1,21 @@ +{ + "Creator": "gocryptfs v1.7-beta1-7-g6b94f5e", + "EncryptedKey": "D0kHfg/pryMO9Ydo15EwpYjNHf3iWKq2GJyNocbjwJt9blEeMoLD5DnoARuDzQs54hblw+9MHwFjCSHYmJrFbA==", + "ScryptObject": { + "Salt": "ehn0LM/Hy/4QkXAMCZq3c3p0O9G7gu5e3OQSR8MiJ6c=", + "N": 65536, + "R": 8, + "P": 1, + "KeyLen": 32 + }, + "Version": 2, + "FeatureFlags": [ + "GCMIV128", + "HKDF", + "DirIV", + "EMENames", + "LongNames", + "Raw64", + "AESSIV" + ] +} diff --git a/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv new file mode 100644 index 0000000..dd57ce1 --- /dev/null +++ b/gocryptfs-xray/xray_tests/aessiv_fs/gocryptfs.diriv @@ -0,0 +1 @@ +.¨Í1Aiõ&Á4—öÉ \ No newline at end of file diff --git a/gocryptfs-xray/xray_tests/xray_test.go b/gocryptfs-xray/xray_tests/xray_test.go index a3374b0..8e5fc0c 100644 --- a/gocryptfs-xray/xray_tests/xray_test.go +++ b/gocryptfs-xray/xray_tests/xray_test.go @@ -24,3 +24,20 @@ func TestAesgcmXray(t *testing.T) { fmt.Printf("have:\n%s", string(out)) } } + +func TestAessivXray(t *testing.T) { + expected, err := ioutil.ReadFile("aessiv_fs.xray.txt") + if err != nil { + t.Fatal(err) + } + cmd := exec.Command("../gocryptfs-xray", "-aessiv", "aessiv_fs/Ldq-c4ADpM5iGSSrPjUAqQ") + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatal(err) + } + if bytes.Compare(out, expected) != 0 { + t.Errorf("Unexpected output") + fmt.Printf("expected:\n%s", string(expected)) + fmt.Printf("have:\n%s", string(out)) + } +}
2019-01-04xray: add tests and example aes-gcm fsJakob Unterwurzacher
The single test compares the gocryptfs-xray output with the expected output.
2019-01-04Update changelogJakob Unterwurzacher
2019-01-04reverse mode: -exclude: filter out excluded .name filesJakob Unterwurzacher
Fixes https://github.com/rfjakob/gocryptfs/issues/286 : While the actual file is properly excluded, the * .name file is still leaked in the directory listing: ``` drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:58 . drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 .. -r-------- 1 sebastian sebastian 408 Dez 17 14:56 gocryptfs.conf -r--r--r-- 1 sebastian sebastian 16 Dez 17 14:58 gocryptfs.diriv -r--r--r-- 1 sebastian sebastian 320 Dez 17 14:58 gocryptfs.longname.3vZ_r3eDPb1_fL3j5VA4rd_bcKWLKT9eaxOVIGK5HFA.name ```
2019-01-04tests: reverse: verify that longname .name files are exluded as wellJakob Unterwurzacher
Currently fails, will be fixed in the next commit. https://github.com/rfjakob/gocryptfs/issues/286
2019-01-04reverse mode: fix "-exclude" in "-plaintextnames" dir listingsJakob Unterwurzacher
Excluded files showed up in directory listing like this: drwxr-xr-x 2 sebastian sebastian 4,0K Dez 17 14:48 . drwxr-xr-x 7 sebastian sebastian 4,0K Dez 17 14:45 .. -????????? ? ? ? ? ? abcd -r-------- 1 sebastian sebastian 366 Dez 17 14:45 gocryptfs.conf Fixes https://github.com/rfjakob/gocryptfs/issues/285
2019-01-04tests: fix VerifyExistence() helper, it missed unstat()able filesJakob Unterwurzacher
VerifyExistence missed unstat()able files in the directory listing because ioutil.ReadDir() filtered them out. https://github.com/rfjakob/gocryptfs/issues/285
2019-01-04fsck: Do not show files/directories without access permissions as corrupt.Sebastian Lackner
When running as a regular user, error EACCES does not necessarily mean that the file/directory/xattr is corrupt, but just that we do not have sufficient access permissions. Add a hint that running as root can be used to check everything. Fixes: https://github.com/rfjakob/gocryptfs/issues/309
2019-01-04fusefrontend: Allow to create sparse file of size 4096.Sebastian Lackner
When the old size is zero, there are no existing blocks to merge the new data with. Directly use Ftruncate if the size is block-aligned. Fixes https://github.com/rfjakob/gocryptfs/issues/305
2019-01-04A few more spelling fixes.Sebastian Lackner
Found with the 'codespell' utility.
2019-01-03Update README for v1.7-beta1v1.7-beta1Jakob Unterwurzacher
2019-01-03travis: bump clone depth to 300Jakob Unterwurzacher
We are hitting the fatal: No names found, cannot describe anything. problem in the symlink_race branch.
2019-01-03tests: overengineer extractloop_plot_csv.m a littleJakob Unterwurzacher
Plot iteration time on second y axis, adjust line styles
2019-01-03Omit syscall.O_RDONLY flag when passing O_PATH.Sebastian Lackner
When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03fusefrontend: Use appropriate flags in decryptPathAt.Sebastian Lackner
2019-01-03fusefrontend: Open directory with syscall.O_DIRECTORY in OpenDir.Sebastian Lackner
2019-01-03fusefrontend: Open directory with syscall.O_DIRECTORY in Rmdir.Sebastian Lackner
2019-01-03fusefrontend: Do not Clear cache at end of Rmdir function.Sebastian Lackner
We already do 'defer fs.dirCache.Clear()', so this is no longer required.
2019-01-03nametransform: Add implicit syscall.O_RDONLY flag.Sebastian Lackner
2019-01-03syscallcompat: Use O_PATH to open base directory.Sebastian Lackner
Also remove some unnecessary flags: When O_PATH is specified in flags, flag bits other than O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW are ignored.
2019-01-03fusefrontend: Remove debug code.Sebastian Lackner
This code was accidentially added in 4f66d66755da63c78b09201c6c72353009251cf2.
2019-01-03tests: extractloop: renice to level 19Jakob Unterwurzacher
Run at low priority to not annoy the user too much.
2019-01-03fusefronted: dirCache: fix bug handling ""Jakob Unterwurzacher
Bug looked like this: $ ls -l . total 0 drwxrwxr-x. 2 jakob jakob 60 Jan 3 15:42 foo -rw-rw-r--. 1 jakob jakob 0 Jan 3 15:46 x $ ls -l . ls: cannot access '.': No such file or directory (only happened when "" was in the dirCache)
2019-01-03tests: allow one extra fd in fd leak detector (dirCache)Jakob Unterwurzacher
The gocryptfs process may keep one fd open for up to one second in the dirCache.
2019-01-03fusefrontend: add dirCacheJakob Unterwurzacher
2019-01-03nametransform: simplify WriteDirIV to WriteDirIVAtJakob Unterwurzacher
Un-spaghettify the function and let the callers open the directory.
2019-01-02fusefrontend: use O_RDONLY in the ListXAttr fallback pathJakob Unterwurzacher
Copy-paste error. https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02fusefrontend: move openBackingDir into its own fileJakob Unterwurzacher
This function is in all fastpaths, will get a cache, and needs its own file. renamed: internal/fusefrontend/names.go -> internal/fusefrontend/openbackingdir.go renamed: internal/fusefrontend/names_test.go -> internal/fusefrontend/openbackingdir_test.go
2019-01-02fusefronted: make EncryptPath symlink-safeJakob Unterwurzacher
Finally allows us to delete EncryptPathDirIV.
2019-01-02tests: retry longer when we see a fd leakJakob Unterwurzacher
Hopefully gets rid of the false positives on travis.
2019-01-02travis: "go get" must descend into subdirs to catch pkg/xattrJakob Unterwurzacher
recent builds all failed with: tests/fsck/fsck_test.go:12:2: cannot find package "github.com/pkg/xattr"
2019-01-02fusefrontend: xattr: fix operations on files without read permissionsJakob Unterwurzacher
* listxattr is fixed via the /proc/self/fd trick * setxattr,removexattr are fixed by opening the file O_WRONLY Fixes https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02tests: xattr: set on 0200 file, list on 0000 fileJakob Unterwurzacher
https://github.com/rfjakob/gocryptfs/issues/308
2019-01-02fusefrontend: don't downgrade type needlesslyJakob Unterwurzacher
2019-01-02travis: fix go get command (needs -t), update depJakob Unterwurzacher
Only with the "-t" flag does go get download dependencies that are only used in tests.
2019-01-02fusefrontend: use Fsetxattr/Fgetxattr/etc on all platformsJakob Unterwurzacher
Darwin now also has these functions, use them. Simplifies the code and makes it symlink-safe on Darwin as well.
2019-01-02syscallcompat: add Fgetxattr / Fsetxattr wrappersJakob Unterwurzacher
These take care of buffer sizing and parsing.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-07 20:28:13 +0000