Go OverlayFS

go-bds/overlayfs is a Go library that provides an abstraction for creating and managing overlay or merge filesystems. It aims to use native operating system functionalities where possible and offers a pure Go fallback implementation.

This allows you to merge multiple directory layers (some read-only, one potentially read-write) into a single, unified view.

Features

• io/fs.FS Compatibility: The pure Go MergeFS implementation fully supports the io/fs.FS interface, including:

  • fs.FS
  • fs.ReadFileFS
  • fs.ReadDirFS
  • fs.StatFS
  • fs.SubFS

• File Operations for Go MergeFS: The Go-native MergeFS also implements common file system operations:

  • Open, Create, ReadFile, WriteFile
  • Stat, Lstat, ReadDir
  • Mkdir, MkdirAll, Remove, RemoveAll
  • Chmod, Chown
  • Rename, Symlink, Readlink, Truncate
  • OpenFile

Mount supported system

	Native	FUSE	FUSE3
Linux	✓	Use Overlayfs	Use Overlayfs
Windows		Winfsp
FreeBSD		✓	✓
MacOS		✓
NetBSD		✓
OpenBSD		✓

• FUSE mounted with sirherobrine23.com.br/Sirherobrine23/cgofuse module with MergeFS support.
• Linux use overlayfs, don't support fuse-overlayfs to mount.

MergeFS

Mergefs is a similar/compatible implementation with fuse-overlyfs but based on the interface at the level of the `` ion/fs base. It has been defined for use of CGOFUSE/FS but can also be used within other modules such as IO/FS.FS It will be very useful in other Golang modules.

As in fuse-overlayfs we have similar limitations and operations, if it has to rename, open a file as writing or self-creation to copy from Lower to Upper first to be able to continue operations normally.

Installation

go get -u sirherobrine23.com.br/go-bds/overlayfs@latest

Usage

Creating an OverlayFS Instance

import "sirherobrine23.com.br/go-bds/overlayfs"

// For a read-write overlay
ovReadWrite := overlayfs.NewOverlayFS(
    "/mnt/merged_target", // Target mount point
    "/path/to/upper_rw",  // Upper, read-write layer
    "/path/to/workdir",   // Workdir (required for Linux kernel overlayfs)
    "/path/to/lower1_ro", // Lower, read-only layer 1
    "/path/to/lower2_ro", // Lower, read-only layer 2
)

// For a read-only merge
ovReadOnly := overlayfs.NewOverlayFS(
    "/mnt/readonly_target", // Target mount point
    "",                     // No upper layer means read-only
    "",                     // No workdir needed if no upper layer (except Linux if it still requires it)
    "/path/to/base1",
    "/path/to/base2",
)

Mounting and Unmounting (Platform-Specific)

// Ensure directories exist
// os.MkdirAll(ovReadWrite.Target, 0755)
// os.MkdirAll(ovReadWrite.Upper, 0755)
// os.MkdirAll(ovReadWrite.Workdir, 0755)
// ... and for lower dirs

ctx, done := context.WithCancel(context.Background())
defer done()

// Mount
err := ovReadWrite.Mount(ctx)
switch err {
case nil:
    fmt.Println("Mounted successfully at:", ovReadWrite.Target)
case overlayfs.ErrNotAvaible:
    log.Fatalf("Overlayfs/FUSE not available on this system.")
case overlayfs.ErrNoCGOAvaible:
    log.Fatalf("CGO is disabled")
case overlayfs.ErrMounted:
    log.Fatalf("Current Target ared mounted")
default:
    log.Fatalf("Failed to mount: %v", err)
}

// ... perform operations on the mounted filesystem ...

done() // Unmount filesystem

Notes:

• Permissions: Mounting usually requires elevated privileges.
• WinFsp: On Windows, WinFsp must be installed for Mount() to succeed.
• Linux: Workdir must be an empty directory on the same filesystem as the Upper directory.

Using the Go-Native MergeFS (io/fs.FS)

If native mounting is not available or not desired, you can use the Go-native MergeFS implementation which provides an io/fs.FS interface and direct file operation methods.

package main

import (
    "fmt"
    "io/fs"
    "log"
    "os"
    "path/filepath"

    "sirherobrine23.com.br/go-bds/overlayfs"
)

func main() {
    // Setup temporary directories for example
    tmpDir, err := os.MkdirTemp("", "mergefs-example")
    if err != nil {
        log.Fatal(err)
    }
    defer os.RemoveAll(tmpDir)

    upperDir := filepath.Join(tmpDir, "upper")
    lower1Dir := filepath.Join(tmpDir, "lower1")
    lower2Dir := filepath.Join(tmpDir, "lower2")

    os.Mkdir(upperDir, 0755)
    os.Mkdir(lower1Dir, 0755)
    os.Mkdir(lower2Dir, 0755)

    // Create some files
    os.WriteFile(filepath.Join(lower1Dir, "file1.txt"), []byte("Content from lower1"), 0644)
    os.WriteFile(filepath.Join(lower2Dir, "file2.txt"), []byte("Content from lower2"), 0644)
    os.WriteFile(filepath.Join(lower1Dir, "override.txt"), []byte("Original in lower1"), 0644)
    os.WriteFile(filepath.Join(upperDir, "file3.txt"), []byte("Content from upper"), 0644)
    os.WriteFile(filepath.Join(upperDir, "override.txt"), []byte("Overridden in upper"), 0644)

    // Create Overlayfs instance (Target and Workdir are not used by MergeFS directly)
    ov := overlayfs.NewOverlayFS(
        "", // Target not used by MergeFS methods directly
        upperDir,
        "", // Workdir not used by MergeFS
        lower1Dir,
        lower2Dir,
    )

    // Get the io/fs.FS compatible interface
    mergedFS := ov.Mergefs()

    // Example: Read a file
    content, err := fs.ReadFile(mergedFS, "file1.txt")
    if err != nil {
        log.Fatalf("ReadFile (file1.txt): %v", err)
    }
    fmt.Printf("file1.txt: %s\n", content) // Output: Content from lower1

    content, err = fs.ReadFile(mergedFS, "file3.txt")
    if err != nil {
        log.Fatalf("ReadFile (file3.txt): %v", err)
    }
    fmt.Printf("file3.txt: %s\n", content) // Output: Content from upper

    content, err = fs.ReadFile(mergedFS, "override.txt")
    if err != nil {
        log.Fatalf("ReadFile (override.txt): %v", err)
    }
    fmt.Printf("override.txt: %s\n", content) // Output: Overridden in upper

    // Example: List directory
    entries, err := fs.ReadDir(mergedFS, ".")
    if err != nil {
        log.Fatalf("ReadDir (.): %v", err)
    }
    fmt.Println("\nDirectory listing:")
    for _, entry := range entries {
        fmt.Printf("- %s (Is dir: %t)\n", entry.Name(), entry.IsDir())
    }

    // Example: Using direct MergeFS methods for writing
    err = ov.WriteFile("newfile.txt", []byte("This is a new file written via MergeFS"), 0644)
    if err != nil {
        log.Fatalf("ov.WriteFile (newfile.txt): %v", err)
    }
    fmt.Println("\nCreated newfile.txt in upper layer.")

    // Verify newfile.txt exists in the actual upper directory
    _, err = os.Stat(filepath.Join(upperDir, "newfile.txt"))
    if err != nil {
        log.Fatalf("Stat on actual upperDir/newfile.txt failed: %v", err)
    }

    // Example: Deleting a file (will create a .wh. file in upper)
    err = ov.Remove("file1.txt") // file1.txt was in lower1
    if err != nil {
        log.Fatalf("ov.Remove (file1.txt): %v", err)
    }
    fmt.Println("Removed file1.txt (created whiteout in upper layer).")

    // Try to read it again, should fail
    _, err = fs.ReadFile(mergedFS, "file1.txt")
    if err == nil || !errors.Is(err, fs.ErrNotExist) {
        log.Fatalf("ReadFile (file1.txt) after remove should be ErrNotExist, got: %v", err)
    }
    fmt.Println("Attempting to read file1.txt after removal correctly results in 'not exist'.")

    // Check for whiteout file
    _, err = os.Stat(filepath.Join(upperDir, ".wh.file1.txt"))
    if err != nil {
        log.Fatalf("Whiteout file .wh.file1.txt not found in upperDir: %v", err)
    }
    fmt.Println("Whiteout file .wh.file1.txt found in upper layer.")
}

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues.