Skip to content

Support mounting local directories in the browser via Filesystem API #196

Closed

Description

@adamziel

https://developer.mozilla.org/en-US/docs/Web/API/File_System_Access_API would make for a 0 to WordPress dev env in 30 seconds. Also, debugging WordPress in Playground would get much easier.

Activity

  1. adamziel commented on Apr 22, 2023

    @adamziel
    CollaboratorAuthor
  2. adamziel commented on May 13, 2023

    @adamziel
    CollaboratorAuthor

    The File System Access API can read local files and directories, and even exposes a createSyncAccessHandle method for files – super convenient for building Emscripten FS adapter:

    // const dirHandle = await window.showDirectoryPicker();
    (async function() {
        const fileHandle = await window.showOpenFilePicker();
        const dirHandle = await window.showDirectoryPicker();
        var blob = new Blob([
        `
            self.onmessage = function(e) {
                console.log(e.data);
            };
            // Rest of your worker code goes here.
        `
        ], { type: "text/javascript" })
        
        // Sync methods can only be used in web workers on https sites
        var worker = new Worker(window.URL.createObjectURL(blob));
        worker.onmessage = function(e) {
            console.log("Received: " + e.data);
        }
        worker.postMessage(fileHandle);
        worker.postMessage(dirHandle);
    })()

    More about synchronous methods. Also, WHATWG Spec.

    I also found an Emscripten PR that proposes a EMSCRIPTEN_NATIVE_FS implementation, although it seems to have a few issues.

    IDBFS Test

  3. adamziel commented on May 15, 2023

    @adamziel
    CollaboratorAuthor

    Exploratory PR.

    A mapping of Emscripten concepts to FS API concepts:

    Emscripten FS API Description
    mkdir FileSystemDirectoryHandle.getDirectory() Creates a new directory.
    symlink Creates a symbolic link.
    rename Renames a file or directory.
    rmdir FileSystemDirectoryHandle.removeEntry() Removes an empty directory.
    readdir FileSystemDirectoryHandle.getEntries() Reads the entries in a directory.
    unlink FileSystemFileHandle.remove() Deletes a file.
    readlink Reads the target of a symbolic link.
    stat FileSystemHandle.getFile() Gets metadata of a file.
    lstat Gets metadata of a symbolic link.
    chmod Changes the permissions of a file.
    fchmod Changes the permissions of a file (file descriptor).
    chown Changes the owner of a file.
    fchown Changes the owner of a file (file descriptor).
    truncate stream.truncate() Truncates or extends a file to a specified length.
    ftruncate Truncates or extends a file (file descriptor).
    utime Changes the access and modification times of a file.
    open FileSystemHandle.getFile() Opens a file.
    createStream .getWriter() Creates a stream for writing data.
    closeStream WritableStream.close() Closes a writable stream.
    close FileSystemSyncAccessHandle.close() Closes a file after it has been opened.
    llseek Repositions the offset of a file descriptor.
    seek FileSystemWritableStream.seek()
    read Blob.text() or Blob.arrayBuffer() Reads data from a file.
    write FileSystemWritableFileStream.write() Writes data to a file.
    allocate Allocates space for a file.
    mmap Maps a file into memory.
    msync Synchronizes a mapped file with the disk.
    munmap Unmaps a mapped file from memory.
    ioctl Controls device parameters of a file.
  4. adamziel commented on May 24, 2023

    @adamziel
    CollaboratorAuthor

    Until we can support local filesystem as Emscripten FS, there could be a “refresh” button that brings all the files from local directory into Playground. It could stat them and load just the updated ones. There could be a “update every 1s” checkbox.

    it’s similar to the explorations in #19

  5. adamziel commented on Jun 3, 2023

    @adamziel
    CollaboratorAuthor

    Here’s some code from ChatGPT that could make a good starting point:

    —-

    Write a TypeScript function that uses the Filesystem API in the browser to read a file tree from the local filesystem into a JavaScript object. Each file name should be a key, its contents should be the value, and directories should be represented as nested objects. Parallelize up to 20 filesystem api operations.

    type FileTree = {
      [fileName: string]: string | FileTree;
    };
    
    async function readFile(file: File): Promise<string> {
      const text = await file.text();
      return text;
    }
    
    const MAX_CONCURRENT_OPERATIONS = 20;
    
    async function readDirectory(directory: any): Promise<FileTree> {
      let result: FileTree = {};
      let operations: Promise<any>[] = [];
    
      for await (const entry of directory.values()) {
        const operation = (async () => {
          if (entry.kind === 'file') {
            const file = await entry.getFile();
            const text = await readFile(file);
            result[entry.name] = text;
          } else if (entry.kind === 'directory') {
            const nestedDirectory = await entry.getDirectory();
            result[entry.name] = await readDirectory(nestedDirectory);
          }
        })();
    
        operations.push(operation);
    
        // If we've hit the limit, wait for one to complete before starting a new one
        if (operations.length >= MAX_CONCURRENT_OPERATIONS) {
          await Promise.race(operations).then(() => {
            operations = operations.filter(op => op !== operation);
          });
        }
      }
    
      // Wait for all operations to complete
      await Promise.all(operations);
    
      return result;
    }
    
    
    async function readFileTree(rootDirectory: any): Promise<FileTree> {
      const fileTree = await readDirectory(rootDirectory);
      return fileTree;
    }

    Usage:

    async function main() {
      try {
        // Request a handle to the directory from the user
        const rootDirectory = await window.showDirectoryPicker();
    
        // Read the file tree
        const fileTree = await readFileTree(rootDirectory);
    
        // Do something with the file tree
        console.log(fileTree);
      } catch (error) {
        console.error('Failed to read file tree:', error);
      }
    }
    
    // Call the main function when the user clicks a button, for example
    document.getElementById('readFilesButton').addEventListener('click', main);

    It could fit the website project in this repo.

  6. adamziel commented on Jun 12, 2023

    @adamziel
    CollaboratorAuthor
  7. adamziel commented on Jun 22, 2023

    @adamziel
    CollaboratorAuthor

    Basic support with manual "sync" button was added in #548

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions