Support Offscreen Canvas, FileReader, Image, createImageBitmap APIs #54

Activity

1. Electroid

   added

   feature requestRequest for Workers team to add a featureRequest for Workers team to add a feature

   on Sep 29, 2022
2. kentonv commented on Sep 29, 2022

   

   kentonv

   on Sep 29, 2022

   Member

   More actions

   Hi Denis,

   We support Blob and File (mainly for use with FormData). I think we could add FileReader pretty easily, but I'm not sure it would actually be useful. It wouldn't actually be backed by a file, but rather data in memory (in a Blob). You still have to fetch that data from somewhere. It might make more sense to implement FileReader as a polyfill so you can have it do what you want in terms of reading the data from a source like Durable Object storage, KV, etc.

   Canvas and Image are complicated APIs. Supporting them is conceivable but would be a very big project.

   😄React with 😄1Reacted by Denis TRUFFAUT
3. dtruffaut

   changed the title [-]Support Offscreen Canvas, Blob, FileReader, Image APIs[/-] [+]Support Offscreen Canvas, FileReader, Image APIs[/+] on Sep 29, 2022
4. dtruffaut

   changed the title [-]Support Offscreen Canvas, FileReader, Image APIs[/-] [+]Support Offscreen Canvas, FileReader, Image, createImageBitmap APIs[/+] on Sep 29, 2022
5. dtruffaut commented on Sep 29, 2022

   

   dtruffaut

   on Sep 29, 2022

   Last edited by dtruffaut

   Author

   More actions

   DataUrl helps browser to display inlined images.
   It is useful for debugging, if you look at the second example below.

   FileReader can be handy for snippets like:

   // Read a blob and returns a dataUrl
   // which is "basically" an inlined thing prefixed with its type
   // data:[<mediatype>][;base64],<data>
   // https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs
   
   const toDataUrl = async (blob) => {
     return new Promise((resolve, reject) => {
       const reader = new FileReader();
       reader.addEventListener('error', () => reject(false), { passive: true });
       reader.addEventListener('load', () => resolve(reader.result), { passive: true });
       reader.readAsDataURL(blob);
     });
   }
   
   console.log(await toDataUrl(blob));

   I also added createImageBitmap to the list.
   Here is an example of createImageBitmap, Offscreen Canvas, FileReader working together:

   const resize = async ({ event, width }) => {
   
     // Get the image
     // While it would be possible to resize the image using this method, 
     // the resize is in reality performed later using OffscreenCanvas
     const image = await createImageBitmap(event.data.data);
   
     const height = Math.ceil(width * image.height / image.width);
     
     // Draw an Offscreen Canvas to convert into webp
     const canvas = new OffscreenCanvas(width, height);
     const contextOptions = {
       alpha: false,
       desynchronized: true,
       preserveDrawingBuffer: true
     };
     const ctx = canvas.getContext('2d', contextOptions);
     ctx['imageSmoothingEnabled'] = true;
     ctx['imageSmoothingQuality'] = 'high';
     
     // Resize happens here (image.width -> width ; image.height -> height)
     ctx['drawImage'](image, 0, 0, image.width, image.height, 0, 0, width, height); // source, destination
     
     // Convert into webp
     const webpBlob = await canvas.convertToBlob({ type: 'image/webp', quality: 0.8 });
     
     // Log the webp to debug
     const dataUrl = await toDataUrl(webpBlob);
     console.log(dataUrl);
   
     // Return the arrayBuffer for analysis, storage, ...etc.
     return webpBlob.arrayBuffer();
   }

   Having this kind of primitives is interesting, because in Machine Learning, developers often need to resize, grayscale, extract alpha... and perform various and lots of granular operations.

   👍React with 👍11Reacted by gut4, Noah Dunbar, Onur Guvenc, Ning Kuang, Felipe Torres (fforres), Lukas Vanagas, Paul Ebose, Marek Hrabe, TowhidKashem, Aleksandar Blažić and 1 more
6. jasnell commented on Oct 3, 2022

   

   jasnell

   on Oct 3, 2022

   Collaborator

   More actions

   Definitely agree with @kentonv here. FileReader doesn't really make a lot of sense in the workerd environment. Our File and Blob instances are always in-memory. FileReader wouldn't add much value, to be honest, and is difficult to justify. A polyfill wouldn't be difficult. Support for data URLs is definitely interesting, I think, and something we should look at but FileReader is not the only way we can produce those.

   Canvas and Image would absolutely require a significant effort. Certainly something we can discuss but not likely to be on a priority list just yet. My key concern for those is that operations on those tend to be compute intensive, which works fine for standalone workerd but may not necessarily translate well into the multi-tenant edge environment where we have stricter cpu and memory limits enforced. Code that works in workerd might not be able to translate reasonably into the edge which could be problematic.
7. CanRau commented on Nov 22, 2022

   

   CanRau

   on Nov 22, 2022

   More actions

   canvas would be so amazing to generate dynamic og:images without depending on external services/APIs

   👍React with 👍16Reacted by Noah Dunbar, Elara Services, Juho Vepsäläinen, Alex Volkov, Mitch, xmflsct, Hyunbin Seo, Matheus Breguêz, Marius, Perny and 6 more

   🚀React with 🚀2Reacted by Juho Vepsäläinen and Matheus Breguêz
8. kentonv commented on Dec 7, 2022

   

   kentonv

   on Dec 7, 2022

   Member

   More actions

   Hey all,

   I know that support for canvas would be amazing! However, this is proposing an enormous project that would require a dedicated team and months (if not years?) of effort to build.

   We prefer this issue tracker to stay focused on immediately actionable items, like bugs. We don't want to accumulate issues that are likely to stay open indefinitely. So, I am going to close this issue. However, I invite you to post discussions (under the discussion tab) to propose and discuss big ideas.
9. kentonv

   closed this as not plannedon Dec 7, 2022
10. klokan commented on Feb 28, 2023

    

    klokan

    on Feb 28, 2023

    More actions

    Relevant discussion continues at #212
11. TowhidKashem commented on Nov 4, 2023

    

    TowhidKashem

    on Nov 4, 2023

    More actions

    Hi Denis,

    We support Blob and File (mainly for use with FormData). I think we could add FileReader pretty easily, but I'm not sure it would actually be useful. It wouldn't actually be backed by a file, but rather data in memory (in a Blob). You still have to fetch that data from somewhere. It might make more sense to implement FileReader as a polyfill so you can have it do what you want in terms of reading the data from a source like Durable Object storage, KV, etc.

    Canvas and Image are complicated APIs. Supporting them is conceivable but would be a very big project.

    for what it's worth I just had a need which could have been solved with FileReader, i need to read an image's EXIF data and to do so I need to load it in file reader before passing it to exif-js lib..
12. git-bruh

    added 2 commits that reference this issue on Sep 15, 2026

    fix(server): facet channel UaF after alarm timeout

    ...

    aba1620

    fix(server): facet channel UaF after alarm timeout

    ...

    95edab0