Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Error on creation of dynamic image sizes #4915

Closed
Closed
Error on creation of dynamic image sizes#4915
Labels
A-Media-RepositoryUploading, downloading images and video, thumbnailing

Description

@vmario89
vmario89
opened on Mar 22, 2019

Hi,

in Riot i got the problem that when i want to post a sticker i cannot see the preview images but posting them works:
grafik

It seems that the Matrix instance does not accept the option dynamic_thumbnails.

grafik

An example of the image: the first one is not available but the last two are working.

As i understand the option the URL should return an image because Matrix should generate a cached image from source by the request with parameters width/height.

the image thingy worked once without problems but i dont know which point made the breaking change.

regards, Mario

Activity

  1. turt2live commented on Mar 22, 2019

    @turt2live
    turt2live
    on Mar 22, 2019
    Member

    I'm suspecting this to be a server-side problem. Do you have any large stack traces in the server logs?

    (Edit: in my haste, I thought this was riot-web, but the question still applies.)

  2. vmario89 commented on Mar 22, 2019

    @vmario89
    vmario89
    on Mar 22, 2019
    Author

    [edited to fix log formatting by @richvdh]

    Hey. no big traces. I put synapse to DEBUG log state but nothing seriousi can see. But Dimension server puts in log:

    Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]: Error: Can't set headers after they are sent.
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at SendStream.headersAlreadySent (/opt/matrix-dimension/node_modules/send/index.js:390:13)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at SendStream.send (/opt/matrix-dimension/node_modules/send/index.js:618:10)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at onstat (/opt/matrix-dimension/node_modules/send/index.js:730:10)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at FSReqWrap.oncomplete (fs.js:155:5)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]: Mar-22-2019 02:38:29.636 +01:00 info [Webserver] Incoming request: GET /api/v1/dimension/media/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true

    https://dimension.fablabchemnitz.de//api/v1/dimension/media/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
    leads to
    https://matrix.fablabchemnitz.de/_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
    which is not available for example

    some images are working partially, some completely not

    so i am guessing this is more related to Dimension instead of Synapse

  3. turt2live commented on Mar 22, 2019

    @turt2live
    turt2live
    on Mar 22, 2019
    Member

    Dimension isn't handling the error gracefully, but the fact that synapse is returning M_NOT_FOUND for a thumbnail is the problem. There should be at least something in the synapse logs to highlight the request and what happened though.

  4. L0ric0 commented on Mar 22, 2019

    @L0ric0
    L0ric0
    on Mar 22, 2019
    Contributor

    I don't know if it is connected but I get this tracebacks since the last update, the config is mostly the same as vmario89s:

    synapse.http.server: [GET-680119] Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7f7bb9a10f60 method='GET' uri='/_matrix/media/v1/thumbnail/web3r.de/oDSvAQybETGHteNLFuZgIOTU?width=120&height=120&method=scale' clientproto='HTTP/1.1' site=8008>
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/synapse/http/server.py", line 81, in wrapped_request_handler
      yield h(self, request)
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/thumbnail_resource.py", line 78, in _async_render_GET
      width, height, method, m_type
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/thumbnail_resource.py", line 221, in _select_or_generate_remote_thumbnail
      desired_height, desired_method, desired_type
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/media_repository.py", line 540, in generate_remote_exact_thumbnail
      t_width, t_height, t_type, t_method, t_len
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 516, in _simple_insert
      self._simple_insert_txn, table, values,
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 408, in runInteraction
      *args, **kwargs
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 457, in runWithConnection
      inner_func, *args, **kwargs
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
      result = inContext.theWork()
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
      inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
      return self.currentContext().callWithContext(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
      return func(*args,**kw)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
      compat.reraise(excValue, excTraceback)
    File "/usr/lib/python3/dist-packages/twisted/python/compat.py", line 464, in reraise
      raise exception.with_traceback(traceback)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
      result = func(conn, *args, **kw)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 453, in inner_func
      return func(conn, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 329, in _new_transaction
      r = func(txn, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 536, in _simple_insert_txn
      txn.execute(sql, vals)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 119, in execute
      self._do_execute(self.txn.execute, sql, *args)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 149, in _do_execute
      sql, *args
psycopg2.IntegrityError: FEHLER:  doppelter Schlüsselwert verletzt Unique-Constraint »remote_media_cache_thumbnails_media_origin_media_id_thumbna_key«
DETAIL:  Schlüssel »(media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(web3r.de, oDSvAQybETGHteNLFuZgIOTU, 120, 120, image/png)« existiert bereits.

  5. richvdh commented on Mar 22, 2019

    @richvdh
    richvdh
    on Mar 22, 2019
    Member

    @turt2live where do you see M_NOT_FOUND?

    @vmario89 please find the "Received request" line in your synapse logs for a URL which doesn't work and share the logs as per https://github.com/matrix-org/synapse/wiki#how-can-i-find-the-lines-corresponding-to-a-given-http-request-in-my-homeserver-log

  6. vmario89 commented on Mar 22, 2019

    @vmario89
    vmario89
    on Mar 22, 2019
    Author

    Hi @richvdh , the following i found in log. seems to be same behaviour like @L0ric0 has got.

    2019-03-22 10:42:59,016 - synapse.http.server - 112 - ERROR - GET-3590- Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7f5490213d68 method='GET' uri='/_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true' clientproto='HTTP/1.1' site=8008>
Traceback (most recent call last):
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 81, in wrapped_request_handler yield h(self, request)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 78, in _async_render_GET width, height, method, m_type
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 221, in _select_or_generate_remote_thumbnail desired_height, desired_method, desired_type
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/media_repository.py", line 540, in generate_remote_exact_thumbnail t_width, t_height, t_type, t_method, t_len
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 516, in _simple_insert self._simple_insert_txn, table, values,
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 408, in runInteraction *args, **kwargs
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 457, in runWithConnection inner_func, *args, **kwargs
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/threadpool.py", line 250, in inContext result = inContext.theWork()
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/threadpool.py", line 266, in <lambda> inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/context.py", line 122, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/context.py", line 85, in callWithContext return func(*args,**kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection compat.reraise(excValue, excTraceback)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/compat.py", line 464, in reraise raise exception.with_traceback(traceback)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection result = func(conn, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 453, in inner_func return func(conn, *args, **kwargs)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 329, in _new_transaction r = func(txn, *args, **kwargs)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 536, in _simple_insert_txn txn.execute(sql, vals)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 119, in execute self._do_execute(self.txn.execute, sql, *args)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 149, in _do_execute sql, *args
psycopg2.IntegrityError: duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL: Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

  7. richvdh commented on Mar 22, 2019

    @richvdh
    richvdh
    on Mar 22, 2019
    Member

    thanks. can you share any other log lines for GET-3590-.

  8. vmario89 commented on Mar 22, 2019

    @vmario89
    vmario89
    on Mar 22, 2019
    Author

    hey. this comes before the error entry:

    2019-03-22 10:42:58,815 - synapse.access.http.8008 - 233 - INFO - GET-3590- 131.246.146.10 - 8008 - Received request: GET /_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
2019-03-22 10:42:58,816 - synapse.util.async_helpers - 205 - DEBUG - GET-3590- Acquired uncontended linearizer lock 'media_remote' for key ('t2bot.io', '193408b58f5e1eb72d9bea13f23914e6')
2019-03-22 10:42:58,816 - synapse.metrics - 372 - INFO - - Collecting gc 0
2019-03-22 10:42:58,817 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {get_cached_remote_media-764a}
2019-03-22 10:42:58,818 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {get_cached_remote_media-764a} 0.001629 sec
2019-03-22 10:42:58,822 - synapse.util.async_helpers - 218 - DEBUG - GET-3590- Releasing linearizer lock 'media_remote' for key ('t2bot.io', '193408b58f5e1eb72d9bea13f23914e6')
2019-03-22 10:42:58,822 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {get_remote_media_thumbnails-764b}
2019-03-22 10:42:58,823 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {get_remote_media_thumbnails-764b} 0.000840 sec
2019-03-22 10:42:58,832 - synapse.rest.media.v1.thumbnail_resource - 216 - DEBUG - GET-3590- We don't have a thumbnail of that size. Generating
2019-03-22 10:42:59,012 - synapse.rest.media.v1.media_repository - 534 - INFO - GET-3590- Stored thumbnail in file '/var/lib/matrix-synapse/media/remote_thumbnail/t2bot.io/19/34/08b58f5e1eb72d9bea13f23914e6/120-120-image-png'
2019-03-22 10:42:59,013 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {store_remote_media_thumbnail-764c}
2019-03-22 10:42:59,015 - synapse.storage._base - 152 - DEBUG - GET-3590- [SQL FAIL] {store_remote_media_thumbnail-764c} duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL:  Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

2019-03-22 10:42:59,015 - synapse.storage._base - 365 - DEBUG - GET-3590- [TXN FAIL] {store_remote_media_thumbnail-764c} duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL:  Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

2019-03-22 10:42:59,015 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {store_remote_media_thumbnail-764c} 0.001923 sec

  9. richvdh commented on Mar 22, 2019

    @richvdh
    richvdh
    on Mar 22, 2019
    Member

    Great, thanks. This looks like a duplicate of #2182 so I will close this and add your logs to that issue.

  10. richvdh
    closed this as completedon Mar 22, 2019

  11. turt2live commented on Mar 22, 2019

    @turt2live
    turt2live
    on Mar 22, 2019
    Member

    @turt2live where do you see M_NOT_FOUND?

    Showed up when I clicked the links yesterday. Seems fixed now though

  12. MadLittleMods
    added
    A-Media-RepositoryUploading, downloading images and video, thumbnailing
    on Jun 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-Media-RepositoryUploading, downloading images and video, thumbnailing

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions