Repository navigation
move slider guis to meshcat #145
Description
Related to #144 . This is a possible work-around for lack of interaction in deepnote (and spotty support for ipywidgets more generally in the jupyter-like cloud notebook ecosystem).
- Can use
set_control, e.g. viathis.set_control("test", this.mycallback, 50, 0, 100, 0.1);in the javascript to add a slider to the ui. -
set_controlmessages are also parsed in the javascript, it's easy enough to pass them through the zmqserver. - Demonstrate that I can send something back to the zmqserver from the callback.
Update: this works:this.set_control("test", '(value) => this.connection.send("slider=" + value)', 50, 0, 100, 0.1); - zmqserver should forward to other websocket clients that are not the sender (and store the values?). perhaps I need
update_controlfor this? Or a variant ofset_controlthat takes only the value. - Design protocol for sending UI updates from zmqserver => python. A few ideas?
1) reuse the existing zmq connection. zmqserver already sends little ack messages back when it receives. I would have to be very careful to not interrupt those handshakes.
2) open a second zmq connection, specifically for the ui callback traffic
3) python could connect to zmqserver as a websocket, and listen only to theupdate_controlmessages.
Note that the python client receiving the ui updates need not be the original meschat.Visualizer() object. I'd be happy to have a separate meshcat.UISubscriber() type object instead. It might be cleaner.
Metadata
Metadata
Assignees
Labels
No labels
Activity
RussTedrake commentedon Aug 11, 2021
Another alternative has always been to avoid the zmqserver. I have a working version of meshcat visualizer in python that relies only on ipython kernel comms as the transport. That works great on local machines, but not on colab / deepnote (both of which use a custom kernel implementation).
But perhaps I should consider a visualizer that hosts the websocket connection directly (as zmqserver does now). This would be instead of kernel comms, would have the advantage that it would work outside of jupyter, but would still cut out the zmq layer. This feels like perhaps the best solution. There are still many decisions to be made (do I support multiple clients? do I store the scene tree on the server and allow a new client to get the history, or act more like drake visualizer and require the viewer to connect before any important message is sent?).
Here's a burn-down list for this version
RussTedrake commentedon Aug 20, 2021
I've decided to run with the Meshcat in C++ solution for this. Closing this since I've moved the discussion to RobotLocomotion/drake#13038