[RFC] New feature: Images in labels

[alexshalamov]

Images inside text-field

This RFC explores possibility of embedding images within a text defined by symbol layer's text-field property.

Style interface

Images within a text could be added via new image expression that takes one mandatory argument that must evaluate to string, for example ["image", "car-15"].

Within 'format' expression, optional image options object may be provided after 'image' expression. Image options properties could be:

• height - sets height of an image
• width - sets width of an image
• scale - proportional scaling
• color - applicable for SDF icons
• sources - array of sources from where image would be tried to be fetched, like <img src=[url1, url2...]> (spritesheet, base url, etc). Example:
  ["spritesheet://icons", "https://www.mapbox.com/images/"]

Support for aforementioned image option properties may be added in the future if found useful.

Example syntax

The new image expression can be used as a section within text-field’s format expression, for instance:

"icon-image": "philadelphia-septa",
"text-field": ["format", 
                    "London ",
                    ["image", "gb-national-rail.london-tfl-rail.london-underground"],
                    " and Taipei ",
                    ["image", "taipei-metro"], { "useful": "image option" } 
                    " metro\n",
                    "SF ", {"text-color": "blue"},
                    ["image", "san-francisco-bart"],
                    " bart"
]

The new image expression could be represented by Image object ('coercable' to string if needed). The Image object would contain image id and image options. If needed, "icon-image" property could be converted to accept "image" expression, so that the text-field and icon-image properties would benefit from new functionality.

"icon-image": ["image", "philadelphia-septa"]

Rendering result

[ryanhamley]

If needed, "icon-image" property could be converted to accept "image" expression, so that the text-field and icon-image properties would benefit from new functionality.

Just wanted to note that this is what we're proposing for default images in #8052 (comment)

[asheemmamoowala]

@alexshalamov this is a really neat way to expand on the formatted text property to provide richer text composing and layout!

The timing of this couldn't be better, because there is alignment on adding an "image" expression operator in #8052 as well.

sources - array of sources from where image would be tried to be fetched, like <img src=[url1, url2...]> (spritesheet, base url, etc). Example:
["spritesheet://icons", "https://www.mapbox.com/images/"]

This has been discussed in the past and is a big change that affects not just the expression and style-spec, but also studio and offline functionality in gl-native.

Third argument is an options object whose properties may be [...] The options argument would be optional. Support for aforementioned object properties may be added in the future if found useful.

Is the options argument needed for the images to be usable inside text fields? In the absence of options, would the images in text-fields always be scaled to fit within the line height?

When the text field is laid out vertically (either using text-writing-mode: "vertical" or symbol-placement: "line", will the image be laid out horizontally or vertically as well? It's not clear to me what the sensible default would be for this.

cc @vakila @ansis @mapbox/studio @mapbox/map-design-team

[alexshalamov]

@asheemmamoowala thanks for feedback! Few comments.

This has been discussed in the past and is a big change that affects not just the expression and style-spec, but also studio and offline functionality in gl-native.

All properties for an options object could be added later if we find any of them useful for map designers. Initially, we could provide 'placeholder' in parser code. Parser would expect optional 'image options' object that in the future could contain new properties, therefore, we could guarantee style parsing backward / forward compatibility.

Is the options argument needed for the images to be usable inside text fields?

I would say yes, I would expect image with same options to produce similar rendering result.

In the absence of options, would the images in text-fields always be scaled to fit within the line height?

Most rich-text editors would change line metrics if image with height that is larger than the line height is embedded. Another option is to proportionally scale image to the line height and if designers want image to be larger, scale option may be used in that case.

When the text field is laid out vertically

I would say that image should be in it's natural orientation, upright, in any writing mode. I will add note to myself to research if there are use-cases for rotated images in vertical writing modes.

[asheemmamoowala]

The image operator options will really only be useful in the usage inside a formatted text-field. If used with icon-image or other image properties, it would be confusing in conjunction with the other layout properties, for ex: icon-size.

Would it work to put the image formatter options into the formatter object, similar to how text-color and text-scale are implemented for text sections?

"text-field": ["format", 
                    "London ",
                    ["image", "london-underground"], {"icon-color": "green", "icon-width":24},
                    " and Taipei ",
                    ["image", "taipei-metro"],
                    " metro\n",
                    "SF ", {"text-color": "blue"},
                    ["image", "san-francisco-bart"],
                    " bart"
]

[alexshalamov]

@asheemmamoowala Good point. Let's move options out of expression, so that it can be reused in 'icon-image'. I will update proposal accordingly.

Still, I would keep one optional 'object' (3rd argument) for an 'image' expression. Would be easy to add common options in the future, for example 'sources'.

[asheemmamoowala]

@mapbox/studio @mapbox/map-design-team - any thoughts on this proposal?

[chloekraw]

cc: mobile engineers in case you have any feedback -- we're planning to start implementation soon! @mapbox/maps-ios @mapbox/maps-android @1ec5

[tobrun]

This is going to be great @alexshalamov, really like the api design.

[ansis]

This looks great!

I agree that sources/width/height/color could be left out in the first version and considered separately so that they also work with icon-image.

In order to avoid reshaping the text on every frame the icon would need to scale together with the text. This means that text-size would affect the icon. How does this work? Let's say we have "text-size": ["step", ["zoom"], 16, 5, 24] and a 10px image. How big would the image be rendered at zoom 4? at zoom 5?

Most rich-text editors would change line metrics if image with height that is larger than the line height is embedded.

Sounds good to me