Feature(app): Add an endpoint to recall generation parameters

[lstein]

Summary

This PR adds a new route endpoint named /api/v1/recall which updates the recallable image generation parameters with new values passed via a POST. The route then triggers the front end to load the updated values. This is intended to make it possible to integrate InvokeAI into desktop workflows where image generation parameters are stored in an external database. It also creates the foundation for remote-controlling the front end.

Related Issues / Discussions

None.

QA Instructions

You can test this PR by sending the following requests to the back end and confirming that the frontend picks up and displays the changed parameters.

Basic Fields:

curl -X POST http://localhost:9090/api/v1/recall/default \
  -H "Content-Type: application/json" \
  -d '{
    "positive_prompt": "a cyberpunk city at night",
    "negative_prompt": "dark, unclear",
    "model": "sd-1.5",
    "steps": 30,
    "seed": 42,
    "cfg_scale": 7.5,
    "width": 512,
    "height": 768
  }'

The full list of recallable parameters that can try for testing is here:

Parameter	Type	Description
positive_prompt	string	Positive prompt text
negative_prompt	string	Negative prompt text
model	string	Main model name/identifier
refiner_model	string	Refiner model name/identifier
vae_model	string	VAE model name/identifier
scheduler	string	Scheduler name
steps	integer	Number of generation steps (≥1)
refiner_steps	integer	Number of refiner steps (≥0)
cfg_scale	number	CFG scale for guidance
cfg_rescale_multiplier	number	CFG rescale multiplier
refiner_cfg_scale	number	Refiner CFG scale
guidance	number	Guidance scale
width	integer	Image width in pixels (≥64)
height	integer	Image height in pixels (≥64)
seed	integer	Random seed (≥0)
denoise_strength	number	Denoising strength (0-1)
refiner_denoise_start	number	Refiner denoising start (0-1)
clip_skip	integer	CLIP skip layers (≥0)
seamless_x	boolean	Enable seamless X tiling
seamless_y	boolean	Enable seamless Y tiling
refiner_positive_aesthetic_score	number	Refiner positive aesthetic score
refiner_negative_aesthetic_score	number	Refiner negative aesthetic score

LoRA Models:

(Change the LoRA names as appropriate to what you have installed).

curl -X POST http://localhost:9090/api/v1/recall/default \
  -H "Content-Type: application/json" \
  -d '{
    "model": "stable-diffusion-xl-base-1-0",
    "loras": [
      {
        "model_name": "Alien Style",
        "weight": 0.8,
        "is_enabled": true
      },
      {
        "model_name": "add-detail-xl",
        "weight": 0.5,
        "is_enabled": true
      }
    ]
  }'

IP Adapters:

Currently you can only use images that have previously been uploaded or generated and are located in the INVOKEAI/outputs/images directory.

curl -X POST http://localhost:9090/api/v1/recall/default -H "Content-Type: application/json" -d '{
    "ip_adapters": [
      {
        "model_name": "FLUX Redux",
        "image_name": "8b5b86dd-c742-4235-b4bc-5ec6b1ff2bc7.png",
        "begin_step_percent": 0.0,
        "end_step_percent": 1.0,
        "image_influence": "high"
      },
      {
        "model_name": "XLabs FLUX IP-Adapter",
        "image_name": "988adc09-8e71-4aef-bd83-f4817bc147a4.png",
        "weight": 0.55,
        "begin_step_percent": 0.0,
        "end_step_percent": 1.0,
        "method": "composition"
      }
    ]
  }'

The API handles both ip_adapters and flux_redux models, even though they take different parameters (irrelevant parameters are ignored). This matches the behavior of the front end.

ControlNets

Again, you need to provide the name of an image that is already in the InvokeAI system. If no image name is provided, then the controlnet parameters will be loaded and the user can upload the image of their choice.

curl -X POST http://localhost:9090/api/v1/recall/default \
  -H "Content-Type: application/json" \
  -d '{
    "control_layers": [
      {
        "model_name": "controlnet-canny-sdxl-1.0",
        "image_name": "8b5b86dd-c742-4235-b4bc-5ec6b1ff2bc7.png",
        "weight": 0.75,
        "begin_step_percent": 0.0,
        "end_step_percent": 0.8,
        "control_mode": "balanced"
      }
    ]
  }'

Full Description of the API

See docs/contributing/RECALL_PARAMETERS/*.md

Merge Plan

Should be a simple merge.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[JPPhoto]

An idea: Could this work with existing saved (i.e. in-database) workflows rather than recalling past queued generations, using linear-exposed parameters as those you could populate?

[lstein]

An idea: Could this work with existing saved (i.e. in-database) workflows rather than recalling past queued generations, using linear-exposed parameters as those you could populate?

This functionality is agnostic about where the parameters are coming from. They could be retrieved from image metadata, come out of an in-database workflow, or be generated de novo by another application. My driving use case is to enable a loose integration with PhotoMapAI such that if you have indexed and clustered a folder of InvokeAI images, then you can click on an image, retrieve its linear-exposed parameters, and pass them to a running InvokeAI for remixing.

Or here's a fun use case. You right click on an image in some photo app or other and send it to a visual LLM to generate a description of the image. You then retrieve the image dimensions and send them along to InvokeAI to generate something that matches the description.

[JPPhoto]

Fair. I am going to defer to frontend folks on this one. But I definitely like the idea and see usefulness in creating another endpoint that lets you execute a specified saved workflow by giving it potential overrides for its exposed linear parameters. This is certainly one path to get to sub-workflow execution if we do it right.

[JPPhoto]

Python code looks fine to me.

[dunkeroni]

Works in my testing.

This feels like something that might become brittle/outdated as more models and UI inputs get added in the future. If we're going to support remote UI control through this API, we should follow up with adding tests and way to keep it from falling behind when other PRs add new features.