Guides
Background removal
֍
An introduction to other image generation techniques
image
Red chair in a room
image
Remove the chair

Let's build a workflow that removes the foreground object from an image. The pipeline:

  • Generates an image using
    GenerateImage

    Generate an image.

    GenerateImage( prompt="hokusai futuristic supercell spiral cloud with glowing core over turbulent ocean", store="hosted", )
    Output
    { "image_uri": "https://assets.substrate.run/84848484.jpg" }
  • Removes the background of the image using
    RemoveBackground

    Remove the background from an image and return the foreground segment as a cut-out or a mask.

    RemoveBackground( image_uri="https://media.substrate.run/apple-forest.jpeg", store="hosted", )
    Output
    { "image_uri": "https://assets.substrate.run/84848484.jpg" }
    , creating a foreground mask
  • Erase content inside the mask using
    EraseImage

    Erase the masked part of an image, e.g. to remove an object by inpainting.

    EraseImage( image_uri="https://media.substrate.run/apple-forest.jpeg", mask_image_uri="https://media.substrate.run/apple-forest-mask.jpeg", store="hosted", )
    Output
    { "image_uri": "https://assets.substrate.run/84848484.jpg" }
  • Fill in details using
    InpaintImage

    Edit an image using image generation inside part of the image or the full image.

    InpaintImage( image_uri="https://media.substrate.run/docs-klimt-park.jpg", mask_image_uri="https://media.substrate.run/spiral-logo.jpeg", prompt="large tropical colorful bright anime birds in a dark jungle full of vines, high resolution", store="hosted", )
    Output
    { "image_uri": "https://assets.substrate.run/84848484.jpg" }
image
GenerateImage
mask
RemoveBackground
erase
EraseImage
inpaint
InpaintImage

First, initialize Substrate:

Python
TypeScript
from substrate import (
Substrate,
GenerateImage,
RemoveBackground,
EraseImage,
InpaintImage,
sb,
)

s = Substrate(api_key=YOUR_API_KEY)

Generate an image using

GenerateImage

Generate an image.

GenerateImage( prompt="hokusai futuristic supercell spiral cloud with glowing core over turbulent ocean", store="hosted", )
Output
{ "image_uri": "https://assets.substrate.run/84848484.jpg" }
.

Advanced image generation
  • For more control over Stable Diffusion XL, use StableDiffusionXLLightning (opens in a new tab).
  • Provide a seed to "pin" the initial noise in the image generation process – this can be a good way to experiment with subtle changes in your prompt.
Python
TypeScript
prompt="by edward hopper, a dark red chesterfield leather wing chair in a dark majestic room, pillars, celestial galaxy wallpaper",
image = GenerateImage(
prompt=prompt,
)

Remove the background from the image, and turn the result into a "mask" (a black and white image). Without return_mask=True,

RemoveBackground

Remove the background from an image and return the foreground segment as a cut-out or a mask.

RemoveBackground( image_uri="https://media.substrate.run/apple-forest.jpeg", store="hosted", )
Output
{ "image_uri": "https://assets.substrate.run/84848484.jpg" }
returns the foreground segment of the image.

Python
TypeScript
mask = RemoveBackground(
image_uri=image.future.image_uri,
return_mask=True,
)

Erase the content inside the mask using

EraseImage

Erase the masked part of an image, e.g. to remove an object by inpainting.

EraseImage( image_uri="https://media.substrate.run/apple-forest.jpeg", mask_image_uri="https://media.substrate.run/apple-forest-mask.jpeg", store="hosted", )
Output
{ "image_uri": "https://assets.substrate.run/84848484.jpg" }
. This produces a rough approximation of erasure, so we follow it with
InpaintImage

Edit an image using image generation inside part of the image or the full image.

InpaintImage( image_uri="https://media.substrate.run/docs-klimt-park.jpg", mask_image_uri="https://media.substrate.run/spiral-logo.jpeg", prompt="large tropical colorful bright anime birds in a dark jungle full of vines, high resolution", store="hosted", )
Output
{ "image_uri": "https://assets.substrate.run/84848484.jpg" }
. (For deeper control of the inpainting process, use
StableDiffusionXLInpaint

Edit an image using Stable Diffusion XL. Supports inpainting (edit part of the image with a mask) and image-to-image (edit the full image).

StableDiffusionXLInpaint( image_uri="https://media.substrate.run/docs-klimt-park.jpg", mask_image_uri="https://media.substrate.run/spiral-logo.jpeg", prompt="large tropical colorful bright birds in a jungle, high resolution oil painting", negative_prompt="dark, cartoon, anime", strength=0.8, num_images=2, store="hosted", seeds=[ 1607280, 1720395, ], )
Output
{ "outputs": [ { "image_uri": "https://assets.substrate.run/84848484.jpg", "seed": 1607326 }, { "image_uri": "https://assets.substrate.run/48484848.jpg", "seed": 1720398 } ] }
.) Finally, call substrate.run with the terminal node of the pipeline, inpaint.

Python
TypeScript
erase = EraseImage(
image_uri=image.future.image_uri,
mask_image_uri=mask.future.image_uri,
)
inpaint = InpaintImage(
image_uri=erase.future.image_uri,
prompt=prompt,
)
res = s.run(inpaint)

Here's a sample result from this pipeline:

image
res.get(image)
image
res.get(mask)
no background
res.get(erase)
image
res.get(inpaint)
Last updated on