How to Regenerate Specific Parts of an Image Using Vary Region on Midjourney

What to know

Vary (Region) is Midjourney’s in-painting tool that allows users to remove unwanted elements from their creations and replace them with something else.
The feature can be used to replace unwanted objects in an image, add new objects in portions where there aren’t any objects, or change the background or lighting of images you’ve already generated.
When activating the Vary (Region) feature, Midjourney will automatically decide what to replace your selected portions of an image with. But you can also specify what elements you want as replacements by combining Vary (Region) with Midjourney’s Remix Mode.
To use this feature, you need to first generate an image within Midjourney, upscale one of the images from the generated image grid, and activate Vary (Region) from the upscaled image. From here, you can select a portion or multiple portions you want to replace and press Submit to create a new set of images.

What is Vary Region on Midjourney?

Midjourney has become quite popular since its debut last year for its creative capabilities but one thing it has lacked was the ability to replace parts of an image which Midjourney calls Vary (Region). Otherwise known as “in-painting” or “generative infill”, the feature works similarly to Adobe’s Generative Fill, allowing users to select a portion from an existing image and then replace this portion with other elements.

Some applications for this feature include removing and replacing unwanted objects from your creations, adding new objects in portions where there aren’t any objects, changing the lighting, background, or mood of the image with a different description, etc. The Vary (Region) feature can be used on existing images you’ve created on Midjourney and to access this option, you’ll first need to upscale an image you wish to modify.

When this feature is activated, Midjourney will show an Editor window from where you can select portions of the image you wish to regenerate. You can either use the rectangular selector tool or the Lasso tool to make a freehand selection of the portion you want to replace.

You can use Vary (Region) in combination with the Remix functionality to replace selected portions with elements you specify. When the feature is used without Remix mode, Midjourney will take creative control over what to replace the selected portion with.

The number of objects you select or the area of selection will affect how differently the new sets of images look relative to the original image. If you select a larger portion, Midjourney will take a more creative role in replacing the elements in the selected area while the regenerated portions will have subtler changes with smaller portions selected.

Midjourney claims that the Vary Region feature works best when replacing larger portions of an image, roughly between 20% and 50%. For replacing minor (<20%) or much larger portions (>50%) of an image, you’re better off using the Vary (Subtle) or Vary (Strong) options instead.

How to use Vary Region to regenerate specific parts of an image on Midjourney

The Vary (Region) is an extension of the Vary options – Vary (Strong) and Vary (Subtle) – that have been available on Midjourney since the release of version 5.2. With Vary (Region), you’re basically giving Midjourney directions by telling which part of the image you want to modify. You can follow the instructions below to regenerate specific parts of an image on Midjourney using the Vary (Region) feature.

Step 1: Find an image you want to modify

Before you can use the Vary (Region) feature, you need to first locate the image you want to modify using this feature on Midjourney. Images can only be in-painted if they were created using Midjourney and are present inside the Discord server or chat where you first generated them. You cannot regenerate parts of an image that’s saved locally on your computer or is shared on the web.

To get started, open the Midjourney server, Bot, or your Discord DM where you created the existing set of images and locate the images you previously generated. You can skip the next set of instructions if you’ve already generated images you want to variate from and move to Step 2 of this guide.

In case you want to start from scratch and generate new images, open any of Midjourney’s servers or the Midjourney Bot on your server or Discord DM. Regardless of where you access Midjourney, click on the text box at the bottom. Here, type /imagine and select the /imagine option from the menu.

Now, enter your desired prompt inside the “prompt” box to request Midjourney to create the original images, and then press the Enter key on your keyboard.

Midjourney will now process your input prompt and create a set of 4 images based on your prompt.

Once you’ve located the image or created an image that you want to regenerate using Midjourney’s inpainting tool, you can proceed to the next step to upscale your desired image and use the Vary (Region) feature on it.

Step 2: Upscale this image and activate Vary (Region)

Before you can regenerate portions of an image on Midjourney, you need to upscale the image that you want to create variations. For this, click on the corresponding upscale option, any one of U1 – U4.

When you choose an image for upscaling, Midjourney will instantly process your request and show you the upscaled version of the selected image. To regenerate a portion of this image, click on the Vary (Region) button underneath the upscaled image on the screen.

When you do that, the Midjourney Bot will show you a new Editor window on the screen with a preview of the upscaled image. You can now move on to Step 3 of this guide to select the portions of the image you wish to replace.

Step 3: Select portions of the image that you want to regenerate

When the Midjourney Editor screen appears, you’re now required to select the portion of the image you wish to regenerate. There are two ways to do that – one using the Rectangle Tool and the other using the Lasso Tool.

Using Rectangle Tool

The Rectangle Tool allows you to choose a rectangular area to mark the portion of the image you want Midjourney to replace; it’s generally suited to select a larger area of an image. To use this tool, click on the Rectangle Tool button (the one marked by a dotted square icon) beneath the image.

With the Rectangle Tool selected, you can create a rectangle to mark an undesired portion of the image by clicking from one point and dragging the pointer diagonally across this portion.

Midjourney allows you to mark multiple portions or use the Rectangle Tool multiple times to fully cover all the areas that you want to AI image generator to recreate. This way, you can click and drag a different rectangle that overlaps with the previously selected area to fully select the subject or element that you wish to replace from the image.

Using Lasso Tool

The Lasso Tool allows you to do a freehand selection that you can drag around with the pointer; generally useful when selecting smaller areas or irregularly shaped objects. To activate it, click on the Lasso Tool (an icon resembling that of a balloon) underneath the image inside Midjourney’s Editor screen.

When the Lass Tool is active, drag through the area of the screen that you want to change. You can drag it from one point to another and then move the pointer towards a third point so that Midjourney marks the area in between all the points.

You can mark circular areas, curved regions, and smaller objects using the Lasso tool as it only selects the region that you want to replace, nothing beyond it.

You can use both tools to mark different portions within the same image inside the Editor screen. When you mark regions inside the Editor window, the selected regions aren’t editable but there’s a way you can unselect a selected region.

If you have selected a region that need not be selected for regeneration, you can use the Undo button (marked by a circular arrow) at the top left corner of the Editor to cancel the last selection. Let’s say, the last selection you made on the Editor is of this potted plant on the floor.

Pressing on the Undo button will remove this selection if you had selected the area in one go previously. Repeatedly pressing the Undo button will remove the previous selections one at a time.

Once you’ve selected the portions to replace, click on Submit at the bottom right corner.

Step 4: Check out your modified images

When you send Midjourney your request with portions of your image selected, it will start processing it. Once processed, you’ll see a set of 4 images on the screen, all different from one another. None of these images will now have the elements you wanted to remove from the selected region.

You can check out each of these images in a larger preview by clicking on the image grid and then clicking on Open in Browser underneath the image.

From here, you can either upscale or variate the images that you generated or use the Vary (Region) option again on the original image to get more regenerated creations.

Here’s what the final set of images looks like in relation to the original image.

Original prompt	Original Image	Selected portion	Final Image
interior design room with a lot of plants

How to regenerate specific parts of an image with modified prompts

The above guide involves using the Vary (Region) feature where you select a region to replace from an image and Midjourney decides what to replace it with. But what if you want to override Midjourney’s decision and want to replace the selected area with elements that you’ve pictured replacing it with? You can do that and request the Midjourney bot to regenerate the selection with your desired objects by specifying it in a modified prompt.

Step 1: Enable Remix mode on Midjourney

This is possible by using the Remix Mode on Midjourney which is the utility that allows you to change the prompts in between variations. This will make sure that the new set of images won’t just be based on the original prompt and its generated images but they also get ideas from the new prompt you use with Remix. You can use this mode to change the input prompt in a way that you change the subject, lighting, environment, and medium of the original composition.

The Remix Mode, thus, makes it easy for you to regenerate parts of an image as you can replace them with elements that you want to add to the existing images. So, to regenerate specific parts of an image with modified prompts, you need to enable Remix Mode on your Midjourney account to proceed.

To enable Remix Mode on Midjourney, open any of Midjourney’s servers on Discord, or access the Midjourney Bot from your Discord Server or Discord DM. Regardless of how you access it, tap on the text box at the bottom. Here, type /settings, click on the /settings option, and then press the Enter key on your keyboard.

In the Midjourney response that shows up, click on Remix mode.

When you do that, the Remix mode option will be highlighted with a green background. You can now start creating images with Remix Mode enabled and when you create variations of a generated image using Vary (Region), Midjourney will ask you to enter a new prompt.

Step 2: Follow Steps 1-3 from the above guide

With Remix Mode enabled you can now use the Vary (Region) option on an image that you like to be able to modify the prompts for the desired image. For this, you will need to locate the image grid that you originally generated or create images with a prompt from scratch, upscale one of the images from it, and then activate the Vary (Region) option from the upscaled image.

You can do that by following the instructions provided in Steps 1-3 from the above guide until you reach Midjourney’s Editor screen and select the area you wish to replace.

Once, you’ve selected the portions of the image you want to change, you can move on to Step 3 below.

Step 3: Modify prompts to regenerate selected portions of the image

Once you’ve activated the Vary (Region) feature and selected portions of the image you wish to regenerate, you can modify the prompt inside the text box that appears underneath the image inside Midjourney’s Editor screen. When modifying the description, your focus should be on the details you want to introduce to the selected portion of the image.

So, you can erase parts of the description that concern the unselected areas of the image and then replace those words with keywords to describe the elements you wish to add to the selected portion. While doing so, you can keep words describing the medium or image type intact but replace other parts with the objects or elements you want Midjourney to imagine in the selected area.

Although the Midjourney Bot considers elements of the existing image, you can modify your prompt with a focus on what you want to happen in the selected area to get the desired results. Midjourney recommends using shorter focused prompts as they will be more effective than describing the whole scene in detail. When regenerating multiple portions of an image, we recommend you work on replacing one portion at a time to prevent Midjourney from adding your desired objects all across the image.

Once you’ve modified the prompt with a different description, click on the Right arrow button at the bottom right corner.

You can check out each of these images in a larger preview by clicking on the image grid and then clicking on Open in Browser underneath the image.

From here, you can either upscale the images that you generated or use the Vary (Region) option again on the original image to get more regenerated creations. You can also upscale one of the output images and then use the Vary (Region) tool to edit the image further with different replacements.

Here’s what the final set of images looks like in relation to the original image.

Original prompt	Original Image	Selected portion	Modified prompt	Final Image
interior design room with a lot of plants			interior chandelier

What happens when you regenerate an image that you’ve already regenerated using Vary Region?

Midjourney’s in-painting tool preserves your selection even after creating regenerated versions of an upscaled image on your account. If you had previously used the Vary (Region) tool on an image and selected an area that you wanted to replace, going back to the Vary (Region) option again will show the existing selection within the Editor screen even at a later time.

If you’re using the Vary (Region) feature once more after activating it previously on an upscaled image, you’ll be able to cancel this selection by clicking on the Undo button at the top left corner of the Editor. This makes it easy for users to select different portions of an image that they want to regenerate without changing other aspects of the original image.

What kind of images can you create using Vary Region?

We tried the Vary (Region) feature on a bunch of images we created on Midjourney with modified prompts and for the most part, the platform was able to produce stunning results by replacing the selected portions with new elements. Here’s a bunch of examples:

Original prompt	Original Image	Modified prompt	Final Image
a street view of a 1960s american cafe in the style of Adrian Donoghue		1964 Ford Mustang
surreal image of a giant Arc de Triomphe gate on a seaside in the style of Ben Goossens		surreal image of Taj Mahal on a seaside in the style of Ben Goossens
Karen Knorr’s photogragh of a room inside a Mughal architecture with intricate details –no reflections		an old glass shelf
Chris Burkard’s photogragh of a hilltop overlooking the sea –no people, person		mountain goat looking at the camera