Trying Generative Fill in Photoshop (beta)

There has been a lot of talk lately about generative AI, and since the beta version of Photoshop with “generative fill” is now available for my Adobe photo plan, I gave it a try. By the way, I am neither an AI expert nor an image processing professional, so this is just a playful report.

Generative Fill is Adobe’s take on the AI image generation based on verbal instructions that has become widely known with Midjourney, Stable Diffusion, and others, and makes it available as a feature of the Repair Brush. While the same feature allows for pinpoint image correction by specifying a selection, the major difference is the ability to freely add elements that were not present in the image.

Let’s get started.

ジェネレーティブ塗りつぶし説明の題材:正面向きのダイサギ Subject of generative fill explanation: front-facing egret
This is the base photo for this post. A Great Egret in a puddle is facing forward and staring at me. I like the atmosphere of the subject, but the cormorant is in an odd position in the background. Also, the background is under construction and covered with a black sheet, so it is not a natural terrain.

First, select the area around the cormorant. After creating the selection, a prompt for generative fill appears, so type in what you want to do.

使用したプロンプトの説明 explanation of the prompt used
“Remove the bird”, simply

A quick flick of the “Generate” button completes the process in about 10 to 20 seconds.

ジェネレーティブ塗りつぶし説明の題材:背景の鳥の除去 Subject of generative fill explanation: removal of a background bird
The cormorant nicely disappeared. There were unnatural versions of the water surface, but this will not look out of place.
ジェネレーティブ塗りつぶし説明の題材:マスキングエリア Subject of generative fill explanation: masking area
The area without red overlay is the selected area.

In this case, there are no other overlapping objects and the surrounding area is water surface, so the conventional restoration brush tool can be used, but it would be a pain if the area is large. Generative fill blends the reflections on the water surface as well as the blurriness with the entire photo in one shot, which is by far the fastest way to do it.

Next, change the bleak construction background to a natural scene. Select the top half of the background, except for the egret, and enter the following prompts.

使用したプロンプトの説明 explanation of the prompt used
Shore of lake which is made of rough and bulky rocks with dark brown colors.
ジェネレーティブ塗りつぶし説明の題材:背景の置換 Subject of generative fill explanation: switching background
Like this. The background was replaced with a background that blends well with the original photo, with little discrepancy in bokeh, color temperature or lighting.
ジェネレーティブ塗りつぶし説明の題材:マスキングエリア Subject of generative fill explanation: masking area
Selected area.

Incidentally, the following two pictures are different versions generated. One of the egret’s head is slightly deformed, but the result is not so bad.

ジェネレーティブ塗りつぶし説明の題材:背景バリエーション Subject of generative fill explanation: variation of the background
ジェネレーティブ塗りつぶし説明の題材:背景バリエーション Subject of generative fill explanation: variation of the background

Next, let’s turn the front-facing face of the great egret to the side. I like the frontal face in this photo, but a side profile would be more common for bird photos.

ジェネレーティブ塗りつぶし説明の題材:右向きのサギ Subject of generative fill explanation: right-facing egret
Result of selecting the entire face from the middle of the neck and instructing the face to turn sideways. Not bad, but the neck is bent in a strange direction from the border selected by the rectangle.

Therefore, I instructed it to modify the image to take into account areas outside the selection.

使用したプロンプトの説明 explanation of the prompt used
Change the egret’s face from straight forward to right. the neck should be smoothly lined with the neck outside the selected area.
ジェネレーティブ塗りつぶし説明の題材:右向きのサギ Subject of generative fill explanation: right-facing egret
It felt better than before. I’m still uncomfortable with the neckline at the throat.

Pinpoint and select the curve of the throat to make it thicker.

使用したプロンプトの説明 explanation of the prompt used
“Broaden the neck, more smoothly”. The center is okay, but what are the others?
ジェネレーティブ塗りつぶし説明の題材:右向きのサギ Subject of generative fill explanation: right-facing egret
It became much more natural. What do you think?

In fact, although it is difficult to see in the overall photo, there are areas that remain uncomfortable when viewed in detail. Due to the selection used to modify the background, an unnatural band can be seen at the border between the neck and the background. Let’s try to remove this.

ジェネレーティブ塗りつぶし説明の題材:拡大図 Subject of generative fill explanation: zoom-in
The middle of the neck, right side border is blurred white.
使用したプロンプトの説明 explanation of the prompt used
Remove the unnatural blur on the border of the neck and the background.

It became like this. The blurring of the borders has disappeared and the image has been cleaned up. The thickness of the neck has also changed slightly, but this would be acceptable if it were not compared with the original image.

ジェネレーティブ塗りつぶし説明の題材:拡大図 Subject of generative fill explanation: zoom-in
After modification of the border between the neck and the background

At least, this is the finished product. If I had to give more orders, I would say that the face of the great egret has a rather flat texture and lacks detail, the beak is highly saturated, and so on. Nevertheless, the quick look of the finished image is not terribly different from the original.

In reality, it takes some trial and error in the way the selection is made and the prompts are drawn, but I have no doubt that this will be further refined in the future.

The AI noise reduction in Lightroom is also extremely powerful, and I feel that the world has truly become a magical place. With this, social networking sites will be flooded with more and more images that look spectacular. I wonder what will happen when the saturation point is reached.

ジェネレーティブ塗りつぶし説明の題材:最終版 Subject of generative fill explanation: finished
This time it’s finished.

This is an example of a case in which I simply instructed the AI to “Modify the border between the neck and the background more naturally”. Something bone-like or a bird’s head popped out from the neck. Scary.

ジェネレーティブ塗りつぶし説明の題材:失敗例 Subject of generative fill explanation: example of failure
I think it’s still a good idea to give it specific instructions on what you want them to do.

An example of trying to add luminance noise to the same level as the surrounding area in order to eliminate the flatness of the egret’s face. Even though I instructed the AI to “add luminance noise,” it may have interpreted “noise” as “randomness in the image (object)”. It became even more chaotic.

ジェネレーティブ塗りつぶし説明の題材:失敗例 Subject of generative fill explanation: example of failure
ジェネレーティブ塗りつぶし説明の題材:失敗例 Subject of generative fill explanation: example of failure

Leave a Reply

Your email address will not be published. Required fields are marked *