AI_Program4 のバックアップ(No.32)

私的AI研究会 > AI_Program4

生成 AI プログラミング４ †

　これまで検証してきた結果をもとに、Python で生成 AI プログラムを書く

▲　目　次

生成 AI プログラミング４
参考資料

※ 最終更新:2025/08/20　

↑

diffusersではじめめる Stable Diffusion （応用編２） †

　顔の崩れを修正する　face_recognition と ADetailer controlnet Paint-By-Example / コントロールネット機能のまとめ

↑

動作環境 †

このプロジェクトは以下の Anaconda 仮想環境とプロジェクト・フォルダで動作する
```
(base) PS > conda activate sd_test
(sd_test) PS > cd workspace_3/sd_test
```

↑

概要 †

この章で作成するプログラム一覧と実行速度の目安

Step		プログラム	GPU					CPU
Step		プログラム	RTX 4070Ti	RTX 4060	RTX 4060L	RTX 3050	GTX 1050	i7-1260P
50	顔の崩れを修正する１	sd_050.py	00:01	00:03	00:10	00:05	00:37	02:12
51	顔の崩れを修正する２「ADetailer」	sd_051.py	00:01	00:03	00:03	00:05	00:34	03:35
52	画像の一部を変換する「Paint-By-Example」	sd_052.py	00:02	00:05	00:13	00:15	02:51	04:03
53	コントロールネットの機能をまとめる	sd_053.py	00:02		00:16		00:40	05:18

　・単位　（時：）分：秒

↑

Step 50：顔の崩れを修正する１ †

はじめに
・全身の画像などの顔の面積が小さいときの画像生成では顔が崩れてしまうことが多い
・顔認識を利用して顔を抽出して拡大再生成した画像を埋め込む方法を実践してみる

処理の流れ
① 元画像から顔認識パッケージ「face_recognition」で科をの領域を検出
② 顔の領域を 512x512 ピクセルサイズに拡大し、「StableDiffusionImg2ImgPipeline」で画像を生成
③ 元の画像の同じ領域に埋め込む
④ 埋め込んだ画像の周辺処理のため 8bit グレイスケールのマスク画像を作成
⑤ マスク画像の周辺をぼかした画像を使って元画像と新しく生成した画像を合成して完成

元画像顔の抽出顔の修正完成画像

周辺の補正前自動で作成したマスクマスクで補正した完成画像

顔認識パッケージ「face_recognition」について
・以前「顔認証アプリケーション基礎編」で検証した（2022/6～）→ 顔認証 (Face recognition)
・現在は「pip face_recognition」コマンドからインストール可能となっている
・Windows 環境下では同時にインストールされる「dlib」パッケージを事前にインストールしておく必要がある（2025/8 現在）
・「dlib」パッケージは「conda-forge」からインストールできる

追加のパッケージ・インストール

 conda install dlib -c conda-forge

 pip install face_recognition

プログラムを実行する（実行時間：約 2秒 RTX 4070 Ti 12GB）

 python sd_050.py

(sd_test) PS > python sd_050.py

Stable Diffusion with diffusers(050)  Ver 0.06: Starting application...

 --result_image             :   results/image_050.png
 --cpu                      :   False
 --log                      :   3
 --model_dir                :   /StabilityMatrix/Data/Models/StableDiffusion
 --model_path               :   SD1.5/beautifulRealistic_brav5.safetensors
 --image_path               :   images/sd_050_test.jpg
 --max_size                 :   0
 --prompt                   :   masterpiece, high quality, very_high_resolution, large_filesize, full color, an extremely cute face, woman, symmetrical, HDR, real, realistic
 --seed                     :   12345678
 --width                    :   512
 --height                   :   512
 --step                     :   20
 --scale                    :   8.5
 --strength                 :   0.4
 --neg_prompt               :   lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, multiple legs, malformation

Fetching 11 files: 100%|███████████████████████████████| 11/11 [00:00<?, ?it/s]
Loading pipeline components...: 100%|████████████| 6/6 [00:00<00:00, 19.56it/s]
100%|██████████████████████████████████████████| 20/20 [00:02<00:00,  9.50it/s]
result_file: results/image_050.png

Finished.

画像ファイル「image_050.png」が生成される
実行例

元画像顔の抽出顔の修正完成画像

モジュール・ソースコード

▼「sd_050.py」

# -*- coding: utf-8 -*-
##--------------------------------------------------
##  Stable Diffusion with diffusers(050)   Ver 0.06
##
##               2025.07.31 Masahiro Izutsu
##--------------------------------------------------
## sd_050.py    顔の崩れを修正する
##  Ver 0.06    2025.07.31  sd_081 IP-Adapter 対応

# タイトル
title = 'Stable Diffusion with diffusers(050)  Ver 0.06'

import warnings
warnings.simplefilter('ignore')

# インポート＆初期設定
import os
import torch
from PIL import Image
from PIL import ImageDraw, ImageFilter
import face_recognition
from diffusers import StableDiffusionUpscalePipeline
from diffusers import StableDiffusionImg2ImgPipeline
from diffusers import logging

import my_logging
import sd_tools as sdt

logging.set_verbosity_error()

# 定数定義
DEF_MODEL_CNTL = 'control_v11p_sd15_inpaint_fp16.safetensors'
DEF_MODEL_BASE = 'SD1.5/beautifulRealistic_brav5.safetensors'
DEF_IMAGE_PATH = 'images/sd_050_test.jpg'
DEF_PROMPT = 'masterpiece, high quality, very_high_resolution, large_filesize, full color, an extremely cute face, woman, symmetrical, HDR, real, realistic'
DEF_NEG_PROMPT = 'lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, multiple legs, malformation'
#FACE_RECOGNITION_MODEL_ID = "hog"                                              # 速度重視の場合
FACE_RECOGNITION_MODEL_ID = "cnn"                                               # 精度重視の場合
UPSCALE_MODEL_ID = "stabilityai/stable-diffusion-x4-upscaler"

# コマンドライン定義
opt_list = [
            ['pros_sel','','sd_050'],                                                                       #  0
            ['result_image', 'results/image_050.png', 'path to output image file'],                         #  1
            ['cpu', 'store_true', 'cpu mode'],                                                              #  2
            ['log', '3', 'Log level(-1/0/1/2/3/4/5) Default value is \'3\''],                               #  3
            ['model_dir', '/StabilityMatrix/Data/Models/StableDiffusion', 'Model directory'],               #  4
            ['model_path', DEF_MODEL_BASE, 'Model Path'],                                                   #  5
            ['image_path', DEF_IMAGE_PATH, 'Sourcs image file path'],                                       #  6
            ['max_size', 0, 'image max size (0=source)'],                                                   #  7
            ['prompt', DEF_PROMPT, 'Prompt text'],                                                          #  8
            ['seed', 12345678, 'Seed parameter (-1 = rundom)'],                                             #  9
            ['width', 512, 'image size width'],                                                             # 10
            ['height', 512, 'image size height'],                                                           # 11
            ['step', 20, 'infer step'],                                                                     # 12
            ['scale', 8.5, 'gaidanse scale'],                                                               # 13
            ['strength', 0.4, 'strength value'],                                                            # 15
            ['neg_prompt', DEF_NEG_PROMPT, 'Negative Prompt text'],                                         # 16
           ]
# 画像確認
def image_log(pil_image, wait_s = -1):
    if wait_s >= 0:
        sdt.image_save2(pil_image, save_path = '', dispname = 'Check image', maxsize = 800, wait_s = wait_s)

# 画像を 512x512 アップスケール
def upscale(image, prompt, device):
    if device == 'cpu':
        pipeline  = StableDiffusionUpscalePipeline.from_pretrained(UPSCALE_MODEL_ID)
    else:
        pipeline  = StableDiffusionUpscalePipeline.from_pretrained(UPSCALE_MODEL_ID, torch_dtype = torch.float16)
    pipeline.to(device)

    low_image = image.convert("RGB")
    low_image = low_image.resize((128, 128))
    new_image = pipeline(prompt = prompt, image = low_image).images[0]
    return new_image

# 顔検出
def face_detection(file_name, offset=20):
    image = face_recognition.load_image_file(file_name)

    #顔部分を検出
    face_locs = face_recognition.face_locations(image, number_of_times_to_upsample = 1, model = FACE_RECOGNITION_MODEL_ID)

    face_org_rects = []
    face_rects = []
    if len(face_locs) == 0:
        return face_rects, face_org_rects                                       # 検出できない

    for face_loc in face_locs:
        top, right, bottom, left  = face_loc
        face_org_rects.append((left, top, right, bottom))

        # 範囲が狭いとモデルが顔を認識できない時があるため、検出範囲の矩形をoffset分広げる。
        top -= offset
        right += offset
        bottom += offset
        left -= offset

        # 検出範囲を正方形にする
        w = right - left
        h = bottom - top
        if w > h:
            bottom += w-h
        else:
            right += h-w

        face_rects.append((left, top, right, bottom))

    return face_rects, face_org_rects

# 顔のスタイル変換
def style_change(model_path, image, prompt, neg_prompt, guidance_scale = 9.5, strength = 0.4, seed = 0, device = 'cpu'):
    if device == 'cpu':
        pipeline  = StableDiffusionImg2ImgPipeline.from_single_file(model_path)
    else:
        pipeline  = StableDiffusionImg2ImgPipeline.from_single_file(model_path, torch_dtype = torch.float16)
    pipeline.to(device)

    generator = torch.Generator(device).manual_seed(seed)
    new_image = pipeline(
                        prompt = prompt,
                        negative_prompt = neg_prompt,
                        image = image,
                        guidance_scale = guidance_scale,
                        strength = strength,
                        generator = generator
                        ).images[0]

    return new_image

# マスク作成
def create_mask(image_width, image_height, rect_width, rect_height, rect_x, rect_y, offset = 10):
    image = Image.new('L', (image_width, image_height), 'black')                # 8bit グレイスケール 黒の画像を作成
    draw = ImageDraw.Draw(image)

    # offset分大きい真っ白の矩形を描画
    draw.rectangle([rect_x-offset, rect_y-offset, rect_x + rect_width + offset, rect_y + rect_height + offset], fill = 'white')

    # offset分小さい真っ黒の矩形を描描画
    draw.rectangle([rect_x+offset, rect_y+offset, rect_x + rect_width - offset, rect_y + rect_height - offset], fill = 'black')

    return image

# 画像の顔修正する
def face_style_change(model_path, file_name, prompt, neg_prompt, guidance_scale = 9.5, strength = 0.3, seed = 0, device = 'cpu', bUp = False):
    face_rects, face_org_rects = face_detection(file_name, offset = 30)
    if face_rects == [] or face_rects == []:
        return None, None, None                                                 # 顔検出なし

    face_rect = face_rects[0]
    face_org_rect = face_org_rects[0]

    left, top, right, bottom = face_rect
    left_org, top_org, right_org, bottom_org = face_org_rect
    w = right - left
    h = bottom - top

    #オリジナル画像から顔部分を切り出す
    init_img = Image.open(file_name)
    new_img = init_img.copy()
    face = new_img.crop(face_rect)

    # 顔をアップスケール
    if bUp:
        upscaled_face = upscale(face, prompt='face', device = device)           # upscale
    else:
        upscaled_face = face.resize((512, 512))                                 # resize
    image_log(upscaled_face, 1)

    # スタイル変更
    new_face = style_change(model_path, upscaled_face, prompt, neg_prompt, guidance_scale = guidance_scale, strength = strength, seed = seed, device = device)
    image_log(new_face, 1)

    # 元の画像に貼り付け
    new_img.paste(new_face.resize((w, h)), (left, top))
#    image_log(new_img, 0)

    # 顔の領域
    draw = ImageDraw.Draw(init_img)
    rectcolor = (0, 0, 255)                                                     # 矩形の色(RGB)
    linewidth = 2                                                               # 線の太さ
    draw.rectangle([(left_org, top_org), (right_org, bottom_org)], outline=rectcolor, width=linewidth)
#    image_log(init_img, 0)

    # エッジ部分の修正のためのマスクを作成
    image_width, image_height = new_img.size
    mask = create_mask(image_width, image_height, h, w, left, top, offset=8)
#    image_log(mask, 0)

    return init_img, new_img, mask

# 画像生成
def image_generation(model_path, image_path, prompt, seed, num_inference_steps=20, width=512, height=512, guidance_scale=8.5, strength=0.4, neg_prompt = '', device='cpu'):
    work_path = sdt.get_work_path(logger = None)
    os.makedirs(work_path, exist_ok = True)                                     # 作業フォルダ作成
    src_path, mask_path = sdt.get_source_mask_path(image_path, logger = None)   # ソース/マスク画像作成

    image, new_img, mask = face_style_change(model_path, image_path, prompt, neg_prompt, guidance_scale = guidance_scale, strength = strength, seed = seed, device = device, bUp = False)
    if image is None or new_img is None or mask is None:
        return None                                                             # Error

    # マスクのエッジをソフトフォーカスにして元の画像と合成しエッジを修正
    mask = mask.filter(ImageFilter.GaussianBlur(10))
    new_img = Image.composite(image, new_img, mask)

    sdt.image_save2(image, save_path = src_path, dispname = src_path, maxsize = 800, wait_s = 1)
    sdt.image_save2(mask, save_path = mask_path, dispname = '', maxsize = 800, wait_s = 1)

    return new_img

# ** main関数 **
def main(opt, logger = None):
    # パラメータ設定
    device = sdt._get_device(opt, logger)
    result_image_path = sdt._get_result_image_path(opt, logger)
    result_path = sdt._get_result_path(opt, logger)
    prompt = sdt._get_prompt(opt, logger)
    src_image = sdt._get_source_image(opt, logger)
    model_path = sdt._get_model_path(opt, logger)
    height, width = sdt._get_image_size(opt, logger)
    seed = sdt._get_seed_value(opt, logger)
    num_inference_steps = sdt._get_inference_steps(opt, logger)
    guidance_scale = sdt._get_guidance_scale(opt, logger)
    strength = sdt._get_strength(opt, logger)
    neg_prompt = sdt._get_negative_prompt(opt, logger)
    image_path = sdt._get_source_image_path(opt, logger)

    # 出力フォルダ
    os.makedirs(result_path, exist_ok = True)

    # 画像生成
    image = image_generation(model_path, image_path, prompt, seed, num_inference_steps, width, height, guidance_scale, strength, neg_prompt = neg_prompt, device = device)

    if image is None:
        logger.info(f'{sdt.RED｝There is no face in the image !!{sdt.NOCOLOR｝')

    else:
        sdt.image_save2(image, result_image_path, result_image_path)
        logger.info(f'result_file: {result_image_path｝')


# main関数エントリーポイント(実行開始)
if __name__ == "__main__":
    parser = sdt.parse_args(None, opt_list)
    opt = parser.parse_args()
    sdt._get_device(opt)
    sdt.display_info(opt, title)

    # アプリケーション・ログ設定
    module = os.path.basename(__file__)
    module_name = os.path.splitext(module)[0]
    logger = my_logging.get_module_logger_sel(module_name, int(opt.log))

    main(opt, logger)

    logger.info('\nFinished.\n')

　※ 上記ソースコードは表示の都合上、半角コード '}' が全角 '｝'になっていることに注意

↑

Step 51：顔の崩れを修正する２「ADetailer」 †

概要
・「Stable Diffusion」の拡張機能である「ADetailer」を「diffusers」で動かす
・以下は「SD1.5」モデル専用のパッケージ「asdff」を使用する
・性別の指定のため「--ext」パラメータを用意する
　男性の場合：--ext 'boy'
　女性の場合：--ext 'girl' またはオプション指定なし

追加のパッケージ・インストール
```
 pip install asdff
```

プログラムを実行する（実行時間：約 3秒 RTX 4070 Ti 12GB）

 python sd_051.py

(sd_test) PS > python sd_051.py

Stable Diffusion with diffusers(sd_051)     Ver 0.06: Starting application...

 --result_image             :   results/image_051.png
 --cpu                      :   False
 --log                      :   3
 --model_dir                :   /StabilityMatrix/Data/Models/StableDiffusion
 --model_path               :   SD1.5/beautifulRealistic_brav5.safetensors
 --image_path               :   D:/anaconda_win/workspace_3/sd_test/images/sd_050_test.jpg
 --max_size                 :   0
 --prompt                   :   masterpiece, best quality, 1girl
 --seed                     :   12345678
 --step                     :   30

= ADetailer =   prompt: 'masterpiece, best quality, 1girl'  model: 'face_yolov8s.pt'
Fetching 11 files: 100%|███████████████████████████████| 11/11 [00:00<?, ?it/s]
Loading pipeline components...: 100%|████████████| 6/6 [00:00<00:00, 19.93it/s]

0: 640x512 1 face, 36.0ms
Speed: 2.3ms preprocess, 36.0ms inference, 61.3ms postprocess per image at shape (1, 3, 640, 512)
100%|██████████████████████████████████████████| 12/12 [00:01<00:00,  9.07it/s]

0: 640x512 1 face, 23.9ms
Speed: 1.3ms preprocess, 23.9ms inference, 2.0ms postprocess per image at shape (1, 3, 640, 512)
100%|██████████████████████████████████████████| 12/12 [00:01<00:00,  9.17it/s]
result_file: results/image_051-sd_050_test.png

Finished.

画像ファイル「image_051-XXXXXXXX.png」が生成される（XXXXXXXX は入力ファイル名）

生成画像例（Step 50 と同じ元画像の場合）

モジュール・ソースコード

▼「sd_051.py」

# -*- coding: utf-8 -*-
##--------------------------------------------------
##  Stable Diffusion with diffusers(051)  Ver 0.06
##
##               2025.08.02 Masahiro Izutsu
##--------------------------------------------------
## sd_051.py
##  Ver 0.00    2025.08.02  asdff(ADetailer)
##  Ver 0.06    2025.08.03  sd_100.py 統合版対応

# asdff
# https://github.com/Bing-su/asdff
# https://github.com/theblackhatmagician/adetailer_sdxl?tab=readme-ov-file
# https://github.com/Bing-su/adetailer?tab=readme-ov-file

import warnings
warnings.simplefilter('ignore')

# インポート＆初期設定
import os
import argparse
import torch
from functools import partial
from asdff import AdPipeline, yolo_detector
from huggingface_hub import hf_hub_download
from diffusers import logging
from diffusers.utils import load_image

import my_logging
import sd_tools as sdt

logging.set_verbosity_error()                                                   # 不要なエラー出力の抑制


# 定数定義
MODEL_DIR = 'Bingsu/adetailer'
MODEL_FACE = 'face_yolov8s.pt'                                                  # 2D /リアルな顔
MODEL_HAND = 'hand_yolov8n.pt'                                                  # 2D /リアルな手
MODEL_PERSON = 'person_yolov8s-seg.pt'                                          # 2D/リアルな人物

def_result_image = 'results/image_051.png'
def_image_path = ''
def_model_dir = '/StabilityMatrix/Data/Models/StableDiffusion'
def_model_path = 'SD1.5/beautifulRealistic_brav5.safetensors'
def_model = def_model_dir + '/' + def_model_path
def_prompt = 'masterpiece, best quality, 1girl'
def_prompt_m = 'masterpiece, best quality, 1boy'
def_seed = 12345678
def_step = 30
def_ext = ''


# タイトル
title = 'Stable Diffusion with diffusers(sd_051)     Ver 0.06'

# コマンドライン・オプション (argparse) 名前/初期値/ヘルプ
opt_list = [
            ['pros_sel','','sd_048'],                                                          #  0
            ['result_image', def_result_image, 'path to output image file'],                                #  1
            ['cpu', 'store_true', 'cpu mode'],                                                              #  2
            ['log', '3', 'Log level(-1/0/1/2/3/4/5) Default value is \'3\''],                               #  3
            ['model_dir', def_model_dir, 'Model directory'],                                                #  4
            ['model_path', def_model_path, 'Model Path'],                                                   #  5
            ['image_path', def_image_path, 'Sourcs image file path'],                                       #  6
            ['max_size', 0, 'image max size (0=source)'],                                                   #  7
            ['prompt', def_prompt, 'Prompt text'],                                                          #  8
            ['seed', def_seed, 'Seed parameter (-1 = rundom)'],                                             #  9
            ['step', def_step, 'infer step'],                                                               # 10
            ['ext', def_ext, 'Extensions \'\' or \'girl\' or \'boy\''],
           ]


# ** 画像生成 **

# 画像生成（画像ファイルパス入力）拡張版
def image_generation2_ex(ext, image_path, device='cpu', prompt=def_prompt, model_path=def_model, seed=def_seed, num_inference_steps=def_step, ad_model=MODEL_FACE):
    # prompt 変更
    if  ext == 'boy':
        prompt = def_prompt_m
    elif ext == 'girl':
        prompt = def_prompt

    src_image = load_image(image_path)
    image = image_generation(src_image, device, prompt, model_path, seed, num_inference_steps, ad_model)
    return image

# 画像生成（PIL 画像イメージ入力）拡張版
def image_generation_ex(ext, src_image, device='cpu', prompt=def_prompt, model_path=def_model, seed=def_seed, num_inference_steps=def_step, ad_model=MODEL_FACE):
    # prompt 変更
    if  ext == 'boy':
        prompt = def_prompt_m
    elif ext == 'girl':
        prompt = def_prompt

    image = image_generation(src_image, device, prompt, model_path, seed, num_inference_steps, ad_model)
    return image



# 画像生成（画像ファイルパス入力）
def image_generation2(image_path, device='cpu', prompt=def_prompt, model_path=def_model, seed=def_seed, num_inference_steps=def_step, ad_model=MODEL_FACE):
    src_image = load_image(image_path)
    image = image_generation(src_image, device, prompt, model_path, seed, num_inference_steps, ad_model)
    return image

# 画像生成（PIL 画像イメージ入力）
def image_generation(src_image, device='cpu', prompt=def_prompt, model_path=def_model, seed=def_seed, num_inference_steps=def_step, ad_model=MODEL_FACE):
    print(f"{sdt.CYAN｝= ADetailer =  {sdt.NOCOLOR｝ prompt: '{prompt｝'  model: '{ad_model｝'")

    # パイプラインを作成
    if device == 'cpu':
        pipeline = AdPipeline.from_single_file(model_path)
    else:
        pipeline = AdPipeline.from_single_file(model_path, torch_dtype=torch.float16)

    pipeline.to(device)

    person_model_path = hf_hub_download(MODEL_DIR, ad_model)
    person_detector = partial(yolo_detector, model_path = person_model_path)

    common = {"prompt": prompt, "num_inference_steps": num_inference_steps｝

    # 画像を生成
    response = pipeline(common=common, detectors=[person_detector, pipeline.default_detector], images=[src_image])
    images = response[0]
    image = None if images == [] else images[0]

    return image


# ** main関数 **
def main(opt, logger):
    # パラメータ設定
    device = sdt._get_device(opt, logger)
    result_image_path = sdt._get_result_image_path(opt, logger)
    result_path = sdt._get_result_path(opt, logger)
    prompt = sdt._get_prompt(opt, logger) if opt.ext != 'boy' else def_prompt_m
    src_image = sdt._get_source_image(opt, logger)
    seed = sdt._get_seed_value(opt, logger)
    num_inference_steps = sdt._get_inference_steps(opt, logger)
    model_path = sdt._get_model_path(opt, logger)
    image_path = sdt._get_source_image_path(opt, logger)

    # 出力フォルダ
    os.makedirs(result_path, exist_ok = True)

    # 画像生成
    image = image_generation(src_image, device, prompt, model_path, seed, num_inference_steps)

    if image is None:
        logger.info(f'{sdt.RED｝There is no face in the image !!{sdt.NOCOLOR｝')

    else:
        s = os.path.splitext(result_image_path)
        s0 = os.path.splitext(os.path.basename(image_path))[0]
        save_path = s[0] + '-' + s0 + s[1]
        sdt.image_save2(image, save_path, save_path)
        logger.info(f'result_file: {save_path｝')

    return


# main関数エントリーポイント(実行開始)
if __name__ == "__main__":
    import my_dialog

    parser = sdt.parse_args(None, opt_list)
    opt = parser.parse_args()
    sdt._get_device(opt)

    if len(opt.image_path) == 0:
        opt.image_path = my_dialog.select_image_file(initdir = './images')
        if len(opt.image_path) == 0:
            exit(0)

    sdt.display_info(opt, title)

    # アプリケーション・ログ設定
    module = os.path.basename(__file__)
    module_name = os.path.splitext(module)[0]
    logger = my_logging.get_module_logger_sel(module_name, int(opt.log))

    main(opt, logger)

    logger.info('\nFinished.\n')

　※ 上記ソースコードは表示の都合上、半角コード '}' が全角 '｝'になっていることに注意

↑

Step 52：画像の一部を変換する「Paint-By-Example」 †

概要
・「diffusers」の別の機能「Paint-By-Example」を試してみる
・プロンプトなしで画像とマスクからマスク部分を別の画像に入れ替える
・簡単にマスクを作成できるプログラムを用意する → マスク作成プログラム「sd_mask.py」

プログラムを実行する（実行時間：約 5秒 RTX 4070 Ti 12GB）

 python sd_052.py

(sd_test) PS > python sd_052.py

Stable Diffusion with diffusers(052)  Ver 0.06: Starting application...

 --result_image             :   results/image_052.png
 --cpu                      :   False
 --log                      :   3
 --model_path               :   Fantasy-Studio/Paint-by-Example
 --image_path               :   images/heron.jpg
 --ctrl_image_path          :   images/kamo.jpg
 --max_size                 :   0
 --seed                     :   -1
 --width                    :   512
 --height                   :   512
 --step                     :   20

Loading pipeline components...:   0%|                                                            | 0/5 [00:00<?, ?it/s]An error occurred while trying to fetch C:\Users\izuts\.cache\huggingface\hub\models--Fantasy-Studio--Paint-by-Example\snapshots\351e6427d8c28a3b24f7c751d43eb4b6735127f7\vae: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\izuts\.cache\huggingface\hub\models--Fantasy-Studio--Paint-by-Example\snapshots\351e6427d8c28a3b24f7c751d43eb4b6735127f7\vae.
An error occurred while trying to fetch C:\Users\izuts\.cache\huggingface\hub\models--Fantasy-Studio--Paint-by-Example\snapshots\351e6427d8c28a3b24f7c751d43eb4b6735127f7\unet: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\izuts\.cache\huggingface\hub\models--Fantasy-Studio--Paint-by-Example\snapshots\351e6427d8c28a3b24f7c751d43eb4b6735127f7\unet.
Loading pipeline components...:  40%|████████████████████▊                               | 2/5 [00:00<00:00,  7.84it/s]You are using a model of type clip_vision_model to instantiate a model of type clip. This is not supported for all configurations of models and can yield errors.
Loading pipeline components...: 100%|█████████████| 5/5 [00:00<00:00,  5.47it/s]
100%|███████████████████████████████████████████| 20/20 [00:05<00:00,  3.76it/s]
result_file: results/image_052-388208840.png

Finished.

画像ファイル「image_052-XXXXXXXX.png」が生成される（XXXXXXXX はシード値）

生成画像例

元画像 ① ② ③ ④

マスク画像 ↓ ↓ ↓ ↓

モジュール・ソースコード

▼「sd_052.py」

# -*- coding: utf-8 -*-
##--------------------------------------------------
##  Stable Diffusion with diffusers(052)   Ver 0.06
##
##               2025.08.07 Masahiro Izutsu
##--------------------------------------------------
## sd_052.py    Paint-By-Example
##              === 画像の一部を変換する ===

##  Ver 0.00    2025.08.07  sd_052.py

# タイトル
title = 'Stable Diffusion with diffusers(052)  Ver 0.06'

import warnings
warnings.simplefilter('ignore')

# インポート＆初期設定
import os
import torch
from diffusers import DiffusionPipeline, logging

import my_logging
import sd_tools as sdt

logging.set_verbosity_error()

# 定数定義
DEF_MODEL_PATH = 'Fantasy-Studio/Paint-by-Example'
DEF_IMAGE_PATH = 'images/heron.jpg'
DEF_CTRL_IMAGE = 'images/kamo.jpg'

# コマンドライン定義
opt_list = [
            ['pros_sel','','sd_052'],                                                                       #  0
            ['result_image', 'results/image_052.png', 'path to output image file'],                         #  1
            ['cpu', 'store_true', 'cpu mode'],                                                              #  2
            ['log', '3', 'Log level(-1/0/1/2/3/4/5) Default value is \'3\''],                               #  3
            ['model_path', DEF_MODEL_PATH, 'Model Path'],                                                   #  4
            ['image_path', DEF_IMAGE_PATH, 'Sourcs image file path'],                                       #  5
            ['ctrl_image_path', DEF_CTRL_IMAGE, 'Control image file path'],                                 #  6
            ['max_size', 0, 'image max size (0=source)'],                                                   #  7
            ['seed', -1, 'Seed parameter (-1 = rundom)'],                                                   #  8
            ['width', 512, 'image size width'],                                                             #  9
            ['height', 512, 'image size height'],                                                           # 10
            ['step', 20, 'infer step'],                                                                     # 11
           ]

# 画像生成
def image_generation(model_path, src_image, msk_image, ex_image, seed, num_inference_steps=20, width=512, height=512, device='cpu'):

    # パイプラインを作成
    if device == 'cpu':
        pipeline = DiffusionPipeline.from_pretrained(model_path)
    else:
        pipeline = DiffusionPipeline.from_pretrained(model_path, type = torch.float16)

    pipeline.to(device)

    # Generatorオブジェクト作成
    generator = torch.Generator(device).manual_seed(seed)

    # 画像を生成
    image = pipeline(
                    image = src_image,
                    mask_image = msk_image,
                    example_image = ex_image,
                    num_inference_steps = num_inference_steps,
                    ).images[0]

    return image


# ** main関数 **
def main(opt, logger = None):
    # パラメータ設定
    device = sdt._get_device(opt, logger)
    result_image_path = sdt._get_result_image_path(opt, logger)
    result_path = sdt._get_result_path(opt, logger)
    model_path = sdt._get_model_path(opt, logger)
    max_size = sdt._get_max_size(opt, logger)
    height, width = sdt._get_image_size(opt, logger)
    seed = sdt._get_seed_value(opt, logger)
    num_inference_steps = sdt._get_inference_steps(opt, logger)
    image_path = sdt._get_source_image_path(opt, logger)
    work_path = sdt.get_work_path(logger)

    src_path, mask_path = sdt.get_source_mask_path(image_path, logger)
    if not os.path.isfile(mask_path):
        import sd_mask

        sd_mask.mask_paint(image_path, work_path, False, logger)
        if not os.path.isfile(mask_path):
            logger.info(f'{sdt.RED｝TThere is no mask image !!{sdt.NOCOLOR｝')
            return

    mask_image = sdt._get_resize_image(mask_path, max_size, logger)
    src_image = sdt._get_source_image(opt, logger)
    ex_image = sdt._get_control_image(opt, logger)
    if mask_image is None or src_image is None or ex_image is None:
        logger.info(f'{sdt.RED｝TThere is no image !!{sdt.NOCOLOR｝')
        return

    # 出力フォルダ
    os.makedirs(result_path, exist_ok = True)

    # 画像生成
    image = image_generation(model_path, src_image, mask_image, ex_image, seed, num_inference_steps, width, height, device)

    if image is None:
        logger.info(f'{sdt.RED｝Unable to generate image !!{sdt.NOCOLOR｝')

    else:
        s = os.path.splitext(result_image_path)
        save_path = s[0] + '-' + str(seed) + s[1]
        logger.info(f'result_file: {save_path｝')
        sdt.image_save2(image, save_path, save_path)

    return


# main関数エントリーポイント(実行開始)
if __name__ == "__main__":
    parser = sdt.parse_args(None, opt_list)
    opt = parser.parse_args()
    sdt._get_device(opt)
    sdt.display_info(opt, title)

    # アプリケーション・ログ設定
    module = os.path.basename(__file__)
    module_name = os.path.splitext(module)[0]
    logger = my_logging.get_module_logger_sel(module_name, int(opt.log))

    main(opt, logger)

    logger.info('\nFinished.\n')

　※ 上記ソースコードは表示の都合上、半角コード '}' が全角 '｝'になっていることに注意

↑

Step 53：コントロールネットの機能をまとめる †

　　canny / inpaint / outpaint / scribble / openpose / pix2pix / txt2img / img2img /
　　lineart / softedge / shuffle / depth / seg / normal / anime / mlsd / tile

概要
・これまでに検証してきた diffusers を利用した ControlNet のプログラムをまとめてみる

・ControlNetのモデル（Stable Diffusion 1.5用）

model	機能	対応済
lllyasviel/control_v11p_sd15_canny	Canny Edge検出	canny
lllyasviel/control_v11p_sd15_mlsd	直線検出	mlsd
lllyasviel/control_v11f1p_sd15_depth	深度推定	depth
lllyasviel/control_v11p_sd15_normalbae	表面法線推定	normal
lllyasviel/control_v11p_sd15_seg	画像セグメンテーション	seg
lllyasviel/control_v11p_sd15_lineart	線画生成	lineart
lllyasviel/control_v11p_sd15_openpose	人間の姿勢推定	openpose
lllyasviel/control_v11p_sd15_scribble	落書きベースの画像生成	scribble
lllyasviel/control_v11p_sd15_softedge	ソフトエッジ画像生成	softedge
lllyasviel/control_v11e_sd15_ip2p	ピクセルからピクセルへの命令	pix2pix
lllyasviel/control_v11p_sd15_inpaint	画像修復	inpaint
lllyasviel/control_v11e_sd15_shuffle	画像シャッフル	shuffle
lllyasviel/control_v11p_sd15s2_lineart_anime	アニメ線画生成	anime
lllyasviel/control_v11u_sd15_tile	画像拡大	tile

・ControlNetのパイプライン（Stable Diffusion 1.5用）

model	機能	対応済
StableDiffusionControlNetPipeline	画像生成	〇
StableDiffusionControlNetImg2ImgPipeline	Img2Img	〇
StableDiffusionControlNetInpaintPipeline	Inpainting	〇

・canny の例

プログラムを実行する（実行時間：約 2秒 RTX 4070 Ti 12GB）

 python sd_053.py --mode 'canny']

(sd_test) PS > python sd_053.py --mode 'canny'

Stable Diffusion with diffusers(053)  Ver 0.02: Starting application...

 --result_image             :   results/image_053.png
 --cpu                      :   False
 --log                      :   3
 --model_dir                :   /StabilityMatrix/Data/Models/StableDiffusion
 --model_path               :   SD1.5/beautifulRealistic_brav5.safetensors
 --ctrl_model_dir           :   /StabilityMatrix/Data/Models/ControlNet
 --ctrl_model_path          :   control_v11p_sd15_canny_fp16.safetensors
 --image_path               :   images/vermeer.png
 --max_size                 :   0
 --prompt                   :   微笑んでいる女性
 --seed                     :   12345678
 --width                    :   512
 --height                   :   512
 --step                     :   20
 --scale                    :   7.0
 --cc_scale                 :   1.0
 --strength                 :   0.6
 --neg_prompt               :   最悪の品質、おかしい人体構造
 --ip_scale                 :   0.5
 --scheduler                :   euler
 --mode                     :   canny

Fetching 11 files: 100%|████████████████████████████████| 11/11 [00:00<?, ?it/s]
Loading pipeline components...: 100%|█████████████| 6/6 [00:00<00:00,  9.42it/s]
100%|███████████████████████████████████████████| 20/20 [00:06<00:00,  3.16it/s]
result_file: results/image_053-vermeer-canny.png

Finished.

画像ファイル「image_053-XXXXXXXX-YYYY.png」が生成される（XXXXXXXX は入力ファイル名, YYYY は処理名）

その他の生成画像例

 python sd_053.py --mode ['canny/inpaint/scribble/lineart/softedge/depth/seg/shuffle/normal/anime/mlsd'] --prompt '微笑んでいる女性' --neg_prompt '最悪の品質、おかしい人体構造' --model_path 'SD1.5/beautifulRealistic_brav5.safetensors'

 python sd_053.py --mode 'tile' --image_path 'images/sd_040_test_64x64.png' --prompt '最高の品質' --neg_prompt '最悪の品質、おかしい人体構造' --model_path 'SD1.5/beautifulRealistic_brav5.safetensors'

元画像	共通設定パラメータ
	prompt	微笑んでいる女性
	prompt	Woman smiling
	neg_prompt	最悪の品質、おかしい人体構造
	neg_prompt	Worst quality, funny body structure
	model	SD1.5/beautifulRealistic_brav5.safetensors
	seed	12345678

canny	inpaint	scribble	lineart	softedge	depth


seg	shuffle	normal (map)	(lineart) anime	mlsd	tile

normal map 「法線マップ」とは
・形状表面の向きを変化させることで凹凸を表現する、一般的な3DCGの手法
・オブジェクトの法線ベクトルのX, Y, Z座標に対応したRGB画像
・法線マップでは、法線方向が左右軸を赤、上下軸を緑、垂直深度を青で表す

MLSD(Mobile LSD) の特性を生かした画像例
・MLSDは、線分検出アルゴリズムの一つで、直線のみを抽出するモデル
```
 python sd_053.py --mode 'mlsd' --image_path 'sample/20060804_190707_001.JPG' --prompt '都市の写真' --neg_prompt '最悪の品質'
```

これまでに検証した生成画像の再現例（デフォールト：パラメータなしで起動する）
```
 python sd_053.py --mode ['pix2pix/inpaint/outpaint/scribble/openpose/txt2img/img2img']
```
pix2pix inpaint outpaint scribble openpose txt2img img2img

Step 42 Step 44 Step 46 Step 47 Step 48 Step 81 Step 91

pix2pix	inpaint	outpaint	scribble	openpose	txt2img	img2img
Step 42	Step 44	Step 46	Step 47	Step 48	Step 81	Step 91

「Pipeline」クラス

機能	戻り値	関数
クラスの初期化	-	__init__(self, mode = , device = , model_dir = , ctrl_model_dir = , scheduler = '')
モード選択	-	select_mode(self, mode, device, model_dir, ctrl_model_dir, scheduler, base_model = '')
ベースモデル選択	-	select_base_model(self, base_model)
ControlNet モデル選択	-	select_ctrl_model(self, ctrl_model)
パイプラインを作成	pipeline	cleate_pipeline(self, logger = None)
IPアダプタを読み込み	pipeline	load_ipadapter(self, pipeline, ip_image = None, ip_scale = 0.5)
スケジューラー設定	pipeline	set_scheduler(self, pipeline)

関数メソッド (詳細はソースコード参照)

機能	戻り値	関数
入力画像の前処理	src_image, msk_image, img_ctrl	pre_generation(opt, logger = None)
画像生成	image	image_generation(pl, ext, model_path, ctrl_model_path, src_image, msk_image, img_ctrl, prompt, seed, num_inference_steps=20, width=512, height=512, guidance_scale=7.0, cc_scale=1.0, strength=0.6, neg_prompt = '', ip_image=None, ip_scale=0.5, device='cpu', logger= None)
パイプライン・オブジェクトの作成	pl	make_pipeline(opt)
初期設定	bf	set_initilal(opt)

モジュール・ソースコード

▼「sd_053.py」

# -*- coding: utf-8 -*-
##--------------------------------------------------
##  Stable Diffusion with diffusers(053)   Ver 0.05
##
##               2025.08.19 Masahiro Izutsu
##--------------------------------------------------
## sd_053_test.py    diffusers 統合版
##
##  Ver 0.00    2025.07.08  sd_053.py
##  Ver 0.01    2025.07.14  コマンドライン入力対応
##  Ver 0.02    2025.08.12  統合 sd_042(pix2pix), sd0_44(inpaint), sd_046(outpaint), sd_047(scribble),
##                          sd_048(openpose), sd_081(txt2img), sd_091(img2img), sd_canny(canny)
##  Ver 0.03    2025.08.14  sd_047 scribble 修正
##  Ver 0.04    2025.08.16  初期化処理 修正
##  Ver 0.05    2025.08.19  sd_101 統合版対応

# タイトル
title = 'Stable Diffusion with diffusers(053)  Ver 0.05'

import warnings
warnings.simplefilter('ignore')

# インポート＆初期設定
import os
import torch
import numpy as np
import cv2
from PIL import Image
from diffusers.utils import load_image
from diffusers import ControlNetModel, logging

import my_logging
import my_imagetool
import sd_tools as sdt

logging.set_verbosity_error()

# 定数定義
MODEL_DIR = '/StabilityMatrix/Data/Models/StableDiffusion'
CTRL_MODEEL_DIR = '/StabilityMatrix/Data/Models/ControlNet'
MODEL_BASE_BRAV5 = 'SD1.5/beautifulRealistic_brav5.safetensors'
MODEL_BASE_V15 = 'SD1.5/v1-5-pruned-emaonly.safetensors'
SCHEDULER_non = 'non'
SCHEDULER_euler = 'euler'
SCHEDULER_uni = 'uni'
SCHEDULER_DPM = 'DPM'
MODE_canny = 'canny'
MODE_inpaint = 'inpaint'
MODE_outpaint = 'outpaint'
MODE_scribble = 'scribble'
MODE_openpose = 'openpose'
MODE_pix2pix = 'pix2pix'
MODE_txt2img = 'txt2img'
MODE_img2img = 'img2img'
MODE_lineart = 'lineart'
MODE_softedge = 'softedge'
MODE_shuffle = 'shuffle'
MODE_depth = 'depth'
MODE_seg = 'seg'
MODE_normal = 'normal'
MODE_anime = 'anime'
MODE_mlsd = 'mlsd'
MODE_tile = 'tile'
ADETAILER_boy = 'boy'
ADETAILER_girl = 'girl'
DEF_IMAGE_canny = 'images/vermeer.png'
DEF_IMAGE_inpaint = 'images/sd_038_test.png'
DEF_IMAGE_outpaint = 'images/sd_046_test.png'
DEF_IMAGE_scribble = 'images/sd_047.png'
DEF_IMAGE_openpose = 'images/sd_048_test1.png'
DEF_IMAGE_pix2pix = 'images/sd_040_test.png'
DEF_IMAGE_img2img = 'images/StableDiffusion_247.png'
DEF_IMAGE_lineart = 'images/sd_040_test.png'
DEF_IMAGE_softedge = 'images/sd_040_test.png'
DEF_IMAGE_shuffle = 'images/sd_040_test.png'
DEF_IMAGE_depth = 'images/sd_040_test.png'
DEF_IMAGE_seg = 'images/sd_040_test.png'
DEF_IMAGE_normal = 'images/sd_040_test.png'
DEF_IMAGE_anime = 'images/sd_040_test.png'
DEF_IMAGE_mlsd = 'images/sd_040_test.png'
DEF_IMAGE_tile = 'images/sd_040_test_64x64.png'
DEF_PROMPT_canny = '微笑んでいる女性'
DEF_PROMPT_inpaint = '微笑んでいる女性'
DEF_PROMPT_outpaint = '庭に立って微笑んでいる女性'
DEF_PROMPT_scribble = 'テーブル上の白いコーヒーカップ'
DEF_PROMPT_openpose = 'ダンスを踊る女性'
DEF_PROMPT_pix2pix = '浜辺の場面にする'
DEF_PROMPT_txt2img = '満開の蘭'
DEF_PROMPT_img2img = '黒髪で短い髪の女性'
DEF_PROMPT_lineart = '微笑んでいる女性'
DEF_PROMPT_softedge = '微笑んでいる女性'
DEF_PROMPT_shuffle = '微笑んでいる女性'
DEF_PROMPT_depth = '微笑んでいる女性'
DEF_PROMPT_seg = '微笑んでいる女性'
DEF_PROMPT_normal = '微笑んでいる女性'
DEF_PROMPT_anime = '微笑んでいる女性'
DEF_PROMPT_mlsd = '微笑んでいる女性'
DEF_PROMPT_tile = '微笑んでいる女性'
DEF_NPROMPT_canny = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_inpaint = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_outpaint = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_scribble = ''
DEF_NPROMPT_openpose = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_pix2pix = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_txt2img = ''
DEF_NPROMPT_img2img = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_lineart = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_softedge = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_shuffle = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_depth = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_seg = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_normal = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_anime = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_mlsd = '最悪の品質、おかしい人体構造'
DEF_NPROMPT_tile = '最悪の品質、おかしい人体構造'

mode_list = [MODE_canny, MODE_scribble, MODE_lineart, MODE_softedge, MODE_anime, MODE_mlsd, MODE_depth, MODE_seg, MODE_normal, MODE_shuffle, MODE_pix2pix, MODE_inpaint, MODE_outpaint, MODE_openpose, MODE_txt2img, MODE_img2img, MODE_tile]

scheduler_list = [SCHEDULER_non, SCHEDULER_euler, SCHEDULER_uni, SCHEDULER_DPM]

adetailer_list = ['', ADETAILER_girl, ADETAILER_boy]

# コマンドライン定義
opt_list = [
            ['pros_sel','','sd_053'],                                                                       #  0
            ['result_image', 'results/image_053.png', 'path to output image file'],                         #  1
            ['cpu', 'store_true', 'cpu mode'],                                                              #  2
            ['log', '3', 'Log level(-1/0/1/2/3/4/5) Default value is \'3\''],                               #  3
            ['model_dir', '', 'Model directory'],                                                           #  4
            ['model_path', '', 'Model Path'],                                                               #  5
            ['ctrl_model_dir', '', 'ControlNet Model directory'],                                           #  6
            ['ctrl_model_path', '', 'ControlNet Model Path'],                                               #  7
            ['image_path', '', 'Sourcs image file path'],                                                   #  8
            ['ctrl_image_path', '', 'Control image file path'],                                             #  9
            ['max_size', 0, 'image max size (0=source)'],                                                   # 10
            ['prompt', '', 'Prompt text'],                                                                  # 11
            ['seed', 12345678, 'Seed parameter (-1 = rundom)'],                                             # 12
            ['width', 512, 'image size width'],                                                             # 13
            ['height', 512, 'image size height'],                                                           # 14
            ['step', 20, 'infer step'],                                                                     # 15
            ['scale', 7.0, 'gaidanse scale'],                                                               # 16
            ['cc_scale', 1.0, 'controlnet conditioning scale'],                                             # 17
            ['strength', 0.6, 'strength value'],                                                            # 18
            ['neg_prompt', '', 'Negative Prompt text'],                                                     # 19
            ['ip_image_path', '', 'IP-Adapter image filr path'],                                            # 20
            ['ip_scale', 0.5, 'IP-Adapter scale'],                                                          # 21
            ['scheduler', '', "Scheduler 'non/euler/uni/DPM'"],                                             # 22
            ['ext', '', "Extensions (ADetailer) '' or 'girl' or 'boy'"],                                    # 23
            ['mode', '', "aplication mode 'canny/inpaint/outpaint/scribble/openpose/pix2pix/txt2img/img2img/lineart/softedge/shuffle/depth'"],
           ]

class Pipeline:
    # クラス内 定数定義
    MODEL_canny = 'control_v11p_sd15_canny_fp16.safetensors'
    MODEL_inpaint = 'control_v11p_sd15_inpaint_fp16.safetensors'
    MODEL_scribble = 'control_v11p_sd15_scribble_fp16.safetensors'
    MODEL_openpose = 'control_v11p_sd15_openpose_fp16.safetensors'
    MODEL_pix2pix = 'control_v11e_sd15_ip2p_fp16.safetensors'
    MODEL_lineart = 'control_v11p_sd15_lineart_fp16.safetensors'
    MODEL_softedge = 'control_v11p_sd15_softedge_fp16.safetensors'
    MODEL_shuffle = 'control_v11e_sd15_shuffle_fp16.safetensors'
    MODEL_depth = 'control_v11f1p_sd15_depth_fp16.safetensors'
    MODEL_seg = 'control_v11p_sd15_seg_fp16.safetensors'
    MODEL_normal = 'control_v11p_sd15_normalbae_fp16.safetensors'
    MODEL_anime = 'control_v11p_sd15s2_lineart_anime_fp16.safetensors'
    MODEL_mlsd = 'control_v11p_sd15_mlsd_fp16.safetensors'
    MODEL_tile = 'control_v11u_sd15_tile_fp16.safetensors'
    IP_CHECKPOINT_DIR = 'h94/IP-Adapter'
    IP_CHECKPOINT_SD15 = 'ip-adapter_sd15.bin'

    # クラスの初期化
    def __init__(self, mode = '', device = '', model_dir = '', ctrl_model_dir = '', scheduler = ''):
        self.mode = MODE_canny if mode == '' else mode
        self.device = 'cpu' if device == '' else device
        self.model_dir = MODEL_DIR if model_dir == '' else model_dir
        self.ctrl_model_dir = CTRL_MODEEL_DIR if ctrl_model_dir == '' else ctrl_model_dir
        self.scheduler = SCHEDULER_non if scheduler == '' else scheduler
        self.select_mode(self.mode, self.device, self.model_dir, self.ctrl_model_dir, self.scheduler)

    def select_mode(self, mode, device, model_dir, ctrl_model_dir, scheduler, base_model = ''):
        self.mode = mode
        self.device = device
        self.model_dir = model_dir
        self.ctrl_model_dir = ctrl_model_dir
        self.scheduler = scheduler

        if mode == MODE_canny:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_canny
        elif mode == MODE_inpaint or self.mode == MODE_outpaint:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_inpaint
        elif mode == MODE_scribble:
            b_model = MODEL_BASE_V15
            c_model = self.MODEL_scribble
        elif mode == MODE_openpose:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_openpose
        elif mode == MODE_pix2pix:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_pix2pix
        elif mode == MODE_txt2img:
            b_model = MODEL_BASE_V15
            c_model = ''
            self.ctrl_model_dir = ''
        elif mode == MODE_img2img:
            b_model = MODEL_BASE_BRAV5
            c_model = ''
            self.ctrl_model_dir = ''
        elif mode == MODE_lineart:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_lineart
        elif mode == MODE_softedge:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_softedge
        elif mode == MODE_shuffle:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_shuffle
        elif mode == MODE_depth:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_depth
        elif mode == MODE_seg:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_seg
        elif mode == MODE_normal:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_normal
        elif mode == MODE_anime:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_anime
        elif mode == MODE_mlsd:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_mlsd
        elif mode == MODE_tile:
            b_model = MODEL_BASE_BRAV5
            c_model = self.MODEL_tile
        else:
            b_model = MODEL_BASE_V15
            c_model = ''
            self.ctrl_model_dir = ''

        if base_model != '':
            b_model = base_model

        self.select_base_model(b_model)
        self.select_ctrl_model(c_model)


    def select_base_model(self, base_model):
        self.base_model = base_model
        self.model_path = base_model if self.model_dir == '' else self.model_dir + '/' + base_model

    def select_ctrl_model(self, ctrl_model):
        if ctrl_model == '':
            self.ctrl_model = ''
            self.ctrl_model_path = ''
        else:
            self.ctrl_model = ctrl_model
            self.ctrl_model_path = ctrl_model if self.ctrl_model_dir == '' else self.ctrl_model_dir + '/' + ctrl_model

    # パイプラインを作成
    def cleate_pipeline(self, logger = None):
        if self.mode == MODE_canny:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_inpaint or self.mode == MODE_outpaint:
            from diffusers import StableDiffusionControlNetInpaintPipeline as StableDiffusionPipeline
        elif self.mode == MODE_scribble:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_openpose:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_pix2pix:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_txt2img:
            from diffusers import StableDiffusionPipeline
            controlnet = None
        elif self.mode == MODE_img2img:
            from diffusers import StableDiffusionImg2ImgPipeline as StableDiffusionPipeline
            controlnet = None
        elif self.mode == MODE_lineart:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_softedge:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_shuffle:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_depth:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_seg:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_normal:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_anime:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_mlsd:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline
        elif self.mode == MODE_tile:
            from diffusers import StableDiffusionControlNetPipeline as StableDiffusionPipeline

        if self.device == 'cpu':
            if self.ctrl_model_path != '':  controlnet = ControlNetModel.from_single_file(self.ctrl_model_path)
            pipeline = StableDiffusionPipeline.from_single_file(self.model_path, controlnet=controlnet)
        else:
            if self.ctrl_model_path != '':  controlnet = ControlNetModel.from_single_file(self.ctrl_model_path, torch_dtype=torch.float16)
            pipeline = StableDiffusionPipeline.from_single_file(
                        self.model_path,
                        controlnet = controlnet,
                        torch_dtype = torch.float16,
            )

        sdt.log_debug(f' ** model_path: {self.model_path｝', logger)
        sdt.log_debug(f' ** select_ctrl_model: {self.ctrl_model_path｝', logger)
        return pipeline


    # IPアダプタを読み込み
    def load_ipadapter(self, pipeline, ip_image = None, ip_scale = 0.5):
        if ip_image != None:
            pipeline.load_ip_adapter(self.IP_CHECKPOINT_DIR, subfolder = 'models', weight_name = self.IP_CHECKPOINT_SD15)
            pipeline.set_ip_adapter_scale(ip_scale)

        return pipeline

    # スケジューラー
    def set_scheduler(self, pipeline):
        if self.scheduler == SCHEDULER_uni:
            from diffusers import UniPCMultistepScheduler
            pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)

        elif self.scheduler == SCHEDULER_euler:
            from diffusers import EulerAncestralDiscreteScheduler
            pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config(pipeline.scheduler.config)

        elif self.scheduler == SCHEDULER_DPM:
            from diffusers import DPMSolverMultistepScheduler
            pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)

        print(f' ** scheduler: {self.scheduler｝')                               # logger出力不可
        return pipeline

# 入力画像の前処理
def pre_generation(opt, logger = None):
    device = sdt._get_device(opt, logger)
    image_path = sdt._get_source_image_path(opt, logger)
    work_path = sdt.get_work_path(logger)
    os.makedirs(work_path, exist_ok = True)                                     # 作業フォルダ作成

    ip_image_path = os.path.basename(opt.ip_image_path)
    if ip_image_path != '':
        sdt.log_info(f' ** IP Adapter: {ip_image_path｝', logger)

    src_image = None
    msk_image = None
    img_ctrl = None

    if opt.mode == MODE_txt2img:                                                # == txt2img ==
        return src_image, msk_image, img_ctrl

    image = sdt._get_source_image(opt, logger)

    if opt.mode == MODE_canny:                                                  # == canny ==
        path = sdt.get_canny_path(image_path, logger)
        if os.path.isfile(path):
            src_image = Image.open(path)
        else:
            src_image = sdt.canny_preprocessor(image, 100, 200)
            src_image.save(path)
        sdt.image_disp(path, path, wait_s = 1)

    elif opt.mode == MODE_inpaint:                                              # == inpaint ==
        _, mask_path = sdt.get_source_mask_path(image_path, logger)
        if not os.path.isfile(mask_path):
            import sd_mask
            sd_mask.mask_paint(image_path, work_path, False, logger)

        if not os.path.isfile(mask_path):
            return None, None, None

        opt.ctrl_image_path = mask_path
        msk_image = sdt._get_control_image(opt, logger)
        src_image = sdt._get_source_image(opt, logger)
        img_ctrl = sdt.make_inpaint_condition(src_image, msk_image)
        sdt.image_disp(mask_path, mask_path, wait_s = 1)

    elif opt.mode == MODE_outpaint:                                             # == outpaint ==
        src_path, mask_path = sdt.get_source_mask_path(image_path, logger)
        size = 512
        img = cv2.imread(image_path)
        msk = sdt.mask_square(img, size)
        msk = my_imagetool.frame_resize(msk, size)
        my_imagetool.image_disp(msk,  mask_path, True, mask_path, wait_s = 1)       # マスク画像保存
        img = my_imagetool.frame_square(img, (0, 0, 0))
        img = my_imagetool.frame_resize(img, size)
        my_imagetool.image_disp(img, src_path, True, src_path, wait_s = 1)          # ソース画像保存

        opt.strength = None                                                         # strength 使用しない
        opt.image_path = src_path
        opt.ctrl_image_path = mask_path
        src_image = sdt._get_source_image(opt, logger)
        msk_image = sdt._get_control_image(opt, logger)
        img_ctrl = sdt.make_inpaint_condition(src_image, msk_image)

    elif opt.mode == MODE_scribble:                                             # == scribble ==
        if sdt.get_image_channel(image_path) == 3:
            img = cv2.imread(image_path)
            msk = sdt.scribble_preprocessor(img)
            mask_path = sdt.get_scribble_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_openpose:                                             # == openpose ==
        pose_path = sdt.get_pose_path(image_path, logger)
        if not os.path.isfile(pose_path):
            from controlnet_aux import OpenposeDetector
            src_image = sdt._get_resize_image(image_path, 0, None)
            openpose_detector = OpenposeDetector.from_pretrained('lllyasviel/ControlNet')
            openpose_image = openpose_detector(src_image)
            openpose_image.save(pose_path)

        opt.image_path = pose_path
        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_pix2pix:                                              # == pix2pix ==
        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None

    elif opt.mode == MODE_img2img:                                              # == img2img ==
        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None

    elif opt.mode == MODE_lineart:                                             # == lineart ==
        if sdt.get_image_channel(image_path) == 3:
            img = cv2.imread(image_path)
            msk = sdt.lineart_preprocessor(img)
            mask_path = sdt.get_lineart_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_softedge:                                             # == softedge ==
        if sdt.get_image_channel(image_path) == 3:
            img = cv2.imread(image_path)
            msk = sdt.softedge_preprocessor(img)
            mask_path = sdt.get_softedge_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_shuffle:                                              # == shuffle ==
        if sdt.get_image_channel(image_path) == 3:
            img = cv2.imread(image_path)
            msk = sdt.shuffle_preprocessor(img)
            mask_path = sdt.get_shuffle_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_depth:                                                # == depth ==
        if sdt.get_image_channel(image_path) == 3:
            img = Image.open(image_path)
            msk = sdt.depth_preprocessor(img)
            mask_path = sdt.get_depth_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_seg:                                                  # == seg ==
        img = Image.open(image_path)
        msk = sdt.seg_preprocessor(img)
        mask_path = sdt.get_seg_path(image_path, logger)
        sdt.image_save2(msk, mask_path)
        opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_normal:                                               # == normal ==
        img = Image.open(image_path)
        msk = sdt.normal_preprocessor(img)
        mask_path = sdt.get_normal_path(image_path, logger)
        sdt.image_save2(msk, mask_path)
        opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_anime:                                                # == anime ==
        if sdt.get_image_channel(image_path) == 3:
            img = Image.open(image_path)
            msk = sdt.anime_preprocessor(img)
            mask_path = sdt.get_anime_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_mlsd:                                                 # == mlsd ==
        if sdt.get_image_channel(image_path) == 3:
            img = Image.open(image_path)
            msk = sdt.mlsd_preprocessor(img)
            mask_path = sdt.get_mlsd_path(image_path, logger)
            sdt.image_save2(msk, mask_path)
            opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    elif opt.mode == MODE_tile:                                                 # == tile ==
        img = Image.open(image_path)
        msk = sdt.tile_preprocessor(img, min(opt.width, opt.height))
        mask_path = sdt.get_tile_path(image_path, logger)
        sdt.image_save2(msk, mask_path)
        opt.image_path = mask_path

        src_image = sdt._get_source_image(opt, logger)
        msk_image = None
        img_ctrl = None
        sdt.image_save2(src_image, dispname = image_path, wait_s = 1)

    return src_image, msk_image, img_ctrl

# 画像生成
def image_generation(pl, ext, model_path, ctrl_model_path, src_image, msk_image, img_ctrl, prompt, seed, num_inference_steps=20, width=512, height=512, guidance_scale=7.0, cc_scale=1.0, strength=0.6, neg_prompt = '', ip_image=None, ip_scale=0.5, device='cpu', logger= None):
    if neg_prompt == '' or neg_prompt == ' ' or neg_prompt == '　':    neg_prompt = None
    if strength == '' or strength == ' ' or strength == '　':    strength = None

    if src_image is not None:
        w, h = src_image.size
        if w != width or h != height:
            src_image = src_image.resize((width, height), resample=Image.BICUBIC)

    # パイプラインを作成
    pipeline = pl.cleate_pipeline(logger)

    # IPアダプタを読み込み
    pipeline = pl.load_ipadapter(pipeline, ip_image, ip_scale)

    # スケジューラー
    pipeline = pl.set_scheduler(pipeline)

    if device == 'cuda':
        pipeline.enable_model_cpu_offload()

    # Generatorオブジェクト作成
    generator = torch.Generator(device).manual_seed(seed)

    # 画像を生成
    if strength == None:
        image = pipeline(
                    prompt = prompt,
                    negative_prompt = neg_prompt,
                    image = src_image,
                    mask_image = msk_image,
                    control_image = img_ctrl,
                    num_inference_steps = num_inference_steps,
                    width = width,
                    height = height,
                    controlnet_conditioning_scale = cc_scale,
                    ip_adapter_image=ip_image,
                    generator = generator
                    ).images[0]
    else:
        image = pipeline(
                    prompt = prompt,
                    negative_prompt = neg_prompt,
                    image = src_image,
                    mask_image = msk_image,
                    control_image = img_ctrl,
                    num_inference_steps = num_inference_steps,
                    width = width,
                    height = height,
                    controlnet_conditioning_scale = cc_scale,
                    ip_adapter_image=ip_image,
                    strength = strength,
                    generator = generator
                    ).images[0]

    # Adetailer
    if image is not None and (ext == ADETAILER_boy or ext == ADETAILER_girl):
        import sd_051
        prompt = sd_051.def_prompt_m if ext == 'boy' else sd_051.def_prompt
        image = sd_051.image_generation(image, device = device, prompt = prompt, model_path = model_path)
        sdt.log_debug(f' ** (Adetailer) prompt: {prompt｝', logger)

    if src_image is not None:
        sdt.log_debug(f' ** w = {w｝, h = {h｝, width = {width｝, height = {height｝', logger)
        if w != width or h != height:
            image = image.resize((w, h), resample=Image.BICUBIC)

    return image

# パイプライン・オブジェクトの作成
def make_pipeline(opt):
    bf = set_initilal(opt)

    # パイプライン・オブジェクト
    if bf:
        device = sdt._get_device(opt)
        pl = Pipeline(mode = opt.mode, device = device, model_dir = opt.model_dir, ctrl_model_dir = opt.ctrl_model_dir, scheduler = opt.scheduler)

        opt.scheduler = pl.scheduler
        opt.model_dir = pl.model_dir
        opt.ctrl_model_dir = pl.ctrl_model_dir

        if opt.model_path == '':
            opt.model_path = pl.base_model
        else:
            pl.select_base_model(opt.model_path)

        if opt.ctrl_model_path == '':
            opt.ctrl_model_path = pl.ctrl_model
        else:
            pi.select_ctrl_model(opt.ctrl_model_path)

    else:
        pl = None

    return pl

# 初期設定
def set_initilal(opt):
    bf = True
    if opt.scheduler =='':
        if opt.mode == MODE_canny:
            opt.scheduler = SCHEDULER_euler
        elif opt.mode == MODE_inpaint:
            opt.scheduler = SCHEDULER_euler
        elif opt.mode == MODE_outpaint:
            opt.scheduler = SCHEDULER_euler
        elif opt.mode == MODE_scribble:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_openpose:
            opt.scheduler = SCHEDULER_non
            if int(opt.step) < 50:   opt.step = 50
        elif opt.mode == MODE_pix2pix:
            opt.scheduler = SCHEDULER_euler
        elif opt.mode == MODE_txt2img:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_img2img:
            opt.scheduler = SCHEDULER_DPM
        elif opt.mode == MODE_lineart:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_softedge:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_shuffle:
            opt.scheduler = SCHEDULER_non
            if int(opt.step) < 50:   opt.step = 50
        elif opt.mode == MODE_depth:
            opt.scheduler = SCHEDULER_non
            if int(opt.step) < 50:   opt.step = 50
        elif opt.mode == MODE_seg:
            opt.scheduler = SCHEDULER_uni
        elif opt.mode == MODE_normal:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_anime:
            opt.scheduler = SCHEDULER_non
        elif opt.mode == MODE_mlsd:
            opt.scheduler = SCHEDULER_non
            if int(opt.step) < 50:   opt.step = 50
        elif opt.mode == MODE_tile:
            opt.scheduler = SCHEDULER_non
        else:
            bf = False

    if opt.image_path =='':
        if opt.mode == MODE_canny:
            opt.image_path = DEF_IMAGE_canny
        elif opt.mode == MODE_inpaint:
            opt.image_path = DEF_IMAGE_inpaint
        elif opt.mode == MODE_outpaint:
            opt.image_path = DEF_IMAGE_outpaint
        elif opt.mode == MODE_scribble:
            opt.image_path = DEF_IMAGE_scribble
        elif opt.mode == MODE_openpose:
            opt.image_path = DEF_IMAGE_openpose
        elif opt.mode == MODE_pix2pix:
            opt.image_path = DEF_IMAGE_pix2pix
        elif opt.mode == MODE_txt2img:
            opt.image_path = ''
        elif opt.mode == MODE_img2img:
            opt.image_path = DEF_IMAGE_img2img
        elif opt.mode == MODE_lineart:
            opt.image_path = DEF_IMAGE_lineart
        elif opt.mode == MODE_softedge:
            opt.image_path = DEF_IMAGE_softedge
        elif opt.mode == MODE_shuffle:
            opt.image_path = DEF_IMAGE_shuffle
        elif opt.mode == MODE_depth:
            opt.image_path = DEF_IMAGE_depth
        elif opt.mode == MODE_seg:
            opt.image_path = DEF_IMAGE_seg
        elif opt.mode == MODE_normal:
            opt.image_path = DEF_IMAGE_normal
        elif opt.mode == MODE_anime:
            opt.image_path = DEF_IMAGE_anime
        elif opt.mode == MODE_mlsd:
            opt.image_path = DEF_IMAGE_mlsd
        elif opt.mode == MODE_tile:
            opt.image_path = DEF_IMAGE_tile
        else:
            bf = False

    if opt.prompt =='':
        if opt.mode == MODE_canny:
            opt.prompt = DEF_PROMPT_canny
        elif opt.mode == MODE_inpaint:
            opt.prompt = DEF_PROMPT_inpaint
        elif opt.mode == MODE_outpaint:
            opt.prompt = DEF_PROMPT_outpaint
        elif opt.mode == MODE_scribble:
            opt.prompt = DEF_PROMPT_scribble
        elif opt.mode == MODE_openpose:
            opt.prompt = DEF_PROMPT_openpose
        elif opt.mode == MODE_pix2pix:
            opt.prompt = DEF_PROMPT_pix2pix
        elif opt.mode == MODE_txt2img:
            opt.prompt = DEF_PROMPT_txt2img
        elif opt.mode == MODE_img2img:
            opt.prompt = DEF_PROMPT_img2img
        elif opt.mode == MODE_lineart:
            opt.prompt = DEF_PROMPT_lineart
        elif opt.mode == MODE_softedge:
            opt.prompt = DEF_PROMPT_softedge
        elif opt.mode == MODE_shuffle:
            opt.prompt = DEF_PROMPT_shuffle
        elif opt.mode == MODE_depth:
            opt.prompt = DEF_PROMPT_depth
        elif opt.mode == MODE_seg:
            opt.prompt = DEF_PROMPT_seg
        elif opt.mode == MODE_normal:
            opt.prompt = DEF_PROMPT_normal
        elif opt.mode == MODE_anime:
            opt.prompt = DEF_PROMPT_anime
        elif opt.mode == MODE_mlsd:
            opt.prompt = DEF_PROMPT_mlsd
        elif opt.mode == MODE_tile:
            opt.prompt = DEF_PROMPT_tile
        else:
            bf = False

    if opt.neg_prompt =='':
        if opt.mode == MODE_canny:
            opt.neg_prompt = DEF_NPROMPT_canny
        elif opt.mode == MODE_inpaint:
            opt.neg_prompt = DEF_NPROMPT_inpaint
        elif opt.mode == MODE_outpaint:
            opt.neg_prompt = DEF_NPROMPT_outpaint
        elif opt.mode == MODE_scribble:
            opt.neg_prompt = DEF_NPROMPT_scribble
        elif opt.mode == MODE_openpose:
            opt.neg_prompt = DEF_NPROMPT_openpose
        elif opt.mode == MODE_pix2pix:
            opt.neg_prompt = DEF_NPROMPT_pix2pix
        elif opt.mode == MODE_txt2img:
            opt.neg_prompt = DEF_NPROMPT_txt2img
        elif opt.mode == MODE_img2img:
            opt.neg_prompt = DEF_NPROMPT_img2img
        elif opt.mode == MODE_lineart:
            opt.neg_prompt = DEF_NPROMPT_lineart
        elif opt.mode == MODE_softedge:
            opt.neg_prompt = DEF_NPROMPT_softedge
        elif opt.mode == MODE_depth:
            opt.neg_prompt = DEF_NPROMPT_depth
        elif opt.mode == MODE_seg:
            opt.neg_prompt = DEF_NPROMPT_seg
        elif opt.mode == MODE_normal:
            opt.neg_prompt = DEF_NPROMPT_normal
        elif opt.mode == MODE_anime:
            opt.neg_prompt = DEF_NPROMPT_anime
        elif opt.mode == MODE_mlsd:
            opt.neg_prompt = DEF_NPROMPT_mlsd
        elif opt.mode == MODE_tile:
            opt.neg_prompt = DEF_NPROMPT_tile
        else:
            bf = False

    return bf


# ** main関数 **
def main(opt, pl, logger = None):
    # パラメータ設定
    device = sdt._get_device(opt, logger)
    result_image_path = sdt._get_result_image_path(opt, logger)
    result_path = sdt._get_result_path(opt, logger)
    prompt = sdt._get_prompt(opt, logger)
    image_path = sdt._get_source_image_path(opt, logger)
    model_path = sdt._get_model_path(opt, logger)
    ctrl_model_path = sdt._get_controlnet_model_path(opt, logger)
    height, width = sdt._get_image_size(opt, logger)
    seed = sdt._get_seed_value(opt, logger)
    num_inference_steps = sdt._get_inference_steps(opt, logger)
    guidance_scale = sdt._get_guidance_scale(opt, logger)
    cc_scale = sdt._get_controlnet_conditioning_scale(opt, logger)
    neg_prompt = sdt._get_negative_prompt(opt, logger)

    ip_image = sdt._get_ip_image(opt, logger)
    ip_scale = sdt._get_ip_scale(opt, logger)

    # 入力画像の前処理
    src_image, msk_image, img_ctrl = pre_generation(opt, logger)
    if opt.mode != MODE_txt2img and src_image is None:
        logger.info(f'{sdt.RED｝Processing will be stopped !!{sdt.NOCOLOR｝')
        return

    strength = sdt._get_strength(opt, logger)

    # 出力フォルダ
    os.makedirs(result_path, exist_ok = True)

    # 画像生成
    image = image_generation(pl, opt.ext, model_path, ctrl_model_path, src_image, msk_image, img_ctrl, prompt, seed, num_inference_steps, width, height, guidance_scale, cc_scale, strength, neg_prompt, ip_image, ip_scale, device)

    if image is None:
        logger.info(f'{sdt.RED｝There is no face in the image !!{sdt.NOCOLOR｝')

    else:
        s = os.path.splitext(result_image_path)
        s0 = os.path.splitext(os.path.basename(image_path))[0]
        s1 = '' if ip_image is None else '-' + os.path.splitext(os.path.basename(opt.ip_image_path))[0]
        save_path = s[0] + '-' + s0 + '-' + opt.mode + s1 + s[1]
        sdt.image_save2(image, save_path, save_path)
        logger.info(f'result_file: {save_path｝')

    return


# main関数エントリーポイント(実行開始)
if __name__ == "__main__":
    parser = sdt.parse_args(None, opt_list)
    opt = parser.parse_args()

    # アプリケーション・ログ設定
    module = os.path.basename(__file__)
    module_name = os.path.splitext(module)[0]
    logger = my_logging.get_module_logger_sel(module_name, int(opt.log))

    pl = make_pipeline(opt)
    if pl is None:
        logger.info(f'{sdt.RED｝Invalid mode name !!{sdt.NOCOLOR｝')
        exit(0)

    sdt.display_info(opt, title)

    main(opt, pl, logger)

    logger.info('\nFinished.\n')

　※ 上記ソースコードは表示の都合上、半角コード '}' が全角 '｝'になっていることに注意

↑

忘備録 †

↑

更新履歴 †

2025/07/26 初版

↑

参考資料 †

Image-to-Image/ControlNet/IP-Adapter

Diffusers

Inpainting
- GitHub: Inpainting
- ドキュメント版 Inpainting に沿って試してみる

face-recognition

Paint-By-Example

Programming

書籍など
- 日経ソフトウエア 2025年7月号「ローカル生成AIプログラミング」
- Interface 2025年3月号「画像による異常検出＆ローカルLLM作り - 仕事のための生成AI」

周辺の補正前	自動で作成したマスク	マスクで補正した完成画像

元画像	顔の抽出	顔の修正	完成画像

元画像	顔の抽出	顔の修正	完成画像

元画像	①	②	③	④

マスク画像	↓	↓	↓	↓