FaceRec1a

顔認証/顔認識パッケージ「Face recognition」のテスト †

　パッケージ(dlib/face_recognition) のサイトにある Pythonコードのサンプルを動かし実用的な「顔認証/顔認識 (Face recognition)」プログラム開発の方法を検討する。

顔認証/顔認識パッケージ「Face recognition」のテスト
- Pythonコードのサンプル
- 更新履歴
参考資料

※ 最終更新:2022/07/20　

Pythonコードのサンプル †

　face_recognition サイトからダウンロードしコードを修正する。
　オリジナル・ディレクトリ「workspace_py37/face_recognition-master」とする。
　プロジェクト・ディレクトリ「workspace_py37/face_rec_try」を実行テストのディレクトリとする。

↑

プロジェクトの準備 †

アップデートファイルを準備する
　update_20220719.zip (97.6MB) <Windows face_rec 追加プロジェクト>
　update_20220719.tar.gz (97.5MB) <Linux face_rec 追加プロジェクト>
　※ 圧縮方法の違いだけで同じ内容のファイル

解凍してできたディレクトリを「workpace_py37」ディレクトリ内に配置する。

/workspace_py37
    ├── face_recognition-master　　　　　← オリジナル
    ├── face_rec_try 　　　　　　　　　　← オリジナルのサンプルコードをコピー/修正
    ├── face_rec 　　　　　　　　　　　　← 新規開発プロジェクト
    :

新しく仮想環境を作成する場合は前項顔認証/顔認識 (Face recognition)参照。
以下の記述で Windows環境下で使用の場合は「python3 → python」に読み替えること。
※ PowerShell ターミナルでエイリアス設定をしてもよい。
```
(base) PS > Set-Alias -Name python3 python
```

仮想環境（py37y）を有効にする

(base) PS > conda activate py37y
(py37y) PS > cd /anaconda_win/workspace_py37/face_rec

↑

[顔検出] 画像から顔を見つける「find_faces_in_picture.py」 †

入力データの準備

「3872x2592」(face_rec_try_full.jpg),「1280x857」(face_rec_try.jpg) 2種類の解像度を用意する。

プログラムソースの修正
入力ファイルパスを修正

from PIL import Image
import face_recognition

# Load the jpg file into a numpy array
image = face_recognition.load_image_file("face_rec_try_full.jpg")

# Find all the faces in the image using the default HOG-based model.
# This method is fairly accurate, but not as accurate as the CNN model and not GPU accelerated.
# See also: find_faces_in_picture_cnn.py
face_locations = face_recognition.face_locations(image)

print("I found {} face(s) in this photograph.".format(len(face_locations)))

for face_location in face_locations:

    # Print the location of each face in this image
    top, right, bottom, left = face_location
    print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))

    # You can access the actual face itself like this:
    face_image = image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.show()

プログラムの実行
・解像度「3872x2592」

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python find_faces_in_picture.py 
I found 5 face(s) in this photograph.
A face is located at pixel location Top: 1427, Left: 810, Bottom: 1889, Right: 1272
A face is located at pixel location Top: 590, Left: 1146, Bottom: 975, Right: 1531
A face is located at pixel location Top: 1488, Left: 2044, Bottom: 1874, Right: 2429
A face is located at pixel location Top: 718, Left: 1830, Bottom: 1103, Right: 2215
A face is located at pixel location Top: 1282, Left: 3510, Bottom: 1505, Right: 3733

・解像度「1280x857」コピーして入力画像を修正 find_faces_in_picture_s.py

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python find_faces_in_picture_s.py 
I found 5 face(s) in this photograph.
A face is located at pixel location Top: 477, Left: 270, Bottom: 632, Right: 425
A face is located at pixel location Top: 182, Left: 368, Bottom: 311, Right: 497
A face is located at pixel location Top: 498, Left: 689, Bottom: 605, Right: 796
A face is located at pixel location Top: 247, Left: 629, Bottom: 355, Right: 736
A face is located at pixel location Top: 420, Left: 1158, Bottom: 495, Right: 1232

※ 解像度の違いによる認識の差はない。

↑

[顔検出] 画像から顔を見つける（ディープラーニングを使用する）「find_faces_in_picture_cnn.py」 †

入力データの準備
前項と同じ画像

プログラムソースの修正
入力ファイルパスを修正

from PIL import Image
import face_recognition

# Load the jpg file into a numpy array
image = face_recognition.load_image_file("face_rec_try_full.jpg")

# Find all the faces in the image using a pre-trained convolutional neural network.
# This method is more accurate than the default HOG model, but it's slower
# unless you have an nvidia GPU and dlib compiled with CUDA extensions. But if you do,
# this will use GPU acceleration and perform well.
# See also: find_faces_in_picture.py
face_locations = face_recognition.face_locations(image, number_of_times_to_upsample=0, model="cnn")

print("I found {} face(s) in this photograph.".format(len(face_locations)))

for face_location in face_locations:

    # Print the location of each face in this image
    top, right, bottom, left = face_location
    print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))

    # You can access the actual face itself like this:
    face_image = image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.show()

プログラムの実行
・解像度「3872x2592」

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python find_faces_in_picture_cnn.py 
I found 7 face(s) in this photograph.
A face is located at pixel location Top: 1412, Left: 834, Bottom: 1820, Right: 1242
A face is located at pixel location Top: 1454, Left: 2031, Bottom: 1861, Right: 2439
A face is located at pixel location Top: 566, Left: 1134, Bottom: 906, Right: 1473
A face is located at pixel location Top: 1292, Left: 3502, Bottom: 1489, Right: 3699
A face is located at pixel location Top: 669, Left: 1866, Bottom: 1077, Right: 2274
A face is located at pixel location Top: 1460, Left: 457, Bottom: 1624, Right: 621
A face is located at pixel location Top: 1356, Left: 283, Bottom: 1451, Right: 378

・解像度「1280x857」コピーして入力画像を修正 find_faces_in_picture_cnn_s.py

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python find_faces_in_picture_cnn_s.py 
I found 5 face(s) in this photograph.
A face is located at pixel location Top: 238, Left: 628, Bottom: 351, Right: 742
A face is located at pixel location Top: 191, Left: 374, Bottom: 305, Right: 488
A face is located at pixel location Top: 470, Left: 671, Bottom: 607, Right: 807
A face is located at pixel location Top: 470, Left: 270, Bottom: 607, Right: 406
A face is located at pixel location Top: 420, Left: 1148, Bottom: 499, Right: 1227

※ 解像度が高い方が認識する顔の数が多い。

↑

[顔認識] k近傍法で顔認識する「face_recognition_knn.py」 †

入力データの準備

・顔画像のデータベース「knn_examples/train」

・テスト対象画像「knn_examples/test」

プログラムソースの修正
入力ファイルをソースに合わせて配置（修正なし）

▼「face_recognition_knn.py」

"""
This is an example of using the k-nearest-neighbors (KNN) algorithm for face recognition.

When should I use this example?
This example is useful when you wish to recognize a large set of known people,
and make a prediction for an unknown person in a feasible computation time.

Algorithm Description:
The knn classifier is first trained on a set of labeled (known) faces and can then predict the person
in an unknown image by finding the k most similar faces (images with closet face-features under euclidean distance)
in its training set, and performing a majority vote (possibly weighted) on their label.

For example, if k=3, and the three closest face images to the given image in the training set are one image of Biden
and two images of Obama, The result would be 'Obama'.

* This implementation uses a weighted vote, such that the votes of closer-neighbors are weighted more heavily.

Usage:

1. Prepare a set of images of the known people you want to recognize. Organize the images in a single directory
   with a sub-directory for each known person.

2. Then, call the 'train' function with the appropriate parameters. Make sure to pass in the 'model_save_path' if you
   want to save the model to disk so you can re-use the model without having to re-train it.

3. Call 'predict' and pass in your trained model to recognize the people in an unknown image.

NOTE: This example requires scikit-learn to be installed! You can install it with pip:

$ pip3 install scikit-learn

"""

import math
from sklearn import neighbors
import os
import os.path
import pickle
from PIL import Image, ImageDraw
import face_recognition
from face_recognition.face_recognition_cli import image_files_in_folder

ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg'}


def train(train_dir, model_save_path=None, n_neighbors=None, knn_algo='ball_tree', verbose=False):
    """
    Trains a k-nearest neighbors classifier for face recognition.

    :param train_dir: directory that contains a sub-directory for each known person, with its name.

     (View in source code to see train_dir example tree structure)

     Structure:
        <train_dir>/
        ├── <person1>/
        │   ├── <somename1>.jpeg
        │   ├── <somename2>.jpeg
        │   ├── ...
        ├── <person2>/
        │   ├── <somename1>.jpeg
        │   └── <somename2>.jpeg
        └── ...

    :param model_save_path: (optional) path to save model on disk
    :param n_neighbors: (optional) number of neighbors to weigh in classification. Chosen automatically if not specified
    :param knn_algo: (optional) underlying data structure to support knn.default is ball_tree
    :param verbose: verbosity of training
    :return: returns knn classifier that was trained on the given data.
    """
    X = []
    y = []

    # Loop through each person in the training set
    for class_dir in os.listdir(train_dir):
        if not os.path.isdir(os.path.join(train_dir, class_dir)):
            continue

        # Loop through each training image for the current person
        for img_path in image_files_in_folder(os.path.join(train_dir, class_dir)):
            image = face_recognition.load_image_file(img_path)
            face_bounding_boxes = face_recognition.face_locations(image)

            if len(face_bounding_boxes) != 1:
                # If there are no people (or too many people) in a training image, skip the image.
                if verbose:
                    print("Image {} not suitable for training: {}".format(img_path, "Didn't find a face" if len(face_bounding_boxes) < 1 else "Found more than one face"))
            else:
                # Add face encoding for current image to the training set
                X.append(face_recognition.face_encodings(image, known_face_locations=face_bounding_boxes)[0])
                y.append(class_dir)

    # Determine how many neighbors to use for weighting in the KNN classifier
    if n_neighbors is None:
        n_neighbors = int(round(math.sqrt(len(X))))
        if verbose:
            print("Chose n_neighbors automatically:", n_neighbors)

    # Create and train the KNN classifier
    knn_clf = neighbors.KNeighborsClassifier(n_neighbors=n_neighbors, algorithm=knn_algo, weights='distance')
    knn_clf.fit(X, y)

    # Save the trained KNN classifier
    if model_save_path is not None:
        with open(model_save_path, 'wb') as f:
            pickle.dump(knn_clf, f)

    return knn_clf


def predict(X_img_path, knn_clf=None, model_path=None, distance_threshold=0.6):
    """
    Recognizes faces in given image using a trained KNN classifier

    :param X_img_path: path to image to be recognized
    :param knn_clf: (optional) a knn classifier object. if not specified, model_save_path must be specified.
    :param model_path: (optional) path to a pickled knn classifier. if not specified, model_save_path must be knn_clf.
    :param distance_threshold: (optional) distance threshold for face classification. the larger it is, the more chance
           of mis-classifying an unknown person as a known one.
    :return: a list of names and face locations for the recognized faces in the image: [(name, bounding box), ...].
        For faces of unrecognized persons, the name 'unknown' will be returned.
    """
    if not os.path.isfile(X_img_path) or os.path.splitext(X_img_path)[1][1:] not in ALLOWED_EXTENSIONS:
        raise Exception("Invalid image path: {}".format(X_img_path))

    if knn_clf is None and model_path is None:
        raise Exception("Must supply knn classifier either thourgh knn_clf or model_path")

    # Load a trained KNN model (if one was passed in)
    if knn_clf is None:
        with open(model_path, 'rb') as f:
            knn_clf = pickle.load(f)

    # Load image file and find face locations
    X_img = face_recognition.load_image_file(X_img_path)
    X_face_locations = face_recognition.face_locations(X_img)

    # If no faces are found in the image, return an empty result.
    if len(X_face_locations) == 0:
        return []

    # Find encodings for faces in the test iamge
    faces_encodings = face_recognition.face_encodings(X_img, known_face_locations=X_face_locations)

    # Use the KNN model to find the best matches for the test face
    closest_distances = knn_clf.kneighbors(faces_encodings, n_neighbors=1)
    are_matches = [closest_distances[0][i][0] <= distance_threshold for i in range(len(X_face_locations))]

    # Predict classes and remove classifications that aren't within the threshold
    return [(pred, loc) if rec else ("unknown", loc) for pred, loc, rec in zip(knn_clf.predict(faces_encodings), X_face_locations, are_matches)]


def show_prediction_labels_on_image(img_path, predictions):
    """
    Shows the face recognition results visually.

    :param img_path: path to image to be recognized
    :param predictions: results of the predict function
    :return:
    """
    pil_image = Image.open(img_path).convert("RGB")
    draw = ImageDraw.Draw(pil_image)

    for name, (top, right, bottom, left) in predictions:
        # Draw a box around the face using the Pillow module
        draw.rectangle(((left, top), (right, bottom)), outline=(0, 0, 255))

        # There's a bug in Pillow where it blows up with non-UTF-8 text
        # when using the default bitmap font
        name = name.encode("UTF-8")

        # Draw a label with a name below the face
        text_width, text_height = draw.textsize(name)
        draw.rectangle(((left, bottom - text_height - 10), (right, bottom)), fill=(0, 0, 255), outline=(0, 0, 255))
        draw.text((left + 6, bottom - text_height - 5), name, fill=(255, 255, 255, 255))

    # Remove the drawing library from memory as per the Pillow docs
    del draw

    # Display the resulting image
    pil_image.show()


if __name__ == "__main__":
    # STEP 1: Train the KNN classifier and save it to disk
    # Once the model is trained and saved, you can skip this step next time.
    print("Training KNN classifier...")
    classifier = train("knn_examples/train", model_save_path="trained_knn_model.clf", n_neighbors=2)
    print("Training complete!")

    # STEP 2: Using the trained classifier, make predictions for unknown images
    for image_file in os.listdir("knn_examples/test"):
        full_file_path = os.path.join("knn_examples/test", image_file)

        print("Looking for faces in {}".format(image_file))

        # Find all people in the image using a trained classifier model
        # Note: You can pass in either a classifier file name or a classifier model instance
        predictions = predict(full_file_path, model_path="trained_knn_model.clf")

        # Print results on the console
        for name, (top, right, bottom, left) in predictions:
            print("- Found {} at ({}, {})".format(name, left, top))

        # Display results overlaid on an image
        show_prediction_labels_on_image(os.path.join("knn_examples/test", image_file), predictions)

プログラムの実行

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python face_recognition_knn.py
Training KNN classifier...
Training complete!
Looking for faces in izutsu_1.jpg
- Found izutsu at (245, 400)
Looking for faces in nitta_1.jpg
- Found nitta at (348, 348)
Looking for faces in face_rec_try.jpg

(eog:19651): EOG-CRITICAL **: 04:19:01.762: eog_list_store_get_pos_by_image: assertion 'EOG_IS_IMAGE (image)' failed

(eog:19651): EOG-CRITICAL **: 04:19:01.809: eog_image_get_file: assertion 'EOG_IS_IMAGE (img)' failed

(eog:19651): GLib-GIO-CRITICAL **: 04:19:01.809: g_file_equal: assertion 'G_IS_FILE (file1)' failed
- Found izutsu at (270, 477)
- Found izutsu at (368, 182)
- Found okegawa at (689, 498)
- Found okegawa at (629, 247)
- Found unknown at (1158, 420)
Looking for faces in okegawa_1.jpg
- Found okegawa at (572, 263)
Looking for faces in yaoi_1.jpg
- Found yaoi at (502, 297)

※ ほぼ認識はできるものの複数の顔の入ったものは誤認識もある。

↑

[顔認識] 人物ごとに複数の画像をトレーニングし、SVM（サポートベクターマシン）を用いて顔認識する「face_recognition_svm.py」 †

入力データの準備

・顔画像のデータベース
前項の「knn_examples/train」内のデータを利用してみる。
・テスト対象画像「face_rec_try_full.jpg」

解像度「3872x2592」のファイルでテスト。

プログラムソースの修正
入力ファイルパスを修正

▼「face_recognition_svm.py」

# Train multiple images per person
# Find and recognize faces in an image using a SVC with scikit-learn

"""
Structure:
        <test_image>.jpg
        <train_dir>/
            <person_1>/
                <person_1_face-1>.jpg
                <person_1_face-2>.jpg
                .
                .
                <person_1_face-n>.jpg
           <person_2>/
                <person_2_face-1>.jpg
                <person_2_face-2>.jpg
                .
                .
                <person_2_face-n>.jpg
            .
            .
            <person_n>/
                <person_n_face-1>.jpg
                <person_n_face-2>.jpg
                .
                .
                <person_n_face-n>.jpg
"""

import face_recognition
from sklearn import svm
import os

# Training the SVC classifier

# The training data would be all the face encodings from all the known images and the labels are their names
encodings = []
names = []

# Training directory
train_dir = os.listdir('knn_examples/train/')

# Loop through each person in the training directory
for person in train_dir:
    pix = os.listdir("knn_examples/train/" + person)

    # Loop through each training image for the current person
    for person_img in pix:
        # Get the face encodings for the face in each image file
        face = face_recognition.load_image_file("knn_examples/train/" + person + "/" + person_img)
        face_bounding_boxes = face_recognition.face_locations(face)

        #If training image contains exactly one face
        if len(face_bounding_boxes) == 1:
            face_enc = face_recognition.face_encodings(face)[0]
            # Add face encoding for current image with corresponding label (name) to the training data
            encodings.append(face_enc)
            names.append(person)
        else:
            print(person + "/" + person_img + " was skipped and can't be used for training")

# Create and train the SVC classifier
clf = svm.SVC(gamma='scale')
clf.fit(encodings,names)

# Load the test image with unknown faces into a numpy array
test_image = face_recognition.load_image_file('face_rec_try_full.jpg')

# Find all the faces in the test image using the default HOG-based model
face_locations = face_recognition.face_locations(test_image)
no = len(face_locations)
print("Number of faces detected: ", no)

# Predict all the faces in the test image using the trained classifier
print("Found:")
for i in range(no):
    test_image_enc = face_recognition.face_encodings(test_image)[i]
    name = clf.predict([test_image_enc])
    print(*name)

プログラムの実行

(py37y) PS > cd /anaconda_win/workspace_py37/face_rec_try/
(py37y) PS > python face_recognition_svm.py 
Number of faces detected:  5
Found:
nitta
nitta
nitta
nitta
nitta

※ トレーニングデータが少なすぎるようで認識結果は✕。

↑

更新履歴 †

2022/07/15 初版
2023/12/19 手順再検証

↑

最新の20件

顔認証/顔認識パッケージ「Face recognition」のテスト †

Pythonコードのサンプル †

プロジェクトの準備 †

[顔検出] 画像から顔を見つける「find_faces_in_picture.py」 †

[顔検出] 画像から顔を見つける（ディープラーニングを使用する）「find_faces_in_picture_cnn.py」 †

[顔認識] k近傍法で顔認識する「face_recognition_knn.py」 †

[顔認識] 人物ごとに複数の画像をトレーニングし、SVM（サポートベクターマシン）を用いて顔認識する「face_recognition_svm.py」 †

更新履歴 †

参考資料 †