引言

利用python开发，对于输入的摄像头视频流，借助Dlib提供的检测识别模式来进行人脸识别，首先，从摄像头中录入（裁剪）人脸图片存蓄到本地，然后提取特征，构建预设人脸特征，根据抠取的/已有的同一个人多张人脸图片提取128D特征值，然后计算该人的128D特征均值，然后和摄像头中实时获取到的人脸提取出特征值，计算欧式距离，断定是否同同一张人脸。

识别模式：

基于 Dlib 的 ResNet 预训练模型（dlib_face_recognition_resnet_model_v1.dat）

识别算法：

ResNet 神经网络（This model is a ResNet network with 29 conv layers. It's 
essentially a version of the ResNet-34 network from the paper Deep 
Residual Learning for Image Recognition by He, Zhang, Ren, and Sun 
with a few layers removed and the number of filters per layer reduced 
by half）

1.人脸检测

faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> -> 

2.计算人脸特征点

shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> -> 

3.人脸特征描述子

facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>

调用的模式以及大概的耗时：

 Features：

·支持人脸数据采集，自行建立人脸数据库/Support face register

·调用摄像头实时人脸检测和识别/Using camera to real-time detect and recognize faces

·支持多张人脸/Support multi-faces

·利用OT来加速识别/Use OT to improve FPS

人脸识别/Face Recognition的说明：

Wikipedia上关于人脸识别系统/Face Recognition System的描述：they work by comparing selected facial features from given image with faces within a database.

本项目中就是比较预设的人脸识别的特征和摄像头实时获取到的人脸的特征，核心就是提取128D人脸特征，然后计算摄像头人脸特征和预设的特征脸的欧式距离，进行比对。

效果如下：

1.总体流程

先说下人脸检测（Face detection）和人脸识别（Face Recognition），前者是达到检测出场景中人脸的目的就可以了，而后者不仅需要检测出人脸，还要和已有人脸数据进行对比，识别出是否在数据库中，或者进行身份标注之类处理，人脸检测和人脸识别两者有时候可能会被人理解混淆。实现人脸识别功能，借助的是Dlib官方网中face_recognition.py这个例程：（Link：http://dlib.net/face_recognition.py.html ）  

我们直接利用“dlib_face_recognition_resnet_model_v1.dat”这个训练好的resnet模型，提取人脸头像的128D特征，然后比对不同人脸图片的128D特征的欧式距离，设定一个阈值来判断是否为同一张脸：

#face recognition model, the object maps human faces into 128D vectors
facerec = dlib.face_recognition_model_v1("dlib_face_recognition_resnet_model_v1.dat")
 
shape = predictor(img, dets[0])
face_descriptor = facerec.compute_face_descriptor(img, shape)

2.源码介绍

主要有

·get_faces_from_camera.py（代号g）

·features_extraction_to_csv.py（代号fe）

·face_reco_from_camera.py（代号fa）

这三个Python文件，接下来会分别介绍实现功能：

2.1 g/人脸注册录入

人脸识别需要将提取到的头像数据和已有图片数据进行比对分析，所有这部分代码实现的功能就是人脸录入。

程序会生成一个窗口，显示调用的摄像头实时获取的图像。

然后根据键盘输入进行人脸捕捉：

·“N”新录入人脸，新建文件夹person_X/用来存蓄某人的人脸图像

·“S”开始捕获人脸，将捕获到的人脸放到person_X/路径下

·“Q”退出窗口

摄像头的调用是利用opencv库的cv2.VideoCapture(0)，此处参数为0代表调用的是笔记本的默认摄像头，你也可以让它调用传入已有的视频文件，

可以参考https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/how_to_use_camera.py如何通过OpenCV调用摄像头。

get faces from camera.py源码：

# Copyright (C) 2020 coneypo
# SPDX-License-Identifier: MIT

# Author:   coneypo
# Blog:     http://www.cnblogs.com/AdaminXie
# GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
# Mail:     coneypo@foxmail.com

# 进行人脸录入 / Face register

import dlib
import numpy as np
import cv2
import os
import shutil
import time

# Dlib 正向人脸检测器 / Use frontal face detector of Dlib
detector = dlib.get_frontal_face_detector()


class Face_Register:
    def __init__(self):
        self.path_photos_from_camera = "data/data_faces_from_camera/"
        self.font = cv2.FONT_ITALIC

        self.existing_faces_cnt = 0         # 已录入的人脸计数器 / cnt for counting saved faces
        self.ss_cnt = 0                     # 录入 personX 人脸时图片计数器 / cnt for screen shots
        self.current_frame_faces_cnt = 0    # 录入人脸计数器 / cnt for counting faces in current frame

        self.save_flag = 1                  # 之后用来控制是否保存图像的 flag / The flag to control if save
        self.press_n_flag = 0               # 之后用来检查是否先按 'n' 再按 's' / The flag to check if press 'n' before 's'

        # FPS
        self.frame_time = 0
        self.frame_start_time = 0
        self.fps = 0

    # 新建保存人脸图像文件和数据CSV文件夹 / Make dir for saving photos and csv
    def pre_work_mkdir(self):
        # 新建文件夹 / Create folders to save faces images and csv
        if os.path.isdir(self.path_photos_from_camera):
            pass
        else:
            os.mkdir(self.path_photos_from_camera)

    # 删除之前存的人脸数据文件夹 / Delete the old data of faces
    def pre_work_del_old_face_folders(self):
        # 删除之前存的人脸数据文件夹, 删除 "/data_faces_from_camera/person_x/"...
        folders_rd = os.listdir(self.path_photos_from_camera)
        for i in range(len(folders_rd)):
            shutil.rmtree(self.path_photos_from_camera+folders_rd[i])
        if os.path.isfile("data/features_all.csv"):
            os.remove("data/features_all.csv")

    # 如果有之前录入的人脸, 在之前 person_x 的序号按照 person_x+1 开始录入 / Start from person_x+1
    def check_existing_faces_cnt(self):
        if os.listdir("data/data_faces_from_camera/"):
            # 获取已录入的最后一个人脸序号 / Get the order of latest person
            person_list = os.listdir("data/data_faces_from_camera/")
            person_num_list = []
            for person in person_list:
                person_num_list.append(int(person.split('_')[-1]))
            self.existing_faces_cnt = max(person_num_list)

        # 如果第一次存储或者没有之前录入的人脸, 按照 person_1 开始录入 / Start from person_1
        else:
            self.existing_faces_cnt = 0

    # 获取处理之后 stream 的帧数 / Update FPS of video stream
    def update_fps(self):
        now = time.time()
        self.frame_time = now - self.frame_start_time
        self.fps = 1.0 / self.frame_time
        self.frame_start_time = now

    # 生成的 cv2 window 上面添加说明文字 / PutText on cv2 window
    def draw_note(self, img_rd):
        # 添加说明 / Add some notes
        cv2.putText(img_rd, "Face Register", (20, 40), self.font, 1, (255, 255, 255), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "FPS:   " + str(self.fps.__round__(2)), (20, 100), self.font, 0.8, (0, 255, 0), 1,
                    cv2.LINE_AA)
        cv2.putText(img_rd, "Faces: " + str(self.current_frame_faces_cnt), (20, 140), self.font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "N: Create face folder", (20, 350), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "S: Save current face", (20, 400), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "Q: Quit", (20, 450), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)

    # 获取人脸 / Main process of face detection and saving
    def process(self, stream):
        # 1. 新建储存人脸图像文件目录 / Create folders to save photos
        self.pre_work_mkdir()

        # 2. 删除 "/data/data_faces_from_camera" 中已有人脸图像文件 / Uncomment if want to delete the saved faces and start from person_1
        if os.path.isdir(self.path_photos_from_camera):
            self.pre_work_del_old_face_folders()

        # 3. 检查 "/data/data_faces_from_camera" 中已有人脸文件
        self.check_existing_faces_cnt()

        while stream.isOpened():
            flag, img_rd = stream.read()        # Get camera video stream
            kk = cv2.waitKey(1)
            faces = detector(img_rd, 0)         # Use Dlib face detector

            # 4. 按下 'n' 新建存储人脸的文件夹 / Press 'n' to create the folders for saving faces
            if kk == ord('n'):
                self.existing_faces_cnt += 1
                current_face_dir = self.path_photos_from_camera + "person_" + str(self.existing_faces_cnt)
                os.makedirs(current_face_dir)
                print('\\n')
                print("新建的人脸文件夹 / Create folders: ", current_face_dir)

                self.ss_cnt = 0                 # 将人脸计数器清零 / Clear the cnt of screen shots
                self.press_n_flag = 1           # 已经按下 'n' / Pressed 'n' already

            # 5. 检测到人脸 / Face detected
            if len(faces) != 0:
                # 矩形框 / Show the ROI of faces
                for k, d in enumerate(faces):
                    # 计算矩形框大小 / Compute the size of rectangle box
                    height = (d.bottom() - d.top())
                    width = (d.right() - d.left())
                    hh = int(height/2)
                    ww = int(width/2)

                    # 6. 判断人脸矩形框是否超出 480x640 / If the size of ROI > 480x640
                    if (d.right()+ww) > 640 or (d.bottom()+hh > 480) or (d.left()-ww < 0) or (d.top()-hh < 0):
                        cv2.putText(img_rd, "OUT OF RANGE", (20, 300), self.font, 0.8, (0, 0, 255), 1, cv2.LINE_AA)
                        color_rectangle = (0, 0, 255)
                        save_flag = 0
                        if kk == ord('s'):
                            print("请调整位置 / Please adjust your position")
                    else:
                        color_rectangle = (255, 255, 255)
                        save_flag = 1

                    cv2.rectangle(img_rd,
                                  tuple([d.left() - ww, d.top() - hh]),
                                  tuple([d.right() + ww, d.bottom() + hh]),
                                  color_rectangle, 2)

                    # 7. 根据人脸大小生成空的图像 / Create blank image according to the size of face detected
                    img_blank = np.zeros((int(height*2), width*2, 3), np.uint8)

                    if save_flag:
                        # 8. 按下 's' 保存摄像头中的人脸到本地 / Press 's' to save faces into local images
                        if kk == ord('s'):
                            # 检查有没有先按'n'新建文件夹 / Check if you have pressed 'n'
                            if self.press_n_flag:
                                self.ss_cnt += 1
                                for ii in range(height*2):
                                    for jj in range(width*2):
                                        img_blank[ii][jj] = img_rd[d.top()-hh + ii][d.left()-ww + jj]
                                cv2.imwrite(current_face_dir + "/img_face_" + str(self.ss_cnt) + ".jpg", img_blank)
                                print("写入本地 / Save into：", str(current_face_dir) + "/img_face_" + str(self.ss_cnt) + ".jpg")
                            else:
                                print("请先按 'N' 来建文件夹, 按 'S' / Please press 'N' and press 'S'")

            self.current_frame_faces_cnt = len(faces)

            # 9. 生成的窗口添加说明文字 / Add note on cv2 window
            self.draw_note(img_rd)

            # 10. 按下 'q' 键退出 / Press 'q' to exit
            if kk == ord('q'):
                break

            # 11. Update FPS
            self.update_fps()

            cv2.namedWindow("camera", 1)
            cv2.imshow("camera", img_rd)

    def run(self):
        cap = cv2.VideoCapture(0)
        self.process(cap)

        cap.release()
        cv2.destroyAllWindows()


def main():
    Face_Register_con = Face_Register()
    Face_Register_con.run()


if __name__ == '__main__':
    main()

 考虑到有可能需要保存的矩形框超出摄像头范围，对于这种异常，如果矩形框超出范围，矩形框就会从白变红，然后提示“OUT OF RANGE”

get_face_from_camera.py的输出log

新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_1
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_2.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_3.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_4.jpg


新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_2
写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_2.jpg


新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_3
写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_2.jpg

2.2 fa/将图像文件中人脸数据提取出来存入CSV

这部分代码实现的功能是将之前捕获到的人脸图像文件，提取出128D特征，然后计算出某人人脸数据的特征均值存入CSV中，方便之后识别时候进行比对，利用numpy.mean()计算特征均值，生成一个存储所有录入人脸数据database的“features_all.csv”。

features extraction to csv.py源码：

# Copyright (C) 2020 coneypo
# SPDX-License-Identifier: MIT

# Author:   coneypo
# Blog:     http://www.cnblogs.com/AdaminXie
# GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
# Mail:     coneypo@foxmail.com

# 从人脸图像文件中提取人脸特征存入 "features_all.csv" / Extract features from images and save into "features_all.csv"

import os
import dlib
from skimage import io
import csv
import numpy as np

# 要读取人脸图像文件的路径 / Path of cropped faces
path_images_from_camera = "data/data_faces_from_camera/"

# Dlib 正向人脸检测器 / Use frontal face detector of Dlib
detector = dlib.get_frontal_face_detector()

# Dlib 人脸 landmark 特征点检测器 / Get face landmarks
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')

# Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor
face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")


# 返回单张图像的 128D 特征 / Return 128D features for single image
# Input:    path_img           <class 'str'>
# Output:   face_descriptor    <class 'dlib.vector'>
def return_128d_features(path_img):
    img_rd = io.imread(path_img)
    faces = detector(img_rd, 1)

    print("%-40s %-20s" % ("检测到人脸的图像 / Image with faces detected:", path_img), '\\n')

    # 因为有可能截下来的人脸再去检测，检测不出来人脸了, 所以要确保是 检测到人脸的人脸图像拿去算特征
    # For photos of faces saved, we need to make sure that we can detect faces from the cropped images
    if len(faces) != 0:
        shape = predictor(img_rd, faces[0])
        face_descriptor = face_reco_model.compute_face_descriptor(img_rd, shape)
    else:
        face_descriptor = 0
        print("no face")
    return face_descriptor


# 返回 personX 的 128D 特征均值 / Return the mean value of 128D face descriptor for person X
# Input:    path_faces_personX       <class 'str'>
# Output:   features_mean_personX    <class 'numpy.ndarray'>
def return_features_mean_personX(path_faces_personX):
    features_list_personX = []
    photos_list = os.listdir(path_faces_personX)
    if photos_list:
        for i in range(len(photos_list)):
            # 调用 return_128d_features() 得到 128D 特征 / Get 128D features for single image of personX
            print("%-40s %-20s" % ("正在读的人脸图像 / Reading image:", path_faces_personX + "/" + photos_list[i]))
            features_128d = return_128d_features(path_faces_personX + "/" + photos_list[i])
            # 遇到没有检测出人脸的图片跳过 / Jump if no face detected from image
            if features_128d == 0:
                i += 1
            else:
                features_list_personX.append(features_128d)
    else:
        print("文件夹内图像文件为空 / Warning: No images in " + path_faces_personX + '/', '\\n')

    # 计算 128D 特征的均值 / Compute the mean
    # personX 的 N 张图像 x 128D -> 1 x 128D
    if features_list_personX:
        features_mean_personX = np.array(features_list_personX).mean(axis=0)
    else:
        features_mean_personX = np.zeros(128, dtype=int, order='C')
    print(type(features_mean_personX))
    return features_mean_personX


# 获取已录入的最后一个人脸序号 / Get the order of latest person
person_list = os.listdir("data/data_faces_from_camera/")
person_num_list = []
for person in person_list:
    person_num_list.append(int(person.split('_')[-1]))
person_cnt = max(person_num_list)

with open("data/features_all.csv", "w", newline="") as csvfile:
    writer = csv.writer(csvfile)
    for person in range(person_cnt):
        # Get the mean/average features of face/personX, it will be a list with a length of 128D
        print(path_images_from_camera + "person_" + str(person + 1))
        features_mean_personX = return_features_mean_personX(path_images_from_camera + "person_" + str(person + 1))
        writer.writerow(features_mean_personX)
        print("特征均值 / The mean of features:", list(features_mean_personX))
        print('\\n')
    print("所有录入人脸数据存入 / Save all the features of faces registered into: data/features_all.csv")

 我们可以看下对于某张图片，face_descriptor这个128D vectors的输出结果：

绿框我们的返回128D特征的函数。

红框调用该函数来计算img_face_13.jpg。

黄框输出为128D的向量。

 之后就需要人脸图像进行批量化操作，提取出128D的特征，然后计算特征均值，存入features_all.csv是一个n行的CSV，n是录入的人脸数，128列是某人的128D特征，这存储的就是录入的人脸数据，之后摄像头捕获的人脸将要过来和这些特征均值进行比对，如果欧式距离比较近的话，就可以认为是同一张人脸。

get_festures_into_CSV.py的输出log：

##### person_1 #####
data/data_csvs_from_camera/person_1.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_2.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_3.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_3.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_4.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_4.jpg 

##### person_2 #####
data/data_csvs_from_camera/person_2.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_2/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_2/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_2/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_2/img_face_2.jpg 

##### person_3 #####
data/data_csvs_from_camera/person_3.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_3/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_3/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_3/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_3/img_face_2.jpg 


...

2.3 fa/实时人脸识别对比分析

这部分源码实现的功能：调用摄像头，捕获摄像头中的人脸，然后如果检测到人脸，将摄像头中的人脸提取出128D的特征，然后和之前录入人脸的128D特征进行计算欧式距离，如果比较小，可以判定为一个人，否则不是一个人：

 所以设计的伪代码如下：

# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")

faces = detector(img_gray, 0)

# 1. 如果检测到人脸
if len(faces) != 0:
    # 遍历所有检测到的人脸
    for i in range(len(faces)):
        # 2. 提取当前帧人脸的特征描述子
        shape = predictor(img_rd, faces[i])
        facerec.compute_face_descriptor(img_rd, shape)
        # 3. 将当前帧人脸特征描述子和数据库的特征描述子进行对比
        for i in range(len(self.features_known_list)):
            e_distance_tmp = self.return_euclidean_distance(self.features_camera_list[k], self.features_known_list[i])

关于  https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/face_reco_from_camera.py里面变量的定义：

关于用到dlib检测器，预测器，识别器：

1.dlib.get_frontal_face_detector

Link:

http://dlib.net/python/index.html#dlib.get_frontal_face_detector

介绍：

返回默认的人检测器，为下面的fhog_object_detectorm/Returns the default face detector

2.class dlib.fhog_object_detector

Link:

http://dlib.net/python/index.html#dlib.fhog_object_detector

介绍：

基于滑动窗的HOG进行目标检测，

This object represents a sliding window histogram-of-oriented-gradients based object detector.

参数：

__call__(self: dlib.fhog_object_detector, image: array, upsample_num_times: int=0L) → dlib.rectangles

3.class dlib.shape_predicter

Link:

http://dlib.net/python/index.html#dlib.shape_predictor

参数/parameters:

__call__(self: dlib.shape_predictor, image: array, box: dlib.rectangle) → dlib.full_object_detection

输入：dlib.rectangle输出：dlib.full_recognition_model_v1

参数/parameters：

compute_face_descriptor(self: dlib.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),uint8], face: dlib.full_object_detection, num_jitters: int=0L, padding: float=0.25) -> dlib.vector

通过print(type())可以更清楚的看到dlib对象的传递：

# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")

faces = detector(img_gray, 0)

# 如果检测到人脸
if len(faces) != 0:
    print(type(faces)                                                # <class 'dlib.dlib.rectangles'>
    # 遍历所有检测到的人脸
    for i in range(len(faces)):
        # 进行人脸比对
        shape = predictor(img_rd, faces[i])
        print(type(shape))                                            # <class 'dlib.dlib.full_object_detection'>
        facerec.compute_face_descriptor(img_rd, shape)
        print(type(facerec.compute_face_descriptor(img_rd, shape))    # <class 'dlib.dlib.vector'>

这样一个对象传递过程：

faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> -> 
shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> -> 
facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>

欧式距离对比的阈值设定，是在return_euclidean_distance函数的dist变量，我这里程序里面的指定的欧式距离判断阈值是0.4，具体阈值可以根据实际情况或者测得结果进行修改。

这边做了一个，让人名跟随显示在头像下方，如果想要在人脸矩形框下方显示人名，首先想要知道Dlib生成的矩形框的尺寸怎么读取。

Dlib返回的dets变量是一系人脸的数据，此处对单张人脸处理，所有取dets[0]的参数，

可以通过dets[0].top()，dets[0].bottom()，dets[0].left()和dets[0].right()来确定要显示的人名的坐标

得到矩形框的坐标，就可以获取人名的相对位置。

face reco from camera.py源码：

# Copyright (C) 2020 coneypo
# SPDX-License-Identifier: MIT

# Author:   coneypo
# Blog:     http://www.cnblogs.com/AdaminXie
# GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
# Mail:     coneypo@foxmail.com

# 摄像头实时人脸识别 / Real-time face detection and recognition

import dlib
import numpy as np
import cv2
import pandas as pd
import os
import time
from PIL import Image, ImageDraw, ImageFont

# Dlib 正向人脸检测器 / Use frontal face detector of Dlib
detector = dlib.get_frontal_face_detector()

# Dlib 人脸 landmark 特征点检测器 / Get face landmarks
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')

# Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor
face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")


class Face_Recognizer:
    def __init__(self):
        self.feature_known_list = []                # 用来存放所有录入人脸特征的数组 / Save the features of faces in the database
        self.name_known_list = []                   # 存储录入人脸名字 / Save the name of faces in the database

        self.current_frame_face_cnt = 0             # 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame
        self.current_frame_feature_list = []        # 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame
        self.current_frame_name_position_list = []  # 存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame
        self.current_frame_name_list = []           # 存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame

        # Update FPS
        self.fps = 0
        self.frame_start_time = 0

    # 从 "features_all.csv" 读取录入人脸特征 / Get known faces from "features_all.csv"
    def get_face_database(self):
        if os.path.exists("data/features_all.csv"):
            path_features_known_csv = "data/features_all.csv"
            csv_rd = pd.read_csv(path_features_known_csv, header=None)
            for i in range(csv_rd.shape[0]):
                features_someone_arr = []
                for j in range(0, 128):
                    if csv_rd.iloc[i][j] == '':
                        features_someone_arr.append('0')
                    else:
                        features_someone_arr.append(csv_rd.iloc[i][j])
                self.feature_known_list.append(features_someone_arr)
                self.name_known_list.append("Person_"+str(i+1))
            print("Faces in Database：", len(self.feature_known_list))
            return 1
        else:
            print('##### Warning #####', '\\n')
            print("'features_all.csv' not found!")
            print(
                "Please run 'get_faces_from_camera.py' and 'features_extraction_to_csv.py' before 'face_reco_from_camera.py'",
                '\\n')
            print('##### End Warning #####')
            return 0

    # 计算两个128D向量间的欧式距离 / Compute the e-distance between two 128D features
    @staticmethod
    def return_euclidean_distance(feature_1, feature_2):
        feature_1 = np.array(feature_1)
        feature_2 = np.array(feature_2)
        dist = np.sqrt(np.sum(np.square(feature_1 - feature_2)))
        return dist

    # 更新 FPS / Update FPS of Video stream
    def update_fps(self):
        now = time.time()
        self.frame_time = now - self.frame_start_time
        self.fps = 1.0 / self.frame_time
        self.frame_start_time = now

    def draw_note(self, img_rd):
        font = cv2.FONT_ITALIC

        cv2.putText(img_rd, "Face Recognizer", (20, 40), font, 1, (255, 255, 255), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "FPS:   " + str(self.fps.__round__(2)), (20, 100), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "Faces: " + str(self.current_frame_face_cnt), (20, 140), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
        cv2.putText(img_rd, "Q: Quit", (20, 450), font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)

    def draw_name(self, img_rd):
        # 在人脸框下面写人脸名字 / Write names under rectangle
        font = ImageFont.truetype("simsun.ttc", 30)
        img = Image.fromarray(cv2.cvtColor(img_rd, cv2.COLOR_BGR2RGB))
        draw = ImageDraw.Draw(img)
        for i in range(self.current_frame_face_cnt):
            # cv2.putText(img_rd, self.current_frame_name_list[i], self.current_frame_name_position_list[i], font, 0.8, (0, 255, 255), 1, cv2.LINE_AA)
            draw.text(xy=self.current_frame_name_position_list[i], text=self.current_frame_name_list[i], font=font)
            img_with_name = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
        return img_with_name

    # 修改显示人名 / Show names in chinese
    def show_chinese_name(self):
        # Default known name: person_1, person_2, person_3
        if self.current_frame_face_cnt >= 1:
            self.name_known_list[0] ='张三'.encode('utf-8').decode()
        # self.name_known_list[1] ='李四'.encode('utf-8').decode()
        # self.name_known_list[2] ='xx'.encode('utf-8').decode()
        # self.name_known_list[3] ='xx'.encode('utf-8').decode()
        # self.name_known_list[4] ='xx'.encode('utf-8').decode()

    # 处理获取的视频流，进行人脸识别 / Face detection and recognition from input video stream
    def process(self, stream):
        # 1. 读取存放所有人脸特征的 csv / Get faces known from "features.all.csv"
        if self.get_face_database():
            while stream.isOpened():
                print(">>> Frame start")
                flag, img_rd = stream.read()
                faces = detector(img_rd, 0)
                kk = cv2.waitKey(1)
                # 按下 q 键退出 / Press 'q' to quit
                if kk == ord('q'):
                    break
                else:
                    self.draw_note(img_rd)
                    self.current_frame_feature_list = []
                    self.current_frame_face_cnt = 0
                    self.current_frame_name_position_list = []
                    self.current_frame_name_list = []

                    # 2. 检测到人脸 / Face detected in current frame
                    if len(faces) != 0:
                        # 3. 获取当前捕获到的图像的所有人脸的特征 / Compute the face descriptors for faces in current frame
                        for i in range(len(faces)):
                            shape = predictor(img_rd, faces[i])
                            self.current_frame_feature_list.append(face_reco_model.compute_face_descriptor(img_rd, shape))
                        # 4. 遍历捕获到的图像中所有的人脸 / Traversal all the faces in the database
                        for k in range(len(faces)):
                            print(">>>>>> For face", k+1, " in camera")
                            # 先默认所有人不认识，是 unknown / Set the default names of faces with "unknown"
                            self.current_frame_name_list.append("unknown")

                            # 每个捕获人脸的名字坐标 / Positions of faces captured
                            self.current_frame_name_position_list.append(tuple(
                                [faces[k].left(), int(faces[k].bottom() + (faces[k].bottom() - faces[k].top()) / 4)]))

                            # 5. 对于某张人脸，遍历所有存储的人脸特征
                            # For every faces detected, compare the faces in the database
                            current_frame_e_distance_list = []
                            for i in range(len(self.feature_known_list)):
                                # 如果 person_X 数据不为空
                                if str(self.feature_known_list[i][0]) != '0.0':
                                    print("   >>> With person", str(i + 1), ", the e distance: ", end='')
                                    e_distance_tmp = self.return_euclidean_distance(self.current_frame_feature_list[k],
                                                                                    self.feature_known_list[i])
                                    print(e_distance_tmp)
                                    current_frame_e_distance_list.append(e_distance_tmp)
                                else:
                                    # 空数据 person_X
                                    current_frame_e_distance_list.append(999999999)
                            # 6. 寻找出最小的欧式距离匹配 / Find the one with minimum e distance
                            similar_person_num = current_frame_e_distance_list.index(min(current_frame_e_distance_list))
                            print("   >>> Minimum e distance with ", self.name_known_list[similar_person_num], ": ", min(current_frame_e_distance_list))

                            if min(current_frame_e_distance_list) < 0.4:
                                self.current_frame_name_list[k] = self.name_known_list[similar_person_num]
                                print("   >>> Face recognition result:  " + str(self.name_known_list[similar_person_num]))
                            else:
                                print("   >>> Face recognition result: Unknown person")

                            # 矩形框 / Draw rectangle
                            for kk, d in enumerate(faces):
                                # 绘制矩形框
                                cv2.rectangle(img_rd, tuple([d.left(), d.top()]), tuple([d.right(), d.bottom()]),
                                              (0, 255, 255), 2)

                        self.current_frame_face_cnt = len(faces)

                        # 7. 在这里更改显示的人名 / Modify name if needed
                        # self.show_chinese_name()

                        # 8. 写名字 / Draw name
                        img_with_name = self.draw_name(img_rd)

                    else:
                        img_with_name = img_rd

                print(">>>>>> Faces in camera now:", self.current_frame_name_list)

                cv2.imshow("camera", img_with_name)

                # 9. 更新 FPS / Update stream FPS
                self.update_fps()
                print(">>> Frame ends\\n\\n")

    # OpenCV 调用摄像头并进行 process
    def run(self):
        cap = cv2.VideoCapture(0)
        # cap = cv2.VideoCapture("video.mp4")
        cap.set(3, 480)     # 640x480
        self.process(cap)

        cap.release()
        cv2.destroyAllWindows()


def main():
    Face_Recognizer_con = Face_Recognizer()
    Face_Recognizer_con.run()


if __name__ == '__main__':
    main()

face_reco_from_camera.py输出log：

Faces in Database： 3
>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24747225595381367
   >>> With person 3 , the e distance: 0.22821104803792178
   >>> Minimum e distance with  Person_3 :  0.22821104803792178
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.2490812900317618
   >>> With person 3 , the e distance: 0.22549497689337802
   >>> Minimum e distance with  Person_3 :  0.22549497689337802
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24569769385882426
   >>> With person 3 , the e distance: 0.2262102554355137
   >>> Minimum e distance with  Person_3 :  0.2262102554355137
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24387949251367172
   >>> With person 3 , the e distance: 0.22636200199905795
   >>> Minimum e distance with  Person_3 :  0.22636200199905795
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.2473446948271673
   >>> With person 3 , the e distance: 0.22534075942468246
   >>> Minimum e distance with  Person_3 :  0.22534075942468246
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24465000646050672
   >>> With person 3 , the e distance: 0.2238005841538998
   >>> Minimum e distance with  Person_3 :  0.2238005841538998
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends

如果对单个人脸，进行实时对比输出：

 通过实时的输出结果，看的比较明显

输出绿色：当是我自己时，计算出来的欧式距离基本都是0.2左右。

输出红色：而换一张图片，明显看到欧式距离计算结果达到0.8，此时就可以判定，后来这张人脸不是一张人脸。

好了今天讲到这了！

拜拜@！