人工智能python3+tensorflow人脸识别_机器学习tensorflow object detection 实现人脸识别...

object detection是Tensorflow很常用的api，功能强大，很有想象空间，人脸识别，花草识别，物品识别等。下面是我做实验的全过程，使用自己收集的胡歌图片，实现人脸识别，找出胡歌。

安装tensorflow

官方的教程已经写得非常好了，这里就不多说，但是有一点必须注意的是，必须安装python3.6版本，不能安装最新的python3.7版本，不然会出现很多不兼容的问题难以处理。尽可能选择一台显卡性能较好的电脑做机器学习，尽量选择gpu训练，不然训练过程非常的慢。

/install/pip

安装object detection api

收集图片

我这里保存了一份胡歌的照片，一共50张，而且已经标记号了，但是我建议我们开发者应该自己动手来标记一份，尽可能的多些图片，越多越好。

/s/13Ln1FinjxX9ANkopBM6kLQ

安装标记工具labelImg

labelImg必须运行在python3.6，不然无法运行起来，这里就不展开了。

/tzutalin/labelImg

brew install qt

brew install libxml2

make qt5py3

python3 labelImg.py

标记图片

打开了标记工具，选择Open Dir把图片添加进来，然后点击Create RectBox勾选我们要选择的目标，输入对应的label，然后ctrl s保存。最终xml文件是和文件名称保存在同一目录。

image.png

把xml转换成csv格式

#!/usr/bin/env python

# -*- coding:utf-8 -*-

# xml2csv.py

import glob

import pandas as pd

import xml.etree.ElementTree as ET

path = 'data/images/train'

def xml_to_csv(path):

xml_list = []

for xml_file in glob.glob(path + '/*.xml'):

tree = ET.parse(xml_file)

root = tree.getroot()

for member in root.findall('object'):

value = (root.find('filename').text,

int(root.find('size')[0].text),

int(root.find('size')[1].text),

member[0].text,

int(member[4][0].text),

int(member[4][1].text),

int(member[4][2].text),

int(member[4][3].text)

)

xml_list.append(value)

column_name = ['filename', 'width', 'height', 'class', 'xmin', 'ymin', 'xmax', 'ymax']

xml_df = pd.DataFrame(xml_list, columns=column_name)

return xml_df

def main():

image_path = path

xml_df = xml_to_csv(image_path)

xml_df.to_csv(path + '/train.csv', index=None)

print('Successfully converted xml to csv.')

main()

把图片和csv转换成tfrecord格式

记得要修改部分地方

#!/usr/bin/env python

# -*- coding:utf-8 -*-

# generate_tfrecord.py

# -*- coding: utf-8 -*-

"""

Usage:

# From tensorflow/models/

# Create train data:

python generate_tfrecord.py --csv_input=data/tv_vehicle_labels.csv --output_path=train.record

# Create test data:

python generate_tfrecord.py --csv_input=data/test_labels.csv --output_path=test.record

"""

import os

import io

import pandas as pd

import tensorflow as tf

import cv2

from PIL import Image

from object_detection.utils import dataset_util

from collections import namedtuple, OrderedDict

os.chdir('data')

flags = tf.app.flags

flags.DEFINE_string('csv_input', 'images/train/train.csv', 'Path to the CSV input')

flags.DEFINE_string('output_path', 'images/train.record', 'Path to output TFRecord')

FLAGS = flags.FLAGS

# TO-DO replace this with label map

def class_text_to_int(row_label):

if row_label == 'huge': # 需改动

return 1

else:

None

def split(df, group):

data = namedtuple('data', ['filename', 'object'])

gb = df.groupby(group)

return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]

def create_tf_example(group, path):

with tf.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:

encoded_jpg = fid.read()

encoded_jpg_io = io.BytesIO(encoded_jpg)

image = Image.open(encoded_jpg_io)

width, height = image.size

filename = group.filename.encode('utf8')

image_format = b'jpg'

xmins = []

xmaxs = []

ymins = []

ymaxs = []

classes_text = []

classes = []

for index, row in group.object.iterrows():

xmins.append(row['xmin'] / width)

xmaxs.append(row['xmax'] / width)

ymins.append(row['ymin'] / height)

ymaxs.append(row['ymax'] / height)

classes_text.append(row['class'].encode('utf8'))

classes.append(class_text_to_int(row['class']))

tf_example = tf.train.Example(features=tf.train.Features(feature={

'image/height': dataset_util.int64_feature(height),

'image/width': dataset_util.int64_feature(width),

'image/filename': dataset_util.bytes_feature(filename),

'image/source_id': dataset_util.bytes_feature(filename),

'image/encoded': dataset_util.bytes_feature(encoded_jpg),

'image/format': dataset_util.bytes_feature(image_format),

'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),

'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),

'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),

'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),

'image/object/class/text': dataset_util.bytes_list_feature(classes_text),

'image/object/class/label': dataset_util.int64_list_feature(classes),

}))

return tf_example

def main(_):

writer = tf.python_io.TFRecordWriter(FLAGS.output_path)

path = os.path.join(os.getcwd(), 'images/train') # 需改动

examples = pd.read_csv(FLAGS.csv_input)

grouped = split(examples, 'filename')

for group in grouped:

tf_example = create_tf_example(group, path)

writer.write(tf_example.SerializeToString())

writer.close()

output_path = os.path.join(os.getcwd(), FLAGS.output_path)

print('Successfully created the TFRecords: {}'.format(output_path))

if __name__ == '__main__':

tf.app.run()

创建.pbtxt文件

在object_detection/data创建一个train.pbtxt文件，和上面生成tfrecord的改动是对应的。

item {

id: 1

name: 'huge'

}

复制修改ssd_mobilenet_v1_coco.config

research/object_detection/samples/configs/ssd_mobilenet_v1_coco.config

在上面的目录可以找到ssd_mobilenet_v1_coco.config配置文件，复制出来放到object_detection/data目录，然后把文件里面带有PATH_TO_BE_CONFIGURED的地方都修改成我们对应的文件路径。

image.png

最终的文件目录结构

创建文件夹data/training，最后文件结构如下

data

├── images

│ ├── train

│ │ ├── 1.jpg

│ │ ├── 1.xml

│ │ ├── 2.jpg

│ │ ├── 2.xml

│ │ ├── ...

│ │ ├── train.csv

│ ├── training

├── train.record

├── train.pbtxt

└── ssd_mobilenet_v1_coco.config

开始训练

cd research/object_detection

python3 model_main.py \

--pipeline_config_path=data/ssd_mobilenet_v1_coco.config \

--model_dir=data/training \

--num_train_steps=60000 \

--num_eval_steps=20 \

--alsologtostderr

启动训练之后，research/object_detection/data/training文件夹就会陆陆续续创建了一些文件。

loss需要低于1.0才可以达到很好的效果，训练过程非常的漫长，这个和电脑的性能有很大关系，我训练了二十多小时才训练了30000多次step，效果才让loss降低到1.0以下，有条件就使用gpu进行训练，记得使用nohup命令后台训练。

监测训练tensorboard

tensorboard --logdir=object_detection/data/training

在浏览器输入地址查看：http://localhost:6006，从右边的图表可以看到训练的loss的值

image.png

监测训练效果

左边的预测的效果，右边是我们设定的正确效果，一定要有耐心，我也曾经一度怀疑是不是我代码写错了，跑了二十几个小时才看到预测正确。

image.png

生成.pb模型文件

下面是训练生成的目录结构

image.png

需要把下面命令的28189改成training文件夹训练的最后数字

cd research/object_detection

python3 export_inference_graph.py \

--input_type=image_tensor \

--pipeline_config_path=data/ssd_mobilenet_v1_coco.config \

--trained_checkpoint_prefix=data/training/model.ckpt-28189 \

--output_directory=data/training

等命令执行完毕后，就可以看到生成了我们要的frozen_inference_graph.pb文件。

测试模型

在data/test_images增加三张胡歌的图片image1.jpg、image2.jpg、image3.jpg，执行下面代码

(在research/object_detection/object_detection_tutorial.ipynb可以看到这些代码)

# object_detection_tutorial.py

import os

from distutils.version import StrictVersion

import numpy as np

import tensorflow as tf

from PIL import Image

import cv2

from object_detection.utils import ops as utils_ops

if StrictVersion(tf.__version__) < StrictVersion('1.9.0'):

raise ImportError('Please upgrade your TensorFlow installation to v1.9.* or later!')

from object_detection.utils import label_map_util

from object_detection.utils import visualization_utils as vis_util

MODEL_NAME = 'data/'

PATH_TO_FROZEN_GRAPH = MODEL_NAME + 'training/frozen_inference_graph.pb'

PATH_TO_LABELS = MODEL_NAME + "train.pbtxt"

detection_graph = tf.Graph()

with detection_graph.as_default():

od_graph_def = tf.GraphDef()

with tf.gfile.GFile(PATH_TO_FROZEN_GRAPH, 'rb') as fid:

serialized_graph = fid.read()

od_graph_def.ParseFromString(serialized_graph)

tf.import_graph_def(od_graph_def, name='')

category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name=True)

def load_image_into_numpy_array(image):

(im_width, im_height) = image.size

return np.array(image.getdata()).reshape(

(im_height, im_width, 3)).astype(np.uint8)

PATH_TO_TEST_IMAGES_DIR = MODEL_NAME + '/test_images'

TEST_IMAGE_PATHS = [os.path.join(PATH_TO_TEST_IMAGES_DIR, 'image{}.jpg'.format(i)) for i in range(1, 4)]

IMAGE_SIZE = (12, 8)

def run_inference_for_single_image(image, graph):

with graph.as_default():

with tf.Session() as sess:

# Get handles to input and output tensors

ops = tf.get_default_graph().get_operations()

all_tensor_names = {output.name for op in ops for output in op.outputs}

tensor_dict = {}

for key in [

'num_detections', 'detection_boxes', 'detection_scores',

'detection_classes', 'detection_masks'

]:

tensor_name = key + ':0'

if tensor_name in all_tensor_names:

tensor_dict[key] = tf.get_default_graph().get_tensor_by_name(

tensor_name)

if 'detection_masks' in tensor_dict:

# The following processing is only for single image

detection_boxes = tf.squeeze(tensor_dict['detection_boxes'], [0])

detection_masks = tf.squeeze(tensor_dict['detection_masks'], [0])

# Reframe is required to translate mask from box coordinates to image coordinates and fit the image size.

real_num_detection = tf.cast(tensor_dict['num_detections'][0], tf.int32)

detection_boxes = tf.slice(detection_boxes, [0, 0], [real_num_detection, -1])

detection_masks = tf.slice(detection_masks, [0, 0, 0], [real_num_detection, -1, -1])

detection_masks_reframed = utils_ops.reframe_box_masks_to_image_masks(

detection_masks, detection_boxes, image.shape[0], image.shape[1])

detection_masks_reframed = tf.cast(

tf.greater(detection_masks_reframed, 0.5), tf.uint8)

# Follow the convention by adding back the batch dimension

tensor_dict['detection_masks'] = tf.expand_dims(

detection_masks_reframed, 0)

image_tensor = tf.get_default_graph().get_tensor_by_name('image_tensor:0')

# Run inference

output_dict = sess.run(tensor_dict,

feed_dict={image_tensor: np.expand_dims(image, 0)})

# all outputs are float32 numpy arrays, so convert types as appropriate

output_dict['num_detections'] = int(output_dict['num_detections'][0])

output_dict['detection_classes'] = output_dict[

'detection_classes'][0].astype(np.uint8)

output_dict['detection_boxes'] = output_dict['detection_boxes'][0]

output_dict['detection_scores'] = output_dict['detection_scores'][0]

if 'detection_masks' in output_dict:

output_dict['detection_masks'] = output_dict['detection_masks'][0]

return output_dict

for image_path in TEST_IMAGE_PATHS:

print(image_path)

image = Image.open(image_path)

image_np = load_image_into_numpy_array(image)

image_np_expanded = np.expand_dims(image_np, axis=0)

output_dict = run_inference_for_single_image(image_np, detection_graph)

vis_util.visualize_boxes_and_labels_on_image_array(

image_np,

output_dict['detection_boxes'],

output_dict['detection_classes'],

output_dict['detection_scores'],

category_index,

instance_masks=output_dict.get('detection_masks'),

use_normalized_coordinates=True,

line_thickness=8)

image = Image.fromarray(image_np.astype('uint8')).convert('RGB')

image.show()

cv2.waitKey(0)

那么就可以看到执行的结果了

image.png

总结

这次探索了几天的时候，一开始是没有安装object detection，想直接在源码上运行，但是这是行不通的，必须安装，这是我遇到的第一个坑。然后就是标记图片的时候部分图片标签错了，导致运行了十几个小时都毫无进展，标记图片必须好好检查一遍。由于我是用mac电脑，无法使用gpu训练，特别的慢，一旦训练起来，我的电脑就不能做其他事情，运行了十几个小时没有结果就一点，一度以为是我训练不对，就没有耐心停掉了。后来弄了一台Linux电脑，但是没有显卡，通过在后台训练了二十几个小时终于看到了成功，后续我会在Android和iOS运用我们这次的训练成功，敬请关注。