The computer engineer researched how they identify the face of a human in an image. For this, we need to identify first where the human face is located in the whole image. The face detector is the method that locates the face of a human in an image and returns it as a bounding box or rectangle box value.
After getting the face position in an image and next, we have to find out small features of the face like eyebrows, lips, etc. Facial landmark detection tells all the required features of a human face which we want.
Dlib's 68 Face Features
The below image is an example of a Dlib's 68 points model. There we can see that points from 1 to 68. But sometimes we don't need all 68 feature points, then for that, we will do in the next post, how we can customize those points according to our requirements. In this post, we only going to see about 68 Dlib's points for clear understanding.
Facial landmark detection with Dlib's 68 Model
There are mostly two steps to detect face landmarks in an image which are given below:
-
Face detection: Face detection is the first method that locates a human face and returns a value in x,y,w,h which is a rectangle.
-
Face landmark: After getting the location of a face in an image, then we have to through points inside of that rectangle.
There are many methods of face detector but we focus in this post on only one which is Dlib's method. Like, Opencv uses methods of LBP cascades, and HAAR and Dlib use methods of HOG (Histogram of Oriented Gradients) and SVM (Support Vector Machine).
Dlib's 68 Facial Landmark Detection in Python
The code in Python is given below and same code you can download it from here
All codes are given with proper comments so that you can understand each and every line of code easily way.
Python: facial_68_landmark.py
This Python code file name is facial_68_landmark.py
Dlib has already a pre-built model which can detect the face. That's why in the below python code facial_68_landmark.py line number 25, we are just accessing directly that model and creating an object faceLandmarkDetector. In the below code, we are first uploading an image and then trying to face that whole image. After getting the face position from the image, we return the rectangle value where the face resides. And on that rectangle is called detection of face. Now, in code line number 54, we are using that rectangle value and image inside of the function to detect face landmarks.
#USAGE: python facial_68_Landmark.py
import dlib,cv2
import numpy as np
from facePoints import facePoints
def writeFaceLandmarksToLocalFile(faceLandmarks, fileName):
with open(fileName, 'w') as f:
for p in faceLandmarks.parts():
f.write("%s %s\n" %(int(p.x),int(p.y)))
f.close()
# location of the model (path of the model).
Model_PATH = "shape_predictor_68_face_landmarks.dat"
# now from the dlib we are extracting the method get_frontal_face_detector()
# and assign that object result to frontalFaceDetector to detect face from the image with
# the help of the 68_face_landmarks.dat model
frontalFaceDetector = dlib.get_frontal_face_detector()
# Now the dlip shape_predictor class will take model and with the help of that, it will show
faceLandmarkDetector = dlib.shape_predictor(Model_PATH)
# We now reading image on which we applied our face detector
image = "test.jpg"
# Now we are reading image using openCV
img= cv2.imread(image)
imageRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# landmarks of the face image will be stored in output/image_k.txt
faceLandmarksOuput= "output/image"
# Now this line will try to detect all faces in an image either 1 or 2 or more faces
allFaces = frontalFaceDetector(imageRGB, 0)
print("List of all faces detected: ",len(allFaces))
# List to store landmarks of all detected faces
allFacesLandmark = []
# Below loop we will use to detect all faces one by one and apply landmarks on them
for k in range(0, len(allFaces)):
# dlib rectangle class will detecting face so that landmark can apply inside of that area
faceRectangleDlib = dlib.rectangle(int(allFaces[k].left()),int(allFaces[k].top()),
int(allFaces[k].right()),int(allFaces[k].bottom()))
# Now we are running loop on every detected face and putting landmark on that with the help of faceLandmarkDetector
detectedLandmarks = faceLandmarkDetector(imageRGB, faceRectangleDlib)
# count number of landmarks we actually detected on image
if k==0:
print("Total number of face landmarks detected ",len(detectedLandmarks.parts()))
# Svaing the landmark one by one to the output folder
allFacesLandmark.append(detectedLandmarks)
# Now finally we drawing landmarks on face
facePoints(img, detectedLandmarks)
fileName = faceLandmarksOuput +"_"+ str(k)+ ".txt"
print("Lanmdark is save into ", fileName)
# Write landmarks to disk
writeFaceLandmarksToLocalFile(detectedLandmarks, fileName)
#Name of the output file
outputNameofImage = "output/image.jpg"
print("Saving output image to", outputNameofImage)
cv2.imwrite(outputNameofImage, img)
cv2.imshow("Face landmark result", img)
# Pause screen to wait key from user to see result
cv2.waitKey(0)
cv2.destroyAllWindows()
In the code below we have defined the method facePoints which is called in the Python code above.
Now to draw landmarks on the face of the detected rectangle, we are passing the landmarks values and image to the facePoints. In the below code, we are passing landmarks and images as a parameter to a method called drawPoints which accessing the coordinates(x,y) of the ith landmarks points using the part(i).x and part(i).y. All landmarks points are saved in a NumPy array and then pass these points to in-built cv2.polylines method to draw the lines on the face using the start point and endpoint parameters.
Here is the basic syntax of the cv2.polylines method:
cv2.polylines(img, array of points in int32 format, bool isClosed, color, lineThickness, lineType)
Now let's see the code,
import cv2
import numpy as np
# This below mehtod will draw all those points which are from 0 to 67 on face one by one.
def drawPoints(image, faceLandmarks, startpoint, endpoint, isClosed=False):
points = []
for i in range(startpoint, endpoint+1):
point = [faceLandmarks.part(i).x, faceLandmarks.part(i).y]
points.append(point)
points = np.array(points, dtype=np.int32)
cv2.polylines(image, [points], isClosed, (255, 200, 0), thickness=2, lineType=cv2.LINE_8)
# Use this function for 70-points facial landmark detector model
# we are checking if points are exactly equal to 68, then we draw all those points on face one by one
def facePoints(image, faceLandmarks):
assert(faceLandmarks.num_parts == 68)
drawPoints(image, faceLandmarks, 0, 16) # Jaw line
drawPoints(image, faceLandmarks, 17, 21) # Left eyebrow
drawPoints(image, faceLandmarks, 22, 26) # Right eyebrow
drawPoints(image, faceLandmarks, 27, 30) # Nose bridge
drawPoints(image, faceLandmarks, 30, 35, True) # Lower nose
drawPoints(image, faceLandmarks, 36, 41, True) # Left eye
drawPoints(image, faceLandmarks, 42, 47, True) # Right Eye
drawPoints(image, faceLandmarks, 48, 59, True) # Outer lip
drawPoints(image, faceLandmarks, 60, 67, True) # Inner lip
# Use this function for any model other than
# 70 points facial_landmark detector model
def facePoints2(image, faceLandmarks, color=(0, 255, 0), radius=4):
for p in faceLandmarks.parts():
cv2.circle(im, (p.x, p.y), radius, color, -1)
The complete code of the above post you can download from the below link:
https://drive.google.com/file/d/1fXlpFVNdGVRszKBxGnjSM4nFLUPnmNrq/view?usp=sharing
The 68-Dlib's point model is not included in that because of the heavy size. So that, we can download it from the below link and keep it inside that folder and you can also set the path of the model from the code.
https://github.com/davisking/dlib-models/blob/master/shape_predictor_68_face_landmarks.dat.bz2
Output:
Conclusion
Dlib's 68-face landmark model shows how we can access face features like eyes, eyebrows, nose, etc. But sometimes, we don't want to access all features of the face and want only some features like lips for lipstick application. So that is also possible using custom training of the Dlib's 68-landmark models and you will get details of that in the next blog.
Frequently Asked Questions(FAQs)
1. What is Dlib 68 points Face landmark Detection?
Dlib 68 points Face landmark Detection is a computer vision algorithm that detects 68 facial landmarks, such as the eyes, nose, and mouth.
2. What is OpenCV?
OpenCV (Open Source Computer Vision Library) is a free and open-source computer vision and machine learning software library.
3. How does Dlib's 68 points Face landmark Detection work?
Dlib 68 points Face landmark Detection uses a machine learning algorithm to detect 68 facial landmarks by analyzing the facial features and geometry.
4. How can I use Dlib 68 points Face landmark Detection with OpenCV and Python?
You can use Dlib 68 points Face landmark Detection with OpenCV and Python by installing the Dlib and OpenCV libraries, loading the pre-trained model, and applying the algorithm to detect the facial landmarks in an image or video.
You may also like:
