fa21-cs242-assignment-1

CS445 Final Project: Facial Detection

Motivation and Impact: 
My interest for this project comes from an interest in facial recognition and its importance in present-day applications. Face recognition is important in areas like security, authentication, and real-time detection, making it useful in both personal and business settings. By working on this project, I hoped to gain a better knowledge of facial characteristics and how they can be used to build a facial detection system. I also wanted to challenge myself by considering other factors like poor lighting and varying facial expressions, which are real challenges in real-world applications. Overcoming these obstacles will not only improve my ability to code, but would also help me create a more reliable facial recognition system.
Approach:
Face identification based on Haar-like characteristics, integral pictures, and sliding windows was implemented in phases. Each part builds upon itself to create a detection system that can identify faces by  facial features.
1. Integral Image Computation
The integral picture was generated as an initial processing step to allow calculation of region sums, which is required for Haar-like feature computation.
The sum of pixels above and to the left of each pixel in the integral picture was calculated.
2. Haar-Like Features
Haar-like features were used to find specific regions resembling eyes and horizontal lines
Features were calculated using templates for accurate detection.
3. Sliding Window Search
A sliding window approach was used to scan the whole picture for feature matches.
Windows of a specific size were moved over the image with a step size, and features were calculated for each window.
4. Detection of Key Components
Eye-like and line-like features were detected using Haar-like features.
5. Filtering and Combining Components
Potential eye pairs were filtered based on vertical alignment and horizontal distance thresholds.
Detected lines were filtered to include only those relevant in position compared to eye location
The selected eye pairs and lines were combined to create a box for the face. 

Results:
1. Integral Image 

The integral image calculates the sum of pixel values, which helps for calculation of Haar-like features.
2. Haar-Like Features
	
Shows all the features matching the Haar-Like Features in the Image
Line: Patterns with a bright regions bordering by two darker regions
3. Sliding Window Search

Rectangles are overlaid on every region analyzed by the sliding window, each rectangle corresponds to a position where the sliding window is calculated for Haar features.

4. Detection of Key Components
	
Shows all the features matching the eye template or line in the Image
5. Filtering and Combining Components

Filtering was used to ensure that just the most important features were kept. These features where used to decide location of box.
6. Random Outputs


Implementation Details: 

Programming Language: Python
Packages: OpenCV, NumPy, Matplotlib, Random

External Resources:
Haar Cascade: eye detection, line detection (see references)
Image Dataset:private dataset of facial images (see references)

Challenge / Innovation:

One of the most challenging parts of this project was creating Haar-like characteristics and integral pictures from scratch. While these are important approaches in computational photography, not having step-by-step instructions  or provided resources  meant I had to explore outside resources as well as constant testing to get the most accurate result. Another problem was merging the eye and line features to create a face detection system that ignored small details and focused on these features. It took multiple trials to fine-tune thresholds for the horizontal and vertical alignment, as well as to filter characteristics to reduce false positives.
The innovative part of this project is how it combines different methods, Haar features, sliding window searches, and the eye-line relationships into a unified system for detecting faces. Adjusting the face box depending on detected features ensures that the whole system is adaptable to diverse facial proportions and fits real-world settings.
Given the complexity of the approaches I used and the time needed to create and test the system, I would give this project 19/20 points in the challenge/innovation category. I believe finding a way to incorporate all these different components of computational photography into a unified system was highly innovative and creative.
References:

Face Detection
"Face Detection Techniques: A Review"
Description: Comprehensive overview of face detection methods.
Kumar, Ashu & Kaur, Amandeep & Kumar, Munish. (2019). Face Detection Techniques: A Review. Artificial Intelligence Review. 52. 10.1007/s10462-018-9650-2. 
Facial Features
"Understanding of facial features in face perception"
Description: Discusses the role of facial features in recognition.
Zhang Q, Zhang Y, Liu N, Sun X. Understanding of facial features in face perception: insights from deep convolutional neural networks. Front Comput Neurosci. 2024 Apr 9;18:1209082. doi: 10.3389/fncom.2024.1209082. PMID: 38655070; PMCID: PMC11035738.
Computing Features
"Image Feature Extraction Using Python"
Description: Techniques for computing image features efficiently.
Link: https://www.geeksforgeeks.org/image-feature-extraction-using-python/
Haar Features
"Python | Haar Cascades for Object Detection"
Description: Explains Haar features and sliding windows for object detection.
Link: https://www.geeksforgeeks.org/python-haar-cascades-for-object-detection/
Integral Images
"Integral Image"
Description: Introduces integral images for efficient feature computation.
Link: https://medium.com/@anubhavroh/integral-image-141f6181db5e
Eye Detection Classifier
eye.xml
Description: Haar cascade for detecting eyes.
Link:https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/
Line Detection Classifier
line.xml
Description: Haar cascade for detecting lines. Developed by Modesto Castrillón-Santana and collaborators, designed for real-time detection applications.
Cite: Castrillón Santana, Modesto & Lorenzo-Navarro, Javier & Deniz, Oscar & Isern, Josep & Falcon, Antonio. (2005). Multiple Face Detection at Different Resolutions for Perceptual User Interfaces. 445-452. 10.1007/11492429_54. 
Facial Database”
“Human Faces
Description: Data set of different faces in different settings
Link: https://www.kaggle.com/datasets/ashwingupta3012/human-faces/data
ChatGPT
Description: Debugging, formatting, runtime optimization