Source-linked AI summary
Tracking of Fingertips and Centres of Palm using KINECT
J. L. Raheja, A. Chaudhary, K Singal
TL;DR
The paper tackles robust fingertip and palm-centre detection for natural hand gestures without markers or sensors. It uses KINECT depth imagery for 3D hand segmentation, fingertip localization, and distance-transform palm detection, reporting near-100% fingertip accuracy with open fingers and around 90% palm-centre accuracy in a lab setup. The detected points are intended to control robotic hands through one- or two-hand gestures.
Problem
Existing methods have limitations involving efficiency, orientation, segmentation conditions, specialized instruments, or markers, motivating natural hand-gesture detection without these constraints.
Method
The method uses KINECT depth information to segment hand blobs, derive fingertip depth maps after palm removal, and locate palm centres with distance transformation on inverted binary hand images.
Results
Near-100% fingertip-detection accuracy was reported with all fingers open, while palm-centre detection was around 90% correct in the laboratory setup.
Takeaways & Limitations
The detected fingertips and palm centres are intended to provide inputs for controlling robotic hands with one or both natural hands without gloves or markers.
Abstract
from arXiv · showhide
Hand Gesture is a popular way to interact or control machines and it has been implemented in many applications. The geometry of hand is such that it is hard to construct in virtual environment and control the joints but the functionality and DOF encourage researchers to make a hand like instrument. This paper presents a novel method for fingertips detection and centres of palms detection distinctly for both hands using MS KINECT in 3D from the input image. KINECT facilitates us by providing the depth information of foreground objects. The hands were segmented using the depth vector and centres of palms were detected using distance transformation on inverse image. This result would be used to feed the inputs to the robotic hands to emulate human hands operation.
I. INTRODUCTION
The paper addresses efficient and orientation-robust fingertip and palm-centre detection for natural hand gestures using depth-capable sensing. It proposes marker-free detection for one or both hands under dynamic conditions.
- Existing fingertip-detection methods target dynamic hand-gesture recognition across different applications.
- Prior approaches faced efficiency, orientation, background, lighting, motion-speed, or specialized-instrument limitations.
- The proposed method detects fingertips and palm centres for one or both hands without sensors, markers, or hand color.
- The approach uses KINECT depth information to improve robustness and reliability over the authors’ earlier 2D segmentation-based method.
II. FINGERTIPS DETECTION IN 3D
The 3D fingertip-detection pipeline obtains depth images from KINECT and uses its calibrated depth output as the basis for processing hand points.
- A. Getting the Depth Image from KINECT: The method begins by obtaining synchronized RGB and depth images from KINECT’s infrared, PrimeSense, and RGB sensing components.KINECT generates depth through Light Coding and provides 11-bit output with 2048 sensitivity levels.
- KINECT depth information supplies the 3D input used by the fingertip-detection approach.
- A. Getting the Depth Image from KINECT: The depth of a 3D point is calibrated with d = K tang(Hdraw +L) – O.The passage defines d as depth in centimetres and gives H, K, L, and O calibration values.
B. Hands Tracking and detecting Hand Point
Hand tracking and hand-point detection use NITE and OpenNI software modules operating on depth-informed imagery.
- NITE modules detect and track hands using Bayesian Object Localization, while OpenNI provides C++ APIs for these operations.The resulting image contains depth information.
C. Segmentation by Depth
The segmentation stage thresholds hand depth and selects the blob containing the detected hand point to isolate the hand region.
- C. Segmentation by Depth: The depth image is segmented by applying a calculated threshold to hand-point depth values.
- C. Segmentation by Depth: The hand is identified by selecting the blob that contains the detected hand point.
- C. Segmentation by Depth: The resulting processing sequence includes threshold imagery, a one-hand image, palm extraction, and a fingers mask.
D. Fingertips detections
Fingertips are extracted by isolating finger regions from the hand’s depth image and identifying their closest points to the camera.
- Fingertips detections: A large circular filter removes the fingers, and subtracting the resulting palm from the hand produces segmented finger masks.The masks are multiplied with the original depth image to form a depth map of the fingers.
- Fingertips detections: The minimum depth values in the finger depth map indicate fingertip locations because fingertips are closest to the camera.
- Fingertips detections: Figure 5 shows the segmented fingers in the depth image.
- Fingertips detections: Figure 6 shows the resulting fingertips detection operating in real time.
III. CENTRE OF PALM DETECTION
Palm centres are located by applying a distance transform to inverted binary hand images, with the transform maximum marking each palm centre.
- III. CENTRE OF PALM DETECTION: Applying a distance transform to inverted binary hand images identifies the centre of each palm at the transform’s maximum.The method supports distinct recognition when both hands are present.
- III. CENTRE OF PALM DETECTION: The approach is applied to both hands so their palm centres can be distinguished separately.The right hand is detected as white and the left hand as pink in the described output.
- III. CENTRE OF PALM DETECTION: Figure 7 illustrates the distance transform applied to the hand.
IV. RESULTS
The real-time system detected fingertips accurately in the lab, including large finger bends, while palm-centre detection was around 90% correct.
- IV. RESULTS: Near 100% fingertip-detection accuracy was achieved when all fingers were open, while fully bent fingers sometimes caused confusion.
- IV. RESULTS: Around 90% of palm-centre detections were correct in the lab setup.
- IV. RESULTS: The complete system operated in real time, with results described as very encouraging.
V. CONCLUSIONS
The paper presents fingertip and palm-centre detection for controlling a robotic hand through natural hand movements, using one or both hands without gloves or markers.
- V. CONCLUSIONS: The detected fingertips and palm centres are intended to provide inputs for controlling a robotic hand with hand gestures.
- V. CONCLUSIONS: Users can present either one hand or both hands, moving in front of a camera without gloves or markers.
- V. CONCLUSIONS: Figure 8 shows the final result with the hand point, palm centre, and fingertips.