Merge pull request #64 from aviralgarg05/main

Hand Game Controller using OpenCV

Computer Vision/Hand Game Controller/README.md

@@ -0,0 +1,71 @@
+# Hand Controller: Control Cars 🚗 Using Hand Gestures in Games
+
+This project leverages Python and OpenCV to create a gesture-based hand controller for games. Using computer vision techniques, the program tracks the user's hand movements and translates them into real-time actions to control a car in any game. This unique and intuitive interface allows players to steer, accelerate, brake, and perform other functions without the need for physical controllers or keyboards.
+
+## Features
+
+- **Real-time Hand Tracking:** Uses OpenCV to detect hand gestures.
+- **Gesture-based Controls:** Move your hand to steer the car, accelerate, or change directions.
+- **Supports PC and Mobile Cameras:** Control the car using either your PC's webcam or your mobile device's camera.
+- **Cross-Game Compatibility:** Can be used to control vehicles in various driving games by mapping hand gestures to game actions.
+
+## How It Works
+
+This project captures the live feed from a camera (either from your PC or mobile) and detects specific hand gestures. These gestures are then translated into game commands (e.g., moving left, right, accelerating, or braking). The gesture detection works using basic image processing techniques, including contour detection and hand tracking algorithms.
+
+### Key Gestures:
+- **Open hand** – Move the car forward
+- **Closed fist** – Stop the car
+- **Move hand left** – Steer left
+- **Move hand right** – Steer right
+
+These gestures are customizable and can be adjusted based on user preferences or the specific game being played.
+
+## Screenshots
+
+![Screenshot 1](https://raw.githubusercontent.com/aviralgarg05/ML-Nexus/f00d9116375685d3ce67852348a2f21d353b56f4/Computer%20Vision/Hand%20Game%20Controller/Screenshot%202024-10-03%20183106.png)
+![Screenshot 2](https://github.com/aviralgarg05/ML-Nexus/blob/f00d9116375685d3ce67852348a2f21d353b56f4/Computer%20Vision/Hand%20Game%20Controller/Screenshot%202024-10-03%20183129.png?raw=true)
+![Screenshot 3](https://github.com/aviralgarg05/ML-Nexus/blob/f00d9116375685d3ce67852348a2f21d353b56f4/Computer%20Vision/Hand%20Game%20Controller/Screenshot%202024-10-03%20183619.png?raw=true)
+![Screenshot 4](https://github.com/aviralgarg05/ML-Nexus/blob/f00d9116375685d3ce67852348a2f21d353b56f4/Computer%20Vision/Hand%20Game%20Controller/Screenshot%202024-10-03%20183644.png?raw=true)
+
+## Installation
+
+To get started, clone the repository and install the required dependencies.
+
+```bash
+git clone https://github.com/your-username/hand-controller.git
+cd hand-controller
+pip install -r requirements.txt
+```
+
+Make sure you have Python 3.x installed along with the required libraries in `requirements.txt`.
+
+### Requirements:
+
+- Python 3.x
+- OpenCV
+- NumPy
+
+## Usage
+
+You can use either your PC's camera or your mobile camera to control the car.
+
+### Using PC Camera:
+1. Run the following command to start the program with your PC webcam:
+    ```bash
+    python main-pc-cam.py
+    ```
+2. Put your hand in front of the camera, and the program will start detecting gestures.
+
+### Using Mobile Camera:
+1. Download the [IP Webcam](https://play.google.com/store/apps/details?id=com.pas.webcam) app on your mobile device.
+2. Launch the app and note the IP address shown on the screen.
+3. In the `main-mobile-cam.py` script, update the `url` variable with the IP address provided by the IP Webcam app.
+4. Run the following command to start the program using your mobile camera:
+    ```bash
+    python main-mobile-cam.py
+    ```
+5. Position your hand in front of the mobile camera to control the car.
+
+### Customizing Gesture Control:
+You can modify the gesture mappings in the main script files (`main-pc-cam.py` or `main-mobile-cam.py`) to assign specific hand gestures to different game actions. Feel free to experiment with different gestures to fit your playstyle.

Computer Vision/Hand Game Controller/main-mobile-cam.py

@@ -0,0 +1,102 @@
+import cv2
+import mediapipe as mp
+from pynput.keyboard import Controller
+
+# Initialize MediaPipe hands with optimizations
+mp_hands = mp.solutions.hands.Hands(static_image_mode=False, max_num_hands=1, min_detection_confidence=0.5, min_tracking_confidence=0.5)
+keyboard = Controller()
+
+# Open the camera
+cp = cv2.VideoCapture(0)
+x1, x2, y1, y2 = 0, 0, 0, 0
+pressed_key = ""
+frame_skip = 1  # Reduce frame skipping to 1 for smoother processing
+
+frame_count = 0
+
+while True:
+    # Capture frame from camera
+    ret, image = cp.read()
+    if not ret:
+        print("Failed to capture frame. Exiting...")
+        break
+
+    frame_count += 1
+
+    # Skip every nth frame to improve performance (reduced skipping)
+    if frame_count % frame_skip != 0:
+        continue
+
+    # Get image dimensions (without resizing)
+    image_height, image_width, _ = image.shape
+
+    # Flip the image horizontally for a selfie-view display
+    image = cv2.flip(image, 1)
+
+    # Convert the image to RGB
+    rgb_img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
+    # Process the image to detect hands
+    output_hands = mp_hands.process(rgb_img)
+    all_hands = output_hands.multi_hand_landmarks
+
+    # Detect keypresses based on hand position
+    if all_hands:
+        hand = all_hands[0]
+        one_hand_landmark = hand.landmark
+
+        for id, lm in enumerate(one_hand_landmark):
+            x = int(lm.x * image_width)
+            y = int(lm.y * image_height)
+
+            if id == 12:  # Finger tip of middle finger
+                x1 = x
+                y1 = y
+
+            if id == 0:  # Wrist point
+                x2 = x
+                y2 = y
+
+        distX = x1 - x2
+        distY = y1 - y2
+
+        if distY > -140 and distY != 0:
+            keyboard.release('d')
+            keyboard.release('a')
+            keyboard.release('w')
+            keyboard.press('s')
+            print("Pressed Key: S")
+        elif distY < -200 and distY != 0:
+            keyboard.release('s')
+            keyboard.release('d')
+            keyboard.release('a')
+            keyboard.press('w')
+            print("Pressed Key: W")
+        elif distX < -100 and distX != 0:
+            keyboard.release('s')
+            keyboard.release('d')
+            keyboard.press('w')
+            keyboard.press('a')
+            print("Pressed Key: A")
+        elif distX > 55 and distX != 0:
+            keyboard.release('a')
+            keyboard.release('s')
+            keyboard.press('w')
+            keyboard.press('d')
+            print("Pressed Key: D")
+    else:
+        keyboard.release('d')
+        keyboard.release('a')
+        keyboard.release('w')
+        keyboard.release('s')
+
+    # Display the camera feed
+    cv2.imshow("Camera Feed", image)
+
+    # Check if 'q' is pressed to quit
+    q = cv2.waitKey(1)
+    if q == ord("q"):
+        break
+
+# Release resources
+cv2.destroyAllWindows()
+cp.release()

Computer Vision/Hand Game Controller/main-pc-cam.py

@@ -0,0 +1,84 @@
+import cv2
+import mediapipe as mp
+
+from pynput.keyboard import Controller
+
+mp_hands = mp.solutions.hands.Hands()
+keyboard = Controller()
+
+cp = cv2.VideoCapture(0)
+x1, x2, y1, y2 =0, 0, 0, 0
+
+while(True):
+
+    _, image = cp.read()
+
+    image_height, image_width, image_depth = image.shape
+    image = cv2.flip(image, 1)
+    rgb_img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
+    output_hands = mp_hands.process(rgb_img)
+    all_hands = output_hands.multi_hand_landmarks
+
+    if all_hands:
+        hand = all_hands[0]
+        one_hand_landmark = hand.landmark
+
+        for id, lm in enumerate(one_hand_landmark):
+            x = int(lm.x * image_width)
+            y = int(lm.y * image_height)
+
+            if id == 12:
+                x1 = x
+                y1 = y
+
+            if id == 0:
+                x2 = x
+                y2 = y
+
+        distX = 0
+        distX = x1 - x2
+        distY = 0
+        distY =y1 - y2
+
+        if distY > -140 and distY !=0:
+            # press S
+            keyboard.release('d')
+            keyboard.release('a')
+            keyboard.release('w')
+            keyboard.press('s')
+            print("S")
+
+        if distY < -200 and distY != 0:
+            keyboard.release('s')
+            keyboard.release('d')
+            keyboard.release('a')
+            keyboard.press('w')
+            print("W")
+
+        if (distX < -100 and distX != 0):
+            keyboard.release('s')
+            keyboard.release('d')
+            keyboard.press('w')
+            keyboard.press('a')
+            print('A')
+
+        if (distX > 55 and distX != 0):
+            keyboard.release('a')
+            keyboard.release('s')
+            keyboard.press('w')
+            keyboard.press('d')
+            print('D')
+
+    else:
+        print('none')
+        keyboard.release('d')
+        keyboard.release('a')
+        keyboard.release('w')
+        keyboard.release('s')
+
+    # if image is not None:
+    #     cv2.imshow("Frame", image)
+    q = cv2.waitKey(1)
+    if q==ord("q"):
+        break
+cv2.destroyAllWindows()

Computer Vision/Hand Game Controller/requirements.txt

@@ -0,0 +1,26 @@
+absl-py==2.1.0
+attrs==23.2.0
+cffi==1.16.0
+contourpy==1.2.1
+cycler==0.12.1
+flatbuffers==24.3.25
+fonttools==4.51.0
+jax==0.4.26
+kiwisolver==1.4.5
+matplotlib==3.8.4
+mediapipe==0.10.11
+ml-dtypes==0.4.0
+numpy==1.26.4
+opencv-contrib-python==4.9.0.80
+opencv-python==4.9.0.80
+opt-einsum==3.3.0
+packaging==24.0
+pillow==10.3.0
+protobuf==3.20.3
+pycparser==2.22
+pynput==1.7.6
+pyparsing==3.1.2
+python-dateutil==2.9.0.post0
+scipy==1.13.0
+six==1.16.0
+sounddevice==0.4.6