New ask Hacker News story: Control PC games with body movements using webcam andPython+ MediaPipe
Control PC games with body movements using webcam andPython+ MediaPipe
3 by TSkavinskyy | 1 comments on Hacker News.
I’ve been experimenting with building a universal gesture-based game controller — the idea is to let you play PC games using your body instead of a traditional controller or keyboard. It uses: MediaPipe for body pose detection OpenCV for image processing A custom Python pipeline I wrote to map gestures → keypresses A standard webcam (placed above the screen) — no VR hardware or special equipment needed Here’s a short demo of it running with Ryse: Son of Rome: https://www.youtube.com/watch?v=0k5jMM-aK1M I’ve also tested it on other 3rd-person action games (Assassin’s Creed, The Witcher, Star Wars Jedi Fallen Order). The system sends configurable keypresses based on detected body movements — for example: Stepping to move the character Arm movements to attack or block Leaning to dodge or strafe It works surprisingly well for games that emphasize movement and timing over precision aiming. I’m sharing this because: I’d love feedback on the technical approach I’m curious what other applications people can think of (beyond gaming?) I’d like to find a few testers to help refine the system and test across more games Eventually I’d like to turn this into an easy-to-configure tool — but for now it’s a fun experimental project. If anyone here is interested in the code, the design tradeoffs, or trying it out, I’d be happy to share more details.
3 by TSkavinskyy | 1 comments on Hacker News.
I’ve been experimenting with building a universal gesture-based game controller — the idea is to let you play PC games using your body instead of a traditional controller or keyboard. It uses: MediaPipe for body pose detection OpenCV for image processing A custom Python pipeline I wrote to map gestures → keypresses A standard webcam (placed above the screen) — no VR hardware or special equipment needed Here’s a short demo of it running with Ryse: Son of Rome: https://www.youtube.com/watch?v=0k5jMM-aK1M I’ve also tested it on other 3rd-person action games (Assassin’s Creed, The Witcher, Star Wars Jedi Fallen Order). The system sends configurable keypresses based on detected body movements — for example: Stepping to move the character Arm movements to attack or block Leaning to dodge or strafe It works surprisingly well for games that emphasize movement and timing over precision aiming. I’m sharing this because: I’d love feedback on the technical approach I’m curious what other applications people can think of (beyond gaming?) I’d like to find a few testers to help refine the system and test across more games Eventually I’d like to turn this into an easy-to-configure tool — but for now it’s a fun experimental project. If anyone here is interested in the code, the design tradeoffs, or trying it out, I’d be happy to share more details.
Comments
Post a Comment