An interactive projection-mapping system that watches what a person does in front of a screen and responds by projecting visuals accordingly.
The project has two core responsibilities:
- Understand user actions – the code should be able to detect and interpret what action the user is performing in front of the screen (e.g. a punch, kick, jump or other gesture/pose).
- Project accordingly – based on the recognised action, the code should project the appropriate visuals onto the screen/surface in real time.
In short: perceive the user's action → decide the response → project it.
The first draft is an interactive, Street Fighter–like game:
- The user performs an action in front of the screen.
- The system recognises the action.
- A character is spawned and projected in response to that action.
This draft serves as a proof of concept for the full loop of action recognition and responsive projection, and as a base to build further interactive experiences on.
Early stage – scope definition and first-draft planning.