Engineering Projects/InteractiveSystem/Howard Community College/fall2012/p1-504-tjgr
Please delete everything in italics after filling it out.
Project documentation focuses on how the team organized itself to tackle the unknown. This is a creative process. The details learned should be captured in tutorials. Project documentation should focus on how the team organized itself to step out into the unknown. Failure is common. The single most effective way to increase engineering respect is to document failures in a creative way that stimulates, inspires and helps others continue the project. Projects are never done. There are always loose ends. There are always decisions that might have been made more clearly. There are always other alternatives that might have been considered. None of this stuff appears in a tutorial. It does appear in the team documentation.
Electronic Sections Expected
editProblem Statement
editTo create an interactive motion and audio system using Xbox 360 Kinect, a projector, and a computer that will project a graphic design with an audio that responds to a person's motion while also projecting the person's silhouette.
Here is a visual demonstration of the problem statement. Although the example in the video is advanced, certain elements of our problem statement are clearly seen such as the graphics being interactive with the person, or the subject, displaying the silhouette, and being sound interactive as well. Example
Team Members
editSummary
editWe plugged a camera and a projector into a computer. Using Po-Motion 30-day-free-trail, we were able to project a scene onto a wall. We put our hands in front of the camera and the projected scene reacted to the motion. However, the range of motion of the camera is very limited. We are replacing the camera with an Xbox 360 Kinect motion sensor. Using certain software, we were able to get the Kinect to work with a computer. The Kinect has several advantageous properties such as having superior technology, full-body sensors, audio-re-activity, and the flexibility of being used with a wide variety of software. Using Somantics Sparklers, we were able to have an interactive system with the Kinect and the graphic. No work has been done on creating a complimentary sound interactive system.
Poster
editStory
editThe team members split up into 3 components. Tamara Barovskaia researched software to create a graphic that is continuously in motion. Gratien researched software that can connect the graphic and the kinect, so that when projected, the graphic would be motion interactive. Jessica "hacked" the Xbox 360 Kinect to get it to be recognized by the computer and be motion sensitive.
We discovered Somantics software and incorporated it in our system. Somantics works well with computer and Kinect. The interactive software is available for free download for Ipads, Windows, and Macs on the homepage. Various tutorials are also available. Somantics reacts to a person's motion, mostly focusing on the limbs. It picks up the input and produces an output on the computer of the person with the interactive graphics. Although, Somantics produces interactive graphics, there is no background necessarily. For example, the background will be of wherever the person will be. If the person is in their room or a classroom while using Somantics, the output will display the person, the interactive graphic, and the room the person is in.
Since we worked with Windows, here is the Somantics for Windows download Somantics Download - Windows
Here is a list of Somantics sample graphics and their tutorials. We used the Sparkles
Although we have not incorporated the interactive sound system in our project, there is a software available called Wekinator that might be helpful. A couple of very important applications of Wekinator include:
Creation of new musical instruments
Create mappings between gesture and computer sounds.
Creation of gesturally-controlled animations and games
Control interactive visual environments like Processing or Quartz Composer, or game engines like Unity, using gestures sensed from webcam, Kinect, Arduino, etc.
Creation of systems for gesture analysis and feedback
Build classifiers to detect which gesture a user is performing. Use the identified gesture to control the computer or to inform the user how he's doing.
Creation of real-time music information retrieval and audio analysis systems
Detect instrument, genre, pitch, rhythm, etc. of audio coming into the mic, and use this to control computer audio, visuals, etc.
Creation of other interactive systems in which the computer responds in real-time to some action performed by a human user.
Anything that can output OSC can be used as a controller Anything that can be controlled by OSC can be controlled by Wekinator
Here is the link to download Wekinator
The following are the requirements to download and operate the Wekinator: Chuck version 1.2.1.2 or later, and Java version 5 or later.
Here is the link to download Chuck
Here are the instructions to navigate the softwares and Wekinator, how to operate them, how to use the Kinect with the software
Instructions
How to Make a Custom Interactive Graphic:
This can be accomplished by using Processing programming.
There are 2 different methods:
Software needed for the first method: 1. Processing.js 2. Anything.html 3. Anything.pde
Software needed for the second method: 1. Processing.js 2. Anything.html
All of these can be downloaded from Processing
What is programming. js? Processing program that makes data visualizations, digital art, interactive animations, graphs, video games, etc. using web standards without plugins. To make a custom graphic, go to the "Reference" tab. There are different categories for the codes used by processing.js. For example, to use a specific shape for a graphic, go under the "Shape" category and select the code for whichever shape needed.
Decision List
editMaterial List
edit1. Computer: Dell, PC, Windows. Quantity: 1. Cost: Provided by Howard Community College
2. Projector: EPSON. Quantity: 1. Cost: Provided by Howard Community College
3. Xbox 360 Kinect Motion Sensor. Quantity: 1. Cost: Provided by Howard Community College Engineering Lab
4. Software. Quantity: 5. Cost: Provided online for free download.
Software List
editThe following is the software used to "hack" into Xbox360 Kinect
1. OpenNI
2. Nite
3. Kinemote
4. Sensorkinect-unstable
Software used for the interactive graphics:
1. Somantics - Sparklers
Time
edit52 hours.
Tutorials
editNext Steps
editWe were able to hack into the Xbox 360 Kinect and have it react with a computer. We used Somantics Sparklers as our interactive graphic. However, there needs to be a sound interactive system. The next group can work on creating a sound interactive system that will compliment the interactive graphic, possible using the Wekinator software. The next group can also try making custom graphics using Processing. Various categories for graphics are provided on Processing including their visual examples and programming codes.
This project relies heavily on the ability to understand programming since almost the entire system is software based. There is a great amount of difficulty in downloading, installing, extracting, and programming Wekinator software in addition to "hacking" the Kinect. This project demands patience and dedication. To assist the programming process, read the instructions carefully and slowly, do not skim through them.