HQ NETWORKXbox One | Xbox | InstallerDeluxe.Com 2,135 HQ Network UsersHQ Network: 2,135 | Members OnlineGuests: 894 |Guests OnlineMembers: 3

  Active Topics  | Achievements  | Cheats  | Downloads  | Feedback  | Forums  | Games  | Hardware  | My Profile  | News  | Reviews  | Tutorials  | Leaderboards  | Videos

How You Become the Controller with Xbox 360 Kinect

Catch up on everything Xbox including the latest Gaming News, Release Dates, DLC, Xbox Live and Modding..

Browse All News |  Write for Us |  Most Popular |  Browsing: 36 SEARCH XBOX NEWS:      

How You Become the Controller with Xbox 360 Kinect

How You Become the Controller with Xbox 360 Kinect
Published by forahobby on Tuesday, January 11, 2011
Tagged: Gaming, Hardware, Xbox 360,
Ron Forbes from Xbox.com talks Xbox 360 Kinect and explains in great detail how you become the controller.

"You are the controller." If you've been following the buzz surrounding Kinect, you've probably heard this phrase tossed around. From plugging leaks with your hands and feet in Kinect Adventures! to changing songs with the flick of a wrist in Zune, Kinect opens up a new way to naturally experience your entertainment

But once you get over the magic of opening the disc tray with the wave of a hand like you're a Jedi, you might start to wonder how it all works under the hood. In this blog post, I'll focus on the secret sauce behind the human tracking system and how it allows game developers to produce Kinect-enabled experiences. Then Arjun Dayal, a program manager on the team, will show you how Kinect enables a gesture-based approach to navigating the Xbox Dashboard and Kinect Hub. But before we get into any of that, let’s start with the conceptual principles that guided Kinect’s development.

It’s an Analog World We Live In

Traditional programming is based on sets of rules and heuristics: cause and effect, zero and one, true and false. Fortunately, this approach works well when modeling simple systems with a limited number of inputs and outputs. Take Halo for example (not a simple system by any means, but suitable for proving the point): pushing A causes Master Chief to jump; moving one stick causes him to walk forward; moving another causes him to look up. If A, then B. Unfortunately, the real world is not digital, but analog.

In the analog world, it’s not just about yes and no, but it’s about maybe. It’s not just about true or false, but it’s about probability. Think briefly about all of the possible variations of a human waving his hand: the range of physical proportions of the body, the global diversity of environmental conditions, differences in clothing properties, cultural nuances to performing even a simple gesture. Quickly, you end up with a search space around 10^23, an unrealistic problem domain to solve through conditional-based programming.

We knew early on that we had to invent a new way of approaching this problem, one that works like the human brain does. When you encounter someone in the world, your brain instantly focuses on him and recognizes him based on years of prior training. It doesn’t crawl a decision tree hundreds of levels deep to discern one human from another. It just knows. Where a baby would have a hard time telling the two apart, you’ve learned to do so in a split second. In fact, you could probably make a reasonable guess about their age, gender, ethnicity, mood, or even their identity (but that’s a blog post for another day). This is part of what makes us human.

Kinect was created in the same way. It sees the world around it. It focuses on you. And even though it’s never seen how you wave your hands before, it instantly approximates your movements to the terabytes of information it’s already learned.

The Kinect Sensor

At the heart of the skeletal tracking pipeline is a CMOS infrared sensor which allows Kinect to perceive the world, regardless of ambient lighting conditions. Think of this as seeing the environment in a monochrome spectrum of black and white: black being infinitely far away and white being infinitely close. The shades of gray in between these two extremes correspond to a physical distance from the sensor. The sensor gathers each point in its field of view and forms it into a depth image that represents the world. A stream of these depth images is produced at a rate of 30 frames per second, creating a real-time 3-D representation of the environment. Another way to think of this is like those pinpoint impression toys that used to be all the rage. By pushing up with your hands (or your face if you were really adventurous), you could create a simple 3-D model of a piece of your body.
Finding the Moving Parts

The next thing Kinect does is to focus on the moving objects most likely to be humans in the image, much like the human eye subconsciously focuses on moving objects that it sees in the world. Later in the pipeline, Kinect will perform a pixel-by-pixel evaluation of the depth image to identify parts of the human body; however, to maintain a responsive refresh rate, this evaluation must be optimized beforehand.

Segmentation is the strategy Kinect uses to separate the humanoid masses from the background, the signal from the noise. Kinect can actively track the full skeletons of up to two human players as well as passively track the shape and position of four passive players at once. In this phase of the pipeline, we create what's called a segmentation mask of each tracked human player in the depth image. This is a modified version of the depth image that has subtracted background objects from the scene such as chairs and pet cats. By sending just the segmentation mask through the rest of the pipeline, we reduce the upcoming computational work that must be done to identify parts of the body.
The Brain Inside Kinect

Here's where the real magic happens. Each pixel of the player segmentation is fed into a machine learning system that’s been trained to recognize parts of the human body. This gives us a probability distribution of the likelihood that a given pixel belongs to a given body part. For example, one pixel may have an 80% chance of belonging to a foot, a 60% chance of belonging to a leg, and a 40% chance of belonging to the chest. It might make sense initially to keep the most probable proposal and throw out the rest, but that would be a bit premature. Rather, we send all of these possibilities (called multiple centroid proposals) down the rest of the pipeline and delay this judgment call until the very end.

As a brief aside, you might start to wonder how we taught this brain to recognize body parts. Training this artificial intelligence (called the Exemplar system) was no small feat: terabytes of data were fed into a machine cluster to teach Kinect a pixel-by-pixel technique for identifying arms, legs, and other body parts it sees. The harlequin-looking figures you see here are some of the data points we used to train and test the Exemplar system.

Model Fitting: Creating the Skeleton

The final step in the pipeline is to use the centroid proposals from the previous phase to create a skeleton of the 20 joints that are tracked. For each joint, Kinect considers every pixel that Exemplar proposed as a likely match. This way it has the full context to make the most accurate assessment of where the human actually is. We also do some additional filtering of the output at this model fitting stage to smooth the output and handle special cases like occluded joints.

One of our goals with the skeletal tracking system was to provide an "a la carte" interface to the different outputs of the pipeline. Game developers can use as many or as few of these components to make enjoyable experiences. For instance, you can use just the segmentation map to create some incredible aesthetic effects (Your Shape: Fitness Evolved is a great example of this).

And there you have it, a complete real-time human tracking system to control your games and entertainment. Hopefully, this look inside Kinect has shown you how we enable game developers to make digital sense of the analog world in which we live. Next, Arjun introduces you to the improved Xbox Dashboard and Kinect Hub. He'll show you how the Dashboard and Kinect Hub leverage the depth stream and 20-joint skeleton to create an intuitive gesture-based way to access your games, movies, music, and other entertainment.

Full Story: www.xbox.com
( Subscribe via )

Associated Topics

GamingHardwareXbox 360
 Latest Kinect Adventures Screenshots (9)
Kinect Adventures screenshot #14904Kinect Adventures screenshot #14903Kinect Adventures screenshot #14902Kinect Adventures screenshot #14901Kinect Adventures screenshot #14900
Kinect Adventures screenshot #11104Kinect Adventures screenshot #11103Kinect Adventures screenshot #11102Kinect Adventures screenshot #11101

Kinect Adventures Achievements (32)

There are 32 achievements with a total of 1000 points available
Court Adjourned AchievementCourt Adjourned
Successfully complete all Rallyball time challenges, or see them completed
3030 gamerpoints
Happily Ever Rafter AchievementHappily Ever Rafter
Successfully complete all River Rush time challenges, or see them completed
3030 gamerpoints
Time Flies AchievementTime Flies
Successfully complete all Reflex Ridge time challenges, or see them completed
3030 gamerpoints
Fulfilled Your Destiny AchievementFulfilled Your Destiny
Participate in every Adventure reward ceremony
5050 gamerpoints
Advanced Degree AchievementAdvanced Degree
Participate in all Advanced Adventures' reward ceremonies
7575 gamerpoints
View All Achievements for Kinect Adventures (32)
 HQ Kinect Adventures Xbox Live Gamers (20)
View All HQ Live Gamers for Kinect Adventures (20)

Other games developed by Good Science Studios (4)

Kinect Fun Labs: Kinect Me

Kinect Fun Labs: Kinect Me

Kinect Fun Labs is a new series of Kinect Enabled Games Exclusively for the Xbox 360...

Kinect Fun Labs: Googly Eyes

Kinect Fun Labs: Googly Eyes

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

Kinect Fun Labs: Build a Buddy

Kinect Fun Labs: Build a Buddy

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

Kinect Fun Labs: Bobble Head

Kinect Fun Labs: Bobble Head

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

View All Games Developed by Good Science Studios (5)

Other games published by Microsoft Games Studios (282)

Halo 3

Halo 3

Bungie's popular first-person shooter makes its debut on the Xbox 360 with Halo 3. The epic saga continues with Halo 3, the hugely anticipated sequel to the highly successful and critically acclaimed Halo franchise. In this third chapter of the Halo trilogy, Master Chief returns to finish the fight, bringing the epic conflict betwe..

Gears of War

Gears of War

“Gears of War” is the first game developed by Epic Games exclusively for Microsoft Game Studios and the Xbox 360 video game and entertainment system. “Gears of War” thrusts gamers into a deep and harrowing story of humankind’s epic battle for survival against the Locust Horde, a nightmarish race of creatures that surface from the bowels of the plan..

Gears of War 2

Gears of War 2

“Gears of War 2” is the highly anticipated sequel to the 5 million-selling blockbuster action game and one of the most popular Xbox games in history. Players continue as Marcus Fenix, a reluctant war hero and leader of Delta Squad, six months after the events of “Gears of War.” The last cities on Sera are sinking, swallowed by a new Locust t..

Halo 3: ODST

Halo 3: ODST

Halo 3: ODST's tale focuses on the legendary ODSTs or "Orbital Drop Shock Troopers" as they drop into the ruined city of New Mombasa, looking for clues behind the Covenant's catastrophic attack on the city. Sgt Mgr. Johnson, the toughest, cigar chomping-est, Marine on the battlefield can be yours to play in Firefight when you p..



"Crackdown" pushes the action-driving hybrid genre into the next generation with the first ever truly 3-D playground. Gamers will enforce justice by any means necessary in Pacific City, a crime-ridden urban center built to encourage the exploration of the full width, depth and height of the city. Coupled with highly innovative co-op gamep..

View All Games Published by Microsoft Games Studios (283)

Rate this Article

Average Score: 4.66 | Votes: 3


 Printer Friendly Printer Friendly  Send to a Friend Send to a Friend

Kinect Adventures

Good Science Studios


- Nov 4, 2010 (US)
- Nov 4, 2010 (US)
Played: 1057  |  Wishlists: 7
Your Rating:

360-HQ Rating:
8.2/10 8.2
Overall Rank: 690 of 2964
Achievements: 32

Total Ratings: 17
Page Views: 51,071
Xbox LIVE Leaderboards
You need to log in or register to rate games.
All Xbox 360 Games | Top 50 Xbox LIVE

Latest Xbox 360 News

Xbox 360 News'Frostpunk: Console Edition' Is Out Now!
Society survival game Frostpunk: Console Edition has launched and is now available on Xbox One.Frostpunk is a society survival game where heat means life and every decision comes at a price. Presentin..

New Monster Hunter World 'Iceborne' Update Adds..

Can Console Games Attract Casino Players and Vi..

Battlefield 5 'Operation Underground' Out Now O..

Blowfish Studios Announces 'Element Space' for ..

Xbox 360 News | Submit News

Xbox 360 News Archive

   Thursday, October 10
 STAR WARS Jedi Fallen Order XBOX ONE Bundles

   Wednesday, October 09
 Xbox360RAZER Nari Ultimate Wireless Headset Is Now A
 CIVILIZATION VI Digital Pre-order And Pre-loa

   Tuesday, October 08
 Xbox360Destiny 2 'Shadowkeep' Is Now Available On Xb
 Alexa and Xbox Join Forces
 CONTRA: ROGUE CORPS Now Available For Xbox On

   Monday, October 07
 Xbox360Alexa Xbox Skill comes to Europe
 Tom Clancy's Ghost Recon Breakpoint: Out Now

   Friday, October 04
 Xbox360The Surge 2 Available Now On Xbox One, PS4 an

   Thursday, October 03
 Xbox360DRAGON BALL Z: KAKAROT now available for digi

   Wednesday, October 02
 Xbox360One Piece: World Seeker DLC Episode 2 'Where

   Tuesday, October 01
 Xbox360Bus Simulator Is Available On Xbox One Right
 NBA 2K20 - The definitive lowdown

   Monday, September 30
 Xbox360'Minecraft Dungeons' Announced for Xbox One,

   Friday, September 27
 Xbox360'Super Dodgeball Beats' is available now on X

   Thursday, September 26
 Xbox360Wreckfest Retro Rammers Pack Is Out Now; Incl

   Wednesday, September 25
 Xbox360Tom Clancy's Ghost Recon Breakpoint Beta Xbox

   Tuesday, September 24
 Xbox360HyperX XBOX ONE Controller charging stations,

   Monday, September 23
 Xbox360TEKKEN 7's third season kicks off with Zafina

   Friday, September 20
 Xbox360KINGDOM HEARTS 3 DLC Arrives This Winter; Wat

   Thursday, September 19
 Xbox360NBA 2K20 Is Available Now on XB1, PS4, SWITCH

   Wednesday, September 18
 Xbox360Monster Hunter World: Iceborne DLC Now Availa

   Tuesday, September 17
 Xbox360Ghost Recon Breakpoint Gameplay: Watch over 4
 Trailmakers releases this week XBOX ONE & PC

   Monday, September 16
 Xbox360Plants vs. Zombies: Battle for Neighborville

   Thursday, September 12
 Xbox360New Blood and Sand DLC and free PvP update fo

   Wednesday, September 11
 Xbox360Fishing Sim World Pro Tour 'Tournament Bass P

   Tuesday, September 10
 Xbox360FINAL FANTASY VIII Remastered finally release
 Blessed Unleashed Open beta dates
 'Control' is now available for Xbox One, PS4

See All Xbox 360 News | Xbox 360 News Archive

HQ Network Affiliates

Xbox One Games
Video Game Consoles
Xbox 360 Cheats
Xbox Cheats
Game Bullets
Xbox 360 Maniac
Xbox One Forums
360 Cheats
Xbox 360 Cheats
Video Codecs
Get 360 Achievements
HD wallpapers
View All HQ Affiliates