HQ NETWORKXbox One | Xbox | InstallerDeluxe.Com 1,241 HQ Network UsersHQ Network: 1,241 | Members OnlineGuests: 673 |Guests OnlineMembers: 1

  Active Topics  | Achievements  | Cheats  | Downloads  | Feedback  | Forums  | Games  | Hardware  | My Profile  | News  | Reviews  | Tutorials  | Leaderboards  | Videos

How You Become the Controller with Xbox 360 Kinect

Catch up on everything Xbox including the latest Gaming News, Release Dates, DLC, Xbox Live and Modding..

Browse All News |  Write for Us |  Most Popular |  Browsing: 29 SEARCH XBOX NEWS:      
Xbox One

How You Become the Controller with Xbox 360 Kinect

How You Become the Controller with Xbox 360 Kinect
Published by forahobby on Tuesday, January 11, 2011
Tagged: Gaming, Hardware, Xbox 360,
Ron Forbes from Xbox.com talks Xbox 360 Kinect and explains in great detail how you become the controller.

"You are the controller." If you've been following the buzz surrounding Kinect, you've probably heard this phrase tossed around. From plugging leaks with your hands and feet in Kinect Adventures! to changing songs with the flick of a wrist in Zune, Kinect opens up a new way to naturally experience your entertainment

But once you get over the magic of opening the disc tray with the wave of a hand like you're a Jedi, you might start to wonder how it all works under the hood. In this blog post, I'll focus on the secret sauce behind the human tracking system and how it allows game developers to produce Kinect-enabled experiences. Then Arjun Dayal, a program manager on the team, will show you how Kinect enables a gesture-based approach to navigating the Xbox Dashboard and Kinect Hub. But before we get into any of that, let’s start with the conceptual principles that guided Kinect’s development.

It’s an Analog World We Live In

Traditional programming is based on sets of rules and heuristics: cause and effect, zero and one, true and false. Fortunately, this approach works well when modeling simple systems with a limited number of inputs and outputs. Take Halo for example (not a simple system by any means, but suitable for proving the point): pushing A causes Master Chief to jump; moving one stick causes him to walk forward; moving another causes him to look up. If A, then B. Unfortunately, the real world is not digital, but analog.

In the analog world, it’s not just about yes and no, but it’s about maybe. It’s not just about true or false, but it’s about probability. Think briefly about all of the possible variations of a human waving his hand: the range of physical proportions of the body, the global diversity of environmental conditions, differences in clothing properties, cultural nuances to performing even a simple gesture. Quickly, you end up with a search space around 10^23, an unrealistic problem domain to solve through conditional-based programming.

We knew early on that we had to invent a new way of approaching this problem, one that works like the human brain does. When you encounter someone in the world, your brain instantly focuses on him and recognizes him based on years of prior training. It doesn’t crawl a decision tree hundreds of levels deep to discern one human from another. It just knows. Where a baby would have a hard time telling the two apart, you’ve learned to do so in a split second. In fact, you could probably make a reasonable guess about their age, gender, ethnicity, mood, or even their identity (but that’s a blog post for another day). This is part of what makes us human.

Kinect was created in the same way. It sees the world around it. It focuses on you. And even though it’s never seen how you wave your hands before, it instantly approximates your movements to the terabytes of information it’s already learned.

The Kinect Sensor

At the heart of the skeletal tracking pipeline is a CMOS infrared sensor which allows Kinect to perceive the world, regardless of ambient lighting conditions. Think of this as seeing the environment in a monochrome spectrum of black and white: black being infinitely far away and white being infinitely close. The shades of gray in between these two extremes correspond to a physical distance from the sensor. The sensor gathers each point in its field of view and forms it into a depth image that represents the world. A stream of these depth images is produced at a rate of 30 frames per second, creating a real-time 3-D representation of the environment. Another way to think of this is like those pinpoint impression toys that used to be all the rage. By pushing up with your hands (or your face if you were really adventurous), you could create a simple 3-D model of a piece of your body.
Finding the Moving Parts

The next thing Kinect does is to focus on the moving objects most likely to be humans in the image, much like the human eye subconsciously focuses on moving objects that it sees in the world. Later in the pipeline, Kinect will perform a pixel-by-pixel evaluation of the depth image to identify parts of the human body; however, to maintain a responsive refresh rate, this evaluation must be optimized beforehand.

Segmentation is the strategy Kinect uses to separate the humanoid masses from the background, the signal from the noise. Kinect can actively track the full skeletons of up to two human players as well as passively track the shape and position of four passive players at once. In this phase of the pipeline, we create what's called a segmentation mask of each tracked human player in the depth image. This is a modified version of the depth image that has subtracted background objects from the scene such as chairs and pet cats. By sending just the segmentation mask through the rest of the pipeline, we reduce the upcoming computational work that must be done to identify parts of the body.
The Brain Inside Kinect

Here's where the real magic happens. Each pixel of the player segmentation is fed into a machine learning system that’s been trained to recognize parts of the human body. This gives us a probability distribution of the likelihood that a given pixel belongs to a given body part. For example, one pixel may have an 80% chance of belonging to a foot, a 60% chance of belonging to a leg, and a 40% chance of belonging to the chest. It might make sense initially to keep the most probable proposal and throw out the rest, but that would be a bit premature. Rather, we send all of these possibilities (called multiple centroid proposals) down the rest of the pipeline and delay this judgment call until the very end.

As a brief aside, you might start to wonder how we taught this brain to recognize body parts. Training this artificial intelligence (called the Exemplar system) was no small feat: terabytes of data were fed into a machine cluster to teach Kinect a pixel-by-pixel technique for identifying arms, legs, and other body parts it sees. The harlequin-looking figures you see here are some of the data points we used to train and test the Exemplar system.

Model Fitting: Creating the Skeleton

The final step in the pipeline is to use the centroid proposals from the previous phase to create a skeleton of the 20 joints that are tracked. For each joint, Kinect considers every pixel that Exemplar proposed as a likely match. This way it has the full context to make the most accurate assessment of where the human actually is. We also do some additional filtering of the output at this model fitting stage to smooth the output and handle special cases like occluded joints.

One of our goals with the skeletal tracking system was to provide an "a la carte" interface to the different outputs of the pipeline. Game developers can use as many or as few of these components to make enjoyable experiences. For instance, you can use just the segmentation map to create some incredible aesthetic effects (Your Shape: Fitness Evolved is a great example of this).

And there you have it, a complete real-time human tracking system to control your games and entertainment. Hopefully, this look inside Kinect has shown you how we enable game developers to make digital sense of the analog world in which we live. Next, Arjun introduces you to the improved Xbox Dashboard and Kinect Hub. He'll show you how the Dashboard and Kinect Hub leverage the depth stream and 20-joint skeleton to create an intuitive gesture-based way to access your games, movies, music, and other entertainment.

Full Story: www.xbox.com
( Subscribe via )

Associated Topics

GamingHardwareXbox 360
 Latest Kinect Adventures Screenshots (9)
Kinect Adventures screenshot #14904Kinect Adventures screenshot #14903Kinect Adventures screenshot #14902Kinect Adventures screenshot #14901Kinect Adventures screenshot #14900
Kinect Adventures screenshot #11104Kinect Adventures screenshot #11103Kinect Adventures screenshot #11102Kinect Adventures screenshot #11101

Kinect Adventures Achievements (32)

There are 32 achievements with a total of 1000 points available
Court Adjourned AchievementCourt Adjourned
Successfully complete all Rallyball time challenges, or see them completed
3030 gamerpoints
Happily Ever Rafter AchievementHappily Ever Rafter
Successfully complete all River Rush time challenges, or see them completed
3030 gamerpoints
Time Flies AchievementTime Flies
Successfully complete all Reflex Ridge time challenges, or see them completed
3030 gamerpoints
Fulfilled Your Destiny AchievementFulfilled Your Destiny
Participate in every Adventure reward ceremony
5050 gamerpoints
Advanced Degree AchievementAdvanced Degree
Participate in all Advanced Adventures' reward ceremonies
7575 gamerpoints
View All Achievements for Kinect Adventures (32)
 HQ Kinect Adventures Xbox Live Gamers (20)
View All HQ Live Gamers for Kinect Adventures (20)

Other games developed by Good Science Studios (4)

Kinect Fun Labs: Kinect Me

Kinect Fun Labs: Kinect Me

Kinect Fun Labs is a new series of Kinect Enabled Games Exclusively for the Xbox 360...

Kinect Fun Labs: Googly Eyes

Kinect Fun Labs: Googly Eyes

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

Kinect Fun Labs: Build a Buddy

Kinect Fun Labs: Build a Buddy

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

Kinect Fun Labs: Bobble Head

Kinect Fun Labs: Bobble Head

Kinect Fun Labs lets you discover and play with the newest advances in Kinect technology. You'll find innovative Gadgets such as "Kinect™ Me" and "Bobble Head" that utilize Kinect's people-scanning tech to turn you into an amazingly realistic avatar or talking bobble head. Other Gadgets use innovations in object scanning, backgr..

View All Games Developed by Good Science Studios (5)

Other games published by Microsoft Games Studios (282)

Halo 3

Halo 3

Bungie's popular first-person shooter makes its debut on the Xbox 360 with Halo 3. The epic saga continues with Halo 3, the hugely anticipated sequel to the highly successful and critically acclaimed Halo franchise. In this third chapter of the Halo trilogy, Master Chief returns to finish the fight, bringing the epic conflict betwe..

Gears of War

Gears of War

“Gears of War” is the first game developed by Epic Games exclusively for Microsoft Game Studios and the Xbox 360 video game and entertainment system. “Gears of War” thrusts gamers into a deep and harrowing story of humankind’s epic battle for survival against the Locust Horde, a nightmarish race of creatures that surface from the bowels of the plan..

Gears of War 2

Gears of War 2

“Gears of War 2” is the highly anticipated sequel to the 5 million-selling blockbuster action game and one of the most popular Xbox games in history. Players continue as Marcus Fenix, a reluctant war hero and leader of Delta Squad, six months after the events of “Gears of War.” The last cities on Sera are sinking, swallowed by a new Locust t..

Halo 3: ODST

Halo 3: ODST

Halo 3: ODST's tale focuses on the legendary ODSTs or "Orbital Drop Shock Troopers" as they drop into the ruined city of New Mombasa, looking for clues behind the Covenant's catastrophic attack on the city. Sgt Mgr. Johnson, the toughest, cigar chomping-est, Marine on the battlefield can be yours to play in Firefight when you p..



"Crackdown" pushes the action-driving hybrid genre into the next generation with the first ever truly 3-D playground. Gamers will enforce justice by any means necessary in Pacific City, a crime-ridden urban center built to encourage the exploration of the full width, depth and height of the city. Coupled with highly innovative co-op gamep..

View All Games Published by Microsoft Games Studios (283)

Rate this Article

Average Score: 4.66 | Votes: 3


 Printer Friendly Printer Friendly  Send to a Friend Send to a Friend

Kinect Adventures

Good Science Studios


- Nov 4, 2010 (US)
- Nov 4, 2010 (US)
Played: 1058  |  Wishlists: 7
Your Rating:

360-HQ Rating:
8.2/10 8.2
Overall Rank: 682 of 2964
Achievements: 32

Total Ratings: 17
Page Views: 54,157
Xbox LIVE Leaderboards
You need to log in or register to rate games.
All Xbox 360 Games | Top 50 Xbox LIVE

Latest Xbox 360 News

Xbox 360 NewsSuperliminal Available Now On Xbox One
First-person puzzler 'Superliminal' is now available for digital download from the Microsoft Store for Xbox One.Superliminal is a first-person puzzle game inspired perspective and optical illusions. P..

“Hellpoint” arrives July 30 on Xbox One; Digita..

Street Power Soccer Heading To All Consoles and..

Party Crasher Simulator announced for XSX, PS5,..

Wolfenstein: Alt History Collection Available N..

Xbox 360 News | Submit News

Xbox 360 News Archive

   Thursday, July 02
 Marvel's Avengers Confirmed for Xbox Series X

   Wednesday, July 01
 Xbox360Sisters Royale: Five Sisters Under now digita
 WW1 FPS Tannenberg coming to XB1 and PS4 cons
 The Otterman Empire hits XBOX, PS4, SWITCH an
 Top 10 Esports games for 2020

   Tuesday, June 30
 Xbox360Yes, Your Grace is OUT NOW on XBOX and SWITCH
 New Xbox Games with Gold for July 2020 Reveal

   Monday, June 29
 Xbox360Control's 'The Foundation' is available now o

   Friday, June 26
 Xbox360New Games Coming to Xbox Game Pass for PC and
 DiRT 5 digital pre-order's go live on Xbox On

   Thursday, June 25
 Xbox360Assetto Corsa Competizione races onto Xbox On

   Wednesday, June 24
 Xbox360“Night Call” Out Now And Included With Xbox G
 West Of Dead: Out Now and included in Xbox Ga

   Tuesday, June 23
 Xbox360Disintegration Launches On Xbox One, PlayStat
 4 Games Bundle: Luc Bernard Collection Availa

   Monday, June 22
 Xbox360STAR WARS: Squadrons Xbox One digital pre-ord

   Friday, June 19
 Xbox360Windbound hits Xbox One, PS4, Switch and PC o
 “Othercide” is now available for digital pre-

   Thursday, June 18
 Xbox360Super Soccer Blast kicks off for Xbox One
 Hard West Ultimate Edition XBOX ONE Digital P

   Wednesday, June 17
 Xbox360“Tower Of Time” now available for digital pre
 Glass Masquerade Double Pack: Available Now O

   Tuesday, June 16
 Xbox360Radical Rabbit Stew out now for Xbox One, PS4
 Pathfinder: Kingmaker XBOX ONE Digital Pre-or
 1971 Project Helios Hits XBOX One, PS4, Switc

   Monday, June 15
 Xbox360Procedurally generated 2D platformer “Pity Pi

   Friday, June 12
 Xbox360Greymoor: The New ESO Chapter Out Now‎

   Thursday, June 11
 Xbox360Charlotte Cracker Joins ONE PIECE: PIRATE WAR

   Wednesday, June 10
 Xbox360'Beyond Blue' digital pre-order is available
 New Super Lucky's Tale is coming to Xbox One

See All Xbox 360 News | Xbox 360 News Archive

HQ Network Affiliates

Xbox One Games
Xbox One X Games
Video Game Consoles
Xbox 360 Cheats
Xbox Cheats
Game Bullets
Xbox One Forums
Xbox 360 Cheats
Xbox 360 Achievements
Xbox 360 Cheats
Video Codecs
Get 360 Achievements
HD wallpapers
View All HQ Affiliates