Duke Integrated Sensing & Processing

Contact | Sitemap

Human Tracking Project

The DISP human tracking project is designed to facilitate the efficient tracking of the position and pose of one or more human subjects within a spatial region. Our end goal is to achieve centimeter resolution detection of point sources, and to estimate the pose of a human [angles of their respective joints] to within 5 degrees. All this using about 150 single-bit detection sensors attached to wireless embedded microprocessor platforms.

Project Design Goals

Our first task is to quantify the information we desire to detect. By modelling the human body as a system of joints, rather than a collection of point sources in three space, we reduce the information we are interested in by at least an order of magnitude. Second, we track information in two domains- visible and far infrared. Joint information are tracked in the visible domain using tagged color fabrics, and in the infrared domain using Reference Structure Tomography.

Reference Structure Tomography is the key to implementing our sensors. Using coded sensor masks to spatially modulate a set of point detectors, we encode complex spatial information of the human subject in relatively simple data measurement. By inverting the mapping, we recover a human body model.

Due to the restriction of available space, our current hardware usage is limited. Hardware is available for up to 64 camera devices but only a dozen are currently devoted to joint tracking. We currently have several prototype development boards for our infrared sensors; in several months we will be mass-producing them in the dozens and hopefully hundreds.

Current Research

Fiber Tracking:

We are constructing a fiber system to allow efficient detection of human position using optical fibers. By distributing the fibers to cross a contiguous path of floor tiles, a human's position in one of N tiles can be determined with only log N sensors.

Human Pose Estimation:

One of our members is currently preparing a Preliminary exam for a Ph.D. degree for work in human pose estimation. This work began as an experiment in 3D imaging techniques for building full scale 3D models. The ARGUS system was designed by Dr. Brady's group at the University of Illinois, to build models of objects by imaging them with a circumscribed ring of cameras. By combining the data from each camera perspective and using cone-beam tomography, models like the one seen to the right were built.

    System Specifications

    Devices # Specifications
    Client 1 1.2GHz Dual-Athlon XP, Windows XP, 512MB Ram
    Master Node 1 450MHz Dual-Pentium III, Redhat Linux 7.1, Kernel 2.4.20
    Distributed Nodes 6 400MHz Dual-Pentium II, Redhat Linux 7.1, Kernel 2.4.17
    Cameras 10 Firewire, 320x240 RGB, 25fps

When Dr. Brady's group moved to Duke University, the ARGUS program moved with it. The ARGUS program was rewritten from a new perspective- the real-time, interactive, multi-viewer streaming of stereograms. Each client program received a pair of images from two neighboring cameras, and displayed the result as a stereo-3D projection, either using red-blue imaging or with appropriate video hardware.

Next, we became interested in human tracking applications, using the camera system as a starting point. The cameras were removed from their circular arrangement, and placed about a large room in such a way that each point in the room was visible to at least two cameras. Using color as a template, we developed a method to detect color, specifically the color of human skin.

Once we could detect color reliably, we tagged each joint of a human subject, and performed simple real-time tracking of the subject's position and pose. Since lighting irregularity is a problem, we used the YUV color domain [ignoring Y] but this also limits the number of colors we could reliably distinguish from one another; we therefore used a five-point model of a human subject, extrapolating other features.

In the course of our development, we build a code framework so that each of the several project goals could be easily achieved with the same hardware. The framework is applied to several test applications, as seen in the pictures above; each were generated using the same underlying server program.