The DISP human tracking project is designed to facilitate the
efficient tracking of the position and pose of one or more human
subjects within a spatial region. Our end goal is to achieve centimeter
resolution detection of point sources, and to estimate the pose
of a human [angles of their respective joints] to within 5 degrees.
All this using about 150 single-bit detection sensors attached
to wireless embedded microprocessor platforms.
Project Design Goals
Our first task is to quantify the information
we desire to detect. By modelling the human body as a system
of joints, rather than a collection of point sources in three
space, we reduce the information we are interested in by at
least an order of magnitude. Second, we track information in
two domains- visible and far infrared. Joint information are
tracked in the visible domain using tagged color fabrics, and
in the infrared domain using Reference Structure Tomography.
Reference Structure Tomography is the
key to implementing our sensors. Using coded sensor masks to
spatially modulate a set of point detectors, we encode complex
spatial information of the human subject in relatively simple
data measurement. By inverting the mapping, we recover a human
Due to the restriction of available space, our current hardware
usage is limited. Hardware is available for up to 64 camera
devices but only a dozen are currently devoted to joint tracking.
We currently have several prototype development boards for our
infrared sensors; in several months we will be mass-producing
them in the dozens and hopefully hundreds.
||We are constructing a fiber
system to allow efficient detection of human position
using optical fibers. By distributing the fibers to cross
a contiguous path of floor tiles, a human's position in
one of N tiles can be determined with only log N sensors.
One of our members is currently preparing a Preliminary exam for
a Ph.D. degree for work in human pose estimation.
This work began as an experiment in 3D imaging
techniques for building full scale 3D models. The ARGUS system
was designed by Dr. Brady's group at the University of Illinois,
to build models of objects by imaging them with a circumscribed
ring of cameras. By combining the data from each camera perspective
and using cone-beam tomography, models like the one seen to
the right were built.
||1.2GHz Dual-Athlon XP, Windows
XP, 512MB Ram
||450MHz Dual-Pentium III, Redhat Linux
7.1, Kernel 2.4.20
||400MHz Dual-Pentium II, Redhat Linux
7.1, Kernel 2.4.17
||Firewire, 320x240 RGB, 25fps
When Dr. Brady's group moved to Duke University,
the ARGUS program moved with it. The ARGUS program was rewritten
from a new perspective- the real-time, interactive, multi-viewer
streaming of stereograms. Each client program received a pair
of images from two neighboring cameras, and displayed the result
as a stereo-3D projection, either using red-blue imaging or
with appropriate video hardware.
we became interested in human tracking applications, using the
camera system as a starting point. The cameras were removed
from their circular arrangement, and placed about a large
room in such a way that each point in the room was visible to
at least two cameras. Using color as a template, we developed
a method to detect color, specifically the color of human skin.
we could detect color reliably, we tagged each joint of a human
subject, and performed simple real-time tracking of the subject's
position and pose. Since lighting irregularity is a problem,
we used the YUV color domain [ignoring Y] but this also limits
the number of colors we could reliably distinguish from one
another; we therefore used a five-point model of a human
subject, extrapolating other features.
In the course of our development, we build a code framework
so that each of the several project goals could be easily achieved
with the same hardware. The framework is applied to several
test applications, as seen in the pictures above; each were
generated using the same underlying server program.