The first part of the project will be to research existing massive multi-player game
engines (such as Second Life, There, Kaneva and others) as well as the challenges of
implementing a simple game inside Second Life. The second part of the project will be the
actual implementation of a game using the selected engine as a proof-of-concept. The
project requires knowledge of C++ and some knowledge of computer graphics. This
project is part of a bigger effort directed toward gaining a better understanding on how
people behave under different conditions (such as for example emergency building
evacuation) by using the millions and millions of hours spent by real people in the rapidly
emerging 3D virtual online communities such as Second Life.
Student: Jared G.
Advisor: Prof. Kostas Daniilidis
Prepare new projects for the CIS 121 class.
Student: Chris J.
Advisor: Prof. Ben Taskar
Chris is helping to develop a system for robust face recognition and tracking in videos and movies in order to enable annotation of videos based on characters and their actions.
Standard face detection techniques degrade outside lab settings, in the presence of motion
blur, occlusion, non-frontal poses, drastic illumination changes. Chris is building a face
detector using boosting with larger and more robust contextual features, including torso
and using color and motion segmentation cues. He is also helping to develop a face and eye
pose estimation system for recognizing the direction of a character gaze. Using robust gaze
and pose tracking will enable following the character's focus to identify salient objects and
actions. By combining this information with screenplay and closed captions of movies and
television, visual elements such as people, objects and actions can be efficiently indexed and
searched.
Student: Leftheris K.
Advisor: Prof. Alla Safanova
Project: Analysis of Naturalness of Motion Capture Data.
This project will require work with our brand new Vicon motion capture system. When
human motion is captured a number of filters is applied to remove noise from the data.
These filters are applied at different levels of motion capturing pipeline. The resulting motion
often looks over smoothed and therefore, not natural. The first part of the project will be
the analysis of the effect filtering of the data has on the naturalness of the final motion. Will
need to perform a set of experiments with different filters and render results on a realistic
looking character in Maya. The second part of the project will be design and implementation
of a data-driven filter - filter based on large amount of "unfiltered" motion capture data
available (which would hopefully not over smooth the motion).
The project requires knowledge of C++ and Maya and also preferably (but not necessary)
knowledge of Matlab.
Student: Dan K. and Jessica O.
Advisor: Prof. Mitch Marcus
Doing computational linguistics as part of Prof. Marcus’ MURI project.
Student: Andrew M.
Advisor: Prof. Boon Loo
Student: Bill M.
Advisor: Prof. Boon Loo
Student: Alex M.
Advisor: Prof. Lyle Ungar
Computers will soon have more raw compute power and memory than human brains do, and
many scientists believe that computers will, in our lifetime, be vastly smarter in all ways than
any human. Many other scientists believe this is ridiculous. The possible rapid transition from
humans to computers as the dominant source of intelligence has been called "the singularity."
This project will collect materials for an undergraduate-level course on the singularity that will
look critically at predictions of when and how the singularity might happen, making
"technology forecasts" for different approaches to artificial general intelligence. The course will
also study scenarios and speculation as to what life might look like during and after the
singularity. (See www.cis.upenn.edu/~ungar/the_singularity.html for example references.)
The summer project will include several components. We will collect a set of core references
on the singularity, design homeworks (e.g., to update the classic studies on the computational
power of the brain and of computers), collect and organize objections to predictions of the
singularity, and, if time permits, conduct some surveys, particularly among AI experts, on
their beliefs about the singularity in order to address the question: "What leads people to
believe or disbelieve in the singularity?"
Student: Noe M.
Advisor: Prof. Kostas Daniilidis
Project 3D Skype:
In the GRASP laboratory, we have designed algorithms and integrated them into a system of
three-dimensional tele-presence, called tele-immersion. Using video from at least two
cameras, we have been able to create 3D renderings of people and their environment in
real time which can be transmitted to any remote place. This 3D stream is displayed
to the remote user in life-size using stereoscopic and free viewpoint projection. Current
implementation necessitates a bandwidth applicable only in campus environments and a
non-portable display. Goal of this summer project is to take the 3D-acquisition algorithm
and combine it with the superior network traversal capability of Skype. Skype has released
the Skype API which enables any two applications to communicate even without voice.
Final goal is to have a user in front of two cameraa at one side and a user with stereo
(red-green) glasses at the remote site and vice-vers. Student's duties include the study of
the Skype API and the development of the interface between 3D-acquisition and transmission
as well as the interface between data reception and display from several viewpoints.
Confidence in programming and in exploring new interfaces is required, but most important
is the student's motivation and enthusiasm in creating new systems and change the way
people communicate today.
Students: Yuval M. and Shai N.
Advisor: Prof. Aravind Joshi
Project: Discourse and PDTB: Aravind Joshi
We have built a 1 million word corpus annotated with some complex discourse information
that goes beyond sentence level syntax. This resource is called the Penn Discourse Treebank
(PDTB). You will find very useful information, including a tutorial, at http://www.seas.upenn.edu/~pdtb. We now want to carry out a variety of experiments to
automatically extract complex discourse information from PDTB, which will be useful to
support many applications in text processing and possibly in text generation.
Prerequisites: A course in Natural Language Processing will be useful but not required.
Strong programming background especially in Java.
Student: Andrew M. and Jiawei P.
Advisor: Prof. Ben Taskar
Project: Scene recognition in movies
Our research is focused on automatic understanding of video (movies and television) – who are the characters (person recognition), what are they
doing (action recognition), where are they (scene recognition) and why (overall semantic
understanding of the plot). To accomplish this, we explore techniques in computer vision,
natural language processing, and machine learning. We make use of dvds and screenplays of
popular TV shows and movies, which give us a huge amount of data to play with. This
project would focus on the subtask "where are they" – identifying scenes in a TV show or
movie which are often revisited. For example, in the TV show Lost, we would like to
automatically recognize when the characters are at the beach, in the jungle, having a flashback,
etc. Coarser categories such as indoor vs. outdoor and night vs. day are also useful. Ideally,
we would like a system which takes as input a movie, episode or whole season of a TV show,
and outputs a list of commonly occurring scenes along with when they occurred. With our
guidance, the student would learn about and use common computer vision techniques (e.g.,
SIFT matching, motion segmentation) and machine learning algorithms (e.g., clustering, SVMs)
to make this system work with high accuracy.
Student: John M.
Advisor: Prof. Norman Badler
Project: Better Looking Models
One of our research projects in the SIG Center for Computer Graphics requires that we create
better looking human models and more appropriate motion capture data to animate them going
about shipboard activities. This work would be supervised by PhD student Joe Kider.
Prerequisites include experience with 3D articulated character models and modeling systems
(Maya), and C++ programming skills for data conversion and code integration.
Student: Zachary M.
Advisor: Prof. Kostas Daniilidis
Working on "Real-time implementation of visual odometry"
