Results of a new study led by CSE alumnus Jacob Whitehill (Ph.D., '12) demonstrates that a real-time, automatic method for identifying and analyzing facial expressions can perform with an accuracy comparable to that of human observers when tracking how engaged students are in the classroom. (Pictured at left: Student engagement levels are tracked in real time by the automatic system for recognizing facial expressions; photo copyright 2014 IEEE.) The study also revealed that engagement levels were a better predictor of students' post-test performance that the students' pre-test scores.
Whitehill is the first author on the paper "The Faces of Engagement: Automatic Recognition of Student Engagement," which was published April 15 in the early online edition of the journal IEEE Transactions on Affective Computing. Whitehill -- who now works at Emotient, Inc., a startup he co-founded with the paper's senior author, Javier Movellan -- did much of his work on the study while part of the Machine Perception Laboratory in Calit2's Qualcomm Institute (co-directed by Movellan). The project was funded, in part, by the UCSD-based Temporal Dynamics of Learning Center (TDLC), led by CSE Prof. Gary Cottrell. TDLC also enabled the key partnership between Movellan and another co-author on the paper, Virginia Commonwealth professor of developmental psychology Zewelanji Serpell, because both are PIs on TDLC's Social Interaction Network. In addition to Movellan, Whitehill and Serpell, the study’s co-authors include Yi-Ching Lin and Aysha Foster from the department of psychology at Virginia State.
“Automatic recognition of student engagement could revolutionize education by increasing understanding of when and why students get disengaged,” said Whitehill (pictured below right). “Automatic engagement detection provides an opportunity for educators to adjust their curriculum for higher impact, either in real time or in subsequent lessons. Automatic engagement detection could be a valuable asset for developing adaptive educational games, improving intelligent tutoring systems and tailoring massive open online courses, or MOOCs.”
The study consisted of training an automatic detector, which measures how engaged a student appears in a webcam video while undergoing cognitive skills training on an iPad®. The study used automatic expression recognition technology to analyze students’ facial expressions on a frame-by-frame basis and estimate their engagement level. “This study is one of the most thorough to date in the application of computer vision and machine learning technologies for automatic student engagement detection,” said Javier Movellan. “The possibilities for its application in education and beyond are tremendous. By understanding what parts of a lecture, conversation, game, advertisement or promotion produced different levels of engagement, an individual or business can obtain valuable feedback to fine-tune the material to something more impactful.”
In its April 14 edition, the UTSanDiego featured an article about "Scientists set to roam the world" this summer. "If you toss a dart at a map of the world, there's a good chance it'll land in a region where scientists from San Diego County will do research this summer," wrote science editor Gary Robbins, adding that "summer field research is a cherished part of science." Case in point: the first researcher featured in the article was CSE Prof. Ryan Kastner, who is photographed (at right) displaying an underwater stereo camera system that he developed for producing 3D reconstructions of underwater objects. Kastner "will use high-resolution imaging tools in June to help archaeologists map Mayan ruins in Guatemala and a sunken ship in Lake Tahoe's Emerald Bay Underwater Park," according to the article. Kastner will be joined on the Lake Tahoe expedition by undergraduate participants in the Engineers for Exploration program, which is co-directed by Kastner.
CSE Prof. Yuanyuan (YY) Zhou (at left) took time out to attend the first-ever Celebration of Women in Computing in Southern California April 5-6 in Carlsbad. And according to a profile in the UCSD Guardian by Raquel Calderon, Zhou spent Saturday afternoon "standing before an audience full of young, aspiring female engineers to share her experiences.""With a doctorate in computer science from Princeton, three startups and volumes of research to her name, Zhou is intimidating in description but humble and talkative in person," wrote Calderon. "Her relentless energy serves her research, her company, her students and her desire to increse the pesence of women in computing."Explaining why she likes to teach and coach students in computer science, Zhou is quoted as saying, "I like how in a 30-minute conversation, you can really help them: you can change their career."“A really interesting observation I made is that many of the event organizers are women,” added Zhou, whose latest venture, Whova, a mobile app for attendees at professional conferences. “Especially in a startup [where] the most important thing is for you to understand your customer... you truly need to listen to the user.”According to the Guardian, "Zhou’s participation in CWIC-SoCal shows her belief in the power of community. She, like many well-established women in her field, believes technology and science need more women."
An article co-written by Cognitive Scientist and CSE Prof. Scott Klemmer (at right) has been read by nearly 30,000 people since it was first posted in late March on LinkedIn. The article, titled "State of Design: How Design Education Must Change," was written with Don Norman of the Nielsen Norman Group (and emeritus professor in Cognitive Science at UC San Diego). They argue that "if design is to live up to its promise it must create new, enduring curricula for design education that merge science and technology, art and business, and indeed, all the knowledge of the university." A major focus of their article is on the university. "To meet the challenges of the 21st century, design and design education must change," they write. "So too must universities." They add that "it would not be difficult for universities to change their evaluation process to encourage both specialists and generalists, in part by valuing broad synthesis, integration, and real-world impact when appropriate. This shift can enable world-class programs that celebrate both craft and theory, and trains students to augment depth with breadth to tackle the multifarious challenges we face."
Faculty Candidate: Stefanie Jegelka (UC Berkeley)
- Start time: 11:00am
- End date: Wednesday, April 16th
- End time: 12:00pm
- Where: CSE 1202
Efficient learning with combinatorial structure
Learning from complex data such as images, text or biological measurements invariably relies on capturing long-range, latent structure. But the combinatorial structure inherent in real-world data can pose significant computational challenges for modeling, learning and inference.
In this talk, I will view these challenges through the lens of submodular set functions. Considered a "discrete analog of convexity", the combinatorial concept of submodularity captures intuitive yet nontrivial dependencies between variables and underlies many widely used concepts in machine learning. Practical use of submodularity, however, requires care. My first example illustrates how to efficiently handle the important class of submodular composite models. The second example combines submodularity and graphs for a new family of combinatorial models that express long-range interactions while still admitting very efficient inference procedures. As a concrete application, our results enable effective realization of combinatorial sparsity priors on real data, significantly improving image segmentation results in settings where state-of-the-art methods fail.
Motivated by good empirical results, we provide a detailed theoretical analysis and identify practically relevant properties that affect complexity and approximation quality of submodular optimization and learning problems.
Stefanie Jegelka is a postdoc at UC Berkeley, working with Michael Jordan and Trevor Darrell. She is also a visitor at the International Computer Science Institute. Before coming to Berkeley, she did her PhD in Bernhard Schölkopf's group at the Max Planck Institutes in beautiful Tübingen and graduated from ETH Zurich. During her PhD, she worked with Jeff Bilmes, and before that with Ulrike von Luxburg and Arthur Gretton.
She likes to spend her brain cycles thinking about combinatorial problems in Machine Learning, in particular efficient (approximation) algorithms. Her interests include submodularity and discrete optimization, graph problems, graphical models, kernel methods and clustering, distributed machine learning, and applications e.g. in computer vision and biology.