Every year, the Summer Undergraduate Research Fellowships (SURF) program allows undergraduates the opportunity to conduct independent scientific research with a professor. SURF projects cover a broad range of disciplines and fields at Caltech, and this past summer, 321 of Caltech undergraduates SURFed. Two of them sat down with us to discuss their research.
Teaching computers to play games
Jagriti Agrawal, senior in computer science
Yisong Yue, assistant professor of computing and mathematical sciences
Jagriti Agrawal's SURF research aimed to discover whether or not a method called imitation learning could be applied to "teach" a computer how to play a 1984 Atari game called Montezuma's Revenge.
"Computers are really good at playing games in which you perform an action and get an immediate reward, such as chess," Agrawal says. "In Montezuma's Revenge, you have to take actions that will reward you in the long run, such as taking detours to pick up a key that you will later use to open a door. Computers that have tried to play this game have used a technique called random exploration, which looks at rewards and takes the best action at each step and is exponential in time. We hope that through imitation learning—learning through human demonstration—the computer can mimic human play and achieve a better play-through more efficiently."
Imitation learning hinges on a person recording their completion of a complicated task, such as playing a game or guiding a robotic arm to move like a real human arm and giving the so-called "training data" to a computer. The computer should then "learn" which actions are optimal in different scenarios.
"Humans learn first by imitation of others and then by self-guided exploration," says Yisong Yue, assistant professor of computing and mathematical sciences and Agrawal's SURF mentor. "In artificial intelligence, the latter has been the predominant approach. But our thought is that in the case of more complicated games, the computer can first learn from the experts—us."
During Agrawal's play-through of the game, she created a series of paired images and commands. "Given any image of a point in the game representing the current game state, I recorded the corresponding best action to take—move left, right, up, down, and so on. So if the computer sees an image of a ladder, it will look at what I did when I encountered this ladder, and it will perform the action that I did."
Their findings are still preliminary. "Given the current amount of data collected, we find that our convolutional neural network tends to over-fit and not generalize well to new game scenarios," Agrawal says. "For example, the network tends to output the 'do nothing' action a disproportionate fraction of the time. Further investigation and data processing will need to be done to address this issue."
Agrawal plans to continue to work on the project throughout the school year. "The next step will be to collect a more diverse set of gameplay from a wider range of human demonstrators," Yue says. "Over the summer, Jagriti developed a pipeline to collect and process human demonstration data, and she will be deploying this at scale during the school year—we may even hold a pizza party so Caltech undergrads can play video games for science!"
Agrawal was a 2016 Rose Hills Foundation SURF Fellow. The Rose Hills Foundation SURF supports 15 Southern California students each summer.
Domestic violence in 19th-century English literature
Grace Lee, senior in applied mathematics and English
Kevin Gilmartin, professor of English and dean of undergraduate students
Grace Lee, a senior double majoring in applied mathematics and English, studied the shifting attitudes toward spousal abuse in 19th-century Britain for her SURF project.
The mid-1800s in England, during the Industrial Revolution, was a period of rapid cultural development, particularly in the area of women's rights. "As a modern woman and feminist, I was interested in learning more about women's rights of the period and, to an extent, what kind of legal changes took place to progress toward the rights women enjoy today," Lee says.
During this period, it was taboo to publicly discuss domestic violence, so authors and illustrators had to communicate their views on the topic symbolically through their publications. By studying works of fiction and satirical newspapers from the 19th century, Lee aimed to understand attitudes toward domestic violence in England.
"Fiction couldn't be published if its focus was solely domestic violence, that would be too controversial," she says. "Violence was often portrayed in a foil character to the main character, such as the character of Isabella in Wuthering Heights or Nancy in Oliver Twist. The gender of the authors is interesting to consider as well. While they both portray domestic violence as wrong, Charles Dickens portrays Nancy as a kind of martyr, he glorifies her subservience and sacrifice. Emily Brontë, on the other hand, shows Isabella as having accidentally entered into an oppressive marriage but still finding her own voice and power."
Violent, sardonic comics in satirical magazines played a role in illustrating the problem of domestic violence to the public. The idea evolved that domestic violence was contradictory to the English ideal of a gentlemanly person.
"To understand the feminist movement of the time, I had to understand its counterpart—what Victorian people thought to be masculine," Lee says. "The public's perception of masculinity underwent a change in the mid-1800s. This made it possible for women's rights under the law to change and expand as well."
The experience of doing research into English literature and history was new and challenging to Lee.
"This SURF was a great opportunity and a lot more challenging than doing a paper for a class because I had so much freedom to explore supplementary materials," she says. "I think the background that I gained in doing independent research will really help in writing my senior thesis."
Lee was the 2016 J. Kent Clark SURF Fellow. This particular endowment was established to support research in the humanities and is named after the late J. Kent Clark, Caltech professor of literature.
Written by Lorinda Dajose