National Academies Press: OpenBook
« Previous: 6 Session 5: Learning from Noisy, Adversarial Inputs
Suggested Citation:"7 Session 6: Learning from Social Media." National Academies of Sciences, Engineering, and Medicine. 2017. Challenges in Machine Generation of Analytic Products from Multi-Source Data: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24900.
×

7

Session 6: Learning from Social Media

LARGE-SCALE MULTI-MODAL DEEP LEARNING

Rob Fergus, New York University and Facebook

Rob Fergus, New York University and Facebook, described Facebook as a multi-modal platform with billions of images, posts, users, comments, and videos uploaded each day. Research is being conducted on a variety of deep learning approaches for handling such a heterogeneous data stream.

The first project Fergus discussed described strategies for handling multi-modal data. Using deep neural networks, World2Vec represents the many different entities found on Facebook as high-dimensional vectors in the same space. Fergus explained that these different types of data (e.g., users and pages) have relations between them, which are then graphed. A complete graph would have 1010 edges1 per day, so this method scales to very large data streams.

Fergus noted that when analyzing these large, noisy data streams, ambiguity creates challenges that can be addressed with these methods—for example, exploring the embedding space can reveal whether a post about “football” is discussing soccer or the National Football League. These methods also allow for Facebook page categorization. Fergus summarized that large-scale embedding methods are very powerful because common space can be used to represent many different types of data, and relationships between data can be represented as distance functions within that space. He explained that these embedding methods can be considered a continuous form of reasoning.

The second project Fergus introduced described ways to train deep models from scratch and thus learn from noisy labels, such as the metadata associated with images found on Flickr. Setting up a successful architecture requires (1) training convolutional neural networks to predict words that appear with an image, (2) treating the words in the comments as targets for the photos, and (3) training convolutional neural networks to predict the words from the images. Training is done using multi-class logistic loss over all image–word pairs and mini-batch stochastic gradient descent with class-uniform sampling so that the common uses do not dominate the training.

For the experimental setup, the networks are trained and then evaluated based on word prediction, transfer learning, and word embeddings. Fergus indicated that this “weak form of supervised training” has been successful; it is possible to learn value representations without explicit human annotation.

___________________

1 Edges are the links that connect the points of a graph.

Suggested Citation:"7 Session 6: Learning from Social Media." National Academies of Sciences, Engineering, and Medicine. 2017. Challenges in Machine Generation of Analytic Products from Multi-Source Data: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24900.
×

The third program Fergus described focused on training big models faster—specifically, training ImageNet in 1 hour. He discovered that combining more data with bigger models leads to higher accuracy, although the training time also increases in such a scenario. In distributed training, as the batch size increases, optimization issues arise, so the learning rate must be scaled linearly to ensure accuracy. Increasing the number of graphical processing units also allows a linear speed-up in the training time.

The fourth and final program that Fergus presented studied training on large data sets that do not fit on a single graphical processing unit. Though the previously discussed program addressed the issue of faster training, it did not address increasing the size of the model itself. A barrier to this is that the memory of a graphical processing unit is limited. The key solution, Fergus explained, is a “hard” mixture-of-experts approach, in which each example is directed to a single expert.

Rama Chellappa, University of Maryland, College Park, noted that the mixture of experts method has also been useful in Janus’s face verification research, as discussed in Chapter 3 of this proceedings. In response to a question from Kathy McKeown, Columbia University, Fergus explained that the representation could be optimized for a particular objective. Darrell Young, Facebook, asked Fergus how he would use fastText vectors between languages. Fergus thought it may be possible to do machine translation with fastText representation, though he was unsure precisely how.

Suggested Citation:"7 Session 6: Learning from Social Media." National Academies of Sciences, Engineering, and Medicine. 2017. Challenges in Machine Generation of Analytic Products from Multi-Source Data: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24900.
×
Page 26
Suggested Citation:"7 Session 6: Learning from Social Media." National Academies of Sciences, Engineering, and Medicine. 2017. Challenges in Machine Generation of Analytic Products from Multi-Source Data: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24900.
×
Page 27
Next: 8 Session 7: Humans and Machines Working Together with Big Data »
Challenges in Machine Generation of Analytic Products from Multi-Source Data: Proceedings of a Workshop Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $44.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Intelligence Community Studies Board of the National Academies of Sciences, Engineering, and Medicine convened a workshop on August 9-10, 2017 to examine challenges in machine generation of analytic products from multi-source data. Workshop speakers and participants discussed research challenges related to machine-based methods for generating analytic products and for automating the evaluation of these products, with special attention to learning from small data, using multi-source data, adversarial learning, and understanding the human-machine relationship. This publication summarizes the presentations and discussions from the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!