The Google Brain team is using deep learning to improve people's lives through long-term research, open-source systems, collaborations with Google and Alphabet, and programs for deep learning research, leading to advancements in machine learning that will improve productivity and the development of robots that can operate in messy environments and interact safely with humans.

• 🤖
  00:00

  Google Brain team uses deep learning to improve people's lives through long-term research, open-source systems, collaborations with Google and Alphabet, and programs for deep learning research.

  • Google Brain team uses deep learning to make machines intelligent and improve people's lives through long-term research, open-source systems, collaborations with Google and Alphabet, and programs for deep learning research.
  • Deep learning and neural networks are causing a shift in machine learning approaches due to the availability of much more compute, making neural nets the best solution for a growing set of problems.
  • Google's focus is on scaling machine learning models and infrastructure to reduce experimental turnaround time and enable deep learning to be used in various Google products.
  • TensorFlow is an open-source platform designed to establish a common ground for expressing all kinds of machine learning ideas, making it great for research and deployment in production settings.
  • TensorFlow is an open-source machine learning package that enables flexible research and production readiness with emphasis on speed and high performance.
  • TensorFlow's performance is good with nearly linear speed-up for different image models on up to 64 GPU cards and supports various platforms including mobile devices and custom machine learning accelerators.
• 🤖
  10:57

  TensorFlow is a popular machine learning platform used by top companies and universities, with computer vision models being applied to various domains for interesting product features.

  • TensorFlow supports multiple programming languages, with Python being the most developed, and has a broad usage base including companies like Apple, Qualcomm, and Google.
  • Machine learning platforms like TensorFlow have gained interest worldwide, with almost a thousand non-Google contributors, and are being used in machine learning classes at top universities.
  • Computer vision can be applied to different domains to create interesting product features by reusing the same basic model structure trained on different data sets.
  • The same computer vision model was used to identify text in Street View images, find rooftops for solar energy potential, and solve medical imaging problems.
  • Identifying symptoms of diabetic retinopathy in retinal images can be done through image classification with the help of ophthalmologists' labels.
  • A model that performs slightly better than the median of eight US board-certified ophthalmologists has been developed to reduce variance in image grading and improve access to eye care in areas with limited ophthalmologists.
• 🤖
  17:51

  Using multiple robots to practice grasping objects and pooling sensor data leads to better grasping mechanisms and models, while deep learning can be applied in various scientific domains to reduce computational expenses.

  • Experiments with real and simulated robotic environments and imitation learning have shown that clear perception of the world around robots is crucial for successful grasping and manipulation.
  • Using multiple robots to practice grasping objects and pooling the sensor data to retrain a model every night leads to better grasping mechanisms and models, as well as transferring actions from simulations to real robots.
  • Deep learning can be applied in various scientific domains by using simulators as training data for an OLAP, which can help in computational science methodology and reduce computational expenses.
  • Using a neural net trained on data from an expensive simulator, quantum chemistry tasks can be completed 300,000 times faster with indistinguishable accuracy.
  • Google has developed a model that predicts depth from an input image, which can be applied to various pixel to pixel learning problems, including predicting depth in portraits and using chemically stained microscope images as targets for the model.
  • Virtual staining allows for longitudinal observation of cell processes without killing the cells, and can also highlight structures that cannot be chemically stained.
• 🤖
  25:14

  Google's neural machine translation has improved translations to a human level.

  • Sequence models can be used for translation by taking input sentences one word at a time and predicting the next word using recurrent neural nets, followed by a beam search to find the most probable output sequence.
  • Smart reply feature uses a feed-forward neural net to predict plausible replies to incoming emails, reducing computational cost.
  • Google's Smart Reply feature, which generates 10% of mobile inbox replies, was initially an April Fool's joke and was later applied to Google Translate with a hundred to a thousand times more training data, resulting in a high-quality model with a deep lsdm stack.
  • The attention module allows for tracking all states and learning to pay attention to different parts of input data when generating output sequences, while data parallelism and shared parameters allow for quick scaling of training.
  • Old phrase-based machine translation system had large statistical models for different sub pieces of problem, while the new system has a substantial jump in quality and is only 500 lines of TensorFlow code.
  • Google's neural machine translation has improved the quality of translations to a human level, making it usable and natural sounding.
• 💻
  32:57

  Google is automating machine learning with neural architecture search, custom hardware, and pre-trained models available on Google Cloud.

  • Automating the solution of machine learning problems through learned to learn approach to eliminate the need for human machine learning experts.
  • Neural architecture search is a way of automatically designing neural architectures to tackle a particular problem without any human knowledge of the underlying architecture.
  • The speaker presented an architecture search that beat the state-of-the-art for language modeling and outperformed traditional update rules for neural optimizers.
  • Deep learning has two nice properties: it is tolerant of reduced precision arithmetic and made up of specific operations, which leads to an opportunity to build custom machine learning hardware for unlocking huge amounts of compute.
  • Google has designed a high-performance chip system called TPU, which is programmable via TensorFlow and will be available through Google Cloud, with 64 pods and 256 chips, providing 11.5 petaflops of compute, and a thousand of these devices will be available for free to researchers around the world.
  • Google Cloud offers pre-trained models and APIs for machine learning tasks, including vision and translation, and is experimenting with higher performance machine learning models using reinforcement learning.
• 🚀
  43:45

  Advancements in machine learning will lead to improved productivity and the development of robots that can operate in messy environments and interact safely with humans.

  • An RL algorithm can find the optimal placement of tensorflow operations on multiple devices, resulting in a 20% increase in speed compared to human expert placement.
  • Advancements in machine learning and increased computing power will lead to the ability to answer complex queries and improve productivity, as well as the development of robots that can operate in messy environments and interact safely with humans.
  • Deep nets are making big changes and it's important to pay attention to their impact.
  • The optimizer update rule in deep learning can be interpreted through the reoccurring sub expression of e to the sign of the gradient times the sign of the momentum, and the use of learn to learn framework can provide human insights into the architecture of neural networks.
  • There is enough labeled data in the world to train a single model on a single problem, but for small model exploration, it depends on the problem and architecture, and current generation GPUs are on the boundary of practical for tiny problems and making it practical for real problems at scale.
  • Building a single giant model that can do thousands of things will improve data efficiency and flexibility in solving new tasks with less data and time.
• 🤖
  53:42

  Train machine learning models with more data and similar architectures to solve multiple problems, but be prepared to experiment with hyperparameters and adapt to changing distributions.

  • Reducing the time spent on idea generation at the whiteboard can significantly improve workflow, and the accuracy of translation models can be improved by training them with more data.
  • The speaker suggests that while there is potential for improvement, supervised tasks with defined input and output and sufficient training data will likely work with the current algorithmic search architecture.
  • Train a model generating models to solve multiple problems and use similar architectures for new problems, with internal development cycles varying depending on the domain.
  • Machine learning models need to be able to adapt to changing distributions quickly, but some problems have more stable distributions than others, and a lot of machine learning research is still empirical.
  • To test an idea, it is necessary to explore hyperparameters and solve interesting problems, and while some ideas may have intuitive success, others require more experimentation.
• 🧠
  59:19

  Using medical notes to improve machine learning interactions, scaling neural nets, and combining automated learning with human-designed solutions can lead to better healthcare decisions.

  • Interpretability of machine learning models is important in some domains, particularly in healthcare, when delivering production models to non-experts.
  • Using medical notes to highlight specific symptoms can improve interactions between machine learning systems and humans, allowing them to play to their strengths.
  • Scaling neural nets with more compute and processors could potentially solve problems that were previously unsolvable, and training them to do multiple tasks could lead to more advanced reasoning abilities.
  • We need a model with a large number of parameters but only activate a small fraction of it for any given task, and memory networks are an interesting emerging area for accomplishing tasks.
  • Combining automated learning to learn systems with human-designed solutions can lead to a hybrid best of both worlds approach in solving interesting problems with machine learning.
  • Neural nets have the potential to make better healthcare decisions by ingesting a lot of data and predicting the most likely diagnoses, despite the heavily regulated environment and privacy issues.