Tensorflow 2.0

[Problems of TF 1.x]

  • TF coding style : "Define and Run" : Building computational graph first, then flowing data VS classical Python coding style : "Define by Run" : Building computational graph while flowing data. (Pytorch style)
  • Complex layer declaration
  • Difficult debugging ! (Debugger indicates TF internal codes)
  • Training loop configuration is formalized. but Do I need to code again and again?
  • TF has too many high level APIs. This makes confusion. for instance, tf.layers (To build various types of ML layers) vs tf.estimator (To train and evaluate models) seem to be very similar -> is replaced by tf.keras (since TF v1.4, 2018 Mar) : Given by the keras grammar and TF native binding, from easy layer definition, to easy training and evaluation.

[Three Keys Toward TF 2.0]

  • Writing eagerly - Defined by run
    • Autograph + default eager mode
    • Graph mode (even you can't use 'if' code
    • Eager mode (no graph, more convinient but less efficient for computation)
    • Combine Graph mode + Eager mode
    • AutoGraph  = Write Eager-style code, generate graphs anyway
    • Code Eager style, then AutoGraph build Graph mode style innerly

  • Fully compatible tf.keras
    • High-level APIs for modeling, training and evaluation
    • use tf.data with tf.keras
    • Export model in SavedModel
    • Build your custom model by subclassing API
    • Very simple distributed strategy to use TPU, MultiGPU and Multi-nodes
  • Better distributed strategy
    • Easier multi- GPU and -nodes utilization under tf.keras

[In a nutshell]

  • High-level : tf.keras
  • High-level custom : tf.keras + subclass + eager
  • Low-level : python-style code + autograph + eager
  • "Eager +autograph" gives you back the "define by run" coding style
  • Unified high level APIs, "tf.keras" for beginners and experts
  • "Distributed strategy" for Multi-GPU