Tensorflow Overview
What does TensorFlow offer?
- Its core is very similar to NumPy, but with GPU support.
- It supports distributed computing (across multiple devices and servers).
- It includes a kind of just-in-time (JIT) compiler that allows it to optimize computations for speed and memory usage. It works by extracting the computation graph from a Python function, then optimizing it (e.g., by pruning unused nodes), and finally running it efficiently (e.g., by automatically running independent operations in parallel).
- Computation graphs can be exported to a portable format, so you can train a TensorFlow model in one environment (e.g., using Python on Linux) and run it in another (e.g., using Java on an Android device).
- It implements autodiff and provides some excellent optimizers so you can easily minimize all sorts of of loss functions.
At the lowest level, each TensorFlow operation (op for short) is implemented using highly efficient C++ code. Many operations have multiple implementations called kernels: each kernel is dedicated to a specific device type, such as CPUs, GPUs, or even TPUs (tensor processing units). GPUs can dramatically speed up computations by splitting them into many smaller chuncks and running them in parallel across many GPU threads. TPUs are even faster: they are custom ASIC chips built specifically for Deep Learning operations.
High-level Deep Learning APIs
Low-level Deep Learning APIs
tf.nn
tf.losses
tf.metrics
tf.optimizers
tf.train
tf.initializers
Autodiff
I/O and Preprocessing
Visualization with TensorBoard
Deployment and Optimization
tf.distribute
tf.saved_model
tf.autograph
tf.graph_util
tf.lite
tf.quantization
tf.tpu
tf.xla