Transitioning to TensorFlow 1.0
The APIs in TensorFlow 1.0 have changed in ways that are not all backwards compatible. That is, TensorFlow programs that worked on TensorFlow 0.n won't necessarily work on TensorFlow 1.0. We have made this API changes to ensure an internally-consistent API, and do not plan to make backwards-breaking changes throughout the 1.N lifecycle.
This guide walks you through the major changes in the API and how to automatically upgrade your programs for TensorFlow 1.0. This guide not only steps you through the changes but also explains why we've made them.
How to upgrade
If you would like to automatically port your code to 1.0, you can try our
tf_upgrade.py
script. While this script handles many cases, manual changes
are sometimes necessary.
Get this script from our
GitHub tree.
To convert a single 0.n TensorFlow source file to 1.0, enter a command of the following format:
$ python tf_upgrade.py --infile InputFile --outfile OutputFile
For example, the following command converts a 0.n TensorFlow
program named test.py
to a 1.0 TensorFlow program named test_1.0.py
:
$ python tf_upgrade.py --infile test.py --outfile test_1.0.py
The tf_upgrade.py
script also generates a file named report.txt
, which
details all the changes it performed and makes additional suggestions about
changes you might need to make manually.
To upgrade a whole directory of 0.n TensorFlow programs to 1.0, enter a command having the following format:
$ python tf_upgrade.py --intree InputDir --outtree OutputDir
For example, the following command converts all the 0.n TensorFlow programs
in the /home/user/cool
directory, creating their 1.0 equivalents in
the /home/user/cool_1.0
directory:
$ python tf_upgrade.py --intree /home/user/cool --outtree /home/user/cool_1.0
Limitations
There are a few things to watch out for. Specifically:
- You must manually fix any instances of
tf.reverse()
. Thetf_upgrade.py
script will warn you abouttf.reverse()
in stdout and in thereport.txt
file. - On reordered arguments,
tf_upgrade.py
tries to minimally reformat your code, so it cannot automatically change the actual argument order. Instead,tf_upgrade.py
makes your function invocations order-independent by introducing keyword arguments. Constructions like
tf.get_variable_scope().reuse_variables()
will likely not work. We recommend deleting those lines and replacing them with lines such as the following:with tf.variable_scope(tf.get_variable_scope(), reuse=True): ...
Analogously to
tf.pack
andtf.unpack
, we're renamedTensorArray.pack
andTensorArray.unpack
toTensorArray.stack
andTensorArray.unstack
. However,TensorArray.pack
andTensorArray.unpack
cannot be detected lexically since they are indirectly related to thetf
namespace e.g.foo = tf.TensorArray(); foo.unpack()
Upgrading your code manually
Instead of running tf_upgrade.py
, you may manually upgrade your code.
The remainder of this document provides a comprehensive list of
all backward incompatible changes made in TensorFlow 1.0.
Variables
Variable functions have been made more consistent and less confusing.
tf.VARIABLES
- should be renamed to
tf.GLOBAL_VARIABLES
- should be renamed to
tf.all_variables
- should be renamed to
tf.global_variables
- should be renamed to
tf.initialize_all_variables
- should be renamed to
tf.global_variables_initializer
- should be renamed to
tf.initialize_local_variables
- should be renamed to
tf.local_variables_initializer
- should be renamed to
tf.initialize_variables
- should be renamed to
tf.variables_initializer
- should be renamed to
Summary functions
Summary functions have been consolidated under the tf.summary
namespace.
tf.audio_summary
- should be renamed to
tf.summary.audio
- should be renamed to
tf.contrib.deprecated.histogram_summary
- should be renamed to
tf.summary.histogram
- should be renamed to
tf.contrib.deprecated.scalar_summary
- should be renamed to
tf.summary.scalar
- should be renamed to
tf.histogram_summary
- should be renamed to
tf.summary.histogram
- should be renamed to
tf.image_summary
- should be renamed to
tf.summary.image
- should be renamed to
tf.merge_all_summaries
- should be renamed to
tf.summary.merge_all
- should be renamed to
tf.merge_summary
- should be renamed to
tf.summary.merge
- should be renamed to
tf.scalar_summary
- should be renamed to
tf.summary.scalar
- should be renamed to
tf.train.SummaryWriter
- should be renamed to
tf.summary.FileWriter
- should be renamed to
Numeric differences
Integer division and tf.floordiv
now uses flooring semantics. This is to
make the results of np.divide
and np.mod
consistent with tf.divide
and
tf.mod
, respectively. In addition we have changed the rounding algorithm
used by tf.round
to match NumPy.
tf.div
The semantics of
tf.divide
division have been changed to match Python semantics completely. That is,/
in Python 3 and future division mode in Python 2 will produce floating point numbers always,//
will produce floored division. However, eventf.div
will produce floored integer division. To force C-style truncation semantics, you must usetf.truncatediv
.Consider changing your code to use
tf.divide
, which follows Python semantics for promotion.
tf.mod
- The semantics of
tf.mod
have been changed to match Python semantics. In particular, flooring semantics are used for integers. If you wish to have C-style truncation mod (remainders), you can usetf.truncatemod
- The semantics of
The old and new behavior of division can be summarized with this table:
Expr | TF 0.11 (py2) | TF 0.11 (py3) | TF 1.0 (py2) | TF 1.0 (py3) |
---|---|---|---|---|
tf.div(3,4) | 0 | 0 | 0 | 0 |
tf.div(-3,4) | 0 | 0 | -1 | -1 |
tf.mod(-3,4) | -3 | -3 | 1 | 1 |
-3/4 | 0 | -0.75 | -1 | -0.75 |
-3/4tf.divide(-3,4) | N/A | N/A | -0.75 | -1 |
The old and new behavior of rounding can be summarized with this table:
Input | Python | NumPy | C++ round() | TensorFlow 0.11(floor(x+.5)) | TensorFlow 1.0 |
---|---|---|---|---|---|
-3.5 | -4 | -4 | -4 | -3 | -4 |
-2.5 | -2 | -2 | -3 | -2 | -2 |
-1.5 | -2 | -2 | -2 | -1 | -2 |
-0.5 | 0 | 0 | -1 | 0 | 0 |
0.5 | 0 | 0 | 1 | 1 | 0 |
1.5 | 2 | 2 | 2 | 2 | 2 |
2.5 | 2 | 2 | 3 | 3 | 2 |
3.5 | 4 | 4 | 4 | 4 | 4 |
NumPy matching names
Many functions have been renamed to match NumPy. This was done to make the transition between NumPy and TensorFlow as easy as possible. There are still numerous cases where functions do not match, so this is far from a hard and fast rule, but we have removed several commonly noticed inconsistencies.
tf.inv
- should be renamed to
tf.reciprocal
- This was done to avoid confusion with NumPy's matrix inverse
np.inv
- should be renamed to
tf.list_diff
- should be renamed to
tf.setdiff1d
- should be renamed to
tf.listdiff
- should be renamed to
tf.setdiff1d
- should be renamed to
tf.mul
- should be renamed to
tf.multiply
- should be renamed to
tf.neg
- should be renamed to
tf.negative
- should be renamed to
tf.select
- should be renamed to
tf.where
tf.where
now takes 3 arguments or 1 argument, just likenp.where
- should be renamed to
tf.sub
- should be renamed to
tf.subtract
- should be renamed to
NumPy matching arguments
Arguments for certain TensorFlow 1.0 methods now match arguments in certain
NumPy methods. To achieve this, TensorFlow 1.0 has changed keyword arguments
and reordered some arguments. Notably, TensorFlow 1.0 now uses axis
rather
than dimension
. TensorFlow 1.0 aims to keep the tensor argument first on
operations that modify Tensors. (see the tf.concat
change).
tf.argmax
- keyword argument
dimension
should be renamed toaxis
- keyword argument
tf.argmin
- keyword argument
dimension
should be renamed toaxis
- keyword argument
tf.concat
- keyword argument
concat_dim
should be renamed toaxis
- arguments have been reordered to
tf.concat(values, axis, name='concat')
.
- keyword argument
tf.count_nonzero
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.expand_dims
- keyword argument
dim
should be renamed toaxis
- keyword argument
tf.reduce_all
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_any
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_join
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_logsumexp
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_max
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_mean
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_min
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_prod
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reduce_sum
- keyword argument
reduction_indices
should be renamed toaxis
- keyword argument
tf.reverse
tf.reverse
used to take a 1Dbool
tensor to control which dimensions were reversed. Now we use a Tensor of axis indices.- For example
tf.reverse(a, [True, False, True])
now must betf.reverse(a, [0, 2])
tf.reverse_sequence
- keyword argument
batch_dim
should be renamed tobatch_axis
- keyword argument
seq_dim
should be renamed toseq_axis
- keyword argument
tf.sparse_concat
- keyword argument
concat_dim
should be renamed toaxis
- keyword argument
tf.sparse_reduce_sum
- keyword argument
reduction_axes
should be renamed toaxis
- keyword argument
tf.sparse_reduce_sum_sparse
- keyword argument
reduction_axes
should be renamed toaxis
- keyword argument
tf.sparse_split
- keyword argument
split_dim
should be renamed toaxis
- arguments have been reordered to
tf.sparse_split(keyword_required=KeywordRequired(), sp_input=None, num_split=None, axis=None, name=None, split_dim=None)
.
- keyword argument
tf.split
- keyword argument
split_dim
should be renamed toaxis
- keyword argument
num_split
should be renamed tonum_or_size_splits
- arguments have been reordered to
tf.split(value, num_or_size_splits, axis=0, num=None, name='split')
.
- keyword argument
tf.squeeze
- keyword argument
squeeze_dims
should be renamed toaxis
- keyword argument
tf.svd
- arguments have been reordered to
tf.svd(tensor, full_matrices=False, compute_uv=True, name=None)
.
- arguments have been reordered to
Simplified math variants
Batched versions of math operations have been removed. Now the functionality is
contained in the non-batched versions. Similarly,tf.complex_abs
has had its
functionality moved to tf.abs
tf.batch_band_part
- should be renamed to
tf.band_part
- should be renamed to
tf.batch_cholesky
- should be renamed to
tf.cholesky
- should be renamed to
tf.batch_cholesky_solve
- should be renamed to
tf.cholesky_solve
- should be renamed to
tf.batch_fft
- should be renamed to
tf.fft
- should be renamed to
tf.batch_fft3d
- should be renamed to
tf.fft3d
- should be renamed to
tf.batch_ifft
- should be renamed to
tf.ifft
- should be renamed to
tf.batch_ifft2d
- should be renamed to
tf.ifft2d
- should be renamed to
tf.batch_ifft3d
- should be renamed to
tf.ifft3d
- should be renamed to
tf.batch_matmul
- should be renamed to
tf.matmul
- should be renamed to
tf.batch_matrix_determinant
- should be renamed to
tf.matrix_determinant
- should be renamed to
tf.batch_matrix_diag
- should be renamed to
tf.matrix_diag
- should be renamed to
tf.batch_matrix_inverse
- should be renamed to
tf.matrix_inverse
- should be renamed to
tf.batch_matrix_solve
- should be renamed to
tf.matrix_solve
- should be renamed to
tf.batch_matrix_solve_ls
- should be renamed to
tf.matrix_solve_ls
- should be renamed to
tf.batch_matrix_transpose
- should be renamed to
tf.matrix_transpose
- should be renamed to
tf.batch_matrix_triangular_solve
- should be renamed to
tf.matrix_triangular_solve
- should be renamed to
tf.batch_self_adjoint_eig
- should be renamed to
tf.self_adjoint_eig
- should be renamed to
tf.batch_self_adjoint_eigvals
- should be renamed to
tf.self_adjoint_eigvals
- should be renamed to
tf.batch_set_diag
- should be renamed to
tf.set_diag
- should be renamed to
tf.batch_svd
- should be renamed to
tf.svd
- should be renamed to
tf.complex_abs
- should be renamed to
tf.abs
- should be renamed to
Misc Changes
Several other changes have been made, including the following:
tf.image.per_image_whitening
- should be renamed to
tf.image.per_image_standardization
- should be renamed to
tf.nn.sigmoid_cross_entropy_with_logits
- arguments have been reordered to
tf.nn.sigmoid_cross_entropy_with_logits(_sentinel=None, labels=None, logits=None, name=None)
.
- arguments have been reordered to
tf.nn.softmax_cross_entropy_with_logits
- arguments have been reordered to
tf.nn.softmax_cross_entropy_with_logits(_sentinel=None, labels=None, logits=None, dim=-1, name=None)
.
- arguments have been reordered to
tf.nn.sparse_softmax_cross_entropy_with_logits
- arguments have been reordered to
tf.nn.sparse_softmax_cross_entropy_with_logits(_sentinel=None, labels=None, logits=None, name=None)
.
- arguments have been reordered to
tf.ones_initializer
- should be changed to a function call i.e.
tf.ones_initializer()
- should be changed to a function call i.e.
tf.pack
- should be renamed to
tf.stack
- should be renamed to
tf.round
- The semantics of
tf.round
now match Banker's rounding.
- The semantics of
tf.unpack
- should be renamed to
tf.unstack
- should be renamed to
tf.zeros_initializer
- should be changed to a function call i.e.
tf.zeros_initializer()
- should be changed to a function call i.e.