Complete Guide to Tensors in Tensorflow.js

Complete Guide to Tensors in Tensorflow.js

Contents

Introduction

In this article, we will go through the tutorial for working with Tensors in Tensorflow.js deep learning library. We will first understand what is tensor and why it is used in the neural network. Then we will go through various tensor functions in tensorflow.js used for creating tensors, linear algebra operations, and memory management. All these will be covered with examples for easy understanding of beginners.

What are Tensors in Neural Network

Tensors are the core of mathematical computational units in deep learning that can be interpreted as an n-dimensional vector or a matrix with n-dimensions.

Each and every data point in a dataset is converted into some type of tensor for neural networks. They are ideal to represent real-world data, for example, an image can be represented in a 3-D Tensor with (height, width, color depth). And a dataset of N images can be represented in a 4-D Tensor as (N, height, width, color depth).

Tensors are used in deep learning because it is computationally efficient to perform linear algebra operations of neural network with tensors. In order to pass them through a neural network and learn, multiple complex transformations are applied to tensors that we will see in the rest of the article.

Types of tensors
Different tensors
A 3-d tensor
A 3-d tensor

Rank of Tensor

Tensors are usually classified using Rank which is nothing but the number of axes a tensor has. For example, A scalar number ‘4’ is a rank-0 tensor whereas a 1-D Array of numbers is a rank-1 tensor as it has numbers along one axis only. Similarly, a matrix is a rank-2 tensor as it has two axes which are its rows and its column.

Ad
Deep Learning Specialization on Coursera

Example of Tensors

 4 #rank zero tensor (Scalar)

[1.0,2.0,3.0] #An array of numbers is a rank-1 tensor

[[1,2],       # A (3,2) matrix is a rank-2 tensor since it has two dimensions
[2,3],
[3,4]]

Difference between a vector and a tensor

Vectors are mathematical quantities that possess magnitude as well as direction whereas tensors are generalizations of mathematical objects. Scalars and vectors both can and are represented as tensors in various cases. To be precise a scalar is a tensor of rank 0 and a vector is a tensor of rank 1. So one can say that every vector is a type of a tensor but every tensor is not a vector.

1. How To Create Tensors in Tensorflow.js

Tensors in Tensorflow.js
Types of tensors

Prerequisites

Add this line to your HTML in order to use tensorflow.js in the browser.

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@2.0.0/dist/tf.min.js"></script>

i) Scalar in Tensorflow.js : tf.scalar()

In order to create a rank-0 tensor (scalar) tf.scalar() function of tensorflow.js is used.

Syntax:

tf.scalar(value,dtype);
value:The value of the scalar
dtype: Data type of the scalar(optional)

Example

tf.scalar(4.0);

ii) 1-D Tensor in Tensorflow.js : tf.tensor1d()

In tensorflow.js tf.tensor1d() function is used to create a tensor of rank 1.

Syntax:

tf.tensor1d(values,dtype);
values: Array of values
dtype: Data type (optional)

Example

tf.tensor1d([1,2,3,4]);

iii) 2-D Tensor in Tensorflow.js : tf.tensor2d()

tf.tensor2d() is used to create a tensor of rank 2 in tensorflow.js.

Syntax:

tf.tensor2d(values,shape,dtype); 
values: Array of values 
shape: shape of the tensor
dtype: Data type (optional)

Example

Here two rows are created with two columns each hence two dimensions.

tf.tensor2d([[6,9],[9,6]]);

iv) 3-D tensor in Tensorflow.js : tf.tensor3d()

tf.tensor3d() is used to create a tensor of rank 3 in tensorflow.js. It can be used to represent a dataset of images.

Syntax:

tf.tensor3d(values,shape,dtype); 
values: Array of values 
shape: shape of the tensor
dtype: Data type (optional)

Example

tf.tensor3d([[[1], [2]], [[3], [4]]]);

v) 4-D Tensor in Tensorflow.js : tf.tensor4d()

In tensorflow.js tf.tensor4d() is used to initialize a tensor of rank 4.

Syntax:

Syntax:tf.tensor4d(values,shape,dtype); 
values: Array of values 
shape: shape of the tensor
dtype: Data type (optional)

Example

Here a tensor with one number in each dimension is created i.e. 1 in the first,2 in the second, and so on.

tf.tensor4d([[[[1],
       [2]],
      [[3],
       [4]]]])

 

vi) General tensor in Tensorflow.js : tf.tensor()

tf.tensor() is used to create a general tensor of a specified shape. The shape attribute decides the rank of the tensor.

Syntax:

tf.tensor(values,shape,dtype);
values: An array of values of the tensor
shape: The shape the tensor is supposed to take(Optional:If not provided shape is as is)

Example

tf.tensor([1,2,3,4],[2,2]);

2. Mathematical Tensor operations in Tensorflow.js

Mathematical Tensor operations in Tensorflow.js
Tensor operations

i) Tensor addition : tf.add()

In order to perform element-wise addition of two tensors in tensorflow.js tf.add() function is used.

const a = tf.tensor1d([1,2,3,4]);
const b = tf.tensor1d([4,3,2,1]);
a.add(b).print()

Output:

Tensor
    [5, 5, 5, 5]

ii) Tensor subtraction : tf.sub()

tf.sub() is tensorflow.js function to perform element-wise subtraction of two tensors.

const a = tf.tensor1d([1,2,3,4]);
const b = tf.tensor1d([4,3,2,1]);
a.sub(b).print()

Output:

Tensor
    [-3, -1, 1, 3]

iii) Tensor multiplication : tf.mul()

Element-wise multiplication of two tensors can be done with tf.mul() in tensorflow.js

const a = tf.tensor1d([1,2,3,4]);
const b = tf.tensor1d([4,3,2,1]);
a.mul(b).print() // or tf.mul(a,b)

Output:

Tensor
    [4, 6, 6, 4]

iv) Tensor division : tf.div()

Element -wise division of two tensors is done with tf.div() in tensorflow.js

const a = tf.tensor1d([1,2,3,4]);
const b = tf.tensor1d([4,3,2,1]);
a.div(b) // or tf.div(a,b)
# Returns [0.25, 0.6666666, 1.5, 4] as a tensor

Output:

Tensor
    [0.25, 0.6666666, 1.5, 4]

v) Dot operation on two tensors : tf.dot()

tf.dot() perform dot operation (element-wise multiplication and addition) on two tensors in tensorflowjs

const a = tf.tensor1d([1,2,3,4]);
const b = tf.tensor1d([4,3,2,1]);
const c=a.dot(b);  // or tf.dot(a, b)

Output:

Tensor [20]

vi) Matrix multiplication : tf.matMul()

tf.matMul() is used for the multiplication of two matrices (rank-2 tensors). Here the number of columns in the first matrix must be equal to the number of rows in the second matrix.

const a = tf.tensor2d([1,2],[1,2]);
const b = tf.tensor2d([1,2,3,4],[2,2]);
const c = a.matMul(b);  // or tf.matMul(a, b)

Output:

Tensor
 [[7,10],]

vi) Transpose of a Tensor : tf.transpose()

tf.transpose() performs transpose operation on a given tensor in tensorflow.js

const a = tf.tensor([1,2,3,4],[2,2]);
a.print() 
a.transpose().print()

Output:

Tensor      
  [[1,2],[3,4]]   #Before transpose
Tensor
 [[1,3],[2,4]]    #After transpose

3. Tensor Utility functions in Tensorflow.js

i) Converting an array into a dataset: tf.data.array()

This function tf.data.array() is used to directly create a dataset from an array of elements or objects. Takes in an array as input and converts them into a dataset object.

const a = tf.data.array([4, 5, 6]); # It can convert an array of elements
const a = tf.data.array([{'item': 1}, {'item': 2}, {'item': 3}]); # Or an array of objects
await a.forEachAsync(e => console.log(e));
# returns a tf.data.Dataset object

ii) Loading data from a CSV file : tf.data.csv()

Lets us use a CSV file to load data to the mainframe. tf.data.csv() needs to be used inside an async function as fetching data can take time and it is required to tell javascript to complete execution before moving forward.

const csvUrl =
'https://storage.googleapis.com/tfjs-examples/multivariate-linear-regression/data/boston-housing-train.csv';

async function run() {
   // We want to predict the column "medv", which represents a median value of
   // a home (in $1000s), so we mark it as a label.
   const csvDataset = tf.data.csv(
     csvUrl, {
       columnConfigs: {
         medv: {
           isLabel: true
         }
       }
     });

iii) One Hot Encoding in Tensorflow.js : tf.oneHot()

tf.oneHot() is used to one-hot encode tensors in tensorflow.js. It is very important for pre-processing categorical data. It is a type of binary encoding where the feature is represented as a vector of values in which the presence of a value is denoted by 1 otherwise it’s zero.

Syntax:

tf.oneHot(tensor to be one hot encoded,number of distinct values)

Example

const z=tf.tensor1d([0,1,2,3], 'int32')
tf.oneHot(z, 4).print();

Output:

Tensor
    [[1, 0, 0, 0],
     [0, 1, 0, 0],
     [0, 0, 1, 0],
     [0, 0, 0, 1]]

iv) Evenly spaced sequence : tf.linspace()

tf.linspace() is tensorflow.js function that returns an equally spaced sequence of numbers for a given interval.

Syntax:

Syntax:tf.linspace(start,stop,number of values)
start: The start value
stop: The end value
number of values: Number of equally spaced values to be generated

Example

tf.linspace(0,5,10).print();

Output:

Tensor
    [0,0.5,1,1.5,2,2.5,3,3.5,4,4.5,5]

v) Ones tensor in Tensorflow.js : tf.ones()

In tensorflow.js the tf.ones() is a function to initialize a tensor of specified shape, fully populated with ones. The only argument is the shape of the tensor.

Example

tf.ones([3,2]).print()

Output:

Tensor
    [[1, 1],
     [1, 1],
     [1, 1]]

vi) Zeros Tensor in Tensorflow.js : tf.zeros()

In tensorflow.js tf.zeros() is used to initialize a tensor of specified shape, fully populated with zeros. It takes in the shape of the tensor to be created.

Example

tf.zeros([2,3]).print()

Output:

Tensor
    [[0, 0, 0],
     [0, 0, 0]]

4. Memory management of Tensors in Tensorflow.js

Memory management
Memory management

Training a neural network takes computation and memory both. And doing it in the web browser also has some drawbacks. One of them is garbage collection, tensors don’t clean up after themselves i.e. they are not automatically garbage collected and can result in the browser drawing a large amount of memory. Thus there are some functions used to rectify the situation.

i) tf.tidy()

tf.tidy() is a cleaner function used as a callback. It cleans up tensors as soon as the tidy function body is done executing.

Example

const y = tf.tidy(() => {
   //a and one will be cleaned up when the tidy ends.
   const one = tf.scalar(1);
   const a = tf.scalar(2);         
   // value of a will be returned to y after the tidy function is done executing
   return a;});

ii) tf.dispose()

tf.dispose() is the dispose function for manual memory free-up in tensorflow.js. It takes in an argument of tensor objects that need to be deleted.

Example

const one = tf.scalar(1);
tf.dispose(one);

 

Conclusion

Coming to the end of this article, where we went through various tensor functions in tensorflow.js used for creating tensors, linear algebra operations, and memory management. All these were covered with examples for easy understanding of beginners.

Reference: Tensorflow.js Documentation

LEAVE A REPLY

Please enter your comment!
Please enter your name here