In this tutorial, we’ll learn about the PyTorch tensor that is the fundamental unit for operations in creating neural network models in PyTorch. We will first understand the basic concept of tensors in general and then see different operations of tensors in PyTorch.

Tensor is nothing but a multi-dimensional matrix consisting of homogeneous data type values. They are ideal to represent real-world data, for example, an image can be represented in a 3-D Tensor with (height, width, color depth). And a dataset of N images can be represented in a 4-D Tensor as (N, height, width, color depth)

Tensors are useful to perform linear algebra operations efficiently, hence they are preferred for mathematical computation in neural networks for faster training. So it makes sense to represent your input data in form of tensors.

PyTorch Tensor

What is PyTorch Tensor

How to create a PyTorch Tensor?

The function for constructing a tensor is torch.tensor, it generally takes a python list or sequence as input.

import torch
import numpy as np

In the below method for creating tensor, we simply pass a python list to torch.tensor function.

torch.tensor([[15., -33.], [-81., -54.]])

tensor([[ 15., -33.],
        [-81., -54.]])

2. Converting NumPy Array to PyTorch Tensor

In this second method, we create the tensor from the NumPy array by passing it as input to the torch.tensor function.

torch.tensor(np.array([[45, 27, 63], [144, 549, 72]]))

tensor([[ 45,  27,  63],
        [144, 549,  72]])

Creating Different Types of PyTorch Tensors

In this section, we look at different types of methods for constructing tensors of zeros, tensor of ones, and also a tensor using random function consisting of normally distributed values.

All these types of tensors will be created by using torch.tensor function.

1. PyTorch Zeros Tensor : touch.zeros()

In a PyTorch zeros tensor, all values consist of zero only. This is created by passing the desired dimension to the torch.zeros function. We can also explicitly mention the data type which will produce the zeros tensor of that data type itself.

torch.zeros([3, 6], dtype=torch.int32)

tensor([[0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0]], dtype=torch.int32)

2. PyTorch Ones Tensor : touch.zeros()

In a PyTorch ones tensor, all values consist of ones only. Here we pass the dimension of the required ones tensor to the torch.ones function.

In this example, we are building the ones tensor using Cuda for leveraging GPU. For this, device parameter is passed to the torch.ones function. (P.S – We have used Cuda just for example, it is not required to use it for creating PyTorch ones tensor)

cuda0 = torch.device('cuda:0')

torch.ones([6, 3], dtype=torch.float64, device=cuda0)

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], device='cuda:0', dtype=torch.float64)

We can create a tensor of random values in PyTorch by using touch.randn function by passing the dimension of the required tensor. The values will be normally distributed values.

# Create PyTorch random tensor from normal distribution randoms
t = torch.randn(3, 7)

print(t)

tensor([[ 1.7181,  0.7267,  1.2087, -1.8400, -0.2532, -0.3306, -2.0244],
        [ 0.2714,  0.7380,  1.8189,  0.5172,  0.2464,  0.8678, -0.5271],
        [ 0.1699, -0.6631,  0.6733, -0.0450, -0.0325,  0.5516, -0.9363]])

How to access elements in PyTorch Tensor

Just like we can access elements in an array, we can access the values of tensors in PyTorch as well.

Here we first build a tensor using torch.tensor function.

x = torch.tensor([[45, 27, 63], [144, 549, 72]])

Here we are using indexing to access a particular index of values.

print(x[1][1])

tensor(549)

The below example shows how we can access a particular index value and then we can alter it.

x[0][1] = 999

print(x)

tensor([[ 45, 999,  63],
        [144, 549,  72]])

# Slicing
t = torch.Tensor([[45, 27, 63], [144, 549, 72], [162, 234, 96]])

# Every row, only the last column
print(t[:, -2])

# First 1 row, all columns
print(t[:1, :])

# Lower right most corner
print(t[-1:, -1:])

tensor([ 27., 549., 234.])
tensor([[45., 27., 63.]])
tensor([[96.]])

Mathematical Operations on Tensors in PyTorch

In this section, we’ll look at mathematical operations using tensor. First, we build the two different tensors in PyTorch.

# Tensor Operations
x = torch.tensor([[45, 27, 63], [144, 549, 72]])
y = torch.tensor([[4, 5, 9], [5.4, 6.3, 9.1]])

# Simple Addition
print(x + y)

# We can also use torch.add()
print(torch.add(x, y))

#Output of Simple Addition.
tensor([[ 49.0000,  32.0000,  72.0000],
        [149.4000, 555.3000,  81.1000]])

For subtracting two tensors in PyTorch can simply use plus operation or use torch.sub function.

 # Simple Subtraction
print(x - y)

# We can also use torch.sub()
print(torch.sub(x, y))

#Output of Simple Subtraction
tensor([[ 41.0000,  22.0000,  54.0000],
        [138.6000, 542.7000,  62.9000]])

3. Cross Product of PyTorch Tensors : cross()

In this example, we have generated two tensors using the torch randn function.

Here the cross product of tensors in PyTorch is calculated using cross() function.

# Compute cross product
t1 = torch.randn(5, 3)
t2 = torch.randn(5, 3)
t1.cross(t2)

tensor([[-0.5692, -0.7240,  0.0919],
        [-1.7224,  0.9906, -1.8802],
        [-3.8744, -1.7378, -0.5835],
        [-1.8263,  0.2717, -0.1073],
        [-2.3449, -0.2191,  2.1565]])

In this example, the matrix multiplication of the two tensors is calculated using mm function.

# Compute matrix product
t = (torch.Tensor([[2, 4], [5, 10]]).mm(torch.Tensor([[10], [20]])))
t

tensor([[100.],
        [250.]])

5. Elementwise Multiplication of PyTorch Tensors : mul()

Element wise multiplication of tensors in PyTorch is done using mul() function.

# Elementwise multiplication of PyTorch Tensors
t = torch.Tensor([[5, 15], [35, 49]])
t.mul(t)

tensor([[  25.,  225.],
        [1225., 2401.]])

Tensor View in PyTorch : view()

Different views of tensors in PyTorch are used for creating a subset or smaller version of the original tensor.

Here view() function is used for creating a view.

import torch 
p = torch.randn(2, 4)

q = p.view(2 * 4)
 
print('Size of p:', p.size())
print('Size of q:', q.size())
 
print(p)
print(q)

Size of p: torch.Size([2, 4])
Size of q: torch.Size([8])
tensor([[ 2.3676, -1.6299, -0.0381,  0.2296],
        [ 0.8290, -0.9862, -1.5253,  0.1887]])
tensor([ 2.3676, -1.6299, -0.0381,  0.2296,  0.8290, -0.9862, -1.5253,  0.1887])

The reshape function has the opposite functionality to the view function.

r = q.reshape(2, 4)
print(r)

tensor([[ 2.3676, -1.6299, -0.0381,  0.2296],
        [ 0.8290, -0.9862, -1.5253,  0.1887]])

PyTorch Tensor To and From Numpy ndarray

Finally, let us see another way of converting a NumPy array into tensor and then how to convert tensor to NumPy error.

1. PyTorch Tensor from NumPy Array : torch.from_numpy()

Let us first create a NumPy array –

# ndarray to tensor
a = np.random.randn(3, 5)

print(a)

print(type(a))

[[ 0.52478144  0.78796427  0.28085501 -0.54082428 -0.42829379]
 [-0.61056479  2.92031748  0.63349779  0.01459906 -0.45633183]
 [-1.96843356 -1.49749835  0.27994694  1.57212136  0.51226665]]
<class 'numpy.ndarray'>

The below function torch.from_numpy will be used for converting an array to a tensor.

t = torch.from_numpy(a)

print(t)

print(type(t))

tensor([[ 0.5248,  0.7880,  0.2809, -0.5408, -0.4283],
        [-0.6106,  2.9203,  0.6335,  0.0146, -0.4563],
        [-1.9684, -1.4975,  0.2799,  1.5721,  0.5123]], dtype=torch.float64)
<class 'torch.Tensor'>

Converting PyTorchTensor to Numpy Array : numpy()

In this case, we are converting a tensor into a NumPy array. Let us first create a tensor with random values –

# tensor to ndarray
t = torch.randn(3, 5)

print(t)

print(type(t))

tensor([[-1.1488,  0.5343,  1.1705,  0.2592,  1.6234],
        [ 0.2604, -0.4572, -0.6114,  1.4538, -0.4086],
        [ 0.8450,  1.1655,  1.4429,  0.2310, -0.6271]])
<class 'torch.Tensor'>

Now we will use numpy() function to convert the tensor to NumPy array.

a = t.numpy()

print(a)

print(type(a))

[[-1.1487507   0.53432685  1.1704543   0.25918666  1.623438  ]
 [ 0.2603834  -0.45724604 -0.61137927  1.453799   -0.40862954]
 [ 0.8449621   1.165458    1.442852    0.23103562 -0.627091  ]]
<class 'numpy.ndarray'>