Introduction
In this tutorial, we’ll learn about the PyTorch tensor that is the fundamental unit for operations in creating neural network models in PyTorch. We will first understand the basic concept of tensors in general and then see different operations of tensors in PyTorch.
What is Tensor and why they are used in Neural Network?
Tensor is nothing but a multi-dimensional matrix consisting of homogeneous data type values. They are ideal to represent real-world data, for example, an image can be represented in a 3-D Tensor with (height, width, color depth). And a dataset of N images can be represented in a 4-D Tensor as (N, height, width, color depth)
Tensors are useful to perform linear algebra operations efficiently, hence they are preferred for mathematical computation in neural networks for faster training. So it makes sense to represent your input data in form of tensors.
What is PyTorch Tensor
PyTorch tensor is the fundamental unit of the PyTorch framework whose operations are similar to Python NumPy arrays. You could have very well used the NumPy array to perform linear algebra operations for the neural network but it can only be done on CPU. But tensors in PyTorch can work with Cuda GPU as well thus giving that nitro boost for training large neural networks.
Syntax
We have the following syntax for creating a tensor in PyTorch.
torch.tensor(data, *, dtype=None, device=None, requires_grad=False, pin_memory=False)
Parameters
data (array_like) – This is the data provided to the tensor initially. It can be provided in various data types like tuple, scalar, etc.
Keyword Arguments
- dtype (torch.dtype, optional) – This is the data type desired for output.
- device (torch.device, optional) – The device desired for result returned.
- requires_grad (bool, optional) – This is to specify whether autograd should record operations or not.
- pin_memory (bool, optional) – If set to true, returned tensor will be stored in pinned memory.
How to create a PyTorch Tensor?
The function for constructing a tensor is torch.tensor, it generally takes a python list or sequence as input.
import torch
import numpy as np
1. Converting Python List to PyTorch Tensor
In the below method for creating tensor, we simply pass a python list to torch.tensor function.
torch.tensor([[15., -33.], [-81., -54.]])
2. Converting NumPy Array to PyTorch Tensor
In this second method, we create the tensor from the NumPy array by passing it as input to the torch.tensor function.
torch.tensor(np.array([[45, 27, 63], [144, 549, 72]]))
Creating Different Types of PyTorch Tensors
In this section, we look at different types of methods for constructing tensors of zeros, tensor of ones, and also a tensor using random function consisting of normally distributed values.
All these types of tensors will be created by using torch.tensor function.
1. PyTorch Zeros Tensor : touch.zeros()
In a PyTorch zeros tensor, all values consist of zero only. This is created by passing the desired dimension to the torch.zeros function. We can also explicitly mention the data type which will produce the zeros tensor of that data type itself.
torch.zeros([3, 6], dtype=torch.int32)
2. PyTorch Ones Tensor : touch.zeros()
In a PyTorch ones tensor, all values consist of ones only. Here we pass the dimension of the required ones tensor to the torch.ones function.
In this example, we are building the ones tensor using Cuda for leveraging GPU. For this, device parameter is passed to the torch.ones function. (P.S – We have used Cuda just for example, it is not required to use it for creating PyTorch ones tensor)
cuda0 = torch.device('cuda:0')
torch.ones([6, 3], dtype=torch.float64, device=cuda0)
3. PyTorch Random Tensor : touch.randn()
We can create a tensor of random values in PyTorch by using touch.randn function by passing the dimension of the required tensor. The values will be normally distributed values.
# Create PyTorch random tensor from normal distribution randoms
t = torch.randn(3, 7)
print(t)
How to access elements in PyTorch Tensor
Just like we can access elements in an array, we can access the values of tensors in PyTorch as well.
Here we first build a tensor using torch.tensor function.
x = torch.tensor([[45, 27, 63], [144, 549, 72]])
1. Access by Index
Here we are using indexing to access a particular index of values.
print(x[1][1])
The below example shows how we can access a particular index value and then we can alter it.
x[0][1] = 999
print(x)
2. Slicing of Tensor
The second method of accessing elements of the tensor in PyTorch is with the help of slicing.
# Slicing
t = torch.Tensor([[45, 27, 63], [144, 549, 72], [162, 234, 96]])
# Every row, only the last column
print(t[:, -2])
# First 1 row, all columns
print(t[:1, :])
# Lower right most corner
print(t[-1:, -1:])
Mathematical Operations on Tensors in PyTorch
In this section, we’ll look at mathematical operations using tensor. First, we build the two different tensors in PyTorch.
# Tensor Operations
x = torch.tensor([[45, 27, 63], [144, 549, 72]])
y = torch.tensor([[4, 5, 9], [5.4, 6.3, 9.1]])
1. Addition of PyTorch Tensors: torch.add()
# Simple Addition
print(x + y)
# We can also use torch.add()
print(torch.add(x, y))
2. Subtraction of PyTorch Tensors : torch.sub()
For subtracting two tensors in PyTorch can simply use plus operation or use torch.sub function.
# Simple Subtraction
print(x - y)
# We can also use torch.sub()
print(torch.sub(x, y))
3. Cross Product of PyTorch Tensors : cross()
In this example, we have generated two tensors using the torch randn function.
Here the cross product of tensors in PyTorch is calculated using cross() function.
# Compute cross product
t1 = torch.randn(5, 3)
t2 = torch.randn(5, 3)
t1.cross(t2)
4. Matrix Multiplication of PyTorch Tensors : mm()
In this example, the matrix multiplication of the two tensors is calculated using mm function.
# Compute matrix product
t = (torch.Tensor([[2, 4], [5, 10]]).mm(torch.Tensor([[10], [20]])))
t
5. Elementwise Multiplication of PyTorch Tensors : mul()
Element wise multiplication of tensors in PyTorch is done using mul() function.
# Elementwise multiplication of PyTorch Tensors
t = torch.Tensor([[5, 15], [35, 49]])
t.mul(t)
Tensor View in PyTorch : view()
Different views of tensors in PyTorch are used for creating a subset or smaller version of the original tensor.
Here view() function is used for creating a view.
import torch
p = torch.randn(2, 4)
q = p.view(2 * 4)
print('Size of p:', p.size())
print('Size of q:', q.size())
print(p)
print(q)
The reshape function has the opposite functionality to the view function.
r = q.reshape(2, 4)
print(r)
PyTorch Tensor To and From Numpy ndarray
Finally, let us see another way of converting a NumPy array into tensor and then how to convert tensor to NumPy error.
1. PyTorch Tensor from NumPy Array : torch.from_numpy()
Let us first create a NumPy array –
# ndarray to tensor
a = np.random.randn(3, 5)
print(a)
print(type(a))
The below function torch.from_numpy will be used for converting an array to a tensor.
t = torch.from_numpy(a)
print(t)
print(type(t))
Converting PyTorchTensor to Numpy Array : numpy()
In this case, we are converting a tensor into a NumPy array. Let us first create a tensor with random values –
# tensor to ndarray
t = torch.randn(3, 5)
print(t)
print(type(t))
Now we will use numpy() function to convert the tensor to NumPy array.
a = t.numpy()
print(a)
print(type(a))
Conclusion
It’s time that we conclude this PyTorch tutorial on tensor where we talked about different examples and different operations.
Reference – PyTorch Documentation
