Tensor Multiplication in PyTorch with torch.matmul() function with Examples

Introduction

In this tutorial, we will explain how to multiply tensors in PyTorch with torch.matmul() function. We will see its syntax and see various examples to understand its functionality in a better way.

Syntax of torch.matmul() function

The syntax of torch matmul function is as follows –

torch.matmul(tensor1, tensor2, out)

  • tensor1 – The first tensor for multiplication
  • tensor2 – The second tensor for multiplication
  • out – Output tensor, result of multiplication of tensor1 with tensor2

Functionality of torch matmul function

The functionality of torch matmul depends on the dimensions of the two input tensors as follows –

  1. When both tensor1 and tensor2 are of 1-Dimension then the dot product is done whose output is a scalar. [See example 1 below]
  2. When both tensor1 and tensor2 are of 2-Dimension then the matrix multiplication is done. However the inner dimension of the two tensors should be the same, i.e. the number of columns in tensor1 should match the number of rows of tensor2. (It is the fundamental rule of matrix multiplication) [See example 2 below]
  3. When tensor1 is 1-Dimension and tensor2 is 2-Dimension then internally one dimension is added to tensor1 to make it 2-Dimension. After that, matrix multiplication is performed between the two tensors, and the extra dimension is removed from the final result. It should be noted that the 1-Dimension tensor should be such that when an extra dimension is added it should match the number of rows of tensor2 to obey the matrix multiplication rule. Otherwise, it will throw an error. [See example 3 below]
  4. When tensor1 is 2-Dimension and tensor2 is 1-Dimension then the matrix-vector multiplication is done. [See example 4 below]
  5. When at least one tensor has dimension N where N>2 then batched matrix multiplication is done where broadcasting logic is used. [See example 5 & 6 below]

Examples

To start with the examples, let us first of all import PyTorch library.

In [0]:

import torch;

 

Example – 1: Multiplying Two 1-Dimension Tensors with torch.matmul()

In the first example, we multiply two 1-D dimension tensors with torch matmul and the resulting output is scalar.

In [1]:

tensor1 = torch.tensor([2,3])

tensor1
Out[1]:
tensor([2, 3])
In[2]:
tensor2 = torch.tensor([4,4])

tensor2

Out[2]:

tensor([4, 4])

In[3]:

output = torch.matmul(tensor1,tensor2)

output

Out[3]:

tensor(20)

Example – 2: Multiplying Two 2-Dimension Tensors with torch.matmul

In this example, we generate two 2-D tensors with randint function of size 4×3 and 3×2 respectively. Do notice that their inner dimension is of the same size i.e. 3 thus making them eligible for matrix multiplication. The output tensor after multiplying with torch matmul is of size 4×2.

In [4]:

tensor1 = torch.randint(high = 20, size=(4,3))

tensor1

Out[4]:

tensor([[13, 15, 12],
        [ 6, 13, 19],
        [ 8, 10, 18],
        [ 8, 17, 17]])
In[5]:
tensor2 = torch.randint(high = 20, size=(3,2))

tensor2

Out[5]:

tensor([[ 2, 18],
        [ 5,  2],
        [ 9, 10]])
In[6]:
output = torch.matmul(tensor1,tensor2)

output
Out[6]:
tensor([[209, 384],
        [248, 324],
        [228, 344],
        [254, 348]])

Example – 3: Multiplying 1D Tensor with 2D Tensor

Here we create a 1D tensor and a 2D tensor of size 2×3 and then multiply them with matmul function.

In [7]:

tensor1 = torch.tensor([6,5])

tensor1
Out[7]:
tensor([6, 5])
In [8]:
tensor2 = torch.randint(high = 20, size=(2,3))

tensor2
Out[8]:
tensor([[10, 12, 12],
        [18, 13, 12]])
In [9]:
output = torch.matmul(tensor1,tensor2)

output
Out[9]:
tensor([150, 137, 132])

Example – 4: Multiplying 2D Tensor with 1D Tensor

Here we create a 2D tensor of size 3×2 and a 1D tensor and then multiply them with PyTorch matmul function.

In [10]:

tensor1 = torch.randint(high = 20, size=(3,2))

tensor1
Out[10]:
tensor([[ 5, 10],
        [13,  0],
        [ 7,  1]])
In [11]:
tensor2 = torch.tensor([2,5])

tensor2
Out[11]:
tensor([2, 5])
In [12]:
output = torch.matmul(tensor1,tensor2)

output
Out [12]:
tensor([60, 26, 19])

Example – 5: Multiplying 2D Tensor with 3D Tensor

In this example, we create a 2D tensor of size 4×3 and a 3D tensor of size 2x3x2 and then multiply them.

In [13]:

tensor1 = torch.randint(high = 20, size=(4,3))

tensor1
Out[13]:
tensor([[16, 18,  8],
        [ 5, 12, 12],
        [17,  7,  5],
        [15,  5, 14]])
In [14]:
tensor2 = torch.randint(high = 20, size=(2,3,2))

tensor2

Out[14]:

tensor([[[14, 15],
         [13,  3],
         [ 8, 13]],

        [[ 2, 12],
         [ 4,  7],
         [ 1,  3]]])
In [15]:
output = torch.matmul(tensor1,tensor2)

output
Out[15]:
tensor([[[522, 398],
         [322, 267],
         [369, 341],
         [387, 422]],

        [[112, 342],
         [ 70, 180],
         [ 67, 268],
         [ 64, 257]]])

Example – 6: Multiplying Two 3D Tensors

In this example, we create two 3D tensors and then multiply them with PyTorch matmul function.

In [16]:

tensor1 = torch.randint(high = 20, size=(2,4,3))

tensor1
Out[16]:
tensor([[[ 6, 14,  8],
         [16, 19,  7],
         [ 2,  1, 13],
         [ 0, 17, 19]],

        [[ 4, 16, 12],
         [14, 16, 15],
         [18, 18,  5],
         [ 4,  8,  8]]])
In [17]:
tensor2 = torch.randint(high = 20, size=(2,3,2))

tensor2
Out[17]:
tensor([[[18, 19],
         [17,  5],
         [ 4,  8]],

        [[ 1,  8],
         [ 2, 13],
         [14,  2]]])
In [18]:
output = torch.matmul(tensor1,tensor2)

output
Out[18]:
tensor([[[378, 248],
         [639, 455],
         [105, 147],
         [365, 237]],

        [[204, 264],
         [256, 350],
         [124, 388],
         [132, 152]]])
  • MLK

    MLK is a knowledge sharing community platform for machine learning enthusiasts, beginners and experts. Let us create a powerful hub together to Make AI Simple for everyone.

Follow Us

Leave a Reply

Your email address will not be published. Required fields are marked *