Pytorch Tensor

A Pytorch Tensor is the most basic data structure in the Pytorch library. In this chapter of Pytorch Tutorial, you will learn about tensors and how you can create them in Pytorch.

Tensor

A Tensor is a data structure that can hold n-dimensional data. An n-dimensional tensor can be simply be considered as an n-dimensional matrix. This “dimensionality” of a tensor is also called its rank. So, a tensor with 1 dimension(also known as a vector) will have a rank of 1, while the tensor with 3 dimensions will have a rank of 5.

Example

[1, 9, 4, 7] is a 1-dimensional tensor, also known as a vector.

[ [ 4, 1, 7, 3],
[ 2, 9, 4, 1],
[ 5, 0, 6, 7],
[ 3, 1, 2, 9] ]
is a 2-dimensional tensor, also referred commonly as matrix.

Any number, 5, for example, is a 0-dimensional tensor, also known as a scalar.

---

Creating a Pytorch Tensor

A Pytorch tensor can be created by simply calling the tensor() function in the pytorch library. This will create a tensor that will store the values passed to it. A list or a numpy array can be passed to the tensor() function as a parameter.

Example

Creating a Pytorch tensor using a list.

tensor_1 = torch.tensor([4, 2, 1, 3])

print(tensor_1)
# Outputs- tensor([4, 2, 1, 3])

Example

Creating a Pytorch tensor using numpy n-dimensional array.

numpy_array = np.array([4, 2, 1, 3])

tensor_2 = torch.tensor(np_array)

print(tensor_2)
# Outputs- tensor([4, 2, 1, 3])

Example

You can pass a nested list or numpy array to create higher dimension tensors.

tensor_3 = torch.tensor([[4, 2], [1, 3]])

print(tensor_3)
# Outputs- tensor([[4, 2], [1, 3]])

The tensor() function returns an object of the class torch.Tensor

---

Pytorch Tensor Types

Tensors in pytorch, are of various types depending on the types of data that they hold.

We can find the data type of a tensor by checking the dtype attribute on the tensor.

Example

print(tensor_3.dtype)
# Outputs- torch.int64

torch.int64 is the default dtype assigned to integer tensors. This simply means that tensor_3 holds 64bit integer data.

The table below lists the most commonly used types of tensors in Pytorch along with the types of data that they hold.

Data Type	dtype	Tensor Type
Boolean	torch.bool	torch.BoolTensor
16 bit Floating Point	torch.float16	torch.HalfTensor
32 bit Floating Point	torch.float32	torch.FloatTensor
64 bit Floating Point	torch.float64	torch.DoubleTensor
16 bit signed integer	torch.int16	torch.ShortTensor
32 bit signed integer	torch.int32	torch.IntTensor
64 bit signed integer	torch.int64	torch.LongTensor

Along with the list or numpy array that we pass to the tensor, we may also pass the dtype parameter. This will instruct pytorch to create that particular type of tensor.

Example

In this example, along with a list, we have also passed the dtype parameter. This will ensure that the data type of the tensor is torch.int16 instead of the default torch.int64

tensor_3 = torch.tensor([[4, 2], [1, 3]], dtype=torch.int16)

print(tensor_3.dtype)
# Outputs- torch.int16

---

Upcasting and Downcasting a Pytorch Tensor

We can upcast and downcast the type of tensor. An upcasting happens when a lower data type, such as an integer is converted to a higher data type, such as a float. A downcasting happens when a higher data type, such as a float is converted to a lower data type, such as an integer. Unlike upcasting, downcasting usually results in a loss of data.

Example

Passing an integer valued list or array and setting its data type to float will create a tensor of the float data type. This is known as upcasting.

tensor_4 = torch.tensor([[4, 2], [1, 3]], dtype=torch.float64)

print(tensor_4)
# Ouputs- tensor([[4., 2.], [1., 3.]], dtype=torch.float64)

Example

Passing a float valued list or array and setting its data type to integer will create a tensor of the integer data type. This is known as downcasting.

tensor_5 = torch.tensor([[4.7, 2.2], [1.4, 7.8]], dtype=torch.int16)

print(tensor_5)
# Ouputs- tensor([[4, 2], [1, 7]], dtype=torch.int16)

Note– Notice the loss of data that occurred due to downcasting.

---

Creating a Pytorch Zeros Tensor

zeros() function is used to create a tensor that contains all values as 0. You have to pass the shape of the required tensor(as a list or a tuple) along with the data type of the tensor.

Example

In this example, we will be creating a tensor of the shape [2, 5] with all values as 0. The data type of this tensor will be torch.int32

tensor_zeros_1 = torch.zeros([2, 5], dtype=torch.int32)

print(tensor_zeros_1)
# Outputs- tensor([[0, 0, 0, 0, 0], [0, 0, 0, 0, 0]], dtype=torch.int32)

print(tensor_zeros_1.dtype)
# Outputs- torch.int32

Example

In this example, we will be creating a tensor of the shape [2, 5] with all values as 0. But this time, the data type of this tensor will be torch.float32

tensor_zeros_2 = torch.zeros([2, 5], dtype=torch.float32)

print(tensor_zeros_2)
# Outputs- tensor([[0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.]])

print(tensor_zeros_2.dtype)
# Outputs- tensor.float32

---

Creating a Pytorch Ones Tensor

ones() function is used to create a tensor that contains all values as 1. You have to pass the shape of the required tensor(as a list or a tuple) along with the data type of the tensor.

Example

In this example, we will be creating a tensor of the shape [2, 5] with all values as 1. The data type of this tensor will be torch.int32

tensor_ones_1 = torch.zeros([2, 5], dtype=torch.int32)

print(tensor_ones_1)
# Outputs- tensor([[1, 1, 1, 1, 1], [1, 1, 1, 1, 1]], dtype=torch.int32)

print(tensor_ones_1.dtype)
# Outputs- torch.int32

Example

In this example, we will be creating a tensor of the shape [2, 5] with all values as 1. But this time, the data type of this tensor will be torch.float32

tensor_ones_2 = torch.zeros([2, 5], dtype=torch.float32)

print(tensor_ones_2)
# Outputs- tensor([[1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.]])

print(tensor_ones_2.dtype)
#Outputs- tensor.float32

---

Creating a Pytorch Tensor from a Numpy Array

from_numpy() function is used to create a pytorch tensor from a numpy array. You pass a numpy n-dimensional array to torch.from_numpy() and a pytorch tensor will be returned.

Example

np_tensor = np.array([1, 2, 3, 4])

pytorch_tensor = torch.from_numpy(np_tensor)

print(type(pytorch_tensor))
# Outputs- torch.Tensor

print(pytorch_tensor)
# Outputs- tensor([1, 2, 3, 4])