Heapsort is one of the general sorting algorithms that performs in O(n.log(n)) in the worst-case, just like merge sort and quicksort, but sorts in place – as quicksort. Although quicksort’s worst-case sorting time is O(n²) it’s often considered that it beats other sorting algorithms in practice. Thus in practice quicksort is “faster” than heapsort. In the same time developers tend to consider heapsort as more difficult to implement than other n.log(n) sorting algorithms.

In the other hand heapsort uses a special data structure, called heap, in order to sort items in place and this data structure is quite useful in some specific cases. Thus to understand heapsort we first need to understand what is a heap.

So first let’s take a look at what is a heap.

Overview

A heap is a complete binary tree, where all the parents are greater than their children (max heap). If all the children are greater than their parents it is considered to call the heap a min-heap. But first what is a complete binary tree? Well, this is a binary tree, where all the levels are full, except the last one, where all the items are placed on the left (just like on the image below).

Combined with the fact that each node contains a greater key than its children, a heap may look like the tree on the following diagram.

The thing is that if we put indices next to each node of this tree, starting from the root (index 1) and continuing from left to right on each level, we’ll get the following tree.

Now if we take a closer look to the picture above we can see that the indices of a node and its children are closely related. Thus for a node of an index i we see that its left child has the index 2*i, while its right child’s index is 2*i + 1.

This particular order gives us the possibility to store each heap in an array.

Since in a heap its greater element is in the root of the tree (for max-heap, respectively in a min-heap its smallest element is the root) we need to answer two questions.

1. How to build a heap out of an ordinary array?
2. After extracting the root, which is the greatest (smallest) item, how can we rebuild the heap in order to keep it a heap again?

First let’s try to answer the first question. How to build a heap? Well, let’s forget about the array for a while and let’s take a look on a ordinary binary tree with only three nodes.

We see that the three green nodes destroy the structure of our heap, because the root (1) is smaller than its children (4) and (5). Thus we need to fix this problem and what we’re going to do is to swap the root with its biggest child.

As you can see on the picture above the i-th item is first compared to its left child. The greater of these two items is compared to the right child. Note that we don’t swap them – we just compare them to get which one is greater. Once we find the greatest of these three values we swap them with the root in case it’s not the root value.

Although now these three elements form a heap, by swapping the root with one of its children may destroy the heap constructed out of this child. That is why we continue the same procedure with it.

This actually gives us the procedure to heapify the three nodes constructed out of the i-th item and its children. However to build a heap from an arbitrary array we should perform this operation starting from floor(len[A] / 2) down to the first item in the array.

Why? Well, a complete binary tree with a full last level contains n/2 + 1 nodes in it. They don’t have children, thus we don’t need to check them – they are “sorted”. Indeed if we start from an item on the right of floor(len[A] / 2) there won’t be items with indices 2*i and 2*i + 1.

Code

So far we know how to build the heap. Next thing is to swap the first and the last element of the array and rebuild the heap. Here’s the PHP code of how to do this.

$a = array(1, 6, 3, 8, 2, 5, 4);

function heapify(&$a, &$i, &$heap_size)
{
    $l = $i*2 + 1;
    $r = $i*2 + 2;
    
    if ($l < $heap_size && $a[$i] < $a[$l]) {
        $largest = $l;
    } else {
        $largest = $i;
    }
    
    if ($r < $heap_size && $a[$largest] < $a[$r]) {
        $largest = $r;
    }
    
    if ($largest != $i) {
        $t = $a[$i];
        $a[$i] = $a[$largest];
        $a[$largest] = $t;
        
        heapify($a, $largest, $heap_size);
    }
}

function build_heap(&$a, &$heap_size)
{
    $len = floor($heap_size / 2);
    for ($i = $len; $i > -1; $i--) {
        heapify($a, $i, $heap_size);
    }
}

function heapsort(&$a)
{
    $heap_size = count($a);
    build_heap($a, $heap_size);
    
    while ($heap_size--) {
        $t = $a[$heap_size];
        $a[$heap_size] = $a[0];
        $a[0] = $t;
        build_heap($a, $heap_size);
    }
}

// 1 2 3 4 5 6 8
heapsort($a);

Complexity

OK, the last question is – how do we know that this algorithm sorts in place in n.log(n) time? Let’s explore the algorithm one more time. The heapify worst-case is when we start from the root down to the lowest level of the tree. In these terms if the tree height is h, the time is O(h), but because the tree is balanced (complete) the time in terms of n is O(log(n)).

In the other hand to build a heap we walk from floor(len[A] / 2) to 0, which makes it run in O(n.log(n)). However there is only one case when the heapify may run in log(n), and that is when it starts from the root, so it’s not absolutely true that building the heap runs in n.log(n).

Indeed heapify depend on the level it has been started. It doesn’t run for the last ceil(n/2) items and it runs in O(1) for another 2^h-1. Thus in practice we can build a heap in O(n).

Once we have the heap built, the only thing to do is to extract its first element and rebuild – heapify from the first item. This makes the sorting algorithm run in O(n.log(n)) – just like quicksort and mergesort.

Application

As I said in the beginning of this post quicksort is often the fastest general purpose algorithm in practice. This makes both merge sort and heapsort not so popular. However heapsort introduces an interesting data structure which can help us in many other cases.

It’s initially used to implement priority queues. What is great about a heap is that after we build it, which we know how to do in linear time, we can extract the greatest value – thus taking the highest priority task. Then with rebuilding the heap we can extract the next priority and so on, without fully sorting the array.

This makes the heapsort the only sorting algorithm that can sort the first k items out of a set of n items without sorting the whole set.

Indeed let’s say we have a set of positive integers and we’d like to get the biggest sum out of three items. Obviously we can sort the array and take the greatest three numbers, but this will cost us n.log(n) time, while using heapsort we can do it much faster! And all this without extra space – in place!

Algorithms always depend on the input. We saw that general purpose sorting algorithms as insertion sort, bubble sort and quicksort can be very efficient in some cases and inefficient in other. Indeed insertion and bubble sort are considered slow, with best-case complexity of O(n²), but they are quite effective when the input is fairly sorted. Thus when you have a sorted array and you add some “new” values to the array you can sort it quite effectively with insertion sort. On the other hand quicksort is considered one of the best general purpose sorting algorithms, but while it’s a great algorithm when the data is randomized it’s practically as slow as bubble sort when the input is almost or fully sorted.

Now we see that depending on the input algorithms may be effective or not. For almost sorted input insertion sort may be preferred instead of quicksort, which in general is a faster algorithm.

Just because the input is so important for an algorithm efficiency we may ask are there any sorting algorithms that are faster than O(n.log(n)), which is the average-case complexity for merge sort and quicksort. And the answer is yes there are faster, linear complexity algorithms, that can sort data faster than quicksort, merge sort and heapsort. But there are some constraints!

Everything sounds great but the thing is that we can’t sort any particular data with linear complexity, so the question is what rules the input must follow in order to be sorted in linear time.

Such an algorithm that is capable of sorting data in linear O(n) time is radix sort and the domain of the input is restricted – it must consist only of integers.

Overview

Let’s say we have an array of integers which is not sorted. Just because it consists only of integers and because array keys are integers in programming languages we can implement radix sort.

First for each value of the input array we put the value of “1” on the key-th place of the temporary array as explained on the following diagram.

Implementation

Implementing radix sort is very easy in fact, which is great. The thing is that old-school programming languages weren’t so flexible and we needed to initialize the entire temporary array. That leads to another problem – we must know the interval of values from the input. Fortunately nowadays programming languages and libraries are more flexible so we can initialize our temporary array even if we don’t know the interval of input values, as on the example bellow. PHP is somewhere in the middle – it’s flexible enough to build-up arrays in the memory without knowing their size in advance, but we still must ksort them.

$list = array(4, 3, 5, 9, 7, 2, 4, 1, 6, 5);
 
function radix_sort($input)
{
    $temp = $output = array();
	$len = count($input);
 
    for ($i = 0; $i < $len; $i++) {
		$temp[$input[$i]] = ($temp[$input[$i]] > 0) 
			? ++$temp[$input[$i]]
			: 1;
    }
    
    ksort($temp);
    
    foreach ($temp as $key => $val) {
		if ($val == 1) {
			$output[] = $key; 
		} else {
			while ($val--) {
				$output[] = $key;
			}
        }
    }
    
    return $output;
}
 
// 1, 2, 3, 4, 4, 5, 5, 6, 7, 9
print_r(radix_sort($list));

The problem is that PHP needs ksort – which is completely foolish as we’re trying to sort an array using “another” sorting method, but to overcome this you must know the interval of values in advance and initialize a temporary array with 0s, as on the example bellow.

define(MIN, 1);
define(MAX, 9);
$list = array(4, 3, 5, 9, 7, 2, 4, 1, 6, 5);

function radix_sort(&$input)
{
    $temp = array();
	$len = count($input);
 
	// initialize with 0s
    $temp = array_fill(MIN, MAX-MIN+1, 0);
    
    foreach ($input as $key => $val) {
    	$temp[$val]++;
    }
    
    $input = array();
    foreach ($temp as $key => $val) {
	if ($val == 1) {
		$input[] = $key;
	} else {
		while ($val--) {
			$input[] = $key;
		}
	}
    }
}

// 4, 3, 5, 9, 7, 2, 4, 1, 6, 5
var_dump($list);

radix_sort(&$list);

// 1, 2, 3, 4, 5, 5, 6, 7, 8, 9
var_dump($list);

Here the input is modified during the sorting process and it’s used as result.

Complexity

The complexity of radix sort is linear, which in terms of omega means O(n). That is a great benefit in performance compared to O(n.log(n)) or even worse with O(n²) as we can see on the following chart.

Why using radix sort

1. It’s fast

Radix sort is very fast compared to other sorting algorithms as we saw on the diagram above. This algorithm is very useful in practice because in practice we often sort sets of integers.

2. It’s easy to understand and implement

Even a beginner can understand and implement radix sort, which is great. You need no more than few loops to implement it.

Why NOT using radix sort

1. Works only with integers

If you’re not sure about the input better do not use radix sort. We may think that our input consists only of integers and we can go for radix sort, but what if in the future someone passes floats or strings to our routine.

2. Requires additional space

Radix sort needs additional space – at least as much as the input.

Final Words

Radix sort is restricted by the input’s domain, but I must say that in practice there are tons of cases where only integers are sorted. This is when we get some data from the db based on primary keys – typically primary in database tables are integers as well. So practically there are lots of cases of sorting integers, so radix sort may be one very, very useful algorithm and it is so cool that it is also easy to implement.

It’s weird that bubble sort is the most famous sorting algorithm in practice since it is one of the worst approaches for data sorting. Why is bubble sort so famous? Perhaps because of its exotic name or because it is so easy to implement. First let’s take a look on its nature.

Bubble sort consists of comparing each pair of adjacent items. Then one of those two items is considered smaller (lighter) and if the lighter element is on the right side of its neighbour, they swap places. Thus the lightest element bubbles to the surface and at the end of each iteration it appears on the top. I’ll try to explain this simple principle with some pictures.

1. Each two adjacent elements are compared

Here “2” appears to be less than “4”, so it is considered lighter and it continues to bubble to the surface (the front of the array).

2. Swap with heavier elements

On his way to the surface the currently lightest item meets a heavier element. Then they swap places.

3. Move forward and swap with each heavier item

The problem with bubble sort is that you may have to swap a lot of elements.

4. If there is a lighter element, then this item begins to bubble to the surface

If the currently lightest element meets another item that is lighter, then the newest currently lightest element starts to bubble to the top.

5. Finally the lightest element is on its place

At the end of each iteration we can be sure that the lightest element is on the right place – at the beginning of the list.

The problem is that this algorithm needs a tremendous number of comaprisons and as we know already this can be slow.

We can easily see how ineffective bubble sort is. Now the question remains – why is it so famous? Maybe indeed the answer lies in the simplicity of its implementation. Let’s see how to implement bubble sort.

Implementation

Implementing bubble sort is easy. The question is how easy? Well, obviously after understanding the principles of this algorithm every developer, even a beginner, can implement it. Here’s a PHP implementation of bubble sort.

$input = array(6, 5, 3, 1, 8, 7, 2, 4);

function bubble_sort($arr)
{
	$length = count($arr);
	
	for ($i = 0; $i < $length; $i++) {
		for ($j = $length-1; $j > $i; $j--) {
			if ($arr[$j] < $arr[$j-1]) {
				$t = $arr[$j];
				$arr[$j] = $arr[$j-1];
				$arr[$j-1] = $t;
			}
		}
	}
	
	return $arr;
}

// 1, 2, 3, 4, 5, 6, 7, 8
$output = bubble_sort($input);

Clearly the implementation consists of few lines of code and two nested loops.

Complexity: Where's Bubble Sort Compared to Other Sorting Algorithms

Compared to other sorting algorithm, bubble sort is really slow. Indeed the complexity of this algoritm is O(n²) which can't be worse. It's weird that the most well known sorting algorithm is the slowest one.

Even for small values of n, the number of comparisons and swaps can be tremendous.

Another problem is that most of the languages (libraries) have built-in sorting functions, that they don't make use of bubble sort and are faster for sure. So why a developer should implement bubble sort at all?

Application: 3 Cool Reasons To Use Bubble Sort

We saw that bubble sort is slow and ineffective, yet it is used in practice. Why? Is there any reason to use this slow, ineffective algorithm with weird name? Yes and here are some of them that might be helpful for any developer.

1. It is easy to implement

Definitely bubble sort is easier to implement than other "complex" sorting algorithms as quicksort. Bubble sort is easy to remember and easy to code and that's great instead of learning and remembering tons of code.

2. Because the library can't help

Let's say you work with JavaScript. Great, there you get array.sort() which can help for this: [3, 1, 2].sort(). But what would happen if you'd rather like to sort more "complex" structures like ... some DOM nodes. Here we have three LI nodes and we want to sort them in some order. Obviously you can't compare them with the "<" operator, so we've to come up with some custom solution.

Click here to sort the list

node 3
node 1
node 2

Here sort() can’t help us and we’ve to code our own function. However we have only few elements (three in our case). Why not using bubble sort?

3. The list is almost sorted

One of the problems with bubble sort is that it consists of too much swapping, but what if we know that the list is almost sorted?

// almost sorted
[1, 2, 4, 3, 5]

We have to swap only 3 with 4.

Note that in the best case bubble sort’s complexity is O(n) – faster than quicksort’s best case!