Heapsort is one of the general sorting algorithms that performs in O(n.log(n)) in the worst-case, just like merge sort and quicksort, but sorts in place – as quicksort. Although quicksort’s worst-case sorting time is O(n²) it’s often considered that it beats other sorting algorithms in practice. Thus in practice quicksort is “faster” than heapsort. In the same time developers tend to consider heapsort as more difficult to implement than other n.log(n) sorting algorithms.

In the other hand heapsort uses a special data structure, called heap, in order to sort items in place and this data structure is quite useful in some specific cases. Thus to understand heapsort we first need to understand what is a heap.

So first let’s take a look at what is a heap.

Overview

A heap is a complete binary tree, where all the parents are greater than their children (max heap). If all the children are greater than their parents it is considered to call the heap a min-heap. But first what is a complete binary tree? Well, this is a binary tree, where all the levels are full, except the last one, where all the items are placed on the left (just like on the image below).

Combined with the fact that each node contains a greater key than its children, a heap may look like the tree on the following diagram.

The thing is that if we put indices next to each node of this tree, starting from the root (index 1) and continuing from left to right on each level, we’ll get the following tree.

Now if we take a closer look to the picture above we can see that the indices of a node and its children are closely related. Thus for a node of an index i we see that its left child has the index 2*i, while its right child’s index is 2*i + 1.

This particular order gives us the possibility to store each heap in an array.

Since in a heap its greater element is in the root of the tree (for max-heap, respectively in a min-heap its smallest element is the root) we need to answer two questions.

1. How to build a heap out of an ordinary array?
2. After extracting the root, which is the greatest (smallest) item, how can we rebuild the heap in order to keep it a heap again?

First let’s try to answer the first question. How to build a heap? Well, let’s forget about the array for a while and let’s take a look on a ordinary binary tree with only three nodes.

We see that the three green nodes destroy the structure of our heap, because the root (1) is smaller than its children (4) and (5). Thus we need to fix this problem and what we’re going to do is to swap the root with its biggest child.

As you can see on the picture above the i-th item is first compared to its left child. The greater of these two items is compared to the right child. Note that we don’t swap them – we just compare them to get which one is greater. Once we find the greatest of these three values we swap them with the root in case it’s not the root value.

Although now these three elements form a heap, by swapping the root with one of its children may destroy the heap constructed out of this child. That is why we continue the same procedure with it.

This actually gives us the procedure to heapify the three nodes constructed out of the i-th item and its children. However to build a heap from an arbitrary array we should perform this operation starting from floor(len[A] / 2) down to the first item in the array.

Why? Well, a complete binary tree with a full last level contains n/2 + 1 nodes in it. They don’t have children, thus we don’t need to check them – they are “sorted”. Indeed if we start from an item on the right of floor(len[A] / 2) there won’t be items with indices 2*i and 2*i + 1.

Code

So far we know how to build the heap. Next thing is to swap the first and the last element of the array and rebuild the heap. Here’s the PHP code of how to do this.

$a = array(1, 6, 3, 8, 2, 5, 4);

function heapify(&$a, &$i, &$heap_size)
{
    $l = $i*2 + 1;
    $r = $i*2 + 2;
    
    if ($l < $heap_size && $a[$i] < $a[$l]) {
        $largest = $l;
    } else {
        $largest = $i;
    }
    
    if ($r < $heap_size && $a[$largest] < $a[$r]) {
        $largest = $r;
    }
    
    if ($largest != $i) {
        $t = $a[$i];
        $a[$i] = $a[$largest];
        $a[$largest] = $t;
        
        heapify($a, $largest, $heap_size);
    }
}

function build_heap(&$a, &$heap_size)
{
    $len = floor($heap_size / 2);
    for ($i = $len; $i > -1; $i--) {
        heapify($a, $i, $heap_size);
    }
}

function heapsort(&$a)
{
    $heap_size = count($a);
    build_heap($a, $heap_size);
    
    while ($heap_size--) {
        $t = $a[$heap_size];
        $a[$heap_size] = $a[0];
        $a[0] = $t;
        build_heap($a, $heap_size);
    }
}

// 1 2 3 4 5 6 8
heapsort($a);

Complexity

OK, the last question is – how do we know that this algorithm sorts in place in n.log(n) time? Let’s explore the algorithm one more time. The heapify worst-case is when we start from the root down to the lowest level of the tree. In these terms if the tree height is h, the time is O(h), but because the tree is balanced (complete) the time in terms of n is O(log(n)).

In the other hand to build a heap we walk from floor(len[A] / 2) to 0, which makes it run in O(n.log(n)). However there is only one case when the heapify may run in log(n), and that is when it starts from the root, so it’s not absolutely true that building the heap runs in n.log(n).

Indeed heapify depend on the level it has been started. It doesn’t run for the last ceil(n/2) items and it runs in O(1) for another 2^h-1. Thus in practice we can build a heap in O(n).

Once we have the heap built, the only thing to do is to extract its first element and rebuild – heapify from the first item. This makes the sorting algorithm run in O(n.log(n)) – just like quicksort and mergesort.

Application

As I said in the beginning of this post quicksort is often the fastest general purpose algorithm in practice. This makes both merge sort and heapsort not so popular. However heapsort introduces an interesting data structure which can help us in many other cases.

It’s initially used to implement priority queues. What is great about a heap is that after we build it, which we know how to do in linear time, we can extract the greatest value – thus taking the highest priority task. Then with rebuilding the heap we can extract the next priority and so on, without fully sorting the array.

This makes the heapsort the only sorting algorithm that can sort the first k items out of a set of n items without sorting the whole set.

Indeed let’s say we have a set of positive integers and we’d like to get the biggest sum out of three items. Obviously we can sort the array and take the greatest three numbers, but this will cost us n.log(n) time, while using heapsort we can do it much faster! And all this without extra space – in place!

The binary search tree is a very useful data structure, where searching can be significantly faster than searching into a linked list. However in some cases searching into a binary tree can be as slow as searching into a linked list and this mainly depends on the input sequence. Indeed in case the input is sorted the binary tree will seem much like a linked list and the search will be slow.

To overcome this we must change a bit the data structure in order to stay well balanced. It’s intuitively clear that the searching process will be better if the tree is well branched. This is when finding an item will become faster with minimal effort.

Since we know how to construct a binary search tree the only thing left is to keep it balanced. Obviously we will need to re-balance the tree on each insert and delete, which will make this data structure more difficult to maintain compared to non-balanced search trees, but searching into it will be significantly faster.

Overview

In order to balance a tree we can go for the very basic and intuitive approach. First let’s take a look of one non-balanced tree.

Compared to the balanced tree on the right from the image above with the same items we see that the root is approximately equal to its middle item. I.e. 4 is the middle item of the sequence [1,2,3,4,5,6,7]!

If we take a look of the sequence [2 3 4], clearly by building a binary tree it will look like a linked list. However if we choose the middle item for a root – we’ll easy build a balanced tree. So the only thing to do is to get the middle item out of a list.

We now see that building a balanced binary tree out of a sorted linked list isn’t that difficult. In the other hand, as I said above, on each insert we’ll have to rebalance the tree. You can think of the tree out of the values [1,2,3,4,5] and the same tree after inserting [44,45,46,47,48]. Clearly the root of the resulting tree will no longer be 3.

So we need to implement the re-balancing in three basic operations. First we need to build a linked list out of a balanced binary tree. On the second place we’ll have to find the middle item and on the third place we’ll have to build again a balanced search tree.

Hopefully the first two tasks are easy to implement, because making out a sorted list out of a binary search tree is very easy. We need just to walk through the tree from left-root-right recursively. Because smaller items are in the left sub-tree and greater items are on the right we’re sure that the resulting list will be sorted. Then finding the middle item is as easy as finding the middle index of an array know its length.

Balancing Optimization

Of course the main problem of re-balancing a tree on each insert/delete is that this operations will be slow and soon or later we’ll have problems. That can happen if we change often our data structure. That’s why we should think of some optimization.

Normally we insert and re-balance on each step, which is slow. In the other hand we can do bulk insert forgetting about the re-balancing for a while. Only after the inserts are done we can go for re-balancing the entire tree.

The same approach we can use with bulk delete. We can just set to NIL the items we want to delete, but we can keep them in memory for a while. Thus the search will stay relatively fast without rebalancing the tree. However this approach can be used carefully because we’ll keep some data in the memory without actually using it.

Implementation

Implementing balanced binary trees is more difficult than just implementing binary search trees. Here’s an example in PHP.

class Node
{
	protected   $_parent = null;
	protected   $_left = null;
	protected   $_right = null;
	protected   $_key;
    protected   $_data = null;
	
    /**
     * @param int $key
     * @param mixed $data 
     */
	public function __construct($key, $data)
	{
		$this->_key = $key;
        $this->_data = $data;
	}
    
    /**
     * Empty the node by keeping up the key, but
     * setting up the data to NULL 
     */
    public function doEmpty() 
    {
        $this->_data = null;
    }
	
    /**
     * Print the key
     * 
     * @return string
     */
	public function __toString()
	{
		return 'First name: ' . $this->_data['f_name']
                . '
'
                . 'Last name: ' . $this->_data['l_name']
                . '
' 
                . 'Birthday: ' . $this->_data['b_day'];
	}
    
    public function &getParent() { return $this->_parent; }
    public function setParent($parent) { $this->_parent = $parent; }
    
    public function &getLeft() { return $this->_left; }
    public function setLeft($left) { $this->_left = $left; }
    
    public function &getRight() { return $this->_right; }
    public function setRight($right) { $this->_right = $right; }
    
    public function &getKey() { return $this->_key; }
    public function setKey($key) { $this->_key = $key; }
    
    public function &getData() { return $this->_data; }
    public function setData($data) { $this->_data = $data; }
}

class BalancedBinaryTree
{
    /**
     * Reference to the root tree
     * 
     * @var Node 
     */
	protected $_root = null;
	
    /**
     * @param type $new
     * @param type $node
     * @return type 
     */
	protected function _insert($new, &$root)
	{
        // in case the tree is empty
        // make the new node the root of
        // the tree
		if ($root == null) {
			$root = $new;
			return;
		}
		
		if ($new->getKey() <= $root->getKey()) {
			if ($root->getLeft() == null) {
				$root->setLeft($new);
				$new->setParent($root);
			} else {
				$this->_insert($new, $root->getLeft());
			}
		} else {
			if ($root->getRight() == null) {
				$root->setRight($new);
				$new->setParent($root);
			} else {
				$this->_insert($new, $root->getRight());
			}
		}		
	}
	
    /**
     * FALSE on not found
     * 
     * @param string $firstName
     * @param BalancedBinaryTree $tree
     * @return boolean 
     */
	protected function _search($firstName, &$tree)
	{
        if ($tree == null) {
            return FALSE;
        }

        $data = $tree->getData();
		
        if ($firstName == $data['f_name']) {
			return $tree;
		}
        
        // search the left sub-tree
        return $this->_search($firstName, $tree->getLeft())
                . $this->_search($firstName, $tree->getRight());
	}
    
    /**
     *
     * @param int $key
     * @param Node $tree
     * @return FALSE or Node 
     */
    protected function _searchByKey($key, &$tree)
    {
        if ($tree == null) {
            return FALSE;
        }
        
        if ($tree->getKey() == $key) {
            return $tree;
        } else if ($tree->getKey() > $key) {
            return $this->_searchByKey($key, $tree->getLeft());
        } else {
            return $this->_searchByKey($key, $tree->getRight());
        }
    }
    
    /**
     * Returns a list out of the tree by emptying the tree. 
     * In other way the tree and the list will allocate memory
     * 
     * @param BalancedBinaryTree $tree 
     */
    protected function _leftRootRight($tree)
    {
        if ($tree == null) {
            return array();
        }
        
        return array_merge(
                $this->_leftRootRight($tree->getLeft()),
                array(array('key' => $tree->getKey(), 'data' => $tree->getData())),
                $this->_leftRootRight($tree->getRight()));
    }
    
    public function _balance($list)
    {
        if (empty($list)) {
            return;
        }
        
        // split the list
        $chunks = array_chunk($list, ceil(count($list) / 2));
        $mid = array_pop($chunks[0]);
        
        $node = new Node($mid['key'], $mid['data']);
        $this->insert($node);
        
        $this->_balance($chunks[0]);
        if (isset($chunks[1]))
            $this->_balance($chunks[1]);
    }
    
    /**
     * Balance a binary search tree 
     */
    public function balance()
    {
        $list = array();
        // make a list out of the tree
        $list = $this->_leftRootRight($this->_root);
        
        // find the medium! Because the list is ordered
        // we can find the middle element in various ways
        $chunks = array_chunk($list, ceil(count($list) / 2));
        $mid = array_pop($chunks[0]);
        
        // empty the tree
        $this->_root = null;
        
        // inser the root
        $node = new Node($mid['key'], $mid['data']);
        $this->insert($node);
        
        $this->_balance($chunks[0]);
        $this->_balance($chunks[1]);
    }
	
    /**
     * Insert a new item into the tree
     * 
     * @param type $node 
     */
	public function insert($newNode)
	{
		$this->_insert($newNode, $this->_root);
	}
	
    /**
     * Search by item key
     * 
     * @param int $key
     * @return Node or FALSE
     */
    public function searchByKey($key)
    {
        return $this->_searchByKey($key, $this->_root);
    }
    
    /**
     * @param BalancedBinary $tree
     * @return string 
     */
    protected function _print($tree)
    {
        if ($tree == null) { return ''; }
        
        return $this->_print($tree->getLeft()) . ' ' 
                . $tree->getKey() . ' ' 
                . $this->_print($tree->getRight());
    }
    
    /**
     * Print the tree from left through the root and the right 
     */
    public function __toString()
    {
        if ($this->_root == null) {
            return 'The tree is empty!';
        }

        return $this->_print($this->_root->getLeft()) . ' '
                . $this->_root->getKey() . ' '
                . $this->_print($this->_root->getRight());
    }
}

$a = new Node(90, array(
    'f_name' => 'W.A.',
    'l_name' => 'Mozart',
    'b_day' => '1756-01-27',
));

$b = new Node(100, array(
    'f_name' => 'John',
    'l_name' => 'Smith',
    'b_day' => '23.05.2039',
));

$c = new Node(80, array(
    'f_name' => 'Sarah',
    'l_name' => 'Johnnes',
    'b_day' => 'tomorrow',
));

$d = new Node(60, array(
    'f_name' => 'Ludwig Van',
    'l_name' => 'Beethoven',
    'b_day' => '1770-12-17',
));

$e = new Node(70, array(
    'f_name' => 'Barbara',
    'l_name' => 'Stefanel',
    'b_day' => 'today',
));

$t = new BalancedBinaryTree();

$t->insert($a);
$t->insert($b);
$t->insert($c);
$t->insert($d);
$t->insert($e);

echo $t;

echo $t->searchByKey(70);

$t->balance();

echo $t->searchByKey(70);

Complexity of Searching

Compared to non-balanced binary search trees we’re sure that searching into a balanced trees is quick enough. The maximum height of the tree is log(n) so the worst-case searching is O(log(n)).

Application

Searching into a balanced binary tree is fast. What is more important is that we’re sure that in the worst-case scenario the search is O(log(n)). The only problem is that keeping a tree balanced is a slow operation that consumes too much resources and must be performed carefully.

PHP by Book

Here’s how PHP developer declare a string, which is something very common in any programming language.

$my_var = 'hello world';
// or
$my_var = "hello world";

PHP Tricks

What if you do the following:

echo hello;

That appears to be correct … Well, it’s not absolutely correct. You’ll be “noticed”.

// Notice: Use of undefined constant hello
echo hello;

However if you disable error reporting, the code will be completely fine.

error_reporting(0);

// no problem now
echo hello;

Variations

What follows from the thing above is that you can use strings without quotes:

// hello
echo hello;

// hello world (concatenated)
echo hello . ' world';

// helloworld
echo hello . world;

However you can’t have spaces and most of the “special” symbols.

// syntax error
echo hello world;

// syntax error
echo hello!;

Final Words

Although you can do this in PHP, that is completely wrong. The code becomes more difficult to read and understand. In the second place you can miss a $ sign in front of a variable declaration and thus the PHP interpreter will assume this is a string. So disable error reporting isn’t so great sometimes.

Have you ever asked yourself which is the algorithm used to find a word after clicking Ctrl+F and typing something? Well I guess you know the answer from the title, but in this article you’ll find out how exactly this is done.

As we saw from the Morris-Pratt string searching we don’t need to compare the text and the pattern character by character. Some comparisons can be skipped in order to improve the performance of the string searching. Indeed the brute force string searching and the Rabin-Karp algorithm are quite slow only because they compare the pattern and the text character by character.

In the other hand the Morris-Pratt algorithm is a very good improvement of the brute force string searching, but the question remains. Is there any algorithm that is faster than Morris-Pratt – is there any way to skip more comparisons and to move the pattern faster.

It’s clear that if we have to find whether a single character is contained into a text we need at least “n” steps, where n is the length of the text. Once we have to find whether a pattern with the length of “m” is contained into a text with length of “n” the case is getting a little more complex.

However the answer is that there is such algorithm that is faster and more suitable than Morris-Pratt. This is the Boyer-Moore string searching.

Overview

Boyer-Moore is an algorithm that improves the performance of pattern searching into a text by considering some observations. It is defined in 1977 by Robert S. Boyer and J Strother Moore and it consist of some specific features.

First of all this algorithm starts comparing the pattern from the leftmost part of text and moves it to the right, as on the picture below.

Unlike other string searching algorithms though, Boyer-Moore compares the pattern against a possible match from right to left as shown below.

The main idea of Boyer-Moore in order to improve the performance are some observations of the pattern. In the terminology of this algorithm they are called good-suffix and bad-character shifts. Let’s see by the following examples what they are standing for.

Good-suffix Shifts

Just like the Morris-Pratt algorithm we start to compare the pattern against some portion of the text where a possible match will occur. In Boyer-Moore as I said this is done from the rightmost letter of the pattern. After some characters have matched we find a mismatch.

So how can we move the pattern to the right in order to skip unusual comparisons. To answer this question we need to explore the pattern. Let’s say there is a portion of the pattern that is repeated inside the pattern itself, like it is shown on the picture below.

In this case we must move the pattern thus the repeated portion must now align with its first occurrence in the pattern.

A variation of this case is when the portion from the pattern A overlaps with another portion that consists of the same characters.

Yet again the shift must align the second portion with its first occurrence.

Finally only a portion of A, let’s say “B”, can happen to occur in the very beginning of the pattern, as on the diagram below.

Now we must align the left end of the pattern with the rightmost occurrence of “B”.

Bad Character Shifts

Beside the good-suffix shifts the Boyer-Moore algorithm make use of the so called bad-character shifts. In case of a mismatch we can skip comparisons in case the character in the text doesn’t happen to appear in the pattern. To become clearer let’s see the following examples.

In the picture above we see that the mismatched character “B” from the text appears only in the beginning of the pattern. Thus we can simply shift the pattern to the right and align both characters B, skipping comparisons. An even better case is described by the following diagram where the mismatched letter isn’t contained into the pattern at all. Then we can shift forward the whole pattern.

Maximum of Good-suffix and Bad-Character shifts

Boyer-Moore needs both good-suffix and bad-character shifts in order to speed up searching performance. After a mismatch the maximum of both is considered in order to move the pattern to the right.

Complexity

It’s clear that Boyer-Moore is faster than Morris-Pratt, but actually its worst-case complexity is O(n+m). The thing is that in natural language search Boyer-Moore does pretty well.

Implementation

Finally let’s see the implementation in PHP, which can be easily “transcribed” into any other programming language. The only thing we need is the structures for bad-character shifts and good-suffixes shifts.

= 0; --$i) {
      if ($i > $g && $suffixes[$i + $m - 1 - $f] < $i - $g) {
         $suffixes[$i] = $suffixes[$i + $m - 1 - $f];
      } else {
         if ($i < $g) {
            $g = $i;
         }
         $f = $i;
         	
         while ($g >= 0 && $pattern[$g] == $pattern[$g + $m - 1 - $f]) {
            $g--;
         }
         $suffixes[$i] = $f - $g;
      }
   }
}

/**
 * Fills in the array of bad characters.
 *
 * @param string $pattern
 * @param array  $badChars
 */
function badCharacters($pattern, &$badChars)
{
   $m = strlen($pattern);

   for ($i = 0; $i < $m - 1; ++$i) {
      $badChars[$pattern{$i}] = $m - $i - 1;
   }
}

/**
 * Fills in the array of good suffixes
 *
 * @param string $pattern
 * @param array  $goodSuffixes
 */
function goodSuffixes($pattern, &$goodSuffixes)
{
   $m 		= strlen($pattern);
   $suff 	= array();

   suffixes($pattern, $suff);

   for ($i = 0; $i < $m; $i++) {
      $goodSuffixes[$i] = $m;
   }

   for ($i = $m - 1; $i >= 0; $i--) {
      if ($suff[$i] == $i + 1) {
         for ($j = 0; $j < $m - $i - 1; $j++) {
            if ($goodSuffixes[$j] == $m) {
               $goodSuffixes[$j] = $m - $i - 1;
            }
         }
      }
   }

   for ($i = 0; $i < $m - 2; $i++) {
      $goodSuffixes[$m - 1 - $suff[$i]] = $m - $i - 1;
   }
}

/**
 * Performs a search of the pattern into a given text
 *
 * @param string $pattern
 * @param string $text
 */
function boyer_moore($pattern, $text)
{
   $n = strlen($text);
   $m = strlen($pattern);

   $goodSuffixes 	= array();
   $badCharacters 	= array();

   goodSuffixes($pattern, &$goodSuffixes);
   badCharacters($pattern, &$badCharacters);

   $j = 0;
   while ($j < $n - $m) {
      for ($i = $m - 1; $i >= 0 && $pattern[$i] == $text[$i + $j]; $i--);
      if ($i < 0) {
         // note that if the substring occurs more
         // than once into the text, the algorithm will
         // print out each position of the substring
         echo $j;
         $j += $goodSuffixes[0];
      } else {
         $j += max($goodSuffixes[$i], $badCharacters[$text[$i + $j]] - $m + $i + 1);
      }
   }
}

// search using Boyer-Moore
// will return 12 and 38
boyer_moore($pattern, $text);

Application

Boyer-Moore is one of the most used string searching algorithm in practice. It is intuitively clear where it can be useful, but yet again I’ll say only that this algorithm is considered as the mostly used in practice for search and replace operations in text editors.

String matching is something crucial for database development and text processing software. Fortunately every modern programming language and library is full of functions for string processing that help us in our everyday work. However is great to understand their principles.

String algorithms can be mainly divided into several categories. One of these categories is string matching.

When we come to string matching the most basic approach is what is known as brute force, which means just to check every single character from the text to match against the pattern. In general we have a text and a pattern (most commonly shorter than the text). What we need to do is to answer the question whether this pattern appears into the text.

Overview

The principles of brute force string matching are quite simple. We must check for a match between the first characters of the pattern with the first character of the text as on the picture bellow.

In case they match we move forward the second character of the pattern comparing it with the “next” character of the text, as on the picture bellow.

Just because we have found a match between the first character from the pattern with some character of the text, doesn’t mean that the pattern appears in the text. We must move forward to see whether the full pattern is contained into the text.

Implementation

Implementation of brute force string matching is easy and here we can see a short PHP example. The bad news is that naturally this algorithm is quite slow.

function sub_string($pattern, $subject) 
{
	$n = strlen($subject);
	$m = strlen($pattern);
	
	for ($i = 0; i < $n-$m; $i++) {
		$j = 0;
		while ($j < $m && $subject[$i+$j] == $pattern[$j]) {
			$j++;
		}
		if ($j == $m) return $i;
	}
	return -1;
}

echo sub_string('o wo', 'hello world!');

Complexity

As I said this algorithm is slow. Actually every algorithm that contains “brute force” in its name is slow, but to show how slow is string matching I can say that its complexity is O(n.m). Here n is the length of the text, while m is the length of the pattern.

In case we fix the length of the text and test against variable length of the pattern, again we get rapidly growing function.

Application

Brute force string matching can be very ineffective, but it can also be very handy in some cases. Just like the sequential search.

It can be very useful ...

1. Doesn't require pre-processing of the text - Indeed if we search the text only once we don't need to pre-process it. Most of the algorithms for string matching need to build an index of the text in order to search quickly. This is great when you've to search more than once into a text, but if you do only once, perhaps (for short texts) brute force matching is great!
2. Doesn't require additional space - Because brute force matching doesn't need pre-processing it also doesn't require more space, which is one cool feature of this algorithm
3. Can be quite effective for short texts and patterns

It can be ineffective ...

1. If we search more than once the text - As I said in the previous section if you perform the search more than once it's perhaps better to use another string matching algorithm that builds an index and it's faster.
2. It's slow - In general brute force algorithms are slow and brute force matching isn't an exception.

Final Words

String matching is something very special in software development and it is used in various cases, so every developer must be familiar with this topic.

When you want to null the sliders on an page, using jQuery UI Slider, somehow the following code works in any browser except IE.

// wrong
$('.my-slider').slider('value', 0);

I say somehow, because according to the documentation this code is simply wrong. It can be used to get the value of the slider as it is a getter. You can null the value with the following snippet.

// correct
$('.my-slider').slider('option', 'value', 0);

IE Fix

Another way to null the value is with an anonymous object.

// correct
$('.my-slider').slider({ value: 0 });

This works on every browser including IE.

Prefix encoding, sometimes called front encoding, is yet another algorithm that tries to remove duplicated data in order to reduce its size. Its principles are simple, however this algorithm tend to be difficult to implement. To understand why, first let’s take a look of its nature.

Please, have a look on the following dictionary.

use
used
useful
usefully
usefulness
useless
uselessly
uselessness

Instead of keeping all these words in plain text or transferring all them over a network, we can compress (encode) them with prefix encoding.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '0fully',
4 => '0less',
5 => '0lessly',
6 => '0lessness',
);

It’s clear that this is not the best compression and we can go even further by using not only the first word as prefix.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '2ly',
4 => '0less',
5 => '4ly',
6 => '4ness',
);

Now the compression is better and the good news is that decompression is a fairly simple process. However the tricky part is compression itself. The problem is that it is quite difficult to chose an appropriate prefix. In our first example this is simple, but most of the times in practice we can have more heterogeneous data. Indeed the process of compression can be very difficult for randomly generated data and the algorithm will be not only slow, but difficult to implement.

The good thing is that this algorithm can be used in many cases once we know the data format in advance. So let’s see three examples where this algorithm can be very handy.

Application

Here are three examples of prefix encoding. As I said above the process of compression can be very difficult for random data, so it is a good practice to use only it if you know in advance the format of the input data.

Date and time prefixes

We humans often skip the first two digits of an year, so for instance we don’t always write 1995 or 1996, but we use the shorter – ‘95 and ‘96. Thus years can be encoded with shorter strings.

input: 	(1991, 1992, 1993, 1994, 1995, 1996)
output:	(91, 92, 93, 94, 95, 96)

The problem is that with small changes of the input stream we can confuse the decoder. Thus if we add years from the 21st century we lose the uniqueness of the data.

input:	(1998, 1992, 1999, 2011, 2012)
output: (98, 92, 99, 11, 12)

Now the decoder can decode the last two values as (1911, 1912) as “19” is considered to be the prefix. So we must know in advance that our prefix is absolutely equal for each of the values. If not the encoding format must be different. For instance we can encode also the prefix, with some special maker.

input:	(1998, 1992, 1932, 1924, 2001, 2012)
output:	(#19, 98, 92, 32, 24, #20, 01, 12)

Once the decoder reads the # character it will know to decode the following number as prefix.

This can be used in practice for date and time formats. Let’s say we have some datetime values, but we know that all of them are in the same day.

2012-01-31 15:33:45
2012-01-31 16:12:11
2012-01-31 17:32:35
2012-01-31 18:54:34

Obviously we can omit the date part of these strings and send (keep) only the time. Once again, we must be absolutely sure that all these values are in the same day. If not, we can use the encoding strategy of the previous example.

Phone numbers

Phone numbers are the typical case of prefix encoding. Not only the international code, but also the mobile network operators use prefixes for their phone numbers. Thus if we have to transfer phone numbers from, let’s say the UK, we can replace the leading “+44” with something shorter.

If you happen to code a phone book for a mobile device you can spend some space by compressing the data using prefix encoding and thus the user will have more space and will store more phone numbers on his mobile.

Phone number prefixes can be also used for database normalization. Thus you can store them in a separate db table and leave only the unique numbers from the phonebook.

Geo Coordinates

Using the same example from my previous post we can send GEO coordinates by removing a common prefix, for large levels of zoom. Indeed when you’ve to send lots of markers to your map application you can expect all of these markers to be fairly close to each other in large zoom level.

Now the coordinates of those points can have a common prefix, like the example bellow with the Subway stations.

LatLon(40.762959,-73.985989)
LatLon(40.761886,-73.983629)
LatLon(40.762861,-73.981612)
LatLon(40.764616,-73.98056)

We can see that all of these GEO points have the same prefix (40.76x, -73.98x), so we can send the prefix only once.

Prefix: (40.76, -73.98)
Data: 
LatLon(2959,5989)
LatLon(1886,3629)
LatLon(2861,1612)
LatLon(4616,056)

These are only three examples of prefix encoding and this algorithm must be considered as very useful when transferring homogeneous data.

Suffix Encoding

Suffix encoding practically the same algorithm as prefix encoding, with the small difference that we use to encode duplicating suffixes. Like the examples bellow suffix encoding can be useful is replacing repeating last name suffixes.

Johnson
Clarkson
Jackson

Or company names.

Apple Inc.
Google Inc.
Yahoo! Inc.

Here we can replace “ Inc.” with something else, but shorter.

Relative encoding is another data compression algorithm. While run-length encoding, bitmap encoding and diagram and pattern substitution were trying to reduce repeating data, with relative encoding the goal is a bit different. Indeed run-length encoding was searching for long runs of repeating elements, while pattern substitution and bitmap encoding were trying to “map” where the repetitions happen to occur.

The only problem with these algorithms is that not always the input stream of data is constructed out of repeating elements. It is clear that if the input stream contains many repeating elements there must be some way of reducing them. However that doesn’t mean that we cannot compress data if there are no repetitions. It all depends on the data. Let’s say we have the following stream to compress.

1, 2, 3, 4, 5, 6, 7

We can hardly imagine how this stream of data can be compressed. The same problem may occur when trying to compress the alphabet. Indeed the alphabet letters the very base of the words so it is the minimal part for word construction and it’s hard to compress them.

Fortunately this isn’t true always. An algorithm that tryies to deal with non repeating data is relative encoding. Let’s see the following input stream – years from a given decade (the 90’s).

1991,1991,1999,1998,1991,1993,1992,1992

Here we have 39 characters and we can reduce them. A natural approach is to remove the leading “19” as we humans often do.

91,91,99,98,91,93,92,92

Now we have a shorter string, but we can go even further with keeping only the first year. All other years will as relative to this year.

91,0,8,7,0,2,1,1

Now the volume of transferred data is reduced a lot (from 39 to 16 – more than 50%). However there are some questions we need to answer first, because the stream wont be always formatted in such pretty way. How about the next character stream?

91,94,95,95,98,100,101,102,105,110

We see that the value 100 is somehow in the middle of the interval and it is handy to use it as a base value for the relative encoding. Thus the stream above will become:

-9,-6,-5,-5,-2,100,1,2,5,10

The problem is that we can’t decide which value will be the base value so easily. What if the data was dispersed in a different way.

96,97,98,99,100,101,102,103,999,1000,1001,1002

Now the value of “100” isn’t useful, because compressing the stream will get something like this:

-4,-3,-2,-1,100,1,2,3,899,900,901,902

To group the relative values around “some” base values will be far more handy.

(-4,-3,-2,-1,100,1,2,3)(-1,1000,1,2)

However to decide which value will be the base value isn’t that easy. Also the encoding format is not so trivial. In the other hand this type of encoding can be useful in som specific cases as we can see bellow.

Implementation

The implementation of this algorithm depends on the specific task and the format of the data stream. Assuming that we’ve to transfer the stream of years in JSON from a web server to a browser, here’s a short PHP snippet.

// JSON: [1991,1991,1999,1998,1999,1998,1995,1997,1994,1993]
$years = array(1991,1991,1999,1998,1999,1998,1995,1997,1994,1993);

function relative_encoding($input)
{
	$output = array();
	$inputLength = count($input);
	
	$base = $input[0];
	
	$output[] = $base;
	
	for ($i = 1; $i < $inputLength; $i++) {
		$output[] = $input[$i] - $base;
	}
	
	return $output;
}

// JSON: [1991,0,8,7,8,7,4,6,3,2]
echo json_encode(relative_encoding($years));

Application

This algorithm may be very useful in many cases, but here’s one of them. There are plenty of map applications around the web. Some products as Google Maps, Yahoo! Maps, Bing Maps are quite famous, while there are very useful open source projects as OpenStreetMap. The web sites using these apps are thousands.

A typical use case is to transfer lots of Geo coordinates from web server to a browser using JSON. Indeed any GEO point on Earth is relative to the point (0,0), which is located near the west coast of Africa, however on large zoom levels, when there are tons of markers we can transfer the information with relative encoding.

For instance the following diagram shows San Francisco with some markers on it. Their coordinates are be relative to the point (0,0) on Earth.

Far more useful may be to encode those markers, relative to the center of the city, thus we can save some space.

However this type of compression can be tricky, for example when dragging the map and updating the marker array. In the other hand we must group markers if we have to load more than one city. That’s why we must be careful when implementing it. But in the other hand it can be very useful - for instance on initial load of the map we can reduce data and speed up the load time.

The thing is that with relative encoding we can save only changes to base value (data) - something like version control systems and thus reducing data transfer and load. Here's a graphical example. In the first case on the diagram bellow we can see that each item is stored on its own. It doesn't depend on the adjacent items and it can be completely independent of them.

However we can keep full info only for the first item and any other item will be relative to it, like on the diagram bellow.

If you have read some preformance tests on JavaScript loops, you may have heard that “while” is faster than “for”. However the question is how faster is “while”? Here are some results, but first let’s take a look on the JavaScript code.

The for experiment

console.time('for');
for (var i = 0; i < 10000000; i++) {
	i / 2;
}
console.timeEnd('for');

The while experiment

console.time('while');
var i = 0;
while (i++ < 10000000) {
	i / 2;
}
console.timeEnd('while');

Note - these tests are performed and measured with Firebug on Firefox.

Results

It's a fact, that you'll get different results as many times as you run this snippet. It depends also on the enviroment and software/hardware specs. That is why I performed them 10 times and then I took the average value. Here are the values of my performance tests. Note that both for and while perform 10,000,000 iterations.

And the Winner Is

While is the winner with an average result of 83.5 milliseconds, while "for" result is 88 average milliseconds.

As the diagram bellow shows, the while loop is slightly faster. However we should be aware that these performance gains are significant for large number of iterations!

Two variants of run-length encoding are the diagram encoding and the pattern substitution algorithms. The diagram encoding is actually a very simple algorithm. Unlike run-length encoding, where the input stream must consists of many repeating elements, as “aaaaaaaa” for instance, which are very rare in a natural language, there are many so called “diagrams” in almost any natural language. In plain English there are some diagrams as “the”, “and”, “ing” (in the word “waiting” for example), “ a”, “ t”, “ e” and many doubled letters. Actually we can extend those diagrams by adding surrounding spaces. Thus we can encode not only “the”, but “ the “, which are 5 characters (2 spaces and 3 letters) with something shorter. In the other hand, as I said, in plain English there are two many doubled letters, which unfortunately aren’t something special for run-length encoding and the compression ratio will be small. Even worse the encoded text may happen to be longer than the input message. Let’s see some examples.

Let’s say we’ve to encode the message “successfully accomplished”, which consists of four doubled letters. However to compress it with run-length encoding we’ll need at least 8 characters, which doesn’t help us a lot.

// 8 chars replaced by 8 chars!?
input: 	"successfully accomplished"
output:	"su2ce2sfu2ly a2complished"

The problem is that if the input text contains numbers, “2” in particular, we’ve to chose an escape symbol (“@” for example), which we’ll use to mark where the encoded run begins. Thus if the input message is “2 successfully accomplished tasks”, it will be encoded as “2 su@2ce@2sfu@2ly a@2complished tasks”. Now the output message is longer!!! than the input string.

// the compressed message is longer!!!
input:	"2 successfully accomplished"
output:	"2 su@2ce@2sfu@2ly a@2complished tasks"

Again if the input stream contains the escape symbol, we have to find another one, and the problem is that it is often too difficult to find short escape symbol that doesn’t appear in the input text, without a full scan of the text.

That is why run-length encoding isn’t a good solution when compressing plain text, where long runs rarely appear. Well, of course, there are exceptions. For example such an exception is the lossy text compression with run-length encoding. It is intuitively clear that compressing text with loss is rarely useful, especially when you’ve to decompress exactly the same text. However there are some cases that lossy compression may be useful. Such case can be removing spaces. Indeed the text “successfully accomplished” brings us exactly the same information as “successfully accomplished”. In this case we can simply remove those spaces. Indeed we can use a marker to indicate the long run of spaces like “successfully@6 accomplished” in order to decompress the input string with absolutely no loss, but we can also throw those symbols away. This desision depends on the goal. Exactly with the same goal in mind we can remove new lines and tabs, only if we’re sure that the sense of the text is preserved. Yet again, a problem is that such long runs don’t happen to occur in random texts. That is why it’s better to use diagram encoding for plain text compression instead of run-length encoding.

Few Questions

After understanding the principles of the diagram encoding, let’s see some examples. In the example above it is better to replace doubled letters with something shorter. Let’s say # for “cc”, @ for “ss” and % for “ll”. Thus the input text will be compressed as “su#e@fu%y a#omplished”, which is shorter. But yet again what will happen if the input message contains one of the substitutions? Also we can’t say if there are many doubled letters and enough reasonable substitutions for them. A better approach is to replace patterns.

Pattern Substitution

The pattern substitution algorithm is a variant of the diagram encoding. As I said above in plain English a very commonly used pattern can be “ the “, which is five characters long. We can now replace it with something like “$%” for example. In this case the message “I send the message” will become “I send$%message”. However there are some obstacles to overcome.

The first problem is that we need to know the language and somehow to define commonly used patterns in a dictionary. What would happen with a message written in some language we don’t know nothing about. Let’s say – Latin like the example bellow.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras venenatis, sapien eget suscipit placerat, justo quam blandit mauris, quis tempor ante sapien sodales augue. Praesent ut mauris quam. Phasellus scelerisque, ante quis consequat tristique, metus turpis consectetur leo, vitae facilisis sapien mi eu sapien. Praesent vitae ligula elit, et faucibus augue. Sed rhoncus sodales dolor ut gravida. In quis augue ac nulla auctor mattis sed sed libero. Donec eget purus eget enim tempor porta vitae eget diam. Mauris aliquet malesuada ipsum, non pulvinar urna vestibulum ac. Donec feugiat velit vitae nunc cursus imperdiet. Donec accumsan faucibus dictum. Phasellus sed mauris sapien. Maecenas mi metus, tincidunt sed rhoncus nec, sodales non sapien.

Clearly without knowing Latin it isn’t easy to define which are those commonly used patterns. The thing is that it’s better to use pattern substitution if you know in advance the set of words and characters.

The second problem is related to decompression. It is obvious that we need to define a dictionary and this dictionary must be used when decoding the message. It will be great also if we find more patterns longer than three characters. If not, the compression ratio will be low. Unfortunately such patterns aren’t very common in any natural language.

Application

It is interesting to answer the question, how to use diagram encoding or patter substitution to compress text in natural language, especially when we don’t know the language in detail? The answer hides in the question. We wont compress natural languages, but machine language. Exactly machine (programming) languages are limited to a smaller sets of words and symbols. Isn’t it true for any programing language? Like PHP, where words like “function”, “while”, “for”, “break”, “switch”, “foreach” happen to be often in use, or HTML with its defined set of tags. Perhaps the best example is CSS, where only the values of the properties can vary. CSS files also tend to have multiple new lines, tabs and spaces, which only humans read.

The question here is why should we compress those file types. It’s clear that after the compression they will be completely useless, both for humans and machines. Yes, that is true, but what if we have to store versions of those files into a DB. Kind of a backup. Imagine you’re working for a web hosting company that has to store daily versions of the sites it’s hosting. Thus the volume of stored information even for small companies hosting only few sites can be enormous. The problem is that compressing those files with some conventional compressing tool isn’t a good idea. Thus we’ve to save a copy of the entire site every day, but as we know the difference between daily versions of a site can be small. A version control system is another solution, but then you’ve to store the plain text of the files.

Perhaps a better approach is to compress the text using pattern substitution and then saving only differences – kind of version control, which can be done with “relative encoding”.

Using the above method we can save lots of disk space and in the same time we can compress/decompress easily. Another good thing is that you can save only changes to the initial files, like version control, which can also be compressed.

Implementation

The implementation of this algorithm is again on PHP and tries only to describe the main principles of compression. In this case I tried to compress a CSS file using the compression above. Although this example is quite primitive we can see some interesting facts. First of all you only need encoding and decoding dictionaries. Practically the encoding and decoding processes are equal, so you don’t need to implement two different functions. Here in this example a native PHP function is used – str_replace, because the purpose of this algorithm is not to describe pattern substitution techniques, but pattern substitution. It assumes that today’s programming languages have string manipulation functions for the purposes of this task.

$str = file_get_contents('large_style_file.css');

$encoding_dict = array(
	"\n" 		=> '$0',
	'text' 		=> '$1',
	'color' 	=> '$2',
	'display' 	=> '$3',
	'font' 		=> '$4',
	'width' 	=> '$5',
	'height'	=> '$6',	
	' '		=> '',
);

function replace_patterns($input, $dict) 
{
	foreach ($dict as $pattern => $replace) {
		$input = str_replace($pattern, $replace, $input);
	}
	
	return $input;
}

$result = replace_patterns($str, $encoding_dict);

By only replacing few CSS properties I achieved almost 40% of compression ratio (as shows the diagram bellow). The initial file is 202 KB, while compressed it’s only 131 KB. Of course, it all depends on the CSS file, but how about replacing all property names with shorter ones. Perhaps then the compression will be even better.