I wrote about binary search in my previous post, which is indeed one very fast searching algorithm, but in some cases we can achieve even faster results. Such an algorithm is the “interpolation search” – perhaps the most interesting of all searching algorithms. However we shouldn’t forget that the data must follow some limitations. In first place the array must be sorted. Also we must know the bounds of the interval.

Why is that? Well, this algorithm tries to follow the way we search a name in a phone book, or a word in the dictionary. We, humans, know in advance that in case the name we’re searching starts with a “B”, like “Bond” for instance, we should start searching near the beginning of the phone book. Thus if we’re searching the word “algorithm” in the dictionary, you know that it should be placed somewhere at the beginning. This is because we know the order of the letters, we know the interval (a-z), and somehow we intuitively know that the words are dispersed equally. These facts are enough to realize that the binary search can be a bad choice. Indeed the binary search algorithm divides the list in two equal sub-lists, which is useless if we know in advance that the searched item is somewhere in the beginning or the end of the list. Yes, we can use also jump search if the item is at the beginning, but not if it is at the end, in that case this algorithm is not so effective.

So the interpolation search is based on some simple facts. The binary search divides the interval on two equal sub-lists, as shown on the image bellow.

What will happen if we don’t use the constant ½, but another more accurate constant “C”, that can lead us closer to the searched item.

The question is how to find this value? Well, we know bounds of the interval and looking closer to the image above we can define the following formula.

C = (x-L)/(R-L)

Now we can be sure that we’re closer to the searched value.

Implementation

Here’s an implementation of interpolation search in PHP.

$list = array(201, 209, 232, 233, 332, 399, 400);
$x = 332;

function interpolation_search($list, $x)
{
	$l = 0;
	$r = count($list) - 1;

	while ($l <= $r) {
		if ($list[$l] == $list[$r]) {
			if ($list[$l] == $x) {
				return $l;
			} else {
				// not found
				return -1;
			}
		}
		
		$k = ($x - $list[$l])/($list[$r] - $list[$l]);
		
		// not found
		if ($k < 0 || $k > 1) {
			return -1;
		}
		
		$mid = round($l + $k*($r - $l));
		
		if ($x < $list[$mid]) {
			$r = $mid - 1;
		} else if ($x > $list[$mid]) {
			$l = $mid + 1;
		} else {
			// success!
			return $mid;
		}
		
		// not found
		return -1;
	}
}

echo interpolation_search($list, $x);

Complexity

The complexity of this algorithm is log₂(log₂(n)) + 1. While I wont cover its proof, I’ll say that this is very slowly growing function as you can see on the following chart.

Indeed when the values are equally dispersed into the interval this search algorithm can be extremely useful – way faster than the binary search. As you can see log₂(log₂(100 M)) ≈ 4.73 !!!

Application

As I said already this algorithm is extremely interesting and very appropriate in many use cases. Here’s an example where interpolation search can be used. Let’s say there’s an array with user data, sorted by their year of birth. We know in advance that all users are born in the 80’s. In this case sequential or even binary search can be slower than interpolation search.

$list = array(
	0 => array('year' => 1980, 'name' => 'John Smith', 'username' => 'John'),
	1 => array('year' => 1980, ...),
	...
	10394 => array('year' => 1981, 'name' => 'Tomas M.', ...),
	...
	348489 => array('year' => '1985', 'name' => 'James Bond', ...),
	...
	2808008 => array('year' => '1990', 'name' => 'W.A. Mozart', ...)
);

Now if we search for somebody born in 1981 a good approach is to use interpolation search.

The binary search is perhaps the most famous and best suitable search algorithm for sorted arrays. Indeed when the array is sorted it is useless to check every single item against the desired value. Of course a better approach is to jump straight to the middle item of the array and if the item’s value is greater than the desired one, we can jump back again to the middle of the interval. Thus the new interval is half the size of the initial one.

If the searched value is greater than the one placed at the middle of the sorted array, we can jump forward. Again on each step the considered list is getting half as long as the list on the previous step, as shown on the image bellow.

Implementation

Here’s a sample implementation of this algorithm on PHP. Obviously the nature of this approach is guiding us to a recursive implementation, but as we know, sometimes recursion can be dangerous. That’s why here we can see either the recursive and iterative solution.

Recursive Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function binary_search($x, $list, $left, $right) 
{
	if ($left > $right)
		return -1;
	
	$mid = ($left + $right) >> 1;

	if ($list[$mid] == $x) {
		return $mid;
	} elseif ($list[$mid] > $x) {
		return binary_search($x, $list, $left, $mid-1);
	} elseif ($list[$mid] < $x) {
		return binary_search($x, $list, $mid+1, $right);
	}
}

echo binary_search($x, $list, 0, count($list)-1);

Iterative Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function iterative_binary_search($x, $list) 
{
	$left = 0;
	$right = count($list)-1;
	
	while ($left <= $right) {
		$mid = ($left + $right) >> 1;
		
		if ($list[$mid] == $x) {
			return $mid;
		} elseif ($list[$mid] > $x) {
			$right = $mid - 1;
		} elseif ($list[$mid] < $x) {
			$left = $mid + 1;
		}
	}
	
	return -1;
}

echo iterative_binary_search($x, $list);

Caution: Optimization

Most of the optimization techniques mentioned online recommend to replace the expensive operation of dividing by 2 with its bitwise equivalent (n >> 1) == n/2. That is not always true and it is very dependant from the programming language. Thus in PHP those operations are fairly similar as PHP is written in C. You’ve to be aware of the language specific features when optimizing code.

Fibonacci Search

Every developer has heard of Fibonacci and his sequence. The Fibonacci search algorithm is practically a variation of the binary search algorithm. In fact the only difference is that the binary search algorithm divides the list into two equal parts, while the Fibonacci search divides it in two but not equal parts. In fact sometimes it is faster to search if you divide the list by such non equal sub-lists. However the length of the sub-lists is not random.

It is clear that the ratio of any two consecutive numbers in the Fibonacci sequence is practically forming the golden ratio. This can lead us to another variation of Fibonacci and binary search - the golden section search. The only different thing is that you’ve to divide the length of the list in two parts exactly by the golden ratio.

The complexity both of the Fibonacci and the golden section search algorithm is identical with the complexity of the binary search. However these two algorithms are rarely used in practice. Also it is more difficult to implement these two algorithms than the binary search and their advantage depends on specifically dispersed data.

Complexity

The complexity of the binary search algorithm is intuitively clear - O(log(n)), which makes it far more effective than the sequential search.

Application

It is useless to mention examples of its use. This algorithm is easy to implement and in the same times it is very fast. Yes, indeed, this algorithm is only possible on sorted lists and this is a limitation. Also, as I said, compared to the jump search here we have more than one jump back in most of the cases, which sometimes can be more expensive than jump forward. However is this the fastest search algorithm? I’ll try to answer this question in my next article.

In my previous article I discussed how the sequential (linear) search can be used on an ordered lists, but then we were limited by the specific features of the given task. Obviously the sequential search on an ordered list is ineffective, because we consecutively check every one of its elements. Is there any way we can optimize this approach? Well, because we know that the list is sorted we can check some of its items, but not all of them. Thus when an item is checked, if it is less than the desired value, we can skip some of the following items of the list by jumping ahead and then check again. Now if the checked element is greater than the desired value, we can be sure that the desired value is hiding somewhere between the previously checked element and the currently checked element. If not, again we can jump ahead. Of course a good approach is to use a fixed step. Let’s say the list length is n and the step’s length is k. Basically we check list(0), then list(k-1), list(2k-1) etc. Once we find the interval where the value might be (m*k-1 < x <= (m+1)*k – 1), we can perform a sequential search between the last two checked positions. By choosing this approach we avoid a lot the weaknesses of the sequential search algorithm. Many comparisons from the sequential search here are eliminated.

How to choose the step’s length

We know that it is a good practice to use a fixed size step. Actually when the step is 1, the algorithm is the traditional sequential search. The question is what should be the length of the step and is there any relation between the length of the list (n) and the length of the step (k)? Indeed there is such a relation and often you can see sources directly saying that the best length k = √n. Why is that?

Well, in the worst case, we do n/k jumps and if the last checked value is greater than the desired one, we do at most k-1 comparisons more. This means n/k + k – 1 comparisons. Now the question is for what values of k this function reaches its minimum. For those of you who remember maths classes this can be found with the formula -n/(k^2) + 1 = 0. Now it’s clear that for k = √n the minimum of the function is reached.

Of course you don’t need to prove this every time you use this algorithm. Instead you can directly assign √n to be the step length. However it is good to be familiar with this approach when trying to optimize an algorithm.

Let’s cosider the following list: (0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610). Its length is 16. Jump search will find the value of 55 with the following steps.

Implementation

Let’s see an example of jump search, written in PHP.

$list = array();

for ($i = 0; $i < 1000; $i++) {
	$list[] = $i;
}

// now we have a sorted list: (0, 1, 2, 3, ..., 999)

function jump_search($x, $list)
{
	// calculate the step
	$len = count($list);
	$step = floor(sqrt($len));
	$prev = 0;
	
	while ($list[($step < $len ? $step : $len)] < $x) {
		$prev = $step;
		$step += floor(sqrt($len));
		
		if ($step >= $len) {
			return FALSE;
		}
	}
	
	while ($list[$prev] < $x) {
		$prev++;
		if ($prev == ($step < $len ? $step : $len)) {
			return FALSE;
		}
	}
	
	if ($list[$prev] == $x) {
		return $prev;
	}
	
	return FALSE;
}

echo (int)jump_search(674, $list);

Here we have a sorted list with 1000 elements that looks like this: (0, 1, 2, ..., 999). Obviously with sequential search we'll find the value of 674 with exactly on the 674-th iteration. Here, with jump search we can reach it on the 44-th iteration, and this shows us the advantage of jump search over the sequential search on ordered lists.

Further Optimization

Although all examples here deal with small lists in practice this is not always true. Sometimes the step itself can be a very large number, so once you know the interval where the desired value could be you can perform jump search again.

We saw that the best size of the step is √n, but it is not a good idea to start from the first element of the list just as we didn’t in the example above. A better option is to begin from kth item. Now we can improve the above solution.

Complexity

Obviously the complexity of the algorithm is O(√n), but once we know the interval where the value is we can improve it by applying jump search again. Indeed let’s say the list length is 1,000,000. The jump interval should be: √1000000=1000. As you can see again, you can use jump search with a new step √1000≈31. Every time we find the desired interval we can apply the jump search algorithm with a smaller step. Of course finally the step will be 1. In this case the complexity of the algorithm is no longer O(√n). Now its complexity is approaching logarithmic value. The problem is that the implementation of this approach is considered to be more difficult than the binary search, where the complexity is also O(log(n)).

Application

As almost every algorithm the jump search is very convinient for a certain kind of tasks. Yes, the binary search is easy to implement and its complexity is O(log(n)), but in case of a very large list the direct jump to the middle can be a bad idea. Then we should make a large step back if the searched value is placed at the beginning of the list.

Perhaps every one of us has performed some sort of a primitive jump search in his life without even knowing it. Do you remember cassette recorders? We used the "fast forward" key and periodically checked whether the tape was on our favorite song. Once we stopped at the middle of the song we used the "rewind" button to find exactly the beginning of the song.

This clumsy example can give us the answer of where jump search can be better than binary search. The advantage of jump search is that you need to jump back only once (in case of the basic implementation).

If jumping back takes you significantly more time than jumping forward then you should use this algorithm.