Since with graphs we can represent real-life problems it’s almost clear why we would need an efficient algorithm that calculates the shortest path between two vertices. Getting back to our example of a road map we can use such an algorithm in order to find the shortest path between two cities. This example, of course, is very basic indeed, but it can give us a clear example of where shortest path can be applied.

In the other hand, we can model an enormous field of real-life problems using graphs – not only road maps. As we already know, whenever we have relations between different abstract objects we can refer an efficient graph algorithm.

OK, so we need a shortest path algorithm, but before we proceed with the exact algorithm first we’ll need to answer some questions and give some definitions.

Overview

First we need a definition of the terms distance and path between two nodes. A path is considered to be the sequence of vertices (or edges if you wish) between two vertices i and j. Of course we assume that there might be no path between any to vertices in the graph! Also we assume that this definition relates both for directed and undirected graphs. After we have the definition of a path we can proceed by defining a “distance”, which is said to be the number of edges in the path between i and j.

Using this terms, if there’s an edge between i and j, the path between them is [i, j], while the distance is 1. Of course, for an undirected graph (i, j) equals to (j, i) and the path [i, j] equals the path [j, i], but that isn’t true for directed graphs where the path [i, j] differs in general from [j, i].

Here we talk about the path between two adjacent vertices, but we can go with the more general case of a path between two vertices that aren’t adjacent.

Now, getting back to the road map example, there might be many paths between city A and city B.

What we actually need to find is the shortest one. This can be very important, because we often want to get from A to B as quickly as possible using the shortest path.

So first, what is a shortest path between i and j. Well, besides the strict definition, I’ll give a simplified one that might be clearer. The shortest path between i and j is such a path, which has the lowest distance compared to all other paths between i and j.

In our algorithm we will use breadth-first search. Why? That is because by using BFS by starting at a given point we expand our search consecutively starting with the closest vertices.

Is breadth-first search enough and will it give us the correct answer – the shortest path between i and j. Actually breadth-first search will gives us even more – the shortest paths to each reachable vertex from a given starting point – the staring vertex.

Why this is correct? Well, because of the nature of the breadth-first search algorithm. As we already know BFS uses a queue in order to store the front of the expansion. Usually as an abstraction BFS colors the vertices in white, gray and black, where the white vertices are those that aren’t visited yet, the gray are in the queue and the black vertices are already visited.

However how can be sure that BFS will give us the shortest paths to each vertex? To answer this question and to be sure that BFS will work for us we must take a closer look at the queue. Clearly by starting at a given point the algorithm is correct – the distance is 0.

Now the second step is to put into the queue all the vertices adjacent to s (where s is the starting point). Clearly this will give us the shortest paths to all adjacent vertices of s.

Continuing by induction we can assume that at level k we have all the shortest paths from s to all the vertices at the level k. It is clear the path between s and the vertices at level k is k, since we assume that each edge adds 1 to the path from s to i. Now by adding all the vertices adjacent (and not visited yet) to the paths of level k we get paths with length k+1 which is again the shortest paths from s to level k+1.

Actually we can talk about a tree built out of the graph by staring at s (which is the root of the tree).

Code

OK, now we know that BFS will find us the shortest paths from s to all the reachable vertices from s. Here’s a simple PHP implementation, that makes use of the Standard PHP Library data structures. Of course, everyone can code and use his own implementation of lists in order to keep the information of the adjacency lists.

The important thing to note is that we keep an additional information in each vertex – the distance between it and s, which is initially infinite. First we go with the modification of BFS in order to find all the distances between s and the other vertices.

Here’s our graph:

class vertex
{
    public $key = null;
    public $color = 'white';
    public $distance = -1;  // infinite
    
    public function __construct($key) 
    {
        $this->key = $key;
    }
}

$v0 = new vertex(0);
$v1 = new vertex(1);
$v2 = new vertex(2);
$v3 = new vertex(3);
$v4 = new vertex(4);
$v5 = new vertex(5);

$list0 = new SplDoublyLinkedList();
$list0->push($v1);
$list0->push($v3);
$list0->rewind();

$list1 = new SplDoublyLinkedList();
$list1->push($v0);
$list1->push($v2);
$list1->rewind();

$list2 = new SplDoublyLinkedList();
$list2->push($v1);
$list2->push($v3);
$list2->push($v4);
$list2->rewind();

$list3 = new SplDoublyLinkedList();
$list3->push($v1);
$list3->push($v2);
$list3->rewind();

$list4 = new SplDoublyLinkedList();
$list4->push($v2);
$list4->push($v5);
$list4->rewind();

$list5 = new SplDoublyLinkedList();
$list5->push($v4);
$list5->rewind();

$adjacencyList = array(
    $list0,
    $list1,
    $list2,
    $list3,
    $list4,
    $list5,
);

function calcDistances(vertex $start, &$adjLists)
{
    // define an empty queue
    $q = array();
    
    // push the starting vertex into the queue
    array_push($q, $start);
    
    // color it gray
    $start->color = 'gray';
    
    // mark the distance to it 0
    $start->distance = 0;
    
    while ($q) {
        // 1. pop from the queue
        $t = array_pop($q);
        
        // 2. foreach poped item find it's adjacent white vertices
        $l = $adjLists[$t->key];
        while ($l->valid()) {
            // 3. mark them gray, increment their length with one from their parent
            if ($l->current()->color == 'white') {
                $l->current()->color = 'gray';
                $l->current()->distance = $t->distance + 1;
                // 4. push them to the queue
                array_push($q, $l->current());
            }
            
            $l->next();
        }
    }
}

calcDistances($v0, $adjacencyList);

print_r($adjacencyList);

Now we can modify the algorithm even more and we add the path property of each vertex. Now each vertex will keep the path from s.

class vertex
{
    public $key         = null;
    public $color       = 'white';
    public $distance    = -1;  // infinite
    public $path        = null;
    
    public function __construct($key) 
    {
        $this->key  = $key;
    }
}

$v0 = new vertex(0);
$v1 = new vertex(1);
$v2 = new vertex(2);
$v3 = new vertex(3);
$v4 = new vertex(4);
$v5 = new vertex(5);

$list0 = new SplDoublyLinkedList();
$list0->push($v1);
$list0->push($v3);
$list0->rewind();

$list1 = new SplDoublyLinkedList();
$list1->push($v0);
$list1->push($v2);
$list1->rewind();

$list2 = new SplDoublyLinkedList();
$list2->push($v1);
$list2->push($v3);
$list2->push($v4);
$list2->rewind();

$list3 = new SplDoublyLinkedList();
$list3->push($v1);
$list3->push($v2);
$list3->rewind();

$list4 = new SplDoublyLinkedList();
$list4->push($v2);
$list4->push($v5);
$list4->rewind();

$list5 = new SplDoublyLinkedList();
$list5->push($v4);
$list5->rewind();

$adjacencyList = array(
    $list0,
    $list1,
    $list2,
    $list3,
    $list4,
    $list5,
);

function calcShortestPaths(vertex $start, &$adjLists)
{
    // define an empty queue
    $q = array();
    
    // push the starting vertex into the queue
    array_push($q, $start);
    
    // color it gray
    $start->color = 'gray';
    
    // mark the distance to it 0
    $start->distance = 0;
    
    // the path to the starting vertex
    $start->path = new SplDoublyLinkedList();
    $start->path->push($start->key);
    
    while ($q) {
        // 1. pop from the queue
        $t = array_pop($q);
        
        // 2. foreach poped item find it's adjacent white vertices
        $l = $adjLists[$t->key];
        while ($l->valid()) {
            // 3. mark them gray, increment their length with one from their parent
            if ($l->current()->color == 'white') {
                $l->current()->color = 'gray';
                $l->current()->distance = $t->distance + 1;
                $l->current()->path = clone $t->path;
                $l->current()->path->push($l->current()->key);
                
                // 4. push them to the queue
                array_push($q, $l->current());
            }
            
            $l->next();
        }
    }
}

calcShortestPaths($v0, $adjacencyList);

print_r($adjacencyList);

Complexity

Clearly the complexity of enqueue and dequeue is O(V), while searching for adjacent vertices is O(E), thus the complexity of this algorithm is O(V + E)!

Application

Finding the shortest path between two nodes is obviousely a very handy algorithm. Applied almost everywhere graphs exists this algorithm is widely used. However there’s one very reasonable question. We’re searching for the shortest path between two vertices and we end with the shortest paths between a starting node an all other vertices? Why we need this “useless” information? Acutally the question should be: is there a faster and more efficient algorithm compared to this one. Well, we’ll see that!

I wrote about binary search in my previous post, which is indeed one very fast searching algorithm, but in some cases we can achieve even faster results. Such an algorithm is the “interpolation search” – perhaps the most interesting of all searching algorithms. However we shouldn’t forget that the data must follow some limitations. In first place the array must be sorted. Also we must know the bounds of the interval.

Why is that? Well, this algorithm tries to follow the way we search a name in a phone book, or a word in the dictionary. We, humans, know in advance that in case the name we’re searching starts with a “B”, like “Bond” for instance, we should start searching near the beginning of the phone book. Thus if we’re searching the word “algorithm” in the dictionary, you know that it should be placed somewhere at the beginning. This is because we know the order of the letters, we know the interval (a-z), and somehow we intuitively know that the words are dispersed equally. These facts are enough to realize that the binary search can be a bad choice. Indeed the binary search algorithm divides the list in two equal sub-lists, which is useless if we know in advance that the searched item is somewhere in the beginning or the end of the list. Yes, we can use also jump search if the item is at the beginning, but not if it is at the end, in that case this algorithm is not so effective.

So the interpolation search is based on some simple facts. The binary search divides the interval on two equal sub-lists, as shown on the image bellow.

What will happen if we don’t use the constant ½, but another more accurate constant “C”, that can lead us closer to the searched item.

The question is how to find this value? Well, we know bounds of the interval and looking closer to the image above we can define the following formula.

C = (x-L)/(R-L)

Now we can be sure that we’re closer to the searched value.

Implementation

Here’s an implementation of interpolation search in PHP.

$list = array(201, 209, 232, 233, 332, 399, 400);
$x = 332;

function interpolation_search($list, $x)
{
	$l = 0;
	$r = count($list) - 1;

	while ($l <= $r) {
		if ($list[$l] == $list[$r]) {
			if ($list[$l] == $x) {
				return $l;
			} else {
				// not found
				return -1;
			}
		}
		
		$k = ($x - $list[$l])/($list[$r] - $list[$l]);
		
		// not found
		if ($k < 0 || $k > 1) {
			return -1;
		}
		
		$mid = round($l + $k*($r - $l));
		
		if ($x < $list[$mid]) {
			$r = $mid - 1;
		} else if ($x > $list[$mid]) {
			$l = $mid + 1;
		} else {
			// success!
			return $mid;
		}
		
		// not found
		return -1;
	}
}

echo interpolation_search($list, $x);

Complexity

The complexity of this algorithm is log₂(log₂(n)) + 1. While I wont cover its proof, I’ll say that this is very slowly growing function as you can see on the following chart.

Indeed when the values are equally dispersed into the interval this search algorithm can be extremely useful – way faster than the binary search. As you can see log₂(log₂(100 M)) ≈ 4.73 !!!

Application

As I said already this algorithm is extremely interesting and very appropriate in many use cases. Here’s an example where interpolation search can be used. Let’s say there’s an array with user data, sorted by their year of birth. We know in advance that all users are born in the 80’s. In this case sequential or even binary search can be slower than interpolation search.

$list = array(
	0 => array('year' => 1980, 'name' => 'John Smith', 'username' => 'John'),
	1 => array('year' => 1980, ...),
	...
	10394 => array('year' => 1981, 'name' => 'Tomas M.', ...),
	...
	348489 => array('year' => '1985', 'name' => 'James Bond', ...),
	...
	2808008 => array('year' => '1990', 'name' => 'W.A. Mozart', ...)
);

Now if we search for somebody born in 1981 a good approach is to use interpolation search.

The binary search is perhaps the most famous and best suitable search algorithm for sorted arrays. Indeed when the array is sorted it is useless to check every single item against the desired value. Of course a better approach is to jump straight to the middle item of the array and if the item’s value is greater than the desired one, we can jump back again to the middle of the interval. Thus the new interval is half the size of the initial one.

If the searched value is greater than the one placed at the middle of the sorted array, we can jump forward. Again on each step the considered list is getting half as long as the list on the previous step, as shown on the image bellow.

Implementation

Here’s a sample implementation of this algorithm on PHP. Obviously the nature of this approach is guiding us to a recursive implementation, but as we know, sometimes recursion can be dangerous. That’s why here we can see either the recursive and iterative solution.

Recursive Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function binary_search($x, $list, $left, $right) 
{
	if ($left > $right)
		return -1;
	
	$mid = ($left + $right) >> 1;

	if ($list[$mid] == $x) {
		return $mid;
	} elseif ($list[$mid] > $x) {
		return binary_search($x, $list, $left, $mid-1);
	} elseif ($list[$mid] < $x) {
		return binary_search($x, $list, $mid+1, $right);
	}
}

echo binary_search($x, $list, 0, count($list)-1);

Iterative Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function iterative_binary_search($x, $list) 
{
	$left = 0;
	$right = count($list)-1;
	
	while ($left <= $right) {
		$mid = ($left + $right) >> 1;
		
		if ($list[$mid] == $x) {
			return $mid;
		} elseif ($list[$mid] > $x) {
			$right = $mid - 1;
		} elseif ($list[$mid] < $x) {
			$left = $mid + 1;
		}
	}
	
	return -1;
}

echo iterative_binary_search($x, $list);

Caution: Optimization

Most of the optimization techniques mentioned online recommend to replace the expensive operation of dividing by 2 with its bitwise equivalent (n >> 1) == n/2. That is not always true and it is very dependant from the programming language. Thus in PHP those operations are fairly similar as PHP is written in C. You’ve to be aware of the language specific features when optimizing code.

Fibonacci Search

Every developer has heard of Fibonacci and his sequence. The Fibonacci search algorithm is practically a variation of the binary search algorithm. In fact the only difference is that the binary search algorithm divides the list into two equal parts, while the Fibonacci search divides it in two but not equal parts. In fact sometimes it is faster to search if you divide the list by such non equal sub-lists. However the length of the sub-lists is not random.

It is clear that the ratio of any two consecutive numbers in the Fibonacci sequence is practically forming the golden ratio. This can lead us to another variation of Fibonacci and binary search - the golden section search. The only different thing is that you’ve to divide the length of the list in two parts exactly by the golden ratio.

The complexity both of the Fibonacci and the golden section search algorithm is identical with the complexity of the binary search. However these two algorithms are rarely used in practice. Also it is more difficult to implement these two algorithms than the binary search and their advantage depends on specifically dispersed data.

Complexity

The complexity of the binary search algorithm is intuitively clear - O(log(n)), which makes it far more effective than the sequential search.

Application

It is useless to mention examples of its use. This algorithm is easy to implement and in the same times it is very fast. Yes, indeed, this algorithm is only possible on sorted lists and this is a limitation. Also, as I said, compared to the jump search here we have more than one jump back in most of the cases, which sometimes can be more expensive than jump forward. However is this the fastest search algorithm? I’ll try to answer this question in my next article.

This is the easiest to implement and the most frequently used search algorithm in practice. Unfortunately the sequential search is also the most ineffective searching algorithm. However, it is so commonly used that it is appropriate to consider several ways to optimize it. In general the sequential search, also called linear search, is the method of consecutively check every value in a list until we find the desired one.

Basic Implementation

The most natural approach is to loop through the list until we find the desired value. Here’s an implementation on PHP using FOR loop, something that can be easily written into any other computer language.

This is really the most ineffective implementation. There are two big mistakes in this code. First of all we calculate the length of the list on every iteration of the array, and secondly after we find the desired element, we don’t break the loop, but continue to loop through the array.

Yes, if the element is repeated without the “break” we can find its last occurrence, but if not the loop will iterate over the end of the array with no practical value.

Optimization of the forward sequential search

… and javascript:

Even with this little optimization the algorithm remains ineffective. As we can see, on every iteration we have two conditional expressions. First we check whether we’ve reached the end of the list, and then we check whether the current element equals to the searched element. So the question is can we reduce the number of the conditional expressions?

Searching in reverse order

Yes, we can reduce the number of comparison instructions from the forward approach of the linear search algorithm by using reverse order searching. Although it seems to be pretty much the same by reversing the order of the search we can discard one of the conditional expressions.

Note that we need to adjust index because of $index—expression.

Indeed here we have only one conditional expression, but the problem is that this implementation is correct ONLY when the element exists in the list, which is not always true. If the element doesn’t appears into the list, then this code can lead to an infinite loop. OK, but how can we stop the loop even when the list doesn’t contain the desired value? The answer is, by adding the searched value to the list.

Sentinel

The above problem can be solved by inserting the desired item as a sentinel value. Thus we’re sure that the list contains the value, so the loop will stop for sure even if at the beginning the value didn’t appear to be part of the list.

This approach can be used to overcome the problem of the reverse linear search approach from the previous section.

Complexity

As I said at the beginning of this post this is one of the most ineffective searching algorithms. Of course the best case is when the searched value is at the very beginning of the list. Thus on the first comparison we can find it. On the other hand the worst case is when the element is located at the very end of the list. Assuming that we don’t know where the element is and the possibility to be anywhere in the list is absolutely equal, then the complexity of this algorithm is O(n).

Different cases

We must remember, however, that the algorithm’s complexity can vary depending on whether the element occurs once.

Is it so ineffective?

Sequential search can be very slow compared to binary search on an ordered list. But actually this is not quite true. Sequential search can be faster than binary search for small arrays, but it is assumed that for n < 8 the sequential search is faster.

Application

The linear search is really very simple to implement and most web developers go to the forward implementation, which is the most ineffective one. On the other hand this algorithm is quite useful when we search in an unordered list. Yes, searching in an ordered list is something that can dramatically change the search algorithm. Actually searching and sorting algorithms are often used together.

A typical case is pulling something from a database, usually in form of a list and then search for some value in it. Unfortunately in most of the cases the database orders the returned result set and yet most of the developers perform a consecutive search over the list. Yet again when the list is ordered it is better to use binary search instead of sequential search.
Let’s say we have a CSV file containing the usernames and the names of our users.

Username,Name
jamesbond007,James Bond
jsmith,John Smith
...

Now we fetch these values into an array.

// work case
$arr = array(
    array('name' => 'James Bond', 'username' => 'jamesbond007'),
    array('name' => 'John Smith', 'username' => 'jsmith')
);

Now using sequential search …

// using a sentinel
$x = 'jsmith';
$arr[] = array('username' => $x, 'name' => '');
$index = 0;

while ($arr[$index++]['username'] != $x);

if ($index < count($arr)) {
    echo "Hello, {$arr[$index-1]['name']}";
} else {
    echo "Hi, guest!";
}