So far we know how to implement graph depth-first and breadth-first search. These two approaches are crucial in order to understand graph traversal algorithms. However they are just explaining how we can walk through in breadth or depth and sometimes this isn’t enough for an efficient solution of graph traversal.

In the examples so far we had an undirected, unweighted graph and we were using adjacency matrices to represent the graphs. By using adjacency matrices we store 1 in the A[i][j] if there’s an edge between vertex i and vertex j. Otherwise we put a 0. However the value of 1 gives us only the information that we have an edge between two vertices, which is not always enough when designing graphs.

Indeed graphs can be weighted. Sometimes the path between two vertices can have a value. Thinking of a road map we know that distances between cities are represented in miles or kilometers. Thus often representing a road map as a graph, we don’t put just 1 between city A and city B, to say that there is a path between them, but also we put some meaningful information – let’s say the distance in miles between A and B.

Note that this value can be the distance in miles, but it can be something else, like the time in hours we’ve to walk between those two cities. In general this value is a function of A and B. So if we keep the distance between A and B we can say this function is F(A, B) = X, or distance(A, B) = X miles.

Of course in this particular example F(A, B) = F(B, A), but this isn’t always true in practice. We can have a directed graph where F(A, B) != F(B, A).

Here I talk about distance between two cities and it is the edge that brings some additional information. However sometimes we have to store the value of the vertices. Let’s say I’m playing a game (like chess) and each move brings me some additional benefit. So each move (vertex) can be evaluated with some particular value. Thus sometimes we don’t have a function of and edge like F(A, B), but function of the vertices, like F(A) and F(B).

In breadth-first search and depth-first search we just pick up a vertex and we consecutively walk through all its successors that haven’t been visited yet.

So in DFS in particular we started from left to right in the array above. So the first node that has to be explored is vertex “1”.

0: [0, 1, 0, 0, 1, 1]

However sometimes, as I said above, we have weighted graphs, so the question is – is there any problem, regarding to the algorithm speed, if we go consecutively through all successors. The answer in general is yes, so we must modify a bit our code in order to continue not with the first but with the best matching successor. By best-matching we mean that the successor should match some criteria like – minimal or maximal value.

Overview

In the following example we see that some of the successors of vertex 0 are very far from it, while others are closer. Thus 4 has the value of 5, while node 1’s value is 2 and 5 is 1.

0: [0, 2, 0, 0, 5, 1]

In this case if we’re searching for the shortest path between 1 and 3, although 1 and 4 are the first two successors in the adjacency matrix of the “start” vertex, we don’t choose them since there’s a better solution – going through node 5.

Problems

The question is – are we sure that by choosing node 5, we’ll find the best path? Even more! Is there a path through node 5? As we see on the image below both cases are possible.

Practically best-first search is identical with depth-first search, with the main difference that we choose the best-matching successor instead of choosing the first matching successor. So we’re sure that we’re going through all the successors but in some particular order, different from DFS. Thus we know that if there’s a path we’ll find it.

However even if we find the path between A and B, we can’t be sure that there is not a better path. We only know that this path is the best so far.

Another question is – how can we find the best matching successor effectively. Well if we’re looking for the minimal or maximal value one possible solution is to sort the array of successors.

0: [0 => 0, 1 => 2, 2 => 0, 3 => 0, 4 => 5, 5 => 1]
// sorted by value
0: [5 => 1, 1 => 2, 4 => 5, 0 => 0, 2 => 0, 3 => 1]

Another good approach will be to use priority queues or heaps.

Thus on every step we’ll get the best matching successor.

Code

In general best-first search uses the ground of depth-first search, so its implementation isn’t more difficult! The following PHP code snippet shows the very small difference between these two algorithms.

class Graph 
{
    protected $_len = 0;
    protected $_g = array();
    protected $_visited = array();
    
    public function __construct()
    {
        $this->_g = array(
            array(0, 2, 0, 0, 5, 1),
            array(1, 0, 3, 0, 0, 0),
            array(0, 2, 0, 8, 0, 0),
            array(0, 0, 3, 0, 5, 0),
            array(1, 0, 0, 8, 0, 1),
            array(1, 0, 0, 0, 5, 0),
        );
        
        $this->_len = count($this->_g);
        
        $this->_initVisited();
    }
    
    protected function _initVisited()
    {
        for ($i = 0; $i < $this->_len; $i++) {
            $this->_visited[$i] = 0;
        }
    }
    
    public function bestFirst($vertex)
    {
        $this->_visited[$vertex] = 1;
    
        echo $vertex . "\n";
        
        asort($this->_g[$vertex]);
        
        foreach ($this->_g[$vertex] as $key => $v) {
            if ($v > 0 && !$this->_visited[$key]) {
                $this->bestFirst($key);
            }
        }
    }
}

$g = new Graph();
// 2 1 0 5 4 3
$g->bestFirst(2);

Application

Best-first search is a typical greedy algorithm. In its principles lies the main greedy approach of chosing the best possible solution so far. It is important to note that depth-first search and breadth-first search are the very basic graph walk through approaches, but they can be also widely extended in order to solve more complex problems.

Here’s one task related to the tree data structure. Given two nodes, can you find their lowest common ancestor?

In a matter of fact this task always has a proper solution, because at least the root node is a common ancestor of all pairs of nodes. However here the task is to find the lowest one, which can be quite far from the root.

We don’t care what kind of trees we have. However the solution, as we will see, can be very different depending on the tree type. Indeed finding the lowest common ancestor can have linear complexity for binary search trees, which isn’t true for ordinary trees.

Overview

Let’s say we have a tree (not binary!) and two nodes from this tree. The task is to find their lowest common ancestor. The thing is that we don’t know much about where they appear to be in the tree.

We can think of this tree as a DOM tree of any single HTML page online. It is not binary or balanced and can’t be sure where these nodes are.

First we must find both paths from the root to each one of the target nodes. Note that this requires additional memory! Then, in linear time we can pass through these “paths” and scan them from the root down to the nodes. We expect these to arrays to be equal at least in their first element (the root). Using this scenario the lowest common ancestor is the last equal element in both arrays.

To see how this algorithm can be dramatically changed depending on the data structure, let’s see another example. Now let’s say we have a binary search tree (BST). We know that in a BST all the elements in the left sub-tree are smaller than the root and all the items on the right sub-tree are greater than the root. This is true also for the left and the right sub-trees.

Now because we’re searching the lowest common ancestor, we don’t need to collect the paths from the root to the nodes in two arrays. We just know that the greater items are on the right, while the smaller items are on the left. This can help us find the lowest ancestor starting directly from the root.

What we do is to compare the two keys of the target nodes with the root key. If one of the keys are smaller, and the other is greater than the root’s key, then obviously the root is the lowest common ancestor. This is true because one of the items will be somewhere in the left sub-tree, while the other will be in the right sub-tree.

In case both values are greater (or smaller) than the root, we can move to the right (or to the left) sub-tree and try again with the same procedure. Thus the first node which key is in between the two target values will be the lowest common ancestor.

Code

Here’s a very simple PHP implementation showing us these two algorithms.

class Tree
{
    public $node = null;
    public $id = null;
    public $parent = null;
    public $children = array();
    
    public function __construct($node, $id = null)
    {
        $this->node = $node;
        $this->id = $id;
    }
    
    public function addChild(Node &$n)
    {
        $n->parent = $this;
        $this->children[] = $n;
    }
    
    /**
     * Returns an element by its id
     * 
     * @param mixed $id
     * @return Node 
     */
    public function search($id)
    {
        if ($this->id == $id) {
            return $this;
        }

        $a = false;
        
        // search all the children starting from the left-most
        foreach ($this->children as $child) {
            $a = $child->search($id);
        }
        
        return $a;
    }
    
    /**
     * Finds a path from the root to the 
     * item and returns it as a list
     * 
     * @param mixed $id 
     * @return array
     */
    public function find_path($id, &$path)
    {
        array_push($path, $this->id);
        
        if ($this->id == $id) {
            return 1;
        }
        
        foreach ($this->children as $child)  {
            if (1 == $child->find_path($id, $path)) return 1;
            array_pop($path);
        }
    }
    
    public function __toString()
    {
        return $this->node . ' ' . $this->id . "\n";
    }
}

$dom = new Tree('DOM', 'ROOT');

$body = new Tree('BODY', 1);
$div1 = new Tree('DIV', 'div-1');
$div2 = new Tree('DIV', 'my-id');

$a = new Tree("A", 'some-link');

$dom->addChild($body);
$body->addChild($div1);
$body->addChild($div2);
$div2->addChild($a);

$path1 = $path2 = array();
$dom->find_path('div-1', $path1);
$dom->find_path('some-link', $path2);

Finding Lowest Common Ancestor in a BST

class Tree
{
    public $key;
    
    public $parent  = null;
    public $left    = null;
    public $right   = null;
    
    public function __construct($key) 
    {
        $this->key = $key;
    }
    
    public function insert(Tree $n) 
    {
        if ($this->key < $n->key) {
            if ($this->right == null) {
                // insert
                $this->right = $n;
                $n->parent = $this;
            } else {
                $this->right->insert($n);
            }
        }
        if ($this->key > $n->key) {
            if ($this->left == null) {
                // insert
                $this->left = $n;
                $n->parent = $this;
            } else {
                $this->left->insert($n);
            }
        }
    }
}

$t = new Tree(10);

$n1 = new Tree(20);
$n2 = new Tree(5);
$n3 = new Tree(7);
$n4 = new Tree(13);

$t->insert($n1);
$t->insert($n2);
$t->insert($n3);

function find_common($node1, $node2, $tree) 
{
    if ($node1->key < $tree->key && $node2->key > $tree->key) {
        return $tree;
    } else if ($node1->key < $tree->key && $node2->key < $tree->key) {
        find_common($node1, $node2, $tree->left);
    } else if ($node1->key > $tree->key && $node2->key > $tree->key) {
        find_common($node1, $node2, $tree->right);
    }
}

$node = find_common($n3, $n4, $t);

Application

A typical use-case of this algorithm is finding the lowest common ancestor of two nodes in a DOM tree. Sometimes we just need to attach an event listener to both items (even before they are attached to the DOM!). Although attaching this event to the “document” will work just fine, all the elements from the nodes up to the root will be “capturing” these events due to event bubbling. Thus attaching the event to the lowest common ancestor is a better solution.

The binary search is perhaps the most famous and best suitable search algorithm for sorted arrays. Indeed when the array is sorted it is useless to check every single item against the desired value. Of course a better approach is to jump straight to the middle item of the array and if the item’s value is greater than the desired one, we can jump back again to the middle of the interval. Thus the new interval is half the size of the initial one.

If the searched value is greater than the one placed at the middle of the sorted array, we can jump forward. Again on each step the considered list is getting half as long as the list on the previous step, as shown on the image bellow.

Implementation

Here’s a sample implementation of this algorithm on PHP. Obviously the nature of this approach is guiding us to a recursive implementation, but as we know, sometimes recursion can be dangerous. That’s why here we can see either the recursive and iterative solution.

Recursive Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function binary_search($x, $list, $left, $right) 
{
	if ($left > $right)
		return -1;
	
	$mid = ($left + $right) >> 1;

	if ($list[$mid] == $x) {
		return $mid;
	} elseif ($list[$mid] > $x) {
		return binary_search($x, $list, $left, $mid-1);
	} elseif ($list[$mid] < $x) {
		return binary_search($x, $list, $mid+1, $right);
	}
}

echo binary_search($x, $list, 0, count($list)-1);

Iterative Binary Search

$list = array(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144);
$x = 55;

function iterative_binary_search($x, $list) 
{
	$left = 0;
	$right = count($list)-1;
	
	while ($left <= $right) {
		$mid = ($left + $right) >> 1;
		
		if ($list[$mid] == $x) {
			return $mid;
		} elseif ($list[$mid] > $x) {
			$right = $mid - 1;
		} elseif ($list[$mid] < $x) {
			$left = $mid + 1;
		}
	}
	
	return -1;
}

echo iterative_binary_search($x, $list);

Caution: Optimization

Most of the optimization techniques mentioned online recommend to replace the expensive operation of dividing by 2 with its bitwise equivalent (n >> 1) == n/2. That is not always true and it is very dependant from the programming language. Thus in PHP those operations are fairly similar as PHP is written in C. You’ve to be aware of the language specific features when optimizing code.

Fibonacci Search

Every developer has heard of Fibonacci and his sequence. The Fibonacci search algorithm is practically a variation of the binary search algorithm. In fact the only difference is that the binary search algorithm divides the list into two equal parts, while the Fibonacci search divides it in two but not equal parts. In fact sometimes it is faster to search if you divide the list by such non equal sub-lists. However the length of the sub-lists is not random.

It is clear that the ratio of any two consecutive numbers in the Fibonacci sequence is practically forming the golden ratio. This can lead us to another variation of Fibonacci and binary search - the golden section search. The only different thing is that you’ve to divide the length of the list in two parts exactly by the golden ratio.

The complexity both of the Fibonacci and the golden section search algorithm is identical with the complexity of the binary search. However these two algorithms are rarely used in practice. Also it is more difficult to implement these two algorithms than the binary search and their advantage depends on specifically dispersed data.

Complexity

The complexity of the binary search algorithm is intuitively clear - O(log(n)), which makes it far more effective than the sequential search.

Application

It is useless to mention examples of its use. This algorithm is easy to implement and in the same times it is very fast. Yes, indeed, this algorithm is only possible on sorted lists and this is a limitation. Also, as I said, compared to the jump search here we have more than one jump back in most of the cases, which sometimes can be more expensive than jump forward. However is this the fastest search algorithm? I’ll try to answer this question in my next article.