A very common problem in computer programming is finding the longest increasing (decreasing) subsequence in a sequence of numbers (usually integers). Actually this is a typical dynamic programming problem.

Dynamic programming can be described as a huge area of computer science problems that can be categorized by the way they can be solved. Unlike divide and conquer, where we were able to merge the fairly equal sub-solutions in order to receive one single solution of the problem, in dynamic programming we usually try to find an optimal sub-solution and then grow it.

Once we have an optimal sub-solution on each step we try to upgrade it in order to cover the whole problem. Thus a typical member of the dynamic programming class is finding the longest subsequence.

However this problem is interesting because it can be related to graph theory. Let’s find out how.

Overview

We already know various ways to calculate the shortest paths in a graph. Indeed finding the single-source shortest path is a typical graph problem. To model such kind of solutions we definitely need a graph represented in our solution.

However the single-source shortest path isn’t a straight-forward problem. It depends on many factors. Thus for positive edges the algorithm of Edsger Dijkstra can be a perfect solution, but when the graph contains negative edges his algorithm is no longer useful.

In the presence of negative edges the Dijkstra’s algorithm doesn’t work and we’d better use the Bellman-Ford algorithm. It is interesting to note, that it was exactly Richard Bellman who first introduced the term “dynamic programming” in the 1940s.

In fact the Bellman-Ford algorithm was able to detect negative cycles. That’s too important, because in presence of negative cycles the shortest path problem is no longer well defined.

In the other hand when we’re talking about shortest paths in a DAG (Directed Acyclic Graph) we can find a faster (linear) solution. That’s because we’re sure that there are no cycles (not even negative cycles)!

Finding the shortest paths in a DAG is closely related to the topological sorting of the DAG. This gives us a linear representation of the vertices of the DAG and we can clearly calculate the distances from the starting node to all other nodes. Note that in a DAG we have one or more nodes that can be considered as starting nodes – which means they don’t have predecessors (incoming edges).

Following the words above is pretty hard to find what all these graph algorithms have to do with finding the longest increasing (decreasing) subsequence. Actually this problem is very closely related to the toplogical sort of the DAG and the problem of finding shortest paths in a DAG.

That’s because we can represent our sequence as a DAG. The only thing we must care about is to “connect” with directed edges those elements that form an increasing (decreasing) pair.

Thus the sequence from our example [1, 8, 2, 7, 3, 4, 1, 6] is going to look like this.

Another important thing to note is that we don’t search for shortest, but for longest path, since our task is to find the longest subsequence.

Pseudo Code

Our pseudo code for finding the shortest paths in a DAG was something like that.

1. Get the toplogically sorted list L of the DAG;
2. The starting node is s;
3. The distance to s equals to 0;
4. All other distances are initialized to ∞
5. For each node (v) in L\{s} do:
5.1. If dist(v) > dist(u) + w(u, v) then
5.1.1.  dist(v) := dist(u) + w(u, v)

In the above pseudo code “u” is every predecessor of “v”!

Now we must “reverse” the solution above in order to find the longest increasing subsequence. Note that we don’t care any more about the weight of the edges, thus we can simply substitute them with 1.

1. Get the sequence (L) as a toplogically sorted DAG;
2. For each (i) in L do:
2.1. S(i) := 1 + max(S(j), where (i, j) is an edge from the DAG);

Application

Finding the longest increasing subsequence can be very useful not only at the Google/Yahoo/Facebook interview, but also in various fields of statistics.

Prefix encoding, sometimes called front encoding, is yet another algorithm that tries to remove duplicated data in order to reduce its size. Its principles are simple, however this algorithm tend to be difficult to implement. To understand why, first let’s take a look of its nature.

Please, have a look on the following dictionary.

use
used
useful
usefully
usefulness
useless
uselessly
uselessness

Instead of keeping all these words in plain text or transferring all them over a network, we can compress (encode) them with prefix encoding.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '0fully',
4 => '0less',
5 => '0lessly',
6 => '0lessness',
);

It’s clear that this is not the best compression and we can go even further by using not only the first word as prefix.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '2ly',
4 => '0less',
5 => '4ly',
6 => '4ness',
);

Now the compression is better and the good news is that decompression is a fairly simple process. However the tricky part is compression itself. The problem is that it is quite difficult to chose an appropriate prefix. In our first example this is simple, but most of the times in practice we can have more heterogeneous data. Indeed the process of compression can be very difficult for randomly generated data and the algorithm will be not only slow, but difficult to implement.

The good thing is that this algorithm can be used in many cases once we know the data format in advance. So let’s see three examples where this algorithm can be very handy.

Application

Here are three examples of prefix encoding. As I said above the process of compression can be very difficult for random data, so it is a good practice to use only it if you know in advance the format of the input data.

Date and time prefixes

We humans often skip the first two digits of an year, so for instance we don’t always write 1995 or 1996, but we use the shorter – ‘95 and ‘96. Thus years can be encoded with shorter strings.

input: 	(1991, 1992, 1993, 1994, 1995, 1996)
output:	(91, 92, 93, 94, 95, 96)

The problem is that with small changes of the input stream we can confuse the decoder. Thus if we add years from the 21st century we lose the uniqueness of the data.

input:	(1998, 1992, 1999, 2011, 2012)
output: (98, 92, 99, 11, 12)

Now the decoder can decode the last two values as (1911, 1912) as “19” is considered to be the prefix. So we must know in advance that our prefix is absolutely equal for each of the values. If not the encoding format must be different. For instance we can encode also the prefix, with some special maker.

input:	(1998, 1992, 1932, 1924, 2001, 2012)
output:	(#19, 98, 92, 32, 24, #20, 01, 12)

Once the decoder reads the # character it will know to decode the following number as prefix.

This can be used in practice for date and time formats. Let’s say we have some datetime values, but we know that all of them are in the same day.

2012-01-31 15:33:45
2012-01-31 16:12:11
2012-01-31 17:32:35
2012-01-31 18:54:34

Obviously we can omit the date part of these strings and send (keep) only the time. Once again, we must be absolutely sure that all these values are in the same day. If not, we can use the encoding strategy of the previous example.

Phone numbers

Phone numbers are the typical case of prefix encoding. Not only the international code, but also the mobile network operators use prefixes for their phone numbers. Thus if we have to transfer phone numbers from, let’s say the UK, we can replace the leading “+44” with something shorter.

If you happen to code a phone book for a mobile device you can spend some space by compressing the data using prefix encoding and thus the user will have more space and will store more phone numbers on his mobile.

Phone number prefixes can be also used for database normalization. Thus you can store them in a separate db table and leave only the unique numbers from the phonebook.

Geo Coordinates

Using the same example from my previous post we can send GEO coordinates by removing a common prefix, for large levels of zoom. Indeed when you’ve to send lots of markers to your map application you can expect all of these markers to be fairly close to each other in large zoom level.

Now the coordinates of those points can have a common prefix, like the example bellow with the Subway stations.

LatLon(40.762959,-73.985989)
LatLon(40.761886,-73.983629)
LatLon(40.762861,-73.981612)
LatLon(40.764616,-73.98056)

We can see that all of these GEO points have the same prefix (40.76x, -73.98x), so we can send the prefix only once.

Prefix: (40.76, -73.98)
Data: 
LatLon(2959,5989)
LatLon(1886,3629)
LatLon(2861,1612)
LatLon(4616,056)

These are only three examples of prefix encoding and this algorithm must be considered as very useful when transferring homogeneous data.

Suffix Encoding

Suffix encoding practically the same algorithm as prefix encoding, with the small difference that we use to encode duplicating suffixes. Like the examples bellow suffix encoding can be useful is replacing repeating last name suffixes.

Johnson
Clarkson
Jackson

Or company names.

Apple Inc.
Google Inc.
Yahoo! Inc.

Here we can replace “ Inc.” with something else, but shorter.

Relative encoding is another data compression algorithm. While run-length encoding, bitmap encoding and diagram and pattern substitution were trying to reduce repeating data, with relative encoding the goal is a bit different. Indeed run-length encoding was searching for long runs of repeating elements, while pattern substitution and bitmap encoding were trying to “map” where the repetitions happen to occur.

The only problem with these algorithms is that not always the input stream of data is constructed out of repeating elements. It is clear that if the input stream contains many repeating elements there must be some way of reducing them. However that doesn’t mean that we cannot compress data if there are no repetitions. It all depends on the data. Let’s say we have the following stream to compress.

1, 2, 3, 4, 5, 6, 7

We can hardly imagine how this stream of data can be compressed. The same problem may occur when trying to compress the alphabet. Indeed the alphabet letters the very base of the words so it is the minimal part for word construction and it’s hard to compress them.

Fortunately this isn’t true always. An algorithm that tryies to deal with non repeating data is relative encoding. Let’s see the following input stream – years from a given decade (the 90’s).

1991,1991,1999,1998,1991,1993,1992,1992

Here we have 39 characters and we can reduce them. A natural approach is to remove the leading “19” as we humans often do.

91,91,99,98,91,93,92,92

Now we have a shorter string, but we can go even further with keeping only the first year. All other years will as relative to this year.

91,0,8,7,0,2,1,1

Now the volume of transferred data is reduced a lot (from 39 to 16 – more than 50%). However there are some questions we need to answer first, because the stream wont be always formatted in such pretty way. How about the next character stream?

91,94,95,95,98,100,101,102,105,110

We see that the value 100 is somehow in the middle of the interval and it is handy to use it as a base value for the relative encoding. Thus the stream above will become:

-9,-6,-5,-5,-2,100,1,2,5,10

The problem is that we can’t decide which value will be the base value so easily. What if the data was dispersed in a different way.

96,97,98,99,100,101,102,103,999,1000,1001,1002

Now the value of “100” isn’t useful, because compressing the stream will get something like this:

-4,-3,-2,-1,100,1,2,3,899,900,901,902

To group the relative values around “some” base values will be far more handy.

(-4,-3,-2,-1,100,1,2,3)(-1,1000,1,2)

However to decide which value will be the base value isn’t that easy. Also the encoding format is not so trivial. In the other hand this type of encoding can be useful in som specific cases as we can see bellow.

Implementation

The implementation of this algorithm depends on the specific task and the format of the data stream. Assuming that we’ve to transfer the stream of years in JSON from a web server to a browser, here’s a short PHP snippet.

// JSON: [1991,1991,1999,1998,1999,1998,1995,1997,1994,1993]
$years = array(1991,1991,1999,1998,1999,1998,1995,1997,1994,1993);

function relative_encoding($input)
{
	$output = array();
	$inputLength = count($input);
	
	$base = $input[0];
	
	$output[] = $base;
	
	for ($i = 1; $i < $inputLength; $i++) {
		$output[] = $input[$i] - $base;
	}
	
	return $output;
}

// JSON: [1991,0,8,7,8,7,4,6,3,2]
echo json_encode(relative_encoding($years));

Application

This algorithm may be very useful in many cases, but here’s one of them. There are plenty of map applications around the web. Some products as Google Maps, Yahoo! Maps, Bing Maps are quite famous, while there are very useful open source projects as OpenStreetMap. The web sites using these apps are thousands.

A typical use case is to transfer lots of Geo coordinates from web server to a browser using JSON. Indeed any GEO point on Earth is relative to the point (0,0), which is located near the west coast of Africa, however on large zoom levels, when there are tons of markers we can transfer the information with relative encoding.

For instance the following diagram shows San Francisco with some markers on it. Their coordinates are be relative to the point (0,0) on Earth.

Far more useful may be to encode those markers, relative to the center of the city, thus we can save some space.

However this type of compression can be tricky, for example when dragging the map and updating the marker array. In the other hand we must group markers if we have to load more than one city. That’s why we must be careful when implementing it. But in the other hand it can be very useful - for instance on initial load of the map we can reduce data and speed up the load time.

The thing is that with relative encoding we can save only changes to base value (data) - something like version control systems and thus reducing data transfer and load. Here's a graphical example. In the first case on the diagram bellow we can see that each item is stored on its own. It doesn't depend on the adjacent items and it can be completely independent of them.

However we can keep full info only for the first item and any other item will be relative to it, like on the diagram bellow.

No matter how fast today’s computers and networks are, the users will constantly need faster and faster services. To reduce the volume of the transferred data we usually use some sort of compression. That is why this computer sciences area will be always interesting to research and develop.

There are many data compression algorithms, some of them lossless, others lossy, but their main goal aways will be to spare storage space and traffic. These algorithms are very useful when talking about data transfer between two distant places. Perhaps the best example is the transfer between a web server and a browser.

In the last few years a lot of research has been done on compressing files, executed on the client side. Such files are javascript, css, htmls and images. In fact servers and clients already have some techniques to compress data, like using GZIP for instance, that can dramatically decrease the transfer. In the other hand there are lots of tools and tricks in order to decrease the size of the data.

Actually when a file is executed by the client’s virtual machine, it doesn’t matter how “beautifully” it is formatted from a programmer’s point of view. Thus the spaces, tabs and the new lines don’t bring any significant information for the environment. That is why such compressing tools like YUI Compressor, Google Closure Compiler, etc. remove those symbols. Well, they can achieve even more in order to improve the compression rate. In this post I won’t cover this, but this shows how important data compression algorithms are.

It would be great if we could just compress data with some tool. Unfortunately this is not the case and usually the compression rate depends on the data itself. It is obvious that the choice of data compression algorithm depends mainly on the data and first of all we must explore the data.

Here I’ll cover one very simple lossless data compression algorithm called “run-length encoding” that can be very useful in some cases.

Overview

This algorithm consists of replacing large sequences of repeating data with only one item of this data followed by a counter showing how many times this item is repeated. To become clearer let’s see a string example.

aaaaaaaaaabbbaxxxxyyyzyx

This string’s length is 24 and as we can see there are lots of repetitions. Using the run-length algorithm, we replace any run with shorter string followed by a counter.

a10b3a1x4y3z1y1x1

The length of this string is 17, which is approximately 70% of the initial length. Obviously this is not the optimal way to compress the given string. For instance we don’t need to use the digit “1” when the character is repeated only once. In some cases this approach can increase the length of the initial string which is exactly the opposite of what we need. In this case we’ll get the string bellow.

a10b3ax4y3zyx

Now the length of the resulting string is 13, which is 54% of the initial length! A variation of the example above is not to keep a counter of the repetitions of the character, but their position instead. Thus the initial string will be compressed as follows.

a0b10a13x14y18z21y22x23

Which of these two approaches you’ll use depends on the goal. In the second case we can achieve a good optimization of binary search.

It is clear that this algorithm is not only applicable on strings. We can achieve very good results on arrays. A typical example is the transfer of JSON from a server to a client. Then if there are large sequences of repeating data we can achieve great results.

Implementation

The implementation bellow is assuming that we’re compressing a string and it’s written on PHP. However the nature of this algorithm doesn’t restrict us to use only strings. As I said before with slight modifications we can use it with other data structures. It is important only to understand that the run-length algorithm is very useful on large sequences of repeating elements, no matter characters or array items.

$message = 'aaaaaaaaaabbbaxxxxyyyzyx';

function run_length_encode($msg)
{
	$i = $j = 0;
	$prev = '';
	$output = '';
	
	while ($msg[$i]) {
		if ($msg[$i] != $prev) {
			
			if ($i) 
				$output .= $j;
				
			$output .= $msg[$i];
				
			$prev = $msg[$i];
			
			$j = 0;
		}
		$j++;
		$i++;
	}
	
	$output .= $j;
	
	return $output;
}

// a10b3a1x4y3z1y1x1
echo run_length_encode($message);

And slightly optimized.

$message = 'aaaaaaaaaabbbaxxxxyyyzyx';

function run_length_encode($msg)
{
	$i = $j = 0;
	$prev = '';
	$output = '';
	
	while ($msg[$i]) {
		if ($msg[$i] != $prev) {
			
			if ($i && $j > 1) 
				$output .= $j;
				
			$output .= $msg[$i];
				
			$prev = $msg[$i];
			
			$j = 0;
		}
		$j++;
		$i++;
	}
	
	if ($j > 1)
		$output .= $j;
	
	return $output;
}

// a10b3ax4y3zyx
echo run_length_encode($message);

Finally a small change – now we store the position of the character.

$message = 'aaaaaaaaaabbbaxxxxyyyzyx';

function run_length_encode($msg)
{
	$i = 0;
	$prev = '';
	$output = '';
	
	while ($msg[$i]) {
		if ($msg[$i] != $prev) {
				
			$output .= $msg[$i] . $i;
				
			$prev = $msg[$i];
			
		}

		$i++;
	}
	
	return $output;
}

// a0b10a13x14y18z21y22x23
echo run_length_encode($message);

Complexity and Data Compression

We’re used to talk about complexity of an algorithm measuring time and we usually try to find the fastest implementation, like in search algorithms. Here it is not so important to compress data quickly, but to compress as much as possible so the output is as small as possible without lossing data. A great feature of run-length encoding is that this algorithm is easy to implement.

Application

We can use run-length encoding in many cases. It is commonly used to compress images and is very successful when we deal only with black and white images. Here I’ll cover another use case that I only mentioned above. Let’s say we have to transfer a very large array of data to our AJAX-powered application using JSON. Let’s say also that the data are some years, for instance the years of the premiere of a movie. There are lots of movies with a premiere in the same year, thus although the data is sorted, we actually can’t have any benefit. More important is that we have large sequences of data. Here we can use run-length encoding.

$data = array(
	0 	=> 1991,
	1 	=> 1991,
	...
	2223 	=> 1991,
	2224 	=> 1992,
	...
	19298 	=> 1995,
	19299 	=> 1996,
	...
);

As you can see to transfer the whole array can be a nightmare, especially on slow networks. It is better to compress it (i.e. with PHP’s json_encode).

// {"0":1991,"1":1991, ..., "2223":1991,"2224":1992, ..., "19298":1995,"19299":1996, ...}
echo json_encode($data);

After running run-length encoding we can receive something like the following array (note that these are only sample data and it’s up to you to decide which is the best format to store data).

$data = array(
	0 => array(1991, 2224),
	1 => array(1992, 3948),
	2 => array(1995, 2398),
	3 => array(1996, 3489),
);

And the JSON output.

// [[1991,2224],[1992,3948],[1995,2398],[1996,3489]]
echo json_encode($data);

Note that if the data is sorted we can achieve great success compressing it!!! This approach can be used for images, graphics or map coordinates.

This is only one example of how data compression can be useful in our daily work. Although the communication between the server and the client can be optimized and compressed, we can improve it. In other words we’re not always sure that the opposite side supports compression.

Well, it’s true that the client has to decompress the data, which can also be slow. Now in the first case we have only the time to transfer, as on the diagram bellow.

In the second case, we should sum the time for compression, transfer and decompression.

All this is important, but in general data compression can be handy in many cases in our daily work.

First thing to say TinyMCE is a very popular WYSIWYG online based editor. It’s very widely used in the web, as may already know it it’s part of the default WordPress installation. Out of the web, of course, there are some other editors as well. The most used and well developed projects are the Yahoo!’s YUI 2 Rich Text Editor and CKEditor also known with his past name – FCKEditor.

Before I proceed with this post, let me say that I’m working and this tutorial is based on version 3.1.1 released on 18 Aug 2008.

Including TinyMCE

The only thing you’ve to do is to include a TinyMCE javascript file and write a simple init. The examples of installing it come with the compressed version of the javascript file:

The result of this code is shown on the picture:

However for this tutorial you’ve to include the uncompressed file, because the compressed one is impossible to read and modify. At the end of all this you can compress yet again the resulting file with some js compressor as YUI Compressor, JSMin or Google Closure Compiler. So instead of including the file above, you’ve to include the _src version as it is:

Adding the Button

Now there is a small configuration object passed to the init function. First you’ve to modify it a bit:

tinyMCE.init({
	mode : "textareas",
	theme : "advanced",
	force_br_newlines : true,
	theme_advanced_buttons1 : "link,unlink,|,bold,italic,underline,strikethrough,|,justifyleft,justifycenter,justifyright,justifyfull,|,mybutton",
	theme_advanced_buttons2 : "",
	theme_advanced_buttons3 : "",
	theme_advanced_buttons4 : "",
	theme_advanced_toolbar_location : "top",
	theme_advanced_toolbar_align : "left",
	theme_advanced_statusbar_location : "",
	theme_advanced_resizing : true
});

Usually in the /tinymce/jscripts/tiny_mce/themes/advanced/editor_template_src.js file there are some buttons described:

tinymce.create('tinymce.themes.AdvancedTheme', {
	// Control name lookup, format: title, command
	controls : {
		bold : ['bold_desc', 'Bold'],
		italic : ['italic_desc', 'Italic'],
		underline : ['underline_desc', 'Underline'],
		strikethrough : ['striketrough_desc', 'Strikethrough'],
		justifyleft : ['justifyleft_desc', 'JustifyLeft'],
		justifycenter : ['justifycenter_desc', 'JustifyCenter'],
		justifyright : ['justifyright_desc', 'JustifyRight'],
		justifyfull : ['justifyfull_desc', 'JustifyFull'],
		bullist : ['bullist_desc', 'InsertUnorderedList'],
		numlist : ['numlist_desc', 'InsertOrderedList'],
		outdent : ['outdent_desc', 'Outdent'],
		indent : ['indent_desc', 'Indent'],
...

What you need to do is simply to add a new line to this config array:

...
        blockquote : ['blockquote_desc', 'mceBlockQuote'],
	mybutton : ['mybutton_desc','myFunc']
},

That’s not enough of course! Now TinyMCE sees this definition, but nothing happens. It cannot build the new button yet. What’s next?

Styling the Button

After giving the new button a name, TinyMCE constructs a markup with the given name into the controlbar:

This name can be simply used in the CSS file (/tinymce/jscripts/tiny_mce/themes/advanced/skins/default/ui.css) to style the button. Because TinyMCE uses sprite for its background images, you can copy/paste the style from some other button and than adjust the background position, only after you’ve added the new icon to the sprite. Here’s simply a copy of the style from another button:

...
.defaultSkin span.mce_numlist {background-position:-80px 0}
.defaultSkin span.mce_mybutton {background-position:-460px 0}
.defaultSkin span.mce_justifyleft {background-position:-460px 0}
...

You can see how the button appears in the control bar!

Binding the Button

The final job to do is to bind the button with some action. This thing happens again in /tinymce/jscripts/tiny_mce/themes/advanced/editor_template_src.js. You just have to search for other bindings – let’s say mceCleanup. There you can write your new function – myFunc:

FormatBlock : function(ui, val) {
	...
},

myFunc : function() {
	this.editor.selection.setContent('Hello World!');
},

mceCleanup : function() {
	...
},

Now after clicking the button the method will be executed.

Note that the name of the method should be exactly the same as described in the controls array in /tinymce/jscripts/tiny_mce/themes/advanced/editor_template_src.js. If not the button wont work ever.

Final Steps

Now you’ve to compress (if you wish) the js files and deploy your editor.

It’s really full of video sharing websites out there, but in fact almost all of them use Flash player to display their video files. This is the reality now, but with the coming of HTML5, perhaps the things are changing a bit!

First of all if you start dealing with video sharing platforms the first thing to do maybe is to find a good Flash (.flv) player and to convert all your video content in FLV.

Few things to know with Flash video players:

1. The user can play those videos only after installing the Flash Plugin to his browser;
2. The video must be encoded either in FLV (or FLash Video) or in MPEG-4 with h.264 codec. Only than the Flash player can play it;

The HTML5, which can be described as variety of new “things” in the HTML comes with a native <video> tag. Something like the <img> tag where you can just point the source of the image in the src attribute.

First Question – How Does it Look Like?

To get more fluent with the <video> tag you must simply thing of it as a “complex” <img> tag. You just point to the video and the browser will do the rest. To be more exact let me send you to the w3schools page, but the important thing as they say:

Until now, there has never been a standard for showing video on a web page.

However to start from somewhere here’s a little snippet showing you the basic, but very basic usage of the HTML5 video tag:

<video src="path_to_the_video" controls="controls"></video>

Second Question – Is Every Video Playable?

But of course not! There are different browser – different video formats. Currently there are three formats, supported by the different browsers.

1. H.264
2. VP8
3. Theora

Note that these are video codecs and a video files does not consists of only a video codec, but also of an audio codec. This makes the video file a container of several things. As you may know already H.264 is used with MPEG-4 video format, which results in .mp4 extended files, Theora is used in OGG, which are all those .ogg files out in the web, and finaly VP8 is used in WebM … or in .webm videos.

First thing to know FLV files (.flv) can also be encoded with H.264. Actually Flash player has a h.264 decoder so this makes it possible to play both .mp4 and .flv files encoded with H.264.

Second thing to notice – where are those .webm files? I’ve never seen them!

Third Question – What is WebM and VP8?

Behind WebM and VP8 is Google. This is a new video format based again on Theora. Actually it pretends to be as good in compressing as H.264, which by the way is meant to be the best. But let me take a look at Nettuts+:

VP8, a relatively new codec, is the brainchild of On2 — the same guys behind Theora. Google acquired On2 in 2010, and opened up all the underlying patents for the codec into the public domain.

WebM, the container of choice for most current browsers, utilizes VP8 for compressing its video content and Vorbis for its audio. It produces content similar in quality to H.264.

It is completely royalty free, now and for the future. On the downside, though, it has limited hardware decoding support as well as third party device/mobile support.

Final Question – How Can I Convert a Video in WebM?

This answer is given by this article, where as it is said after having the ffmpeg version >= 0.6 you simply can use this command:

ffmpeg -i girltalk.mp4 -f webm -vcodec libvpx -acodec libvorbis -aq 90 -ac 2 girltalk.webm

Conclusion

It’s interesting to see what will happen one day. There is going to be either a harmonious life in the web video field or the codecs/formats will be as much as the browser? Who knows?

The last few months I was searching about some kind of mobile interface of Google Analytics for my Nokia e71. However I did not find one?! Please correct me if I’m wrong.

In the same time I knew there is a GA API, which is free to use, so finally I wrote a tiny web app allowing the reading of basic information about your Google Analytics accounts.

Note that your username and password are NOT SAVED! However it’s up to you to try.

An important note is that the API has a limitation in requests so yet again, it’s possible to be unavailable in case of too many requests.

Visit with Your Mobile Browser

this link:

http://www.stoimen.com/ga/

Note that I made it primary about my Nokia, so any screenshots will be welcomed as well.

It is still beta so be careful. In fact any suggestions are welcome. Either for improving the security and error handling as well as new data exports.

There are lots of things to be done, but this is really basic.

First of all the use of some software is needed not because of itself, but because of the need of some specific functionality. In my case this was the need of real time news feed. Of course there is a way to make this without Node.js, as I’ll describe later in this post, but there are several disadvantages. However to begin from somewhere, let me explain what is Node.js.

Introducing Node.js

Perhaps the best way do describe what is Node.js is from its about page.

Node’s goal is to provide an easy way to build scalable network programs. In the “hello world” web server example above, many client connections can be handled concurrently. Node tells the operating system (through epoll, kqueue, /dev/poll, or select) that it should be notified when a new connection is made, and then it goes to sleep. If someone new connects, then it executes the callback. Each connection is only a small heap allocation.

…

In general Node is a program using the Google Chrome’s V8 JavaScript engine, which in turn is a program that can parse and execute code written in JavaScript. V8 is a very very interesting project itself. First of all from Google have developed this engine especially for one of his products – their browser Chrome. It pretends to be and by no means is one of the masterpieces of Google. It is fast and reliable engine, written in C++ and JavaScript, as Wikipedia’s page says. Actually this code is open source and can be embedded in whatever application written in C++. Thus you can have in your application a JavaScript engine.

Such kind of application/server is Node.js. With V8 embedded into, Node is intended to run on a machine and to serve some requests. Just like a normal web server, but with few differences. First of all Node is not only serving HTTP requests, but also TCP. Well I’m not so deep into this yet, but so far that works for me. What I need for now is its web serving functionality.

The way Node works, however is different from what we know from every other web server. Here it is a normal server in general:

Whenever a request comes to the server, it tries to return the response as fast as possible. This is one of the main goals of every web developer team – to make the application as fast as possible. However in Node the things work different:

Here the server doesn’t return immediately the response, but holds as long as possible until some “event” occurs. You can thing of Node as a classical chat server. Until one of the peers doesn’t write a message, the other doesn’t receive anything.

Some may be confused as how you can use both the typical web server, as Apache, with Node as another web server within the same page. The answer is – with AJAX. As you know thus you can have only one chunk of your page talking with the server – in this case the entire page is returned by Apache, but inside the page you can have a chat app using Node as a server, working perhaps on a different port on the same machine.

Typically this scenario can be simulated without Node, but as I said, with some disadvantages. Let’s say you have two browsers and one server – you can infinitely loop AJAX calls to the server and whenever there is new message on the server return it to the browser – this is quite inefficient. There are two approaches in this case.

1. You can make those AJAX calls too recently and you can have a call every N seconds. In that case if N is to short you’ll have too many calls and in case the other peer doesn’t make anything the server will be unusually loaded with no practical effect.
2. In the second case you can choose to make N too long – for instance 20 seconds. This is also bad, because during this period of time the peers may want to interact and chat.

In that case Node is a web server, but simply doesn’t return the response until a specific event doesn’t occur. How does this look like in a web browser. You may know how looking into the Firebug console for every AJAX call you see the request and how many seconds it has taken. If you make an AJAX call to a Node server app the request just doesn’t respond immediately.

Installing Node.js

The process of installation is described quite well, so I’m going to describe it in few steps.

1. Download the application from Git, note that V8 is embedded inside. The version while this post was written and of course was stable is v0.2.4
2. For those familiar with Unix/Linux systems, there are three simple steps following, however without GCC (Gnu Compiler Collection) you wont be able to proceed! I’m writing this especially for the Mac OS users, as they may know there is no built-in GCC there.
3. ./configure
4. make
5. make install
6. make test (optional)

This in general is enough. After all that you can write the node command in your terminal. However if there are some problems while compiling it you may refer to Git and the Node community for more help. Don’t forget that V8 can work only on x86, x64 and ARM architectures.

Summary

There are few things that can be summarized now:

1. First of all Node can be a web server, as Apache is, but with the option to respond on event firing.
2. You can request it either with a web page loaded in your browser or within a chunk of the page via AJAX.
3. To install Node you need GCC. It comes with V8 JavaScript engine inside, which can run only on x86, x64 and ARM.

So far so good! Now we have Node installed, practically useless, because there is no application running on it. In my next post I’ll describe how to run your first application.

First of all there are several things to know about Zend Framework and caching. Whenever you work on a big web application caching is one of the mostly used mechanisms of speeding up the app and improve user performance. In general the task and the solution are pretty simple and natural.

As the application grows up the visitors become more and more impatient about what they receive. A single page is becoming slower and slower and the result is painful. First of all every time a user hits a page the application server uses the web server, a script interpreter, a database server and potentially the file system. But that’s not all. After all this output is generated on the server, as HTML in the most cases, it is sent to the client where again CSS and JavaScript engines parse and execute them.

In this scenario it’s easy to imagine how many time is spent. While there are several techniques to optimize the client side by optimizing JavaScript, CSS and the static images used for the design of the site, here I’m going to talk more about the backend.

Beside the Optimization

Let’s assume we’ve one of the very used combination between Apache (as a webserver), PHP (as server scripting language) and MySQL (as database server). Here you can choose to optimize all three of them. However beside the optimization of them one of the most simple steps you can do is to cache the output generated by these three branches of your web app.

Caching the Content

In fact you can cache only single parts of the whole process. For instance you can cache only the result returned by some slow database query. Let’s imagine a query takes about 2 seconds to execute. Now you can cache the result into a file and during the cache is active, i.e. it has not expired, the application takes it from a file stored somewhere in the file system.

In a typical Zend Framework scenario you can first setup the frontend and backend options of the cache.

$frontendOptions = array(
	'lifetime'                => 600, // in seconds - this is 10 seconds
	'automatic_serialization' => true,
);
$backendOptions = array('cache_dir' => 'cache/');
$cache = Zend_Cache::factory('Core', 'File', $frontendOptions, $backendOptions);
		
$cacheKey = md5('mykey');

if (!$cache->load($cacheKey)) {
	$slowQueryResult = $article->fetchAll();
	$cache->save($slowQueryResult, $cacheKey);
} else {
	$slowQueryResult = $cache->load($cacheKey);
}

You can setup different options here by setting up the cache directory, lifetime, etc.

Note that the cache directory must exists and with write permissions and Zend Framework doesn’t create it for you and will throw error.

The problem here is that now you cache only part of the generated content and in many cases this is still too slow for most of the users. However all the work (in most of the cases) of the web server, the script interpreter and the database server result in a simple HTML output. What if you have this output generated or cached for you and when the user hit the page the server will return this pre-generated code?

This is indeed very fast, because it’s similar to return a text file, as the HTML is simply formatted text.

Caching the Entire Page

Before I proceed, I’d like to say that I work with Zend Framework 1.9.x. Now in the latest versions of ZF there are new mechanisms of caching the output even Zend_Cache_Frontend_Page works fine on them.

You can simply setup the page cache within few simple lines of code:

$fo = array(
    'lifetime' => 600,
    'regexps' => array(
        '^/' => array(
	'cache' => true,
         'cache_with_cookie_variables' => true,
        ),
    )
);

$bo = array(
    'cache_dir' => 'cache/'
);

$cache = Zend_Cache::factory('Page', 'File', $fo, $bo);
$cache->start();

However my advise is to place this code as high as possible, because this will cache everything generated as output. It’s a good practice if you place this even in the bootstrap before you make the connection with the database. Actually you don’t need a database connection when you’ve to return a simple text(html) file.

This will improve your app’s performance a lot!

However there are few things to know. When you setup the cache to work even with cookie variables, you can see that hitting the page with different browsers Zend Framework will generated different cache pages. This is quite useless because than you don’t have any benefit of caching the content.

First of all let me say that THIS IS NOT A BUG! of ZF. Simply the framework will use the cookie variables to generate the cache key. It’s obvious that different browsers, even more different users, will have different cookie set and the framework will generate different cache keys for them.

Thus you’ve to change the setting to generate the cache key from cookie variable by explicitly set this option to false:

$fo = array(
    'lifetime' => 600,
    'regexps' => array(
        '^/' => array(
		 'cache' => true,
         'cache_with_cookie_variables' => true,
         'make_id_with_cookie_variables' => false,
        ),
    )
);

$bo = array(
    'cache_dir' => 'cache/'
);

$cache = Zend_Cache::factory('Page', 'File', $fo, $bo);
$cache->start();

Note that in this example we cache every single page generated by the framework explained in the regexps. This is not so good especially when the users have the possibility to login and to see customized content for them, so you can be careful what you cache.

A typical problem that this solution solves is when your application uses Google Analytics. As you may know Google Analytics sets up a cookie every time when an user hits the page, so every time the framework will generate a different cache for him and in result he won’t see any benefit and performance improvement from your site.

What is list()?

Let’s assume you’ve two variables and one array with two elements:

$arr = array(1, 2);
$a = $arr[0];
$b = $arr[1];
// now a == 1, and b == 2

Maybe the easiest way to assign to the first variable the value of the first element of the array, and to the second variable the value of the second element of the array is to use list():

list($a, $b) = array(1, 2);
// again a == 1, and b == 2

Note that you can do the same with as many variables/array elements you want.

The funny thing is that nobody use it, while I see at least one perfect usage. You can think of a typical admin panel of any web system. For sure any developer has programmed the typical table page containing a list of “things” and a paging to switch the result set page. It’s like the posts page from the WordPress admin panel, or a search result page in Google, where you’ve one page of results and at the bottom – a number of pages where you can go to another page of the search results.

What I’ve seen in almost every project is that typically the developers prefer to set the results for the page into one list returned from one method and the count of the entire result set is returned by another method.

Thus most of the times there are two methods, performing most commonly a database queries. Something like – getList($offset, $limit) and getCount() where the first one returns the page and the second one returns the count of the result set.

This will result into two methods, two calls and two lines of code:

function getList($offset, $limit)
{
	...
	// although the entire result set contains
	// 5 elements by limiting it from the parameters
	// only three are returned
	return $list; // where $list = array('title1', 'title2', 'title3');	
}

function getCount()
{
	...
	// here's returned the count
	// of the entire result set
	return $count; // where $count = 5;	
}

$list  = getList(0, 3);
$count = getCount();

What actually you can do is to perform both queries in one method, perhaps called getItems($offset, $limit) where the returned value is an array containing both the list of results and the count:

function getItems($offset, $limit)
{
	...
	return array('list' => $list, 'count' => $count);
}

And finally when calling this method to list() that into the two variables – single lined.

list($list, $count) = getItems(0, 3);