A very common problem in computer programming is finding the longest increasing (decreasing) subsequence in a sequence of numbers (usually integers). Actually this is a typical dynamic programming problem.

Dynamic programming can be described as a huge area of computer science problems that can be categorized by the way they can be solved. Unlike divide and conquer, where we were able to merge the fairly equal sub-solutions in order to receive one single solution of the problem, in dynamic programming we usually try to find an optimal sub-solution and then grow it.

Once we have an optimal sub-solution on each step we try to upgrade it in order to cover the whole problem. Thus a typical member of the dynamic programming class is finding the longest subsequence.

However this problem is interesting because it can be related to graph theory. Let’s find out how.

Overview

We already know various ways to calculate the shortest paths in a graph. Indeed finding the single-source shortest path is a typical graph problem. To model such kind of solutions we definitely need a graph represented in our solution.

However the single-source shortest path isn’t a straight-forward problem. It depends on many factors. Thus for positive edges the algorithm of Edsger Dijkstra can be a perfect solution, but when the graph contains negative edges his algorithm is no longer useful.

In the presence of negative edges the Dijkstra’s algorithm doesn’t work and we’d better use the Bellman-Ford algorithm. It is interesting to note, that it was exactly Richard Bellman who first introduced the term “dynamic programming” in the 1940s.

In fact the Bellman-Ford algorithm was able to detect negative cycles. That’s too important, because in presence of negative cycles the shortest path problem is no longer well defined.

In the other hand when we’re talking about shortest paths in a DAG (Directed Acyclic Graph) we can find a faster (linear) solution. That’s because we’re sure that there are no cycles (not even negative cycles)!

Finding the shortest paths in a DAG is closely related to the topological sorting of the DAG. This gives us a linear representation of the vertices of the DAG and we can clearly calculate the distances from the starting node to all other nodes. Note that in a DAG we have one or more nodes that can be considered as starting nodes – which means they don’t have predecessors (incoming edges).

Following the words above is pretty hard to find what all these graph algorithms have to do with finding the longest increasing (decreasing) subsequence. Actually this problem is very closely related to the toplogical sort of the DAG and the problem of finding shortest paths in a DAG.

That’s because we can represent our sequence as a DAG. The only thing we must care about is to “connect” with directed edges those elements that form an increasing (decreasing) pair.

Thus the sequence from our example [1, 8, 2, 7, 3, 4, 1, 6] is going to look like this.

Another important thing to note is that we don’t search for shortest, but for longest path, since our task is to find the longest subsequence.

Pseudo Code

Our pseudo code for finding the shortest paths in a DAG was something like that.

1. Get the toplogically sorted list L of the DAG;
2. The starting node is s;
3. The distance to s equals to 0;
4. All other distances are initialized to ∞
5. For each node (v) in L\{s} do:
5.1. If dist(v) > dist(u) + w(u, v) then
5.1.1.  dist(v) := dist(u) + w(u, v)

In the above pseudo code “u” is every predecessor of “v”!

Now we must “reverse” the solution above in order to find the longest increasing subsequence. Note that we don’t care any more about the weight of the edges, thus we can simply substitute them with 1.

1. Get the sequence (L) as a toplogically sorted DAG;
2. For each (i) in L do:
2.1. S(i) := 1 + max(S(j), where (i, j) is an edge from the DAG);

Application

Finding the longest increasing subsequence can be very useful not only at the Google/Yahoo/Facebook interview, but also in various fields of statistics.

Prefix encoding, sometimes called front encoding, is yet another algorithm that tries to remove duplicated data in order to reduce its size. Its principles are simple, however this algorithm tend to be difficult to implement. To understand why, first let’s take a look of its nature.

Please, have a look on the following dictionary.

use
used
useful
usefully
usefulness
useless
uselessly
uselessness

Instead of keeping all these words in plain text or transferring all them over a network, we can compress (encode) them with prefix encoding.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '0fully',
4 => '0less',
5 => '0lessly',
6 => '0lessness',
);

It’s clear that this is not the best compression and we can go even further by using not only the first word as prefix.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '2ly',
4 => '0less',
5 => '4ly',
6 => '4ness',
);

Now the compression is better and the good news is that decompression is a fairly simple process. However the tricky part is compression itself. The problem is that it is quite difficult to chose an appropriate prefix. In our first example this is simple, but most of the times in practice we can have more heterogeneous data. Indeed the process of compression can be very difficult for randomly generated data and the algorithm will be not only slow, but difficult to implement.

The good thing is that this algorithm can be used in many cases once we know the data format in advance. So let’s see three examples where this algorithm can be very handy.

Application

Here are three examples of prefix encoding. As I said above the process of compression can be very difficult for random data, so it is a good practice to use only it if you know in advance the format of the input data.

Date and time prefixes

We humans often skip the first two digits of an year, so for instance we don’t always write 1995 or 1996, but we use the shorter – ‘95 and ‘96. Thus years can be encoded with shorter strings.

input: 	(1991, 1992, 1993, 1994, 1995, 1996)
output:	(91, 92, 93, 94, 95, 96)

The problem is that with small changes of the input stream we can confuse the decoder. Thus if we add years from the 21st century we lose the uniqueness of the data.

input:	(1998, 1992, 1999, 2011, 2012)
output: (98, 92, 99, 11, 12)

Now the decoder can decode the last two values as (1911, 1912) as “19” is considered to be the prefix. So we must know in advance that our prefix is absolutely equal for each of the values. If not the encoding format must be different. For instance we can encode also the prefix, with some special maker.

input:	(1998, 1992, 1932, 1924, 2001, 2012)
output:	(#19, 98, 92, 32, 24, #20, 01, 12)

Once the decoder reads the # character it will know to decode the following number as prefix.

This can be used in practice for date and time formats. Let’s say we have some datetime values, but we know that all of them are in the same day.

2012-01-31 15:33:45
2012-01-31 16:12:11
2012-01-31 17:32:35
2012-01-31 18:54:34

Obviously we can omit the date part of these strings and send (keep) only the time. Once again, we must be absolutely sure that all these values are in the same day. If not, we can use the encoding strategy of the previous example.

Phone numbers

Phone numbers are the typical case of prefix encoding. Not only the international code, but also the mobile network operators use prefixes for their phone numbers. Thus if we have to transfer phone numbers from, let’s say the UK, we can replace the leading “+44” with something shorter.

If you happen to code a phone book for a mobile device you can spend some space by compressing the data using prefix encoding and thus the user will have more space and will store more phone numbers on his mobile.

Phone number prefixes can be also used for database normalization. Thus you can store them in a separate db table and leave only the unique numbers from the phonebook.

Geo Coordinates

Using the same example from my previous post we can send GEO coordinates by removing a common prefix, for large levels of zoom. Indeed when you’ve to send lots of markers to your map application you can expect all of these markers to be fairly close to each other in large zoom level.

Now the coordinates of those points can have a common prefix, like the example bellow with the Subway stations.

LatLon(40.762959,-73.985989)
LatLon(40.761886,-73.983629)
LatLon(40.762861,-73.981612)
LatLon(40.764616,-73.98056)

We can see that all of these GEO points have the same prefix (40.76x, -73.98x), so we can send the prefix only once.

Prefix: (40.76, -73.98)
Data: 
LatLon(2959,5989)
LatLon(1886,3629)
LatLon(2861,1612)
LatLon(4616,056)

These are only three examples of prefix encoding and this algorithm must be considered as very useful when transferring homogeneous data.

Suffix Encoding

Suffix encoding practically the same algorithm as prefix encoding, with the small difference that we use to encode duplicating suffixes. Like the examples bellow suffix encoding can be useful is replacing repeating last name suffixes.

Johnson
Clarkson
Jackson

Or company names.

Apple Inc.
Google Inc.
Yahoo! Inc.

Here we can replace “ Inc.” with something else, but shorter.

Lately most of the web developers are talking more and more about optimization. One of the practices everybody’s supporting is to use cookie free domains for static content. First of all, what’s static content. That, in breve, are all images, JavaScripts and CSS. That’s everything that’s transmitted to the client with no change from the server at all. In a typical PHP/MySQL site everything generated on the server site is considered dynamic, while every component that’s given to the client with no change is static. That’s why they don’t need cookies in the request.

That’s what Yahoo! YSlow says:

In a short example, lets say you’ve a web page with 10 background images used by its CSS file. Here’s a good practice to combine them in one or even use base64 for them, but that’s another talk. So in that scenario you’ll send all the cookies you’ve on the site with this images, but actually they don’t profit at all from this. The question is why you should send all this data with no need? Won’t you benefit from sending it with no cookies.

As it sound logical I read some articles recently describing that the benefit from putting the static content on a non-cookie domain doesn’t pays back. OK it may be strange, however every 40 ms or whatever of page load is important, aren’t they?

Setup a Cookie Free Domain

The problem is that if you’d like to setup a cookie free domain the things are becoming a bit harder. You’ve two options:

1. Move all your static content on a different domain, where no cookies are set.
2. Move your static content on a different sub domain and set all the cookies to the www subdomain. (Here’s a bit tricky).

All this indeed a bit tricky! So let me proceed with the next topic.

Benefits from CDN

A CDN or Content Delivery Network is a term become famous with the growing web. Now big sites have servers in almost every continent and perhaps country. CDN is an abstraction of all this. The good thing is that there’s supposed to be stored static content. Think about the YouTube’s video files. Another good thing is that this domains are cookie free by default. The thing is …

Why don’t You Combine Them?

You’ll benefit from both ideas. Cookie free domains with CDN. In one side the web page will benefit from the closest location of the server and the CDN and in other side all this will come with no cookies to you. That’s really nice and most of the time people thing of CDN for only storing large scale data, such as video files, but no one says you cannot put your CSS, JavaScripts and background images there!

If someone asks you what is the javascript in modern web developer, probably you should answer it’s almost the half of the traffic of your site and it’s almost everything when dealing with user experience.

Today’s big web apps are useless without it AJAX/JS part. Think about Facebook, Google products, Twitter, Yahoo!, Youtube.

Actually they made this a standart. The times when you used to use JavaScript as a helper language just to figure out how a drop down menu will work are far way in the past. Now JavaScript is everything in a rich web sites. With no JS they will be no rich at all! Then it comes the question how to improve sites with lots of javascript. Actually one of the main problems in the web now is that JavaScript is blocking content.

Blocked by the JavaScript

What that means at all is that until the browser receives and parses the JavaScript file/s it doesn’t process anything else, it even doesn’t load any other resources. Now you can imagine how big the problem is. When it comes to large files more than 100K the user experience will be very bad!

Optimize the web? What about optimizing JavaScript!

If the problem is so big as I described above why should we doubt about where to start with the optimization process? But of course with the JavaScript and the natural question that rises is how this can be optimized at all?

Three steps

Like me, many of you may heart of some techniques that improve javascript performance. Here are some I was able to select as important:

1. minify/compress

That’s rule number one. If everything that is a JavaScript file is processed once it arrives to the client than make it smaller and make it go faster trough the wire. Actually one technique to speed up things more is to concatenated all the files you have into one single big file. Although this will not spend you some space will reduce the HTTP requests and therefore speed up the loading process. Good tools that you can use to minify javascript files are the Yahoo’s YUI Compressor, that beside that compresses CSS and Google Closure Compiler. Both are extremely useful and by the last measurements the Google’s Compiler is even better, but it’s up to you to decide which one to pick up.

2 .write fast and smart code

You can minify everything that’s JavaScript and the code may remain slow. But why is that? That’s because the JavaScript is written in a bad way. You know there are many resources in the web describing what is a bad and what is a good practice when writing JS. Even more you can measure it by yourself if you’d like to use the Firebug’s profiler. That’s a good start to avoid bad practices.

3. improve loading – lazy loading

That’s a bit more complicated. If there’s a really big javascript file after compressing and concatenating all the JS functionality remains big. How to avoid that? Simply let’s the user to see the most important functionality and than load the entire app. Some good tutorials about lazy loading again can be found online. Don’t hesitate to search about.

Beside this really basic advices I’ll continue to write in my blog about specific techniques how to improve the JavaScript as this is one of the most interesting parts of web development by me.

Optimize every image

The first thing that you can do to improve your site image is to optimize them. Sometimes the files you put on the site as they are JPEG, GIF and PNGs can be improved just by using some software that strips useless information from their headers and using various algorithms to smooth the similar pixels. As you may know in the case of PNG and GIF this is particularly natural. Stoyan Stefanov a lead Yahoo! developer is know as guru when it comes image optimization. You can check more detailed information about software and tools on his blog here. The reality is that you don’t need extra info into the images, as useless information about the camera, which is often setup into the image header, and when it comes to the web that’s really good. In fact according to some researches this can spend you more than 30% of traffic.

Use CSS sprites

What are CSS sprites? Well you possibly already know. I’ve written some articles about that, so I won’t mention it again, but what’s the point. The CSS sprite usage helps you merge all the images you use into your site into one single image and to manipulate the site’s outlook only by changing the element background position. That’s really nice, it requires a bit concentration to track different elements background, but it’s not difficult at all. In fact there are some tools that can help to automate this. The good thing is that thus you replace all the background images with one single image and thus with one single request. So if you’ve one CSS file with twenty background images by using one CSS sprite there is only two requests instead of twenty one requests. Even more, you can go even further with this optimization technique and to replace the CSS sprite call by http by built into the CSS file base64 string. With that simple trick you got only one! request. Yes this becomes more difficult to maintain, but it’s really close to extreme optimization.

Use base64 to improve http request number when possible

That’s not a news, but however if you like to switch from multiple to one request the answer is base64. The good thing with base64 is that it can send to the client multiple images with single request and it natively eliminates the extra info from the image you convert. It has to be used with care, because sometimes even if it’s slower using multiple requests via HTTP for the images gives the user the feeling of faster site, just because the user sees response from the server earlier.

That’s all I can say about image optimization in one site. You can search in detail about tools and software, as well as other techniques explained here.