Prefix encoding, sometimes called front encoding, is yet another algorithm that tries to remove duplicated data in order to reduce its size. Its principles are simple, however this algorithm tend to be difficult to implement. To understand why, first let’s take a look of its nature.

Please, have a look on the following dictionary.

use
used
useful
usefully
usefulness
useless
uselessly
uselessness

Instead of keeping all these words in plain text or transferring all them over a network, we can compress (encode) them with prefix encoding.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '0fully',
4 => '0less',
5 => '0lessly',
6 => '0lessness',
);

It’s clear that this is not the best compression and we can go even further by using not only the first word as prefix.

$data = array(
0 => 'use',
1 => '0d',
2 => '0ful',
3 => '2ly',
4 => '0less',
5 => '4ly',
6 => '4ness',
);

Now the compression is better and the good news is that decompression is a fairly simple process. However the tricky part is compression itself. The problem is that it is quite difficult to chose an appropriate prefix. In our first example this is simple, but most of the times in practice we can have more heterogeneous data. Indeed the process of compression can be very difficult for randomly generated data and the algorithm will be not only slow, but difficult to implement.

The good thing is that this algorithm can be used in many cases once we know the data format in advance. So let’s see three examples where this algorithm can be very handy.

Application

Here are three examples of prefix encoding. As I said above the process of compression can be very difficult for random data, so it is a good practice to use only it if you know in advance the format of the input data.

Date and time prefixes

We humans often skip the first two digits of an year, so for instance we don’t always write 1995 or 1996, but we use the shorter – ‘95 and ‘96. Thus years can be encoded with shorter strings.

input: 	(1991, 1992, 1993, 1994, 1995, 1996)
output:	(91, 92, 93, 94, 95, 96)

The problem is that with small changes of the input stream we can confuse the decoder. Thus if we add years from the 21st century we lose the uniqueness of the data.

input:	(1998, 1992, 1999, 2011, 2012)
output: (98, 92, 99, 11, 12)

Now the decoder can decode the last two values as (1911, 1912) as “19” is considered to be the prefix. So we must know in advance that our prefix is absolutely equal for each of the values. If not the encoding format must be different. For instance we can encode also the prefix, with some special maker.

input:	(1998, 1992, 1932, 1924, 2001, 2012)
output:	(#19, 98, 92, 32, 24, #20, 01, 12)

Once the decoder reads the # character it will know to decode the following number as prefix.

This can be used in practice for date and time formats. Let’s say we have some datetime values, but we know that all of them are in the same day.

2012-01-31 15:33:45
2012-01-31 16:12:11
2012-01-31 17:32:35
2012-01-31 18:54:34

Obviously we can omit the date part of these strings and send (keep) only the time. Once again, we must be absolutely sure that all these values are in the same day. If not, we can use the encoding strategy of the previous example.

Phone numbers

Phone numbers are the typical case of prefix encoding. Not only the international code, but also the mobile network operators use prefixes for their phone numbers. Thus if we have to transfer phone numbers from, let’s say the UK, we can replace the leading “+44” with something shorter.

If you happen to code a phone book for a mobile device you can spend some space by compressing the data using prefix encoding and thus the user will have more space and will store more phone numbers on his mobile.

Phone number prefixes can be also used for database normalization. Thus you can store them in a separate db table and leave only the unique numbers from the phonebook.

Geo Coordinates

Using the same example from my previous post we can send GEO coordinates by removing a common prefix, for large levels of zoom. Indeed when you’ve to send lots of markers to your map application you can expect all of these markers to be fairly close to each other in large zoom level.

Now the coordinates of those points can have a common prefix, like the example bellow with the Subway stations.

LatLon(40.762959,-73.985989)
LatLon(40.761886,-73.983629)
LatLon(40.762861,-73.981612)
LatLon(40.764616,-73.98056)

We can see that all of these GEO points have the same prefix (40.76x, -73.98x), so we can send the prefix only once.

Prefix: (40.76, -73.98)
Data: 
LatLon(2959,5989)
LatLon(1886,3629)
LatLon(2861,1612)
LatLon(4616,056)

These are only three examples of prefix encoding and this algorithm must be considered as very useful when transferring homogeneous data.

Suffix Encoding

Suffix encoding practically the same algorithm as prefix encoding, with the small difference that we use to encode duplicating suffixes. Like the examples bellow suffix encoding can be useful is replacing repeating last name suffixes.

Johnson
Clarkson
Jackson

Or company names.

Apple Inc.
Google Inc.
Yahoo! Inc.

Here we can replace “ Inc.” with something else, but shorter.

Most of the mobile devices today support GPS geo tagging. In fact most of them come bundled with navigation software that uses GPS and therefore all the pictures and (maybe) videos can be geo tagged. But as expected different vendors come with different support and formats.

iPhone OS comes with geotagging both on video and image files, while the latest Android and Symbian (the Nokia main OS for smartphones) can geo tag only images.

Even more – until recently Symbian didn’t support any geotagging before the installation of an additional software – such as Location Tagger . So generally the things are quite simple:

1. iPhone OS geotags both video and image files;
2. Android geotags only images;
3. Symbian geotags only images – and on some devices this is possible only after installing a software;

This is in breve the state of mobile device geotagging!

Why Use GPS Data?

Perhaps one of the main reasons why not support geotagging especially on video files can be the usage of those geo tags. First of all what a geotag means?





You may know that even Android doesn’t “geotag” videos this is not quite true. Because after using you gallery you can see where those videos are shot. This is fantastic, but actually the real information about where the video has been taken is not into the video file, but it’s in an additional log file that keeps it. Thus actually the video files doesn’t know the geo coordinates.

Here comes the problem with video format, because you cannot be sure that every format supports tags that can keep geo coordinates. Actually quicktime’s MOV can store them, while Symbian’s 3GP cannot. In fact Symbian cannot store any geo information about video files!

So now we’ve at least three different formats for each of those three vendors.

1. quicktime for iPhone
2. mpeg-4 with Android
3. 3gp with Symbian

For now I can only say that iPhone can keep the geotag into his video files.





But let me return to the question – why we need this geo tags? Until the video file is on the mobile device – there’s no problem. But once you try to download it – whether on Flickr, YouTube, Picasa, etc. you’ll lose any geo information if it’s not into the file tags. And of course if the above sites can’t read it!

The general reason to store this data into the file is to move it along with the file. Once you move this file from your mobile device to a web platform you’ll see where the file has been created.

EXIF, Exiftool and PHP’s exif_read_data

There are several tools to read geotags. For images, and here we talk only for JPEGs, this is the EXIF information. You can download the exif command line program and try to reed data with it:

exif filename.jpg

Which can result in something like that:

Manufacturer        |Nokia                                                     
Model               |E71                                                       
Orientation         |top - left                                                
x-Resolution        |300.00                                                    
y-Resolution        |300.00                                                    
Resolution Unit     |Inch                                                      
YCbCr Positioning   |centered                                                  
Compression         |JPEG compression                                          
x-Resolution        |72.00                                                     
y-Resolution        |72.00                                                     
Resolution Unit     |Inch                                                      
FNumber             |f/3.2                                                     
Exif Version        |Exif Version 2.2                                          
Date and Time (origi|2011:01:24 12:15:01                                       
Date and Time (digit|2011:01:24 12:15:01                                       
ComponentsConfigurat|Y Cb Cr -                                                 
Aperture            |3.20 EV (f/3.0)                                           
Light Source        |0                                                         
Flash               |Flash did not fire, auto mode.                            
Focal Length        |4.9 mm                                                    
FlashPixVersion     |FlashPix Version 1.0                                      
Color Space         |sRGB                                                      
PixelXDimension     |2048                                                      
PixelYDimension     |1536                                                      
Custom Rendered     |Normal process                                            
Exposure Mode       |Auto exposure                                             
White Balance       |Auto white balance                                        
Digital Zoom Ratio  |1.00                                                      
Scene Capture Type  |Standard                                                  
GPS tag version     |0x02, 0x02, 0x00, 0x00                                    
North or South Latit|N                                                         
Latitude            |42.00, 39.00, 24.67                                       
East or West Longitu|E                                                         
Longitude           |23.00, 21.00, 28.45                                       
Altitude reference  |0x00                                                      
Altitude            |625.50 

While for video files (iPhone) exif cannot help you, this is possible through Exiftool. With it you can read even iPhone video geotags:

./exiftool filname.mov

In addition this information can be read with PHP via exif_read_data:

$info = exif_read_data('filename.jpg');
var_dump($info);

And this will give you not only standard tags, but geo tags too.

However FFMPEG is a converting program and many things depend on what kind of video files come as an input and what do you want to achieve as an output. Here comes the codec question, where various libraries can help. In my case, as I wrote few posts ago, I use MP4 as output encoded with h.264, both encoded with ffmpeg/libx264. So far so good, but there is one main problem that anyone used this pair knows.

Flash Player and MP4

As far as I know Flash Player 9 and beyond plays video files encoded with MP4/h.264 except the typical FLV (flash video files). This can help video sites not only to share their video content via web – within browser based flash players, but also to show them via some mobile device interface where MP4/h.264 is perhaps the only one reasonable solution. In this case the file is accessible in both medias.

By default Flash Player can play such FFMPEG encoded .mp4 files, but fist the whole file must be downloaded by the browser and only than the playback can begin. That’s because the FFMPEG puts meta info at the end of the video file, while the flash player needs it at the beginning and only than to begin playing while the file is still downloading.

qt-faststart

As the name of this software says this program helps you move the important meta info from the end to the beginning of the file. This helps the video to playback as early as possible. In an ideal circumstances everything works just fine, but there is a serious problem in fact.

qt-faststart and Server Crash

qt-faststart is responsible to move the meta info where it must be – at the beginning of the video file, but cannot finish execution when the video file is broken or the meta info doesn’t exists. Actually this is not a problem of qt-faststart, but to the video file. However this was a critical problem in my case, because the video files remains stuck for hours of execution and 99% of CPU usage.

Thankfully there is a solution and it can be found in the qt-faststart wrapper – qtfaststart.py

qtfaststart.py

written on Python this script overcomes the disadvantages of qt-faststart. As described by his author Daniel G. Taylor:

Works everywhere Python can be installed

Handles both 32-bit (stco) and 64-bit (co64) atoms

Handles any file where the mdat atom is before the moov atom

Preserves the order of other atoms

Can replace the original file (if given no output file)

With its help not only qt-faststart doesn’t crash, but also the video files remain progressive as desired. The installation process is as difficult as copy/paste on the server where Python must be installed.

It’s nice that at least you’re moving your site to mobile device. It’s really important. Now if you’re using PHP the question is how to detect the mobile browser.

HTTP_USER_AGENT

Every PHP developer knows that $_SERVER[‘HTTP_USER_AGENT’] string optionally contain the browser that access your site, but what are the possible values of this string?

The Answer is …

in that really really full list of user agent strings: http://www.zytrax.com/tech/web/mobile_ids.html

It isn’t strange that the internet is becoming more and more mobile. When it matters to news sites I think most of the mobile versions are better than the “desktop” versions of the site. Take a look at Huffingtonpost.com or LeMonde.fr.

Mobile Ads

The first thing that makes impression when comparing both mobile and desktop versions of a site is the layout. Of course the lack of ads on the mobile version, or if not the absolute lack at least the small number of appearing ads, seems to be quite pleasant.

The thing is that as the web grow and as the web 2.0 is becoming a reality most of us will prefer to use it on a mobile device, just because it’s

1. easier to use
2. easier to find
3. easier to follow

Here’s a nice example of what mobile phones are now.

Now lets take a look on how the mobile web will grow in the near future.

That graphics describe quite well what will happen in the upcoming years. Is there now a question: should I make a mobile version of my site?