Saturday, 27 April 2013

10 things to do after installing Linux Mint 13

Linux Mint 13 just got released and I have been using it (MATE edition) on my laptop for nearly a week and from my experience, this is a very nice and stable release. However, depending on personal need and preference, users still need to tweak and customize a few things to make it more usable. Here are 10 things that I did after installing Linux Mint 13 - MATE edition.

Update System

As always on a freshly installed system, the first thing you need to do is to update the packages. To do so, you can click on the shield icon on the notification area, enter the password and the update manager will appear. Click on the Install Updates button to do the system updating. It will take a while depending on your network speed.

10 things to do after installing Linux Mint 13

You can also open the terminal and run the following command for system updating: 
 sudo apt-get update && sudo apt-get upgrade  

Install proprietary driver for your graphic card

If your computer has a dedicated graphic card, you should consider installing the proprietary driver for it. When the open source driver does get the job done, it still cannot hold a candle to the proprietary one. To check for available proprietary drivers, open the Mint menu, go to control center then Additional Drivers then select the driver to install. If you are using an ATI card, I recommend you to pick the second driver in the additional drivers window since the first one had caused my system to crash.

 things to do after installing Linux Mint 13

After installing the driver, you need to reboot the system for the new driver to take effect.

Change the search engines of Firefox


In Linux Mint 13, Firefox uses two search engines, Yahoo search if you use the search box and DuckDuckGo if you search in the address box. I always prefer Google over these search engines so one of the first things I did after installing Linux Mint 13 was to change the Firefox search engine back to Google search.

To change the search engine in the search box of Firefox to Google, you just need to open Firefox then go to the following address:
 http://www.linuxmint.com/searchengine_google.php  

After that, click on the drop down list in the search box and select "Add Google"

change search engine firefox Linux mint

To change the search engine of the address bar from DuckDuckGo to Google, you open Firefox , type about:config on the address bar. You will see a warning, just click on the "I'll be careful" button to go to the config page. In this config page, scroll down until you find the line that begins with "keyword.URL":

change firefox Linux mint search engine

Right click on this line and select the Modify option. The box to edit the search engine will appear, in this box, you just need to replace the DuckDuckGo url with Google:
 http://www.google.com/search?q=  

change Linux mint firefox search engine

Click ok and everything is done.  Now both the address bar and the search box use Google as the default search engine.

Get more themes and icons

Linux Mint 13 comes with very few themes and icon sets so I think everybody would love to get more themes and icon to decorate the desktop. The best place to look for nice themes and icons in my opinion is Gnome-look. Just go there and search for the themes and icons you like. Also remember to read the installation instruction of each theme and icon set.

Here is my Linux Mint 13 desktop with Delorean Noir theme and ubo icons

beautiful Linux Mint desktop


Get a conky setup

Conky is an amazing linux application. A neat conky will make your desktop look amazing. I may be accused of being flamboyant but a conky setup was the first thing I got for my Linux Mint 13 desktop.

Take a look at the awesome Reloj Conky config:



If you are interested in conky like me, you can check my articles about beautiful conky configs.

Customize the Mint Menu

You can use the Mint Menu to search for and launch applications or access quickly to preferred folders. To customize the Mint Menu, right click on it and choose the Preferences option and the Menu preferences window will open. To add shortcut to your favorite folders, just go to the Places tab and click on the New button to add new folders to the Mint Menu:

customize linux mint menu

To customize the Favorites list on the Mint Menu, you can right click on each application to remove or add it into the Favorite list:

how to customize linux mint menu

You may find that after you remove some applications out of and add new ones into the Favorite List, the icons arent arranged into the order you want and you cannot drag these icons around on the menu. To change the order of the icons on the Favorites list, you need to edit the file~/.linuxmint/mintMenu/applications.lst. Here is how this file looks to get the Mint Menu in the screenshot below. ( Note: after you edit the applications.list file, you need to right click on the Mint Menu and select Reload plugins to restart the Mint Menu)
 location:/usr/share/applications/firefox.desktop  
 location:/usr/share/applications/thunderbird.desktop  
 location:/usr/share/applications/pidgin.desktop  
 location:/usr/share/applications/xchat.desktop  
 separator  
 location:/usr/share/applications/mate-appearance-properties.desktop  
 location:/usr/share/applications/libreoffice-writer.desktop  
 location:/usr/share/applications/mate-terminal.desktop  
 location:/usr/share/applications/pluma.desktop  
 separator  
 location:/usr/share/applications/ccsm.desktop  
 location:/usr/share/applications/gimp.desktop  
 separator  
 location:/usr/share/applications/banshee.desktop  
 location:/usr/share/applications/vlc.desktop  

how to customize mint menu

Set auto mount for all the partitions

Besides the root partition, there are several other partitions in the hard drive of my laptop. These partitions are where I keep my important stuffs. And I hate it when I tried to access a file from an application, I realized that I forgot to mount these partitions. That's why I always prefer auto mounting all the partitions in my Linux box.

There are actually many ways to auto mount partitions in Linux, one of them is to use a tool called  "pysdm". First, you need to install it:
 sudo apt-get install pysdm  

Next, open the terminal and run pysdm as root:
 sudo pysdm  

The configuration window of pysdm will appear, on which you just need to select the partition to configure:


After that, click on the Assistant button on the right and you will have a menu to customize the behavior of each partition. The option to auto mount partition at booting is the second one.


Just do that to all the partitions you want to mount at booting up and next time, you dont need to manually mount these partition anymore.

Hide the partition icons on the desktop

By default, the desktop of Linux Mint 13 always displays the icons of mounted partitions and USB drives and I think that makes the desktop ugly. I always want my desktop to be nice and clean. To hide these icons, hit Alt+F2 and type mateconf-editor then hit enter:


The MATE configuration window will appear. On which, you go to apps > caja > desktop then uncheck the volumes_visible option like the image below:


And you wont see these partition and usb icons on your desktop anymore.

Install Compiz Config Settings Manager

Compiz is an awesome tool, you can do a lot of cool things with it so I recommend everybody to install Compiz. You can tweak a lot of things with CCSM and of course, it is always fun to show the cubic desktops to your friends :D

To install Compiz, you can search for it in the Software Manager or run the following command:
 sudo apt-get install compizconfig-settings-manager  

Install Wine

I myself at times need to run some Windows applications so Wine is always a must-have tool for me. To install Wine, you can search for it in the Software Manager or use this command:
 sudo apt-get install wine  

-------------------------------------------------------------------------------------------

These are 10 things I did after installing Linux Mint 13 - MATE edition. If you are using Linux Mint 13 too and have any other ideas, please tell me by giving a comment.

Changing Folder Icons Mint Linux


With the Mate Desktop customizing is a great way to quickly find a special folder or file. You may have noticed that the folder icons are predetermined by the theme you set. So if you wish to change a folder icon to another so that it will stand out from the system-wide icons. There are tones of sites to get icons and you can use them to customize your folders in your user area and find your files a little bit faster.

Changing Folder Icons


1. Open the File Browser

Go to Menu > System Tools > File Browser, right click a folder icon, select Properties.

2. Navigate to the folder

Find a folder or make one for testing.

3. Right click the folder icon.

Left click Properties to open the settings window.

4. Left click the icon image.

Left click on the folder image at the top to choose a new icon.


Saturday, 20 April 2013

Map Reduce: A really simple introduction

Ever since google published its research paper on map reduce, you have been hearing about it. Here and there. If you have uptil now considered map-reduce a mysterious buzzword, and ignored it, Know that its not. The basic concept is really very simple. and in this tutorial I try to explain it in the simplest way that I can. Note that I have intentionally missed out some deeper details to make it really friendly to a beginner.

Chapter 1: Your CEO’s Strange itch:

Imagine this. You work in a really big company. Your company is planning to launch the next big “Blogging platform”. Tommorow morning you go to your office and there’s a mail from your CEO regarding a new work:
 
Dear  <Your Name>,
 As you know we are building the blogging platform blogger2.com, I need some statistics. I need to find out, Acorss all blogs ever wrriten on blogger.com, how many times 1 character words occur(like 'a', 'I'), How many times two character words occur (like 'be', 'is').. and so on till how many times do ten character words occur.

 I know its a really big job. So, I will assign, all 50,000 employees working in our company to work with you on this for a week.  I am going on a vacation for a week, and its really important that I've this when I return. Good luck.

regds,
The CEO

P.s : and one more thing. Everything has to be done manually, except going to the blog and copy pasting it on notepad. I read somewhere that if you write programs, google can find out about it

Picture yourself in that position for a moment. You have 50,000 people to work for you for a week. And you need to find out the number of 1 character words, No. of 2 character words etc., covering the maximum number of blogs in blogspot. Finally you need to give a report to your CEO with something like this:
  • Occurance of one character words – Around 937688399933
  • Occurance of two chracter words – Around 23388383830753434
  • .. hence forth till 10
If homicide, suicide or resigining the job is not an option, how would you solve it? How would you avoid the chaos of so many people working. How will you co-ordinate those many since the output of one has to be merged with another?
You decide to take leave for the day, go home, sleep over it, and the next day wake up with the greatest Idea ever. “S**t! i wasted a day!”

Chapter 2: Your proclamation: Let there be caste

The next day, You stand with a mike on the dias before 50,000 and proclaim. For a week, you will all be divided into many groups:
  • The Mappers (tens of Thousands of people will be in this group)
  • The Grouper (Assume just one guy for now)
  • The Reducers( Around 10 of em.) and..
  • The Master(That’s you).
Then you talk to each one of the groups.

Chapter 3: Your talk with The Mappers

Each mapper will get a set of 50 blog urls and really Big sheet of paper. Each one of you need to go to each of that url. and for each word in those blogs, write one line on the paper. The format of that line should be the number of characters in the word, then a commna, and then the actual word.

For example, if you find the word “a”, you write “1,a”, in a new line in your paper. since the word “a” has only 1 character. If you find the word “hello”, you write “5,hello” on the new line.

Each take 4 days. So, After 4 days, your sheet might look like this
  • “1,a”
  • “5,hello”
  • “2,if”
  • .. and a million more lines
At the end of the 4th day. each one of you will give your sheet completely filled to the Grouper

Chapter 4: Your talk with the Grouper

I will give you 10 papers. The first paper will be marked 1, the second paper will be marked 2, and so on, till 10.
You collect the output from mappers and for each line in the mapper’s sheet, if it says “1,”, your write the on sheet 1, if it says “2, ”, you write it on sheet two.

For example, if the first line of a mapper’s sheet says “1,a”, you write “a” on sheet 1. if it says “2,if”, your write “if” on sheet 2. If it says “5,hello”, you write hello on sheet 5.

So at the end of your work, the 10 sheets you have might look like this
  • Sheet 1: a, a ,a , I, I , i, a, i, i, i…. millions more
  • Sheet 2: if, of, it, of, of, if, at, im, is,is, of, of … millions more
  • Sheet 3 :the, the, and, for, met, bet, the, the, and, … millions more
  • ..
  • Sheet 10: ……
once you are done, you distribute, each sheet to one reducer. For example sheet 1 goes to reducer 1, sheet 2 goes to reducer 2 and so on.

Chapter 5: Your talk with the Reducers:

Each one of you gets one sheet from the grouper. for each sheet you count the number of words written on it and write it in big bold letters on the back side of the paper.

For ex, if you are reducer 2. You get sheet 2 from the grouper that looks like this:
“Sheet 2: if, of, it, of, of, if, at, im, is,is, of, of …”

You count the number of words on that sheet, say the number of words is 28838380044, You write it on the back side of the paper , in big bold letters and give it to me(the master).

Chapter 6: The controlled Chaos and the climax:

At the end of this process you have 10 sheets, Sheet 1, having the count of the number of words with 1 character on the back side. Sheet2, having the count of the number words with 2 characetrs on the back side. You did it. Genius.
You essentially did map reduce. The greatest advantage in your approach was this
  • The mappers can work independently
  • The reducers can work independently
  • The Grouper can work really fast, because, he din’t have to do any counting of words, all the had to do was to look at the first number and put that word in the appropriate sheet.
The process can be easily applied to other kinds of problems. In such a case :
  • The work of the Master(dividing the work) and the Grouper(Grouping the values by key[the value before commna]), remains the same. This is what any map-reduce library provides.
  • The work of the mappers and reducers differ according to the problem. This is what you should write.
You can optimize this a little bit. And I am skipping those. For example, you don’t even have to mention the words, every where, you could have just written down “x”, instead of the actual word, since in the end, we are just counting. And everything need not happen in a sequence like First Mapper, the Grouper and then reducer. Moreover, one person can be sometime do the job of a mapper and some other time the job of a grouper. Give all this a thought and you will get more answers.
So You solve the biggest challenge ever posed to you.

After a week You collect the sheet of papers from the reducers. The back side of sheet 1 will have the number of occurences of words with 1 character. The back side of sheet 2 will have the number of occurances of words with 2 characters and so on..

You put this information in a excel, Take a printout in a neat sheet of paper and take it to your CEO with a big smile. “Good job “, he says, “put it on the desk, I will take a look at it in a month” :)

Wednesday, 17 April 2013

Confused About Map/Reduce ?



 I was working on some Hadoop stuff recently, and as a total beginner, I found that the Map/Reduce concept was not easy to understand, despite the huge number of tutorials.
The Wordcount example is the ‘Hello World’ of Hadoop, but when I prepared a small presentation for my team, I realized it was not clear enough to explain Map/Reduce in 5 minutes.
As you may already know, the Map/Reduce pattern is a pattern that is very good for embarrassingly parallel algorithms.
Okayyyy but… What is an embarrassingly parallel algorithm?
Answer: It is an algorithm that is very well fit to be executed multiple times in parallel.
Ok then… what is very well suited for a parallel execution?
Answer: Any algorithm that’s working on data that can be isolated.
When writing an application, if you execute multiple occurrences of it at the same time, and they need to access some common data, there will be some clash, and you will have to handles cases like when one occurrence is changing some data while another other is reading it. You’re doingconcurrency.
But if your occurrence is working on some data that no other occurrence will need, then you’re doingparallelism. Obviously you can scale further, since you do not have concurrency issues.
So let’s take an example, let’s say you have a list of cities, and each one has two attributes : the state it belongs to, and its yearly average temperature. E.g. : San Francisco : {CA, 58}
Now you want to calculate the yearly average temperature BY STATE.
Since you can group cities by state, and calculate the average temperature of a state without caring about cities of other states, you have a great embarrassingly parallel algorithm candidate.
If you wanted to do it sequentially, you would start with an empty list of yearly state average temperatures. Then you would iterate through the list of cities, and for each city, look at the state, then update the relevant yearly state average temperature.
Fortunately, it’s very easy to do it in parallel instead.
Let’s have a look at this map:
This is a map of India. There are several states : MP, CG, OR… And several cities, each one having {State, City average temperature} as value.
We want here to calculate the yearly average per state. In order to do that, we should group the city average temperatures by state, then calculate the average of each group.
We don’t really care about the city names, so we will discard those and keep only the state names and cities Temperatures.
Now we have only the data we need, and we can regroup the temperatures values by state. We’re going to get a list of temperatures averages for each state.
At this point, we have the data in good shape to actually do the maths… All we have to do is to calculate the average temperature for each state
That wasn’t hard.
We had some input data. We did a little regrouping, then we did the calculation. And all this could be executed in parallel (One parallel task for each state).
Well… That was Map/Reduce!
Let’s do it again
Map/Reduce has 3 stages : Map/Shuffle/Reduce
The Shuffle part is done automatically by Hadoop, you just need to implement the Map and Reduce parts.
You get input data as <Key,Value>  for the Map part.
In this example, the Key is the City name, and the Value is the set of  attributes : State and City yearly average temperature.
Since you want to regroup your temperatures by state, you’re going to get rid of the city name, and the State will become the Key, while the Temperature will become the Value.
Now, the shuffle task will run on the output of the Map task. It is going to group all the values by Key, and you’ll get a List<Value>
And this is what the Reduce task will get as input : the Key, List<Value> from the Shuffle task.
The Reduce task is the one that does the logic on the data, in our case this is the calculation of the State yearly average temperature.
And that’s what we will get as final output
This is how the data is shaped across Map/Reduce:
Mapper <K1, V1> —> <K2, V2>
Reducer <K2, List<V2>> —><K3, V3>
I hope this helped makes things a bit clearer about Map/Reduce, if you’re interested in explanations about Map Reduce v2/YARN, just leave a comment and I’ll post another entry.
PS: The java code for this example can be found here: