1. Fairly is used in a moderate sense before the adjectives used in good sense.
e.g. - He is fairly well.

Rather is also used in the moderate sense before the adjectives used in bad sense.
e.g. - Ram is rather hopeless.

2. Fairly is used in pleasant conditions.
e.g. - She is fairly beautiful.

Rather is used in unpleasant conditions.
e.g. - They are rather mean and cunning.

3. Fairly is never preceded by Too wherever we use rather before the adverb " too'.
e.g. - John is rather too unwise.

4. Fairly doesn't take the article A immediately after it, but this is not the case with Rather.
e.g. - Abraham is rather a stupid boy. ( Right )