If everything we are given -- cover to cover -- in Fandango is a clue, the compass is made up of exactly 3 colors -- Red, White, and Yellow.

Red & Yellow make "Orange", which could tell us that those are the important pages (think the MAP page), and maybe Blue and Green pages are the real "Red Herrings".

It would also explain why all the keys in the illustrations are "Orange"...