I use something like
\usepackage{xcolor}
\hypersetup{
colorlinks,
linkcolor={red!50!black},
citecolor={blue!50!black},
urlcolor={blue!80!black}
}
This gets rid of the ugly color boxes, but uses dark colored fonts which still make it clear that they are clickable.