It sounds like you have a good understanding of my problem:
I can't open the .acsm file on either the Mac or the
Nook (I get the 'error getting license, server communication problem' message)
I can copy an .epub file from ADE to the Nook
I can get the .epub file to the Nook and can read it there