Here's what your lookin' for: Trelane's phylactery, Trelane's orb, an' Trelane's ember agate.
All three should be in chests within the tower an' you shouldn't have a problem opening the chests on your own; you just have to be careful of the ogres.
The tower's defenses won't take care of 'em on their own; you'll need to have your weapons at the ready. Those ogres find out you're there and stealing from 'em, an' they'll be sure to put the hurt on you.