We consider two quasi-exactly solvable problems in one dimension for which
the Schr\"odinger equation can be converted to Heun's equation. We show that in
neither case the Bender-Dunne polynomials form an orthogonal set. Using the
anti-isopectral transformation we also discover a new quasi-exactly solvable
problem and show that even in this case the polynomials do not form an
orthogonal set.