I definitely recommend going inside the Parliament building; it's free to European citizens (so I think a definite must for the budget traveller!) and is a stunning building. Turn up in the morning, with your passport, and then get your tickets for a predetermined time later that day (closed Mondays). While waiting we went into the Jewish museum (varying opening times so check) and synagogue, which was very impressive - biggest synagogue in Europe - as was the Silver Tree holocaust memorial.

A great budget suggestion for all English speaking travellers in Budapest is to go on a free tour in the Museum of Fine Arts. Nowadays admission is free to the permanent exhibition and the Docent Program offers free 1 hr tours at 11am, Tuesday - Saturday. It is an excellent budget event and you are in for a cultural treat!