When a page is loaded in a Modal Dialog, its URL ends with «IsDlg=1», that means that it’s being opened in a SharePoint PopUp. If you need to hide some elements when that happens (e.g. hide master page elements) you need to add a new class to these elements:
Adding class «ms-dialogHidden» to an element (div, img… anything) it’ll be hidden in Modal Dialogs and visible when it’s not a dialog.
Furthermore, Modal Dialog size it’s calculated depending on the inside page, so it can help if you initially hide everything from the master page and then load only your page.