A fast computational algorithm is presented for the analysis of multilayered nanolithography masks. The technique used is an exact field-theoretical approach which can model the diffraction effects in subwavelength propagation regimes. The field scattered by the mask pattern is obtained in two steps. First, a surface impedance generating operator (SIGO) that relates the tangential electric field on the boundary of each etched area to its equivalent surface electric current is computed. Second, the exterior problem is formulated based on the equivalence theorem in electromagnetics and is combined with the SIGO model. These two steps may be executed in parallel, making the lithography simulation fast and numerically efficient. For an arbitrary 2D mask illuminated by a TMy-polarized incident wave, the required Green's functions are obtained. The Green's function of the interior problem is calculated directly in the spatial domain while the complex images method is used for computing the Green's functions of the exterior multilayer problem. Based on this forward modeling procedure, a parameter sweep is performed and a binary mask pattern under normal incident coherent illumination is analyzed.