Nonlinear ultrasonic guided waves have been investigated widely in closed waveguides such as plates, pipes, etc. However, the description of nonlinear ultrasonic guided waves remains challenging for open waveguides, as energy may leak into the surrounding medium. In this work, the properties of nonlinear ultrasonic guided waves in open waveguides are investigated. Mathematical framework is first established based on real reciprocity relation and modal expansion with perfectly matched layers. Numerical models are then implemented, including nonlinear semi-analytical finite element (SAFE) method to predict the properties of nonlinear ultrasonic guided waves, and time domain finite element models to simulate the nonlinear guided wave propagation and cross validate the predictions from the nonlinear SAFE method. Two examples, an aluminum plate attached to an elastomer and an aluminum plate with water loaded on one side, are studied to demonstrate the proposed methods and reveal some interesting phenomena that only exist in open waveguides. It is interesting to find out that the amplitude of the attenuated second harmonic wave in immersed waveguides can keep constant with propagation distance, only if the primary wave is non-leaky, which may bring potential non-destructive test applications for underwater inspections. Such a feature is validated by well-designed experiments in one-sidedly immersed plates.