Recycling is one of the most efficient methods for environmental friendly waste management. Among municipal wastes, plastics are the most common material that can be easily recycled and polyethylene terephthalate (PET) is one of its major types. PET material is used in consumer goods packaging such as drinking bottles, toiletry containers, food packaging and many more. Usually, a recycling process is tailored to a specific material for optimal purification and decontamination to obtain high grade recyclable material. The quantity and quality of the sorting process are limited by the capacity of human workers that suffer from fatigue and boredom. Several automated sorting systems have been proposed in the literature that include using chemical, proximity and vision sensors. The main advantages of vision based sensors are its environmentally friendly approach, non-intrusive detection and capability of high throughput. However, the existing methods rely heavily on deterministic approaches that make them less accurate as the variations in PET plastic waste appearance are too high. We proposed a probabilistic approach of modeling the PET material by analyzing the reflection region and its surrounding. Three parameters are modeled by Gaussian and exponential distributions: color, size and distance of the reflection region. The final classification is made through a supervised training method of likelihood ratio test. The main novelty of the proposed method is the probabilistic approach in integrating various PET material signatures that are contaminated by stains under constant lighting changes. The system is evaluated by using four performance metrics: precision, recall, accuracy and error. Our system performed the best in all evaluation metrics compared to the benchmark methods. The system can be further improved by fusing all neighborhood information in decision making and by implementing the system in a graphics processing unit for faster processing speed.