Skip to main content
Log in

A novel hypergraph convolution network for wafer defect patterns identification based on an unbalanced dataset

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In semiconductor industry, various wafer defect patterns represent different causes of manufacturing failures. Identification of specific defect patterns is important to wafer fabrication process. Recently, many studies concentrate on developing Deep Learning algorithms such as Convolution Neural Network, most of which neglect hyper-relations among dataset. Therefore, in this work, a novel Hypergraph Convolution Network is proposed for the automatic Wafer Defect Identification (HCN-WDI). The main contributions include: (1) The detailed theoretical formulation and NP-Completeness proof of normalized cut for (hyper-)edge segmentation is firstly discussed. (2) The data augmentation techniques are applied to balance the number inequality of different patterns in wafer defect dataset WM-811K. (3) The Hyper Convolution Network is implemented as an end-to-end operator to identify wafer defect patterns and three conventional image classifiers are used as feature extractors and reference baselines for the proposed HCN-WDI model. The experimental results show that the proposed HCN-WDI model outperforms other three fine-tuning conventional image classifiers and obtains the highest \(96.44\%\) averaged classification accuracy. Besides, by comparing the results from various combinations of extracted features, it is concluded that the accuracy of the HCN-WDI model is also dependent on the quality rather than quantities of feature extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon request.

Code availability

The feature extractors are available from the corresponding author upon request.

References

  • Bai, S., Zhang, F., & Torr, P. H. (2021). Hypergraph convolution and hypergraph attention. Pattern Recognition, 110(107), 637.

    Google Scholar 

  • Berge, C. (1973). Graphs and hypergraphs. AGRIS.

  • Cheon, S., Lee, H., Kim, C. O., & Lee, S. H. (2019). Convolutional neural network for wafer surface defect classification and the detection of unknown defect class. IEEE Transactions on Semiconductor Manufacturing, 32(2), 163–170.

    Article  Google Scholar 

  • Chou, W., Shao, J., Chung, R., Chen, L., Chen, A., & Zhou, L. (2019). Semiconductors—The next wave: Opportunities and winning strategies for semiconductor companies.

  • Di Palma, F., De Nicolao, G., Miraglia, G., Pasquinetti, E., & Piccinini, F. (2005). Unsupervised spatial pattern classification of electrical-wafer-sorting maps in semiconductor manufacturing. Pattern Recognition Letters, 26(12), 1857–1865.

    Article  Google Scholar 

  • Feng, Y., You, H., Zhang, Z., Ji, R., & Gao, Y. (2019). Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence (pp 3558–3565).

  • Garey, M. R., & Johnson, D. S. (1979). Computers and intractability (Vol. 174). San Francisco: Freeman.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778).

  • Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429–449.

    Article  Google Scholar 

  • Jeong, Y. S., Kim, S. J., & Jeong, M. K. (2008). Automatic identification of defect patterns in semiconductor wafer maps using spatial correlogram and dynamic time warping. IEEE Transactions on Semiconductor manufacturing, 21(4), 625–637.

    Article  Google Scholar 

  • Jin, C. H., Na, H. J., Piao, M., Pok, G., & Ryu, K. H. (2019). A novel DBSCAN-based defect pattern detection and classification framework for wafer bin map. IEEE Transactions on Semiconductor Manufacturing, 32(3), 286–292.

    Article  Google Scholar 

  • Khan, S. H., Hayat, M., Bennamoun, M., Sohel, F. A., & Togneri, R. (2017). Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3573–3587.

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25.

  • Kyeong, K., & Kim, H. (2018). Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks. IEEE Transactions on Semiconductor Manufacturing, 31(3), 395–402.

    Article  Google Scholar 

  • Li, P., & Milenkovic, O. (2017). Inhomogeneous hypergraph clustering with applications. Advances in Neural Information Processing Systems, 30.

  • Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., & Malossi, C (2018). Bagan: Data augmentation with balancing gan. arXiv:1803.09655

  • Milor, L. (2013). A survey of yield modeling and yield enhancement methods. IEEE Transactions on Semiconductor Manufacturing, 26(2), 196–213.

    Article  Google Scholar 

  • Nakazawa, T., & Kulkarni, D. V. (2018). Wafer map defect pattern classification and image retrieval using convolutional neural network. IEEE Transactions on Semiconductor Manufacturing, 31(2), 309–314.

    Article  Google Scholar 

  • Saqlain, M., Abbas, Q., & Lee, J. Y. (2020). A deep convolutional neural network for wafer defect identification on an imbalanced dataset in semiconductor manufacturing processes. IEEE Transactions on Semiconductor Manufacturing, 33(3), 436–444.

    Article  Google Scholar 

  • Saqlain, M., Jargalsaikhan, B., & Lee, J. Y. (2019). A voting ensemble classifier for wafer map defect patterns identification in semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 32(2), 171–182.

    Article  Google Scholar 

  • Semiconductor, G. (2019). Semiconductors: As the backbone of the connected world, the industry’s future is bright. In KMPG’s 14th Annu Glob Semicond outlook Rep (pp. 1–24).

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  • STATISTA. (2019). Monthly semiconductor sales worldwide from 2012 to 2020 (in billion u.s. dollars). Statista Gmbh: Hamburg, Germany.

  • Tello, G., Al-Jarrah, O. Y., Yoo, P. D., Al-Hammadi, Y., Muhaidat, S., & Lee, U. (2018). Deep-structured machine learning model for the recognition of mixed-defect patterns in semiconductor fabrication processes. IEEE Transactions on Semiconductor Manufacturing, 31(2), 315–322.

    Article  Google Scholar 

  • Wang, J., Yang, Z., Zhang, J., Zhang, Q., & Chien, W. T. K. (2019). Adabalgan: An improved generative adversarial network with imbalanced learning for wafer defective pattern recognition. IEEE Transactions on Semiconductor Manufacturing, 32(3), 310–319.

    Article  Google Scholar 

  • Wu, M. J., Jang, J. S. R., & Chen, J. L. (2014). Wafer map failure pattern recognition and similarity ranking for large-scale data sets. IEEE Transactions on Semiconductor Manufacturing, 28(1), 1–12.

    Google Scholar 

  • Wu, Z., & Leahy, R. (1993). An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11), 1101–1113.

    Article  Google Scholar 

  • Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Philip, S. Y. (2020). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4–24.

    Article  Google Scholar 

  • Zhang, Z., Cui, P., & Zhu, W. (2020). Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering.

  • Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020). Graph neural networks: A review of methods and applications. AI Open, 1, 57–81.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yuxi Xie.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Proof of NP-completeness

Appendix A: Proof of NP-completeness

Proof

To prove NP-completeness of optimal normalized cuts in hypergraph, i.e.,

$$\begin{aligned} \mathop {{{\,\mathrm{arg\,min}\,}}}_{\emptyset \ne S \subset V} c\left( S \right) := \textrm{vol}\ \partial S \left( \frac{1}{\textrm{vol}\ S} + \frac{1}{\textrm{vol}\ S^c} \right) \end{aligned}$$
(A1)

Two restrictions are introduced:

  • edge weight keeps constant, i.e., \(\textrm{w}(e)=1\).

  • each hyperedge only connects two nodes.

Based on the two restrictions above, the definitions of volumes of vertex subset S and \(S^c\) can be reduced as,

$$\begin{aligned}{} & {} \textrm{Vol}\ S := \sum _{v\in S} \sum _{e \in E} \textrm{w}\left( e \right) h\left( v, e \right) = \sum _{v \in S} \sum _{t \in V} \textrm{w}\left( v,t \right) \\{} & {} \textrm{Vol}\ S^c := \sum _{u \in S^c} \sum _{e \in E} \textrm{w} \left( e \right) h\left( u,e \right) = \sum _{u \in S^c} \sum _{t \in V} \textrm{w} \left( u,t \right) \end{aligned}$$

Then the volume of hyperedge boundary \(\partial S\) is defined as

$$\begin{aligned} \textrm{vol}\ \partial S:= & {} \sum _{e \in \partial S} \textrm{w}\left( e \right) \frac{\left| e \cap S \right| \left| e \cap S^c \right| }{\delta \left( e \right) } \\ {}= & {} \sum _{e \in \partial S} \textrm{w}\left( e \right) \frac{\left| e \cap S \right| \left| e \cap S^c\right| }{\sum _{v \in V} h\left( v,e \right) } \\= & {} \frac{1}{2} \sum _{e \in \partial S} \textrm{w}\left( e \right) = \frac{1}{2} \sum _{v \in S, u \in S^c} \textrm{w}\left( u,v \right) \end{aligned}$$

in which, \(\left| e \cap S \right| = \left| e \cap S^c \right| =1\) based on the two restrictions above.

Summarily, the Eq. (A1) can be converted to

$$\begin{aligned}{} & {} \mathop {{{\,\mathrm{arg\,min}\,}}}_{\emptyset \ne S \subset V} c \left( S \right) := \textrm{vol} \partial S \left( \frac{1}{\textrm{vol} S } + \frac{1}{\textrm{vol} S^c} \right) \nonumber \\{} & {} \quad = \frac{1}{2} \sum _{v \in S, u \in S^c} \textrm{w}\left( u,v \right) \nonumber \\{} & {} \qquad \left[ \frac{1}{\sum \limits _{v \in S} \sum \limits _{t \in V} \textrm{w}\left( v,t \right) } + \frac{1}{\sum \limits _{u \in S^c} \sum \limits _{t \in V} \textrm{w}\left( u,t \right) } \right] \end{aligned}$$
(A2)

It denotes that minimizing normalized cuts for hypergraph can be reduced to simple graph which is NP-complete.

According to NP-Completeness proof techniques Garey and Johnson (1979), by applying some specifications of additional restrictions on the problem \(\Pi \) which is to be proved as NP-Completeness, the modified problem with additional restrictions can be reduced to be identical another problem \(\Pi ^{\prime }\) which has been known as NP-Complete.

Therefore, the NP-completeness of the problem that minimizing normalized cuts of hypergraph has been proved.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Y., Li, S., Wu, C.T. et al. A novel hypergraph convolution network for wafer defect patterns identification based on an unbalanced dataset. J Intell Manuf 35, 633–646 (2024). https://doi.org/10.1007/s10845-022-02067-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-02067-z

Keywords

Navigation