Skip to main content

Advertisement

SpringerLink
Log in

Realization of superhuman intelligence in microstrip filter design based on clustering-reinforcement learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Microstrip filters are widely used in signal processing because of their light weight, compact structure and high reliability. Designing these filters is very time consuming, and a designer generally needs much knowledge of electromagnetic theory. In recent years, artificial intelligence (AI) technology has been used to accelerate the design process. However, current AI models retain the human design mindset (adopt a regular structure) and thus cannot be applied to the automatic design of irregular structures. We proposed a clustering-reinforcement learning model named parallel advantage actor-critic with K-means (PAAC-K). The PAAC-K model is based on a reinforcement learning model, in which the size of the overlapping area is used as the reward function, and a clustering algorithm was added to extract characteristics for learning. We used the stepwise training method to avoid repeated exploration in a design with different frequencies. The PAAC-K model realized superhuman intelligence that automated the design of irregular structures, which was proven with four application examples. This work presents an AI model and a design mindset for irregular structures, which is of great significance in promoting the development of filter devices.

Graphic Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data generated and analysed during the current study are available from the corresponding author upon reasonable request.

References

  1. Guo X, Zhu L, Wu W (2019) Design of complex weighted feeding network based on generalized coupled-resonator filter theory. IEEE Transactions on Microwave Theory and Techniques 67(11):4376–4385. https://doi.org/10.1109/TMTT.2019.2939238

    Article  Google Scholar 

  2. Kishore S, Arora A, V. Phani Kumar K, et al (2021) Compact dual-band bandpass filter with high-passband isolation using coupled lines and open stub. Microwave and Optical Technology Letters 63(11):2710–2714. https://doi.org/10.1002/mop.32940

  3. Chen W, Wu Y, Wang W (2020) Planar wideband high-selectivity impedance-transforming differential bandpass filter with deep common-mode suppression. IEEE Transactions on Circuits and Systems II-Express Briefs 67(10):1914–1918. https://doi.org/10.1109/TCSII.2019.2959814

    Article  Google Scholar 

  4. Freno BA, Matula NR, Johnson WA (2021) Manufactured solutions for the method-of-moments implementation of the electric-field integral equation. Journal of Computational Physics 443. https://doi.org/10.1016/j.jcp.2021.110538

  5. Olm M, Badia S, Martin AF (2019) On a general implementation of h- and p-adaptive curl-conforming finite elements. Advances in Engineering Software 132:74–91. https://doi.org/10.1016/j.advengsoft.2019.03.006

    Article  Google Scholar 

  6. Jansson E, Thiringer T, Grunditz E (2020) Convergence of core losses in a permanent magnet machine, as function of mesh density distribution, a case-study using finite-element analysis. IEEE Transactions on Energy Conversion 35(3):1667–1675. https://doi.org/10.1109/TEC.2020.2982265

    Article  Google Scholar 

  7. Luo HY, Shao W, Ding X et al (2022) Shape modeling of microstrip filters based on convolutional neural network. IEEE Microwave and Wireless Components Letters 32(9):1019–1022. https://doi.org/10.1109/LMWC.2022.3162414

    Article  Google Scholar 

  8. Wu Y, Pan G, Lu D et al (2022) Artificial neural network for dimensionality reduction and its application to microwave filters inverse modeling. IEEE Transactions on Microwave Theory and Techniques 70(11):4683–4693. https://doi.org/10.1109/TMTT.2022.3161928

    Article  Google Scholar 

  9. Zhao P, Wu K (2020) Homotopy optimization of microwave and millimeter-wave filters based on neural network model. IEEE Transactions on Microwave Theory and Techniques 68(4):1390–1400. https://doi.org/10.1109/tmtt.2019.2963639

    Article  Google Scholar 

  10. Feng F, Na W, Jin J et al (2022) Artificial neural networks for microwave computer-aided design: The state of the art. IEEE Transactions on Microwave Theory and Techniques 70(11):4597–4619. https://doi.org/10.1109/TMTT.2022.3197751

    Article  Google Scholar 

  11. Dai X, Yang Q, Du H et al (2021) Direct synthesis approach for designing high selectivity microstrip distributed bandpass filters combined with deep learning. AEU - International Journal of Electronics and Communications 131(153):499. https://doi.org/10.1016/j.aeue.2020.153499

    Article  Google Scholar 

  12. Yahya SI, Rezaei A, Nouri L (2021) The use of artificial neural network to design and fabricate one of the most compact microstrip diplexers for broadband l-band and s-band wireless applications. Wireless Networks 27(1):663–676. https://doi.org/10.1007/s11276-020-02478-x

    Article  Google Scholar 

  13. Salehi MR, Noori L, Abiri E (2016) Prediction of matching condition for a microstrip subsystem using artificial neural network and adaptive neuro-fuzzy inference system. International Journal of Electronics 103(11):1882–1893. https://doi.org/10.1080/00207217.2016.1138539

    Article  Google Scholar 

  14. Na WC, Zhang QJ (2014) Automated knowledge-based neural network modeling for microwave applications. IEEE Microwave and Wireless Components Letters 24(7):499–501. https://doi.org/10.1109/LMWC.2014.2316251

    Article  Google Scholar 

  15. Pinchuk P, Margot JL (2022) A machine learning-based direction-of-origin filter for the identification of radio frequency interference in the search for techno signatures. Astronomical Journal 163(2). https://doi.org/10.3847/1538-3881/ac426f

  16. Singh P, Singh VK, Lala A, et al (2018) Design and analysis of microstrip antenna using multilayer feed-forward back-propagation neural network (MLPFFBP-ANN). In: Advances in Communication, Devices and Networking. Springer Singapore, pp 393–398, https://doi.org/10.1007/978-981-10-7901-6_43

  17. Sharma K, Pandey GP (2021) Efficient modelling of compact microstrip antenna using machine learning. AEU - International Journal of Electronics and Communications 135(153):739. https://doi.org/10.1016/j.aeue.2021.153739

    Article  Google Scholar 

  18. Ai MX, Xie YF, Tang ZH et al (2021) Deep learning feature-based setpoint generation and optimal control for flotation processes. Information Sciences 578:644–658. https://doi.org/10.1016/j.ins.2021.07.060

    Article  MathSciNet  Google Scholar 

  19. Shen CY, Gonzalez Y, Chen LY et al (2018) Intelligent parameter tuning in optimization-based iterative ct reconstruction via deep reinforcement learning. IEEE Transactions on Medical Imaging 37(6):1430–1439. https://doi.org/10.1109/Tmi.2018.2823679

    Article  Google Scholar 

  20. Harandi MT, Nili Ahmadabadi M, Araabi BN (2009) Optimal local basis: A reinforcement learning approach for face recognition. International Journal of Computer Vision 81(2):191–204. https://doi.org/10.1007/s11263-008-0161-5

    Article  Google Scholar 

  21. Hong RC, Liu DQ, Mo XY et al (2022) Learning to compose and reason with language tree structures for visual grounding. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(2):684–696. https://doi.org/10.1109/Tpami.2019.2911066

    Article  Google Scholar 

  22. Chen L, Cui J, Tang X et al (2022) Rlpath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning. Applied Intelligence 52(4):4715–4726. https://doi.org/10.1007/s10489-021-02672-0

    Article  Google Scholar 

  23. Lee Y, Shin J, Kim Y (2021) Simultaneous neural machine translation with a reinforced attention mechanism. Etri Journal 43(5):775–786. https://doi.org/10.4218/etrij.2020-0358

    Article  Google Scholar 

  24. Xu C, Li Q, Zhang D et al (2020) Deep successor feature learning for text generation. Neurocomputing 396:495–500. https://doi.org/10.1016/j.neucom.2018.11.116

    Article  Google Scholar 

  25. Afshar M, Phillips A, Karnik N et al (2019) Natural language processing and machine learning to identify alcohol misuse from the electronic health record in trauma patients: development and internal validation. Journal of the American Medical Informatics Association 26(3):254–261. https://doi.org/10.1093/jamia/ocy166

    Article  Google Scholar 

  26. Prollochs N, Feuerriegel S, Lutz B et al (2020) Negation scope detection for sentiment analysis: A reinforcement learning framework for replicating human interpretations. Information Sciences 536:205–221. https://doi.org/10.1016/j.ins.2020.05.022

    Article  Google Scholar 

  27. Li GH, Dong M, Ming LF et al (2022) Deep reinforcement learning based ensemble model for rumor tracking. Information Systems 103(101):772. https://doi.org/10.1016/j.is.2021.101772

    Article  Google Scholar 

  28. Swetha NG, Karpagam GR (2022) Reinforcement learning infused intelligent framework for semantic web service composition rl infused intelligent framework for swsc. Applied Intelligence 52(2):1979–2000. https://doi.org/10.1007/s10489-021-02351-0

    Article  Google Scholar 

  29. Xiao Y, Nazarian S, Bogdan P (2021) Plasticity-on-chip design: Exploiting self-similarity for data communications. IEEE Transactions on Computers 70(6):950–962. https://doi.org/10.1109/Tc.2021.3071507

    Article  MATH  Google Scholar 

  30. Lee J, Koh H, Choe HJ (2021) Learning to trade in financial time series using high-frequency through wavelet transformation and deep reinforcement learning. Applied Intelligence 51(8):6202–6223. https://doi.org/10.1007/s10489-021-02218-4

    Article  Google Scholar 

  31. Licks GP, Couto JC, Miehe PD et al (2020) Smartix: A database indexing agent based on reinforcement learning. Applied Intelligence 50(8):2575–2588. https://doi.org/10.1007/s10489-020-01674-8

    Article  Google Scholar 

  32. Ishita SZ, Ahmed CF, Leung CK (2022) New approaches for mining regular high utility sequential patterns. Applied Intelligence 52(4):3781–3806. https://doi.org/10.1007/s10489-021-02536-7

    Article  Google Scholar 

  33. Xu JY, Yao L, Li L et al (2020) Argumentation based reinforcement learning for meta-knowledge extraction. Information Sciences 506:258–272. https://doi.org/10.1016/j.ins.2019.07.094

    Article  MATH  Google Scholar 

  34. Li T, Wang ZJ, Yang GY et al (2021) Semi-selfish mining based on hidden markov decision process. International Journal of Intelligent Systems 36(7):3596–3612. https://doi.org/10.1002/int.22428

    Article  Google Scholar 

  35. Karimi M, Hasanzadeh A, Shen Y (2020) Network-principled deep generative models for designing drug combinations as graph sets. Bioinformatics 36:445–454. https://doi.org/10.1093/bioinformatics/btaa317

    Article  Google Scholar 

  36. Chong B, Yang YG, Wang ZL et al (2021) Reinforcement learning to boost molecular docking upon protein conformational ensemble. Physical Chemistry Chemical Physics 23(11):6800–6806. https://doi.org/10.1039/d0cp06378a

  37. Tan RK, Liu Y, Xie L (2022) Reinforcement learning for systems pharmacology-oriented and personalized drug design. Expert Opinion on Drug Discovery 17(8):849–863. https://doi.org/10.1080/17460441.2022.2072288

    Article  Google Scholar 

  38. Paiva Tomaz LB, Silva Julia RM, Duarte VA (2018) A multiagent player system composed by expert agents in specific game stages operating in high performance environment. Applied Intelligence 48(1):1–22. https://doi.org/10.1007/s10489-017-0952-x

    Article  Google Scholar 

  39. Li SX, Li O, Liu GY, et al (2021) Trajectory based prioritized double experience buffer for sample-efficient policy optimization. IEEE Access 9:101,424–101,432. https://doi.org/10.1109/ACCESS.2021.3097357

  40. Xu R, Lieberherr K (2020) Learning self-play agents for combinatorial optimization problems. The Knowledge Engineering Review 35:11. https://doi.org/10.1017/S026988892000020X

    Article  Google Scholar 

  41. Liu J, Chen ZX, Dong WH, et al (2019) Microwave Integrated Circuits Design with Relational Induction Neural Network. arXiv e-prints arXiv:1901.02069. https://arxiv.org/abs/1901.02069

  42. Clemente AV (2017) Decoupling deep learning and reinforcement learning for stable and efficient deep policy gradient algorithms. Master’s thesis, Norwegian University of Science and Technology Trondheim

  43. Sinaga KP, Yang MS (2020) Unsupervised k-means clustering algorithm. IEEE Access 8:80,716–80,727. 10.1109/ACCESS.2020.2988796

  44. Zheng Y, Li X, Xu L (2020) Balance control for the first-order inverted pendulum based on the advantage actor-critic algorithm. International Journal of Control Automation and Systems 18(12):3093–3100. https://doi.org/10.1007/s12555-019-0278-z

    Article  Google Scholar 

  45. Arulkumaran K, Deisenroth MP, Brundage M et al (2017) Deep reinforcement learning a brief survey. IEEE Signal Processing Magazine 34(6):26–38. https://doi.org/10.1109/MSP.2017.2743240

    Article  Google Scholar 

  46. Yang L, Zhu L, Choi WW et al (2018) Wideband balanced-to-unbalanced bandpass filters synthetically designed with chebyshev filtering response. IEEE Transactions on Microwave Theory and Techniques 66(10):4528–4539. https://doi.org/10.1109/tmtt.2018.2860949

    Article  Google Scholar 

  47. Shuang W, Yan L, Jiu-sheng L (2017) Compact dual-band bandpass filter using a stepped impedance resonator for wlan/wimax application. In: 2017 7th IEEE International Symposium on Microwave, Antenna, Propagation, and EMC Technologies (MAPE), pp 180–183, https://doi.org/10.1109/MAPE.2017.8250830

  48. Liu XB, Yang SH, Wang HQ, et al (2022) New lithium bismuth phosphate ceramic: crystal structure, microstructure, microwave dielectric properties and co-firing compatibility with aluminum electrode. Journal of Materials Science-Materials in Electronics 33(13):10,114–10,120. https://doi.org/10.1007/s10854-022-08001-6

  49. Krishna VN, Padmasine KG (2023) A review on microwave band pass filters: Materials and design optimization techniques for wireless communication systems. Materials Science in Semiconductor Processing 154. https://doi.org/10.1016/j.mssp.2022.107181

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Guangxi (No. 2019GXNSFBA185037), Science and Technology Base and Talent Special Project of Guangxi (No. AD19110017) and Guangxi Key Laboratory of Embedded Technology and Intelligent Systems (No. 2020-2-1).

Author information

Authors and Affiliations

  1. Guangxi Key Laboratory of Embedded Technology and Intelligent Systems, Guilin University of Technology, Guilin, 514004, Guangxi, China

    Sheng-Hui Yang, Xiao-Bin Liu, Tian-Jian Tan, Lei Zhang, Chang Su, Huan-Fu Zhou & Xiao-Lan Xie

  2. College of Information Science and Engineering, Guilin University of Technology, Guilin, 514004, Guangxi, China

    Sheng-Hui Yang, Xiao-Bin Liu, Tian-Jian Tan, Lei Zhang, Chang Su & Xiao-Lan Xie

  3. School of Materials Science and Engineering, Guilin University of Technology, Guilin, 514004, Guangxi, China

    Huan-Fu Zhou

Authors
  1. Sheng-Hui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao-Bin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tian-Jian Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chang Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huan-Fu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiao-Lan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Sheng-Hui Yang: Formal Analysis; Methodology; Software; Validation; Visualization; Writing-original draft preparation. Xiao-Bin Liu: Conceptualization; Formal Analysis; Funding acquisition; Methodology; Supervision; Writing-original draft preparation; Writing-review and editing. Tian-Jian Tan: Software; Validation; Visualization. Lei Zhang: Formal Analysis; Software; Validation. Chang Su: Software; Validation. Huan-Fu Zhou: Writing-review and editing. Xiao-Lan Xie: Methodology; Writing-review and editing.

Corresponding author

Correspondence to Xiao-Bin Liu.

Ethics declarations

Ethical and informed consent for data used

This article does not contain any studies with human participants or animals performed by any of the authors. Informed consent was obtained from all individual participants included in the study.

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, SH., Liu, XB., Tan, TJ. et al. Realization of superhuman intelligence in microstrip filter design based on clustering-reinforcement learning. Appl Intell 53, 22938–22951 (2023). https://doi.org/10.1007/s10489-023-04638-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04638-w

Keywords

Search

Navigation