Skip to main content
SpringerLink
Log in

OR-Benchmark: An Open and Reconfigurable Digital Watermarking Benchmarking Framework

  • Conference paper
  • First Online:
Computer Security (ESORICS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12580))

Included in the following conference series:

  • 796 Accesses

Abstract

Benchmarking digital watermarking algorithms is not an easy task because different applications of digital watermarking often have very different sets of requirements and trade-offs between conflicting requirements. While there have been some general-purpose digital watermarking benchmarking systems available, they normally do not support complicated benchmarking tasks and cannot be easily reconfigured to work with different watermarking algorithms and testing conditions. In this paper, we propose OR-Benchmark, an open and highly reconfigurable general-purpose digital watermarking benchmarking framework, which has the following two key features: 1) all the interfaces are public and general enough to support all watermarking applications and benchmarking tasks we can think of; 2) end users can easily extend the functionalities and freely configure what watermarking algorithms are tested, what system components are used, how the benchmarking process runs, and what results should be produced. We implemented a prototype of this framework as a MATLAB software package and demonstrate how it can be used in three typical use cases. The first two use cases show how easily we can define benchmarking profiles for some robust image watermarking algorithms. The third use case shows how OR-Benchmark can be configured to benchmark some image watermarking algorithms for content authentication and self-restoration, which cannot be easily supported by other digital watermarking benchmarking systems.

A prototype of OR-Benchmark as a MATLAB software package is implemented and the source code will be released soon under an open source license.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cox, I., Miller, M., Bloom, J., Fridrich, J., Kalker, T.: Digital Watermarking and Steganography. Morgan Kaufmann Publishers, Burlington (2007)

    Google Scholar 

  2. Guitart, O., Kim, H.C., III, E.J.D.: Watermark evaluation testbed. J. Electron. Imaging 041106 (2006). https://doi.org/10.1117/1.2400067

  3. He, H., Chen, F., Tai, H.M., Kalker, T., Zhang, J.: Performance analysis of a block-neighborhood based self-recovery fragile watermarking scheme. IEEE Trans. Inf. Forensics Secur. 7(1), 185–196 (2012). https://doi.org/10.1109/TIFS.2011.2162950

    Article  Google Scholar 

  4. Ho, A.T.S., Zhu, X., Shen, J., Marziliano, P.: Fragile watermarking based on encoding of the zeroes of the \(z\)-transform. IEEE Trans. Inf. Forensics Secur. 3(3), 567–569 (2008). https://doi.org/10.1109/TIFS.2008.926994

    Article  Google Scholar 

  5. Houmansadr, A., Kiyavash, N., Borisov, N.: Non-blind watermarking of network flows. IEEE/ACM Trans. Networking 22(4), 1232–1244 (2014). https://doi.org/10.1109/TNET.2013.2272740

    Article  Google Scholar 

  6. Kim, H.C., Lin, E.T., Guitart, O., Delp III, E.J.: Further progress in watermark evaluation testbed (WET). In: Security, Steganography, and Watermarking of Multimedia Contents VII. Proceedings of SPIE, vol. 5681, pp. 241–251 (2005). https://doi.org/10.1117/12.593803

  7. Kuhn, M.: StirMark - image-watermarking robustness test. http://www.cl.cam.ac.uk/~mgk25/stirmark.html

  8. Kutter, M., Petitcolas, F.A.P.: A fair benchmark for image watermarking systems. In: Security and Watermarking of Multimedia Contents. Proceedings of SPIE, vol. 3657, pp. 226–239 (1999). https://doi.org/10.1117/12.344672

  9. Lang, A.: StirMark Benchmark for Audio - SMBA. http://omen.cs.uni-magdeburg.de/alang/smba.php

  10. Li, G., Pei, S., Chen, G., Cao, W., Wu, B.: A self-embedded watermarking scheme based on relationship function of corresponding inter-blocks DCT coefficient. In: Proceedings of CSCWD 2009, pp. 107–112. https://doi.org/10.1109/CSCWD.2009.4968043

  11. Li, W., Xue, X., Lu, P.: Localized audio watermarking technique robust against time-scale modification. IEEE Trans. Multimedia 8(1), 60–69 (2006). https://doi.org/10.1109/TMM.2005.861291

    Article  Google Scholar 

  12. Lin, C.Y., Chang, S.F.: Semi-fragile watermarking for authenticating JPEG visual content. In: Security and Watermarking of Multimedia Contents II. Processings of SPIE, vol. 3971, pp. 140–151 (2000). https://doi.org/10.1117/12.384968

  13. Lin, C.Y., Wu, M., Bloom, J.A., Cox, I.J., Miller, M.L., Lui, Y.M.: Rotation, scale, and translation resilient watermarking for images. IEEE Trans. Image Process. 10(5), 767–782 (2001). https://doi.org/10.1109/83.918569

    Article  MATH  Google Scholar 

  14. Lugan, S., Macq, B.: Thread-based benchmarking deployment. In: Security, Steganography and Watermarking of Multimedia Contents VI. Proceedings of SPIE, vol. 5306, pp. 248–255 (2004). https://doi.org/10.1117/12.538692

  15. Macq, B., Dittmann, J., Delp, E.J.: Benchmarking of image watermarking algorithms for digital rights management. Proc. IEEE 92, 971–984 (2004). https://doi.org/10.1109/JPROC.2004.827361

    Article  Google Scholar 

  16. Maeno, K., Sun, Q., Chang, S.F., Suto, M.: New semi-fragile image authentication watermarking techniques using random bias and nonuniform quantization. IEEE Trans. Multimedia 8(1), 32–45 (2006). https://doi.org/10.1109/TMM.2005.861293

    Article  Google Scholar 

  17. Michiels, B., Macq, B.: Benchmarking image watermarking algorithms with Openwatermark. In: Proceedings of EUSIPCO 2006 (2006)

    Google Scholar 

  18. Ni, Z., Shi, Y.Q., Ansari, N., Su, W., Sun, Q., Lin, X.: Robust lossless image data hiding designed for semi-fragile image authentication. IEEE Trans. Circuits Syst. Video Technol. 18(4), 497–509 (2008). https://doi.org/10.1109/TCSVT.2008.918761

    Article  Google Scholar 

  19. Pereira, S., Voloshynovskiy, S., Madueno, M., Marchand-Maillet, S., Pun, T.: Second generation benchmarking and application oriented evaluation. In: Moskowitz, I.S. (ed.) IH 2001. LNCS, vol. 2137, pp. 340–353. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45496-9_25

    Chapter  Google Scholar 

  20. Petitcolas, F.A.P., Steinebach, M., Raynal, F., Dittmann, J., Fontaine, C., Fates, N.: A public automated web-based evaluation service for watermarking schemes: StirMark benchmark. In: Security and Watermarking of Multimedia Contents III. Proceedings of SPIE, vol. 4314, pp. 575–584 (2001). https://doi.org/10.1117/12.435442

  21. Petitcolas, F.: Stirmark benchmark 4.0. http://www.cl.cam.ac.uk/~fapp2/watermarking/stirmark/

  22. Petitcolas, F.A.P.: Watermarking scheme evaluation. IEEE Signal Process. Mag. 17(5), 58–64 (2000). https://doi.org/10.1109/79.879339

    Article  Google Scholar 

  23. Petitcolas, F.A.P., Anderson, R.J., Kuhn, M.G.: Attacks on copyright marking systems. In: Aucsmith, D. (ed.) IH 1998. LNCS, vol. 1525, pp. 218–238. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49380-8_16

    Chapter  Google Scholar 

  24. Podilchuk, C.I., Delp, E.J.: Digital watermarking: algorithms and applications. IEEE Signal Process. Mag. 18(4), 33–46 (2001). https://doi.org/10.1109/79.939835

    Article  Google Scholar 

  25. Solachidis, V., Tefas, A., Nikolaidis, N., Tsekeridou, S., Nikolaidis, A., Pitas, I.: A benchmarking protocol for watermarking methods. In: Proceedings of ICIP 2001, pp. 1023–1026. https://doi.org/10.1109/ICIP.2001.958300

  26. Steinebach, M., et al.: StirMark benchmark: audio watermarking attacks. In: Proceedings of ITCC 2001, pp. 49–54. https://doi.org/10.1109/ITCC.2001.918764

  27. Stütz, T., Autrusseau, F., Uhl, A.: Non-blind structure-preserving substitution watermarking of H.264/CAVLC inter-frames. IEEE Trans. Multimedia 16(5), 1337–1349 (2014). https://doi.org/10.1109/TMM.2014.2310595

  28. Vorbrüggen, J.C., Cayre, F.: The Certimark benchmark: architecture and future perspectives. In: Proceedings of ICME 2002, pp. 485–488. https://doi.org/10.1109/ICME.2002.1035651

  29. Wang, H., Ho, A.T.S., Zhao, X.: a novel fast self-restoration semi-fragile watermarking algorithm for image content authentication resistant to jpeg compression. In: Shi, Y.Q., Kim, H.-J., Perez-Gonzalez, F. (eds.) IWDW 2011. LNCS, vol. 7128, pp. 72–85. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32205-1_8

    Chapter  Google Scholar 

  30. Wang, K., Lavoué, G., Denis, F., Baskurt, A., He, X.: A benchmark for 3D mesh watermarking. In: Proceedings of SMI, vol. 2010, pp. 231–235 (2010)

    Google Scholar 

  31. Wang, K., Lavoué, G., Denis, F., Baskurt, A.: A comprehensive survey on three-dimensional mesh watermarking. IEEE Trans. Multimedia 10(8), 1513–1527 (2008). https://doi.org/10.1109/TMM.2008.2007350

    Article  Google Scholar 

  32. Zhu, X., Ding, J., Dong, H., Hu, K., Zhang, X.: Normalized correlation-based quantization modulation for robust watermarking. IEEE Trans. Multimedia 16(7), 1888–1904 (2014). https://doi.org/10.1109/TMM.2014.2340695

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hangzhou Dianzi University, Hangzhou, China

    Hui Wang

  2. Tianjin University of Science and Technology, Tianjin, China

    Anthony T. S. Ho

  3. Wuhan University of Technology, Wuhan, China

    Anthony T. S. Ho

  4. University of Surrey, Guildford, UK

    Anthony T. S. Ho & Shujun Li

  5. University of Kent, Canterbury, UK

    Shujun Li

Authors
  1. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony T. S. Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shujun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Wang .

Editor information

Editors and Affiliations

  1. University of Surrey, Guildford, UK

    Ioana Boureanu

  2. University of Surrey, Guildford, UK

    Constantin Cătălin Drăgan

  3. University of Surrey, Guildford, UK

    Mark Manulis

  4. Technical University of Denmark, Kongens Lyngby, Denmark

    Thanassis Giannetsos

  5. Ionian University, Kerkira, Greece

    Christoforos Dadoyan

  6. UBITECH Ltd., Chalandri, Greece

    Panagiotis Gouvas

  7. Dartmouth College and Naval Information Warfare Center, Hanover, NH, USA

    Roger A. Hallman

  8. University of Kent, Canterbury, Kent, UK

    Shujun Li

  9. Teessside University, Middlesbrough, UK

    Victor Chang

  10. Technical University of Berlin, Berlin, Germany

    Frank Pallas

  11. Alexander von Humboldt Institute for Internet and Society, Berlin, Germany

    Jörg Pohle

  12. Ruhr University Bochum, Bochum, Germany

    Angela Sasse

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, H., Ho, A.T.S., Li, S. (2020). OR-Benchmark: An Open and Reconfigurable Digital Watermarking Benchmarking Framework. In: Boureanu, I., et al. Computer Security. ESORICS 2020. Lecture Notes in Computer Science(), vol 12580. Springer, Cham. https://doi.org/10.1007/978-3-030-66504-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-66504-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-66503-6

  • Online ISBN: 978-3-030-66504-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation