Skip to main content
SpringerLink
Log in

Copilot: monitoring embedded systems

  • SI: SwHM
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

Runtime verification (RV) is a natural fit for ultra-critical systems that require correct software behavior. Due to the low reliability of commodity hardware and the adversity of operational environments, it is common in ultra-critical systems to replicate processing units (and their hosted software) and incorporate fault-tolerant algorithms to compare the outputs, even if the software is considered to be fault-free. In this paper, we investigate the use of software monitoring in distributed fault-tolerant systems and the implementation of fault-tolerance mechanisms using RV techniques. We describe the Copilot language and compiler that generates monitors for distributed real-time systems, and we discuss two case-studies in which Copilot-generated monitors were used to detect onboard software and hardware faults and monitor air-ground data link messaging protocols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. To use conditionals (if-then-else’s) in Copilot specifications, as in Figs. 1 and 2, the GHC language extension RebindableSyntax must be set on.

  2. The function permutations comes from the Haskell standard library Data.list.

  3. http://hackage.haskell.org/package/atom.

  4. http://hackage.haskell.org/package/sbv.

  5. Two explanations are in order: (1) reify allows sharing in the expressions to be compiled [19], and >>= is a higher-order operator that takes the result of reification and “feeds” it to the compile function.

  6. http://www.cprover.org/cbmc/LICENSE. It is the user’s responsibility to ensure their use conforms to the license.

  7. Tape left on the static pitot tube of Aeroperú Flight 603 in 1996 resulted in the death of 70 passengers and crew [28].

  8. http://qgroundcontrol.org/mavlink/start.

  9. At the time of this writing, Copilot did not handle streams of arrays. Modeling the protocol as a stream of Word32s, as we explain herein, is inefficient, resulting in a large specification.

  10. Copilot’s nscanl is a fixed-length (of n) analogue of the Haskell scanl function in Haskell, such that scanl f z [x1, x2, ...] == [z, z ‘f‘ x1, (z ‘f‘ x1) ‘f‘ x2, ...].

  11. We could incorporate further analysis of the packets as well, like checking for the correct length of certain MAVLink packet types or inspection of the payload. Some of these tests could be derived from the MAVLink XML protocol description automatically.

  12. Latitude and longitude in degrees, altitude in meters.

  13. When streams of arrays are implemented in Copilot, the CRC can be derived from a Copilot specification.

References

  1. (2000) FAA system handbook. http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/

  2. (2010) Aeronautical radio: avionics application software standard interface: ARINC specification 653p1-3. ARINC, Inc., Annapolis. ARINC 653 Part 1

  3. (2012) Aeronautical radio: avionics application software standard interface: ARINC specification 653p2-2 extended services. ARINC Inc., Annapolis. ARINC 653 Part 2

  4. (2011) Aviation Today: more pitot tube incidents revealed. Aviation Today. http://www.aviationtoday.com/regions/usa/More-Pitot-Tube-Incidents-Revealed_72414.html

  5. Axelsson E, Claessen K, Dévai G, Horváth Z, Keijzer K, Lyckegård B, Persson A, Sheeran M, Svenningsson J, Vajda A (2010) Feldspar: a domain specific language for digital signal processing algorithms. In: 8th ACM/IEEE international conference on formal methods and models for codesign

  6. Barrett C, Sebastiani R, Seshia S, Tinelli C (2009) Satisfiability modulo theories, chap. 26, pp 825–885. In: Frontiers in artificial intelligence and applications. IOS Press, Amsterdam

  7. Bergin C (2008) Faulty MDM removed. NASA Spaceflight.com. http://www.nasaspaceflight.com/2008/05/sts-124-frr-debate-outstanding-issues-faulty-mdm-removed/. Downloaded 28 Nov 2008

  8. Bonakdarpour B, Kulkarni SS (2008) SYCRAFT: a tool for synthesizing distributed fault-tolerant programs. In: International conference on concurrency theory (CONCUR ’08). Springer, Berlin, pp 167–171

  9. Bonakdarpour B, Navabpour S, Fischmeister S (2011) Sampling-based runtime verification. In: 17th International symposium on formal methods (FM)

  10. Bureau ATS (2007) In-flight upset event 240 Km North-West of Perth, WA Boeing Company 777-200, 9M-MRG 1 August 2005. ATSB Transport Safety Investigation Report. Aviation Occurrace Report-200503722

  11. Butler RW, Finelli GB (1993) The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Trans Softw Eng 19:3–12

    Article  Google Scholar 

  12. Chen F, d’Amorim M, Roşu G (2006) Checking and correcting behaviors of java programs at runtime with Java-MOP. Electron Notes Theor Comput Sci 144:3–20

    Article  Google Scholar 

  13. Chen F, Roşu G (2005) Java-MOP: a monitoring oriented programming environment for Java. In: 11th International conference on tools and algorithms for the construction and analysis of systems (TACAS’05). LNCS, vol 3440. Springer, Berlin, pp 546–550

  14. Claessen K, Hughes J (2000) Quickcheck: a lightweight tool for random testing of haskell programs. In: ACM SIGPLAN notices. ACM, New York, pp 268–279

  15. Clarke E, Kroening D, Lerda F (2004) A tool for checking ANSI-C programs. In: Tools and algorithms for the construction and analysis of systems (TACAS). LNCS. Springer, Berlin, pp 168–176

  16. Dwyer M, Diep M, Elbaum S (2008) Reducing the cost of path property monitoring through sampling. In: Proceedings of the 23rd international conference on automated software engineering, pp 228–237

  17. Farhat H (2004) Digital design and computer organization, 1st edn. In: Digital Design and Computer Organization. CRC Press, Boca Raton

  18. Fischmeister S, Ba Y (2010) Sampling-based program execution monitoring. In: ACM International conference on Languages, compilers, and tools for embedded systems (LCTES), pp 133–142

  19. Gill A (2009) Type-safe observable sharing in Haskell. In: Proceedings of the 2009 ACM SIGPLAN Haskell Symposium

  20. Halbwachs N, Raymond P (1999) Validation of synchronous reactive systems: from formal verification to automatic testing. In: ASIAN’99 Asian computing science conference. LNCS, vol 1742. Springer, Berlin

  21. Havelund K (2008) Runtime verification of C programs. In: Testing of software and communicating systems (TestCom/FATES). Springer, Berlin, pp 7–22

  22. Hawkins T (2008) Controlling hybrid vehicles with Haskell. Presentation. Commercial Users of Functional Programming (CUFP). http://cufp.galois.com/2008/schedule.html

  23. Hesselink WH (2005) The Boyer–Moore majority vote algorithm

  24. Jones SP (ed) (2002) Haskell 98 language and libraries: the revised report. http://haskell.org/

  25. Kim M, Viswanathan M, Ben-Abdallah H, Kannan S, Lee I, Sokolsky O (1999) Formally specified monitoring of temporal properties. In: 11th euromicro conference on real-time systems, pp 114–122

  26. Klein G, Andronick J, Elphinstone K, Heiser G, Cock D, Derrin P, Elkaduwe D, Engelhardt K, Kolanski R, Norrish M, Sewell T, Tuch H, Winwood S (2010) seL4: Formal verification of an OS kernel. Commun ACM 53(6):107–115

    Article  Google Scholar 

  27. Krüger IH, Meisinger M, Menarini M (2007) Runtime verification of interactions: from MSCs to aspects. In: International conference on runtime verification. Springer, Berlin, pp 63–74

  28. Ladkin PB (2002) News and comment on the Aeroperu b757 accident; AeroPeru Flight 603, 2 October 1996. Online article RVS-RR-96-16. http://www.rvs.uni-bielefeld.de/publications/Reports/aeroperu-news.html

  29. Lamport L, Shostak R, Pease M (1982) The Byzantine generals problem. ACM Trans Program Lang Syst 4:382–401

    Article  MATH  Google Scholar 

  30. Leveson NG, Turner CS (1993) An investigation of the Therac-25 accidents. Computer 26:18–41

    Article  Google Scholar 

  31. Macaulay K (2008) ATSB preliminary factual report, in-flight upset, Qantas Airbus A330, 154 Km West of Learmonth, WA, 7 October 2008. Australian Transport Safety Bureau Media Release. http://www.atsb.gov.au/newsroom/2008/release/2008_45.aspx

  32. Mikác J, Caspi P (2005) Formal system development with Lustre: framework and example. Technical Report TR-2005-11, Verimag Technical Report. http://www-verimag.imag.fr/index.php?page=techrep-list&lang=en

  33. Moore SJ, Boyer RS (1981) MJRTY—a fast majority vote algorithm. Technical Report 1981-32, Institute for Computing Science, University of Texas

  34. Nuseibeh B (1997) Soapbox: Ariane 5: Who dunnit? IEEE Softw 14(3):15–16

    Article  Google Scholar 

  35. Pike L, Goodloe A, Morisset R, Niller S (2010) Copilot: a hard real-time runtime monitor. In: Runtime verification (RV), vol 6418. Springer, Berlin, pp 345–359

  36. Pike L, Wegmann N, Niller S, Goodloe A (2012) Experience report: do-it-yourself high-assurance compiler. In: Proceedings of the 17th ACM SIGPLAN conference on functional programming. ACM, New York

  37. RTCA (1992) Software considerations in airborne systems and equipment certification. RTCA, Inc., USA. RCTA/DO- 178B

  38. Rushby J (2008) Runtime certification. In: RV’08: Proceedings of runtime verification, Budapest, Hungary, March 30, 2008. Selected Papers. Springer, Berlin, pp 21–35

  39. Rushby J (2009) Software verification and system assurance. In: International conference on software engineering and formal methods (SEFM). IEEE, New York, pp 3–10

  40. Sammapun U, Lee I, Sokolsky O (2005) RT-MaC: runtime monitoring and checking of quantitative and probabilistic properties. In: Proceedings of the 11th IEEE international conference on embedded and real-time computing systems and applications, pp 147–153

  41. Stoller SD, Bartocci E, Seyster J, Grosu R, Havelund K, Smolka SA, Zadok E (2011) Runtime verification with state estimation. In: Proceedings of the 2nd international conference on runtime verification (RV’11)

Download references

Acknowledgments

This work was supported by NASA Contract NNL08AD13T. Portions of this work have been published as conference papers in Runtime Verification, 2010 and Runtime Verification, 2011. We wish to especially thank the following individuals: Ben Di Vito at the NASA Langley Research Center (NASA LaRC) monitored this contract, and Paul Miner and Eric Cooper, also at NASA LaRC, provided valuable input. Robin Morisset developed an earlier version of Copilot.

Author information

Authors and Affiliations

  1. Galois, Inc., Portland, USA

    Lee Pike

  2. University of Copenhagen, Copenhagen, Denmark

    Nis Wegmann

  3. Evertz Microsystems, Burlington, Canada

    Sebastian Niller

  4. NASA, Hampton, USA

    Alwyn Goodloe

Authors
  1. Lee Pike
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nis Wegmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian Niller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alwyn Goodloe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lee Pike.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pike, L., Wegmann, N., Niller, S. et al. Copilot: monitoring embedded systems. Innovations Syst Softw Eng 9, 235–255 (2013). https://doi.org/10.1007/s11334-013-0223-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-013-0223-x

Keywords

Search

Navigation